Why GPU Capacity Comes in Three-Year Rectangles — Erik Bernhardsson, Modal

Vibhu welcomes Erik Bernhardsson from Modal and asks him to explain the business. Erik’s short version: they build AI infrastructure. Modal offers a compute platform and a storage platform that companies such as Decagon, Suno, Runway and Cognition use to build AI applications.

In practice that means a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs. Customers get a software SDK so they can move fast; Modal manages the capacity underneath and makes it easier to deploy, scale and run those workloads in production.

Vibhu notes that Erik was on Latent Space about two years earlier and asks how much bigger the company is now, whatever numbers he can share. Erik does not remember the exact baseline, but he puts the growth at at least 50× in revenue and in users.

Vibhu says he actually uses Modal for a side project that turns 2D images into 3D models — just hosting a couple of models. Erik asks for feedback. Vibhu likes it; that is why he uses it over the competitors. Before Modal, he says, Erik was at Spotify and also at Better.com.

That is right, Erik says. At Spotify he did a lot of different things, but his main contribution was the music recommendation system. He joined after school around 2008 or 2009, stayed seven years, and spent most of that time building recommendations from scratch with a small team.

They launched related artists, Discover Weekly and other features — a lot of matrix factorization, a lot of compute-intensive work. The genesis of Modal, he argues, was building those AI applications and realizing the infrastructure was terrible. There had to be a better way. Modal is the tool he always wanted at Spotify.

Vibhu asks whether Modal is mostly an offshoot of Spotify, not Better.com. Better was a side tour, Erik says, a roller coaster they have talked about separately. He was CTO for five years and grew the tech team from a handful of people to three or four hundred. He learned management, hiring and developer productivity, and did more full-stack work, then came back to infrastructure and machine learning, which was closer to his heart.

Better.com did mortgages. How did he get into that? Erik does not want to linger. Zach Frankel, whom he did not know at the time, recruited him as CTO when the company had ten people. The founder, Vishal, was very smart and a little crazy in mostly positive ways; they got along. They started building in 2015. Mortgages turned out to be hard, and interest rates jumped after the pandemic, which caused a lot of trouble — but Erik had already left.

That was around 2020 or 2021. Did he leave with the Modal idea already in hand? It is hard to say. After Better he had a lot of random ideas and was not sure he wanted to start a company. During the pandemic he took something like a sabbatical with two kids, then two and four, hacking part-time, considering consulting. He got pumped about the idea that became Modal — but it was maybe the thirtieth thing he built. A few attempts before that went nowhere.

The first idea that felt like a real platform was Modal. Were the earlier ideas also developer tools? The first was some kind of web scraping, which went nowhere — though in hindsight, Erik says, maybe it could have gone somewhere. This was 2021, before ChatGPT, which launched in late 2022.

He rewinds to Spotify. There he had released Luigi, an open-source workflow orchestrator in the same category as Airflow and Prefect — a dated category now, but it got a lot of attention for a while. He never turned it into a business. In 2021, looking for something to work on, he came back to orchestration: running code in the cloud.

What he realized was that the core puzzle was not just orchestrating compute. It was executing it. That was the bigger problem. He left scheduling on the shelf, telling himself he would come back to it, and never did. Modal’s focus became making it easy to run code in the cloud, and at first it was fairly general purpose.

They have gone full circle more recently, but the original idea was simple: make it super easy to run code in the cloud, GPU and AI workloads or data jobs. They obsessed over scale and developer experience, and realized early that they would have to go very deep in the stack. They built their own filesystem and container runtime and took on hard engineering problems.

ChatGPT was not quite the killer app. That same fall, Stable Diffusion came out, and it was. Modal had four or five people. Suddenly a lot of users wanted to run a Stable Diffusion model, and that made them realize generative-AI inference was a really good use case.

How did they find those users? Erik is a compulsive tweeter — less so now that the company distracts him — and he blogged a lot, so he had a following. Early growth was him tweeting about Modal, getting people excited, and writing long posts about where the cloud was going. He calls it founder-led marketing: lean into what you are good at. What he is good at, he says, is shitposting and being terminally online. A lot of growth still comes from being good at social media.

How long until the first customer, and did they raise early? It took a while, and they did not raise for a while. Erik self-funded through the first few employees, which made him frugal and focused on ROI. He did not take a salary for a long time; at some point that became a problem because he needed the money. The seed round came in early 2022, after about a year of work.

Akshat Bubna joined as CTO and cofounder around August 2021, about five years before this conversation. Meaningful revenue did not show up until late 2022. Stable Diffusion was the thing. They went from about $100 a month in November 2022 to about $2,000 the next month, then about $30,000 in January 2023. Erik felt a massive anxiety lift: maybe they were onto something. They have been growing very fast since.

Was that $30k-a-month moment a couple of big customers or a broad base? It was one. Erik is not sure he has named them, or if they even know, but he thinks they suspect it: a New York Stable Diffusion company called Mage. In the first quarter of 2023 they were 80 or 90 percent of revenue. Then it diversified. Late 2023, Suno became a big customer and a large chunk of revenue for a while. Over time they have much better diversification.

Vibhu remembers using Modal for ComfyUI and assumes a lot of people were running ComfyUI workflows on it. Some were, Erik says, but Modal users tended to be more low-level: custom workflows, adapters, their own inference, lots of optimizations. Modal has always been pretty hackable. The work they had done for developer experience turned out to be good for production too, because autoscaling is a similar problem: spinning containers up and down very fast. That is how they were positioned to catch those early workloads.

Did tools for autoscaling not exist? Why build them? Erik’s thesis is that every new era of technology runs up against the limits of the last generation of tools — the web, the cloud, mobile, PCs, now AI. The tools exist, but they are often not particularly good. You have to rethink them to really turbocharge the new world. The cloud needed Kubernetes and Docker. Generative AI can be built on that prior generation, but those systems were built for a different workload.

They were built for steady backend services: an even number of requests, running at pretty low utilization, because CPUs are cheap and people do not worry much about cost. Now GPUs are expensive, so you care about capacity and utilization. Models are large and have to be loaded over the network onto GPU. The old tools were not built for that. Modal felt they had to rebuild a lot of it rather than wrap the old stack.

Besides the filesystem, they spent years on CPU snapshotting, GPU snapshotting, an autoscaler, a scheduler and capacity-management tools. You cannot build this generation of software as a wrapper on the last one. You kind of need to start over.

Did they see weird behavior or breakages while scaling? Capacity has been a challenge from time to time: how do you get hundreds, then thousands, then tens of thousands of GPUs? They had to go multi-region very early, a fateful decision in hindsight. They always treated capacity as one big pool over the entire world, not separate clusters in each region. That was one of the more consequential early choices.

There was no single cartoonish incident — they have had bugs and outages, and Erik would love 100 percent uptime. Recently they have basically hit four or five nines, but through Modal’s history they have run up against scaling issues. Vibhu turns to the GPU market: do they use AWS, GCP and the other hyperscalers, and do they also buy infrastructure?

Erik wants to talk about cloud capacity, because it has changed a lot. For the first few years almost all of it came from the hyperscalers, mostly spot, some on-demand, in many regions. They started on AWS, added GCP, and later Azure. They rented from Oracle pretty early because they wanted A10s and A100s and got really good rates. Over time they added more clouds and some reservations. Until the end of last year, most capacity was still either spot or one-year reservations.

Spot means you are renting it right then, and it is preemptible, which they had to deal with. It worked, in aggregate, because they could pull capacity from hundreds of regions. That model worked until two things happened at once: Modal started growing very fast and needed more GPUs, and the cloud market itself changed.

In the last year, to get the capacity they need, they have had to work with a much larger set of clouds — almost 25 at this point. In many cases, to get what they want, they have to sign three-plus-year deals, pay a meaningful amount upfront, and lock in pricing on a three-year term. The last six to nine months have been spent rethinking how they get capacity.

They had to build a procurement desk: about four people working full time, calling neoclouds, doing deals, vetting capacity. They also built an engineering team of about ten just to manage bare metal around the world — network issues, bad drives, GPUs falling off the bus. They still do not own capacity. They rent it, typically on those three-year deals. Whether to buy or keep renting is a live decision.

Erik is not mega-dogmatic. Renting is capital-efficient and avoids the operational mess of vendors, power and cables. Buying has financial complexity too: raise equity and spend it on hardware, which is very dilutive, or raise debt, which usually wants locked-in long-term revenue to cover the interest. They look at the economics, the operational load and the risks. They are open to buying. Locking up 2027 capacity will take a lot of work and a lot of avenues.

Does that mean hiring a lot of finance people? Not really, though Erik has had to spend a lot of time on it. These are very expensive assets. AI investment is starting to look like several percentage points of US GDP, and he has watched US interest rates shoot up in the last few weeks. Look at the neoclouds: they are very sophisticated financial-engineering companies. Modal might have to do similar things.

He would rather stay a lean software company doing what they are good at. In general he does not think software companies should own hard assets, any more than they should own their office. But to get the capacity they want, everything is on the table.

Vibhu suggests supply will be constrained for a while, so they kind of have to. Does Erik agree, or is this a moment in time? He does not necessarily agree. NVIDIA is delivering several hundred thousand Blackwells a month. The market is not perfectly efficient, but if you are willing to pay more than everyone else, you can get GPUs. Anthropic and OpenAI are buying as much as they can. Supply is constantly coming onto the market. It is not a fully cornered resource.

Prices are going up, and that is worth reflecting on, because the normal state of hardware is gravity: eventually prices go down. This is a weird period of rising hardware prices. It could last another six months or another 36. He does not buy the idea that you must own capacity — that sounds a bit like the pandemic toilet-paper run. Capacity will be tight for a while, and prices are going up. Luckily, that is not even necessarily bad for Modal.

Scarcity means they can charge customers more, and with reservations locked in at a rate they are already effectively long GPU prices. He does not worry much about prices going up. He worries more about prices collapsing. He does not think that happens soon, but it would be quite bad for a lot of players.

Why would falling prices be bad? Because anyone who locked in a lot of supply paid too much. There is a risk of buying at the top. Hardware prices tend to fall in the very long run, so it is worth keeping that in mind. Right now constrained capacity creates a scarcity rent. Most of it goes to NVIDIA; some of it goes to neoclouds and anyone who owns capacity. Those better economics are undeniable today.

Long term he is less sure. There will always be a little premium to owning GPUs, but is it worth holding very expensive assets on the balance sheet? They will see over a few years. They might own some, rent some, do joint ventures, complex SPVs — everything is on the table.

How do they compete with OpenAI and Anthropic for GPUs, if those labs want to buy literally everything? They are. That is part of why prices are going up. NVIDIA is also shipping a lot, and Erik thinks NVIDIA prefers a little diversification: not a small number of huge buyers. Hyperscalers already get constrained allocation. He thinks NVIDIA likes seeing neoclouds and alternative clouds get more chips.

More speculation: smaller, slightly riskier clouds are an opportunity for Modal. His feeling is that Anthropic and OpenAI prefer to buy from the tried-and-true, partly because training runs need very large blocks. There is plenty of other capacity. You can get 32 nodes in Armenia, 128 nodes in Indonesia. Modal loves that. They will go get smaller pools all over the world at pretty good prices.

If someone offers GPUs, is that all they care about, or do they worry that it is a janky warehouse? Sometimes it feels like a janky warehouse. They send vendors a spec sheet: IOPS, a CPU PassMark floor, north-south bandwidth, egress and ingress. They try to do an acceptance period with one node and tests over time. Signing tens or hundreds of millions of dollars with a vendor they still do not fully know still takes a leap of faith.

Over time they build something like SemiAnalysis’s ClusterMAX, an internal Modal-max score. Erik knows which vendors he likes. They prefer more deals with the ones that have been reliable — and not just uptime: how responsive they are on support, whether they fix what Modal asks for, and how fast they recover when something goes down.

Vibhu asks whether demand for intelligence is basically infinite, and if so why not buy as much capacity as possible and resell whatever they overbuy. Erik says that is easy to say when a resource is scarce. If they bought everything today they could not utilize all of it — a lot of it, yes, but not all. Vibhu says they could sell the rest. The challenge is correlation: if prices fall, utilization falls, and the resale market vanishes at the same time, you are stuck.

He is extremely aggressive about buying capacity and raising money to do it. Overall he is very bullish. He still wants to point out the tail risks, and that when they hit they can get very bad. Part of it is age: his first job was in 1999, so he saw the dot-com crash and later the financial crisis. He wants a company that is around for 20, 30, 50 more years. They could try to corner the market. Does Vibhu know the Hunt brothers? They tried to corner silver and failed spectacularly. Cornering is generally not a good idea.

They are still very aggressive, ramping on a plan that matches demand growth and where they want to be at the end of next year on run rate. Are they only buying NVIDIA, or looking at other accelerators? They are looking. Alternative accelerators are starting to get interesting. Ironically, every year for five or even ten years he has said next year will be the year of non-NVIDIA silicon, and he has always been wrong. He thinks 2027 might finally be that year. We will see if he is wrong again.

The scores on certain workloads are getting really good. TPUs are used widely inside Google and seem to work well; he does not know as much about the training side. Hardware is hard; software is easier, which makes him think that if the software layer for these accelerators gets fixed, demand will follow. NVIDIA is a bottleneck, so more alternatives are likely — though TSMC and ASML may be deeper bottlenecks further down the pipe.

Long term, monopolies do not always last. It would be healthy for the market to have more diversity. He wants to be clear that he loves NVIDIA: they are an amazing company with an amazing product. He still expects more diversity, including names like Cerebras and Groq.

What does he think of those providers? They are pretty exciting, and they do really well on certain workloads. There is interesting hybrid, disaggregated inference: NVIDIA for prefill, Cerebras for decode. Vibhu has not heard of that. Groq too, because some phases are compute-bound and some are memory-bound, so you can mix and match. Deploying it in practice gets very complex, but he thinks there is a lot of opportunity in those more complicated hardware setups.

Are there learnings from neoclouds such as Nebius? The interesting question is how they expand so quickly. The craziest example is Nscale, which went from nowhere to hundreds of billions of dollars in the pipeline despite barely having any capacity. How? They go to a big buyer — Anthropic, OpenAI — and ask if they want five years of chips at a good rate. They take that contract to Wall Street: look, a committed $10 billion buyer, we just need the money. Then they go to Super Micro, Dell, NVIDIA, get the hardware, install it, and deliver.

It is a nice financial construct: pre-sell things you have not even bought, and you can scale extremely fast. That is what CoreWeave and the big neoclouds have done. They are very savvy about raising money and treating GPUs as collateral. Modal has always been more product-led and less able to lock in long-term contracts the same way, so it is harder to scale that fast financially, even if Erik thinks they get to the same place in the long run.

Vibhu asks about turning GPUs into an asset class — he thinks NVIDIA said something like that. Erik points to startups such as Silicon Data, Compute Exchange and others trying to build price indices, indexes and derivatives. It is interesting. Compute is not as homogeneous as other asset classes: different networks, locations, disks, CPUs, reliability. Then again, people said the same about oil, and we got tradeable grades. At this scale of AI data centers, they do need smarter risk hedging, risk-sharing and insurance. He is glad people are thinking about it. He is not sure GPUs become Henry Hub or Brent crude.

Vibhu wants to shift to Modal sandboxes, then backs up: what are the products? GPUs, sandboxes, anything else? Erik thinks in two big products. Number one is Modal Functions, launched in 2021: serverless functions that are easy to deploy and scale, with Modal managing capacity. That turned out to be really good for inference and for managing GPUs. They never had a dedicated inference SKU or a GPU product as such. They never thought of Modal as a neocloud, and he thinks there is a risk of getting sucked that way — selling a three-year rectangle of GPUs.

Modal has always been good at deploy-and-scale: cyclical, bursty workloads, capacity managed for you. Because they have many thousands of customers, they can aggregate that into very smooth demand and run it efficiently. Functions were the core. Product-market fit showed up in 2023. Functions are now about two-thirds of revenue, mostly inference in practice, though you can use them for all kinds of other work.

Modal Sandboxes launched in 2023 and, like functions, took a while to find product-market fit. Why launch them? Erik and Akshat were talking over lunch. A lot of people they knew were using LLMs to output code. Maybe those people needed a place to run it. Modal already had the primitives: containers, isolation, gVisor. Akshat wrote a prototype and launched it in a day or two.

Growth was close to zero. Erik does not want to trivialize it — there were early users who liked it for certain things — but into the summer of 2023 it was a pretty insignificant source of revenue. Two years later they even discussed shutting the product down. Then it exploded.

What changed is that LLMs got really good at generating code: from refactoring functions and writing snippets to generating full apps. Growth came from vibe-coding companies such as Lovable, then from companies like Ramp using Modal for background agents, then from neolabs doing reinforcement learning on Modal sandboxes. Sandboxes are now about a third of revenue, a second product.

That is exciting to Erik because even in the early days he wanted Modal to be a whole toolkit — storage, compute, different ways to work with them, almost a cloud. They always wanted to be a multi-product vendor. It was not true in practice until now. They have functions and sandboxes, and they are working on training, batch and other pieces. That evolution was in the early decks. It only became visible to the outside world recently.

Vibhu asks whether anything in the works looks like the next product, or any market predictions. Erik is very bullish on training. It is underserved. Everyone focuses on huge training runs; Modal sees a lot of demand in the small and mid-size segment. He will talk more about that later in the week at a conference they are organizing. Batch is interesting too. A customer the other day needed to transcribe 5,000 years of audio — millions of files. Modal had the capacity; it took a few days.

He is also bullish on low latency. Once speech-to-speech and voice models really work, that should be a very interesting market. Then all the agentic stuff. He has mixed feelings about the word: in a way all software will become agentic, or at least AI-powered. Still, there is a common set of building blocks for LLM-powered software across use cases.

He thinks of Modal as being in the business of building blocks: generalizable, battle-tested, composable. There is interesting work on the agent side too. Vibhu notes that on another podcast a year or two ago Erik said they were not focused on training. What changed? They have been building a training product on and off for years. They were never not focused on it. A year or two ago they had far fewer than today’s 150 engineers, so inference had to be the sole focus. They are bigger now and can place other bets.

He is also interested in post-training and reinforcement learning, and in storage, data warehousing, feature stores, real-time streaming pipelines. The long-term view is: you want to build some cool AI stuff, Modal has the building blocks, they will help you compose them. That is the beauty of software, and that is the company he wants to build.

When he says training, is he going after enterprises doing huge runs? No. Those customers will do hyperscaler or neocloud mega-deals for 100,000 GPUs. The underserved market is researchers iterating, people training smaller models, text-to-speech and speech-to-text, computational biotech — a market where they already see a lot of demand — recommendation systems, traditional machine learning. You do not need 100,000 GPUs. Setting up training clusters is still very hard.

Has he noticed people fine-tuning models such as GLM 5.2 or Kimi 2.5? If anything he has seen the other direction. A couple of years ago the pitch was that open-source models were not quite as good, but a small model fine-tuned on your domain could reach the frontier on quality and cost. Today the open models are so good you can swap them in: Kimi K3, DeepSeek 4.1, GLM 5.3, almost as good as the frontier at much lower cost.

They still see a lot of demand for post-training, and they are working on products in that space, both SFT and reinforcement learning. Fine-tuning is no longer something you kind of have to do. It is optional. Open-source models work really well off the shelf too.

Back to sandboxes: what did they have to update that existing sandboxes did not allow? Not a whole lot. They are infra geeks, good at scale and reliability, and that is why Erik thinks they have by far the largest market share in the space — he does not have numbers — running hundreds of thousands or even millions of sandboxes. They have had to rewrite systems a bunch of times to handle that scale. They were always good at it.

The sandbox space is not the most differentiated. Over time they have added VMs: the product used to be on gVisor, and they now offer full virtual machines as well, because some use cases need full Linux compatibility. They have built better primitives for piping data back and forth, working with the filesystem, and a lot of snapshotting. What will differentiate in the long run is forking, snapshotting, resuming snapshots, live migration.

The novelty of sandboxes is ephemerality: spin one up for a few seconds. That is a string he wants to pull on. Decouple the agent and the harness from the sandbox. Let agents control many sandboxes, fork them, snapshot them, maybe one sandbox per command. They have already spent a lot of time on snapshots. There is more to build.

Vibhu asks what snapshotting actually gives you. Today a lot of sandbox workloads are lift-and-shift: I have a dev box, I shove it in the cloud, an agent controls it, maybe the harness runs inside. Erik does not think that is the long-term model for agent compute. A better picture is long-running agents that look like durable-execution workflows, running 24/7, occasionally firing commands.

If you can spin up a sandbox in 100 milliseconds, run a few commands, shut it down and snapshot it, the faster that wake-and-sleep cycle gets, the more you can have sandboxes that only live for a brief period. That makes compute more efficient. It also lets you fork, move work around, and imagine Claude Code not as a linear flow but as hundreds of agents branching at once.

CI is the illustrative example. For most of GitHub Actions’ history you had a sequential flow: slowly install a bunch of stuff, run the test suite, generate reports. You can shard, but it is annoying. What Erik would love is to take the entire test suite and run every single test in its own sandbox — parallelize the hell out of it across a thousand sandboxes.

Why can’t that be done? Too much overhead starting and stopping sandboxes. The more frictionless that becomes, the more you get extremely bursty, large-scale parallelism. He is an infra geek; he gets excited about this. Once the primitives are right, it unlocks software we could not build before.

Why has that not already been built? People have always wanted to parallelize CI. You can, but the overhead is Amdahl’s law. In Modal’s own test suite they parallelize the way GitHub Actions does: split the suite in two and each shard still does all the installs and setup. That can take minutes. If you could do the setup once, snapshot the container, clone it and run the tests independently, you would get benefits you cannot materialize until the transaction cost is down.

Vibhu brings up a partnership: Anthropic using Modal sandboxes. Yes, Erik says — the managed agents with Anthropic, a bring-your-own-sandbox integration. You can use Modal for those. How did that happen? Anthropic is trying to create a thriving ecosystem. They make money from inference. It is in their interest to promote as much of an ecosystem as possible for everything else, as long as you use Anthropic for inference. Partnering with Modal as a sandbox provider is rational.

Vibhu asks if Erik is willing to talk about the Hugging Face incident. He is; it is all out there. They were using a Modal sandbox, but it was not a fault of the sandbox. Erik does not remember every detail of the exploit. Agents that exploited Hugging Face found a vulnerability in an app deployed on Modal and used it as a springboard to get full internet access, as he understands it.

To be clear: an app running on Modal was hacked, not Modal. Lots of people run stuff on AWS; occasionally those apps get hacked. That does not mean AWS got hacked. One of many apps on Modal got hacked. There are still lessons. They have been asking whether they can put more sensible security restrictions on by default.

He uses the S3 analogy. It used to be very easy to make a bucket public. That was not really AWS’s security problem, but customers kept leaking data, so S3 made public buckets much harder. In a similar way, Modal is exploring whether they can make security a bit more restricted by default without destroying the magic that makes the product good.

A separate thread: agents are getting really good at cybersecurity. As a learning from the incident, Modal started using agents internally to find vulnerabilities. So far they have not found anything, but they are good at probing deep parts of the system, trying to exploit things, escalate, and get out of sandboxes. The team is exploring that actively.

How do they get the agents to run through the codebase — just “find security vulnerabilities”? Not quite. They put them in a sandbox and tell them to try to escape. Escape the room, but for agents. They started a few months ago, around when Fable came out. Erik’s view is that agents are very good at finding vulnerabilities, and you can also use them defensively. Pretty much every serious shop is thinking about the same thing. Over time he is not that nervous. If anything, the breakthroughs will make the world safer.

Are the big cybersecurity companies just cooked? That is far outside his core expertise. A lot of what they sell is fear — branding and marketing as much as product. Brand may be even more valuable in the AI era. He would not jump to “cooked.” They have deep enterprise relationships, big sales teams, and they are very good at telling spooky stories.

Vibhu asks because Modal is doing this themselves, not hiring Palo Alto Networks. Maybe only giant enterprises care, and they want someone to blame if something goes wrong. That is true, Erik says. You hire an established vendor in part so you can blame them. Agents may reduce how often startups need human pentests — maybe from every six months to once a year. He does not know. Humans with AI are more powerful. Maybe you bring in pentestors who are really good at using agents and find issues faster. He is an optimist. He started a company. He does not think humans or those companies are cooked.

Vibhu wants to talk about developer experience. Modal has always focused on it. Does DX even matter anymore? Erik thinks it still does. The implied question is agents: the docs are already there, the agents will read them. In practice, almost everything that makes developer experience good also makes agent experience good: good docs, descriptive error messages, fast feedback loops. Some things have changed.

The other day he was setting something up with Cloudflare, which he finds notoriously annoying. He threw an agent at it and it configured a bunch of things and pulled him through. Agents do make it a bit easier to work with things that used to be painful. The human toll of dealing with YAML has gone down, because you can tell the agent to deal with it. For the most part, though, what is good DX is still good agent experience.

Infrastructure also needs to be more programmatic over time. That has always been the sandbox thesis: things you used to configure in YAML should be an SDK — launch a sandbox, do stuff, tear it down, imperative rather than a declarative DSL. SDKs are as relevant as they ever were, if not more.

Have they changed the product because most people are now running agents on it? They run a lot of internal tests where they tell agents to build a Modal app and run it. A concrete, funny finding: because of agent training data, it is very hard to deprecate things. Agents love old versions of Modal syntax, and it is hard to nudge them off it. Over the last year they have had to be more conservative about removing features, and careful to update every example that is out there, plus skills.

There is also an insane amount of Modal code online now going into training data. Similar to SEO, there is agent-engine optimization, or whatever people call it. They run internal benchmarks: ask an agent to deploy Stable Diffusion with whatever tool it wants, and see if it recommends Modal. Or spin up sandboxes. Or rank the sandbox vendors. They track that over time.

What do you do if it does not recommend Modal? Harder. Like SEO, it is primarily content: you have to put a lot out. You can also probe perception: rank sandbox vendors by reliability or by price. Agents seem prone to misunderstanding pricing, so they spend time making content clearer and more understandable for agents, because those systems index everything said about Modal.

Has he thought about making a bunch of GitHub repos and having agents write Modal code into them — a slop cannon? You probably can. His guess is that today’s training data is quite insensitive to whether something was AI-generated in the first place, so you could create a lot of slop. They have thought about it. He does not love the idea. In theory, like auto-generated content in early SEO, it could be effective.

Vibhu switches to forward-deployed engineers. Modal hires them. What do they do? We are in a new era of applications people could not build a few years ago, and a lot of companies need help. Modal started investing in FDEs a couple of years ago and really scaled it this year: from about four at the start of the year to something like 24. It is not exactly the Palantir model. They do heavy customer engagement.

They sit with customers on performance tuning, especially with LLMs — training draft models, tweaking parameters, vLLM and SGLang. When customers hit issues they work around them. They also watch for over-provisioning: too much CPU or memory, which is expensive. They will sit down and say, you are spending a lot on Modal, we can help you save money. On live deals they help migrate code from other platforms.

They never intend to become a services-heavy business. The goals for FDEs, as Erik ranks them: A, part of the sales process and closing deals; B, serving customers, a form of support; C, the most important, bridging gaps in the product that they later want as real features. Anything an FDE does, his question is: long term, how do we turn this into product so we do not need an FDE for it?

Do they hire PMs? Product managers? No. Are the FDEs kind of pseudo-PMs? It is a different function. On the customer-facing side there is overlap. On the inward side, product management is more like: what building blocks should Modal invest in? In between is looking at ten or twenty customers asking for slightly different things and finding the pattern. They do have people doing product. Some came from DevRel or engineering and showed a strong aptitude for thinking strategically about what to build.

He is not anti-product-management. He does think a lot of companies hired PMs too eagerly. You can tell in some products: eager managers who thought of widgety things and shipped them. Doing product well at an infra company is very hard. It takes deep strategic thinking about foundational building blocks and where you want to be in one to two years, because infrastructure is slow to build. You do not want 35 similar features that all do slightly different things. That kind of thinking is rare, which is why they have not aggressively built a PM team yet.

How have teams evolved since gen AI — and Erik saw a certain structure at Spotify? In a funny way, yes, structures have changed a lot, but he is not convinced gen AI drove it. What he has seen is a swing back toward being an IC, which he thinks is very positive. Ten or twenty years ago everyone wanted to be in management because that was how you got money, recognition and prestige. That shift started before gen AI and has been heavily accelerated by it.

They make it very clear at Modal that manager or IC, there is no difference in prestige. Everyone is a member of technical staff. It is about contributions. They see a lot of former managers becoming ICs because they want to go back and write code. There has been a shift away from non-technical managers. Everyone, including Erik, still writes a lot of code; Akshat too; pretty much the whole tech team, and a lot of people who are not on the tech team still write code. Gen AI made writing code even cheaper, which accelerated that.

If non-technical people are committing code, how do they review it? They do code reviews, but they think of them as optional. Erik’s view is that in many cases people use review to escape the accountability of merging to production. If that is the culture, you need to invest way more in CI and CD. When he writes code he will occasionally ask authors whether it is in line with how they think, and he hopes he is not changing the architecture. Internally they make it clear that reviews are quite optional.

If you are changing docs or doing something basic, do not wait for a human — especially in the era of AI. Why does a human need to look at every piece of code? You do not. That does require very good internal controls and testing. Infra companies need one more nine than the average startup. You have to build systems that are resilient: rate limits everywhere, decoupled systems, no single point of failure, extremely good test coverage.

Agents are actually really good at writing tests now. A few years ago he thought that was a major gap, because good tests require stepping out of what you are doing and being a little bit the devil’s advocate. Now they are quite good at that too. Do they use frontier models for coding, or have they tried open source? They experiment with everything. A lot of Cursor, of course Claude and Codex, and now a lot of people using Night Shift and other open-source stacks. He does not have a strong point of view. He wants people to try things.

They are trying to dogfood more internally now that they run inference for a lot of open-source models at very large scale. Using internal endpoints for agentic development is helpful, so they try to promote that. Vibhu had read, a year or two ago, that Erik did not think there would be a big on-prem business. Has he changed his mind? No. For Modal, he is a cloud maximalist.

He takes a step back. When the cloud arrived in 2008 or 2009, his first reaction was that it was crazy: why would I run my code on someone else’s computer? Six months later he was doing it. Enterprises took another 15 to 20 years. Look at Snowflake, AWS, Datadog: the cloud is clearly winning, and multi-tenant systems are clearly winning. On-prem looks increasingly like a legacy business model.

BYOC and VPC peering, to him, reflect an outdated security model: protect everything inside our own network, the network as a moat. That is not true anymore. The industry has moved to more robust paradigms such as zero trust. From that security angle, BYOC looks increasingly outdated.

From Modal’s angle, full control of the stack — they rent the hardware, deploy it, manage health and metrics on every host — means they can iterate very fast. That is valuable. Vendors who had to ship enterprise editions into customer environments had to think carefully about release management. Modal does not. If they want to introduce a new database, they can, because they control it. They do not have to think about what customers have access to. Multi-tenancy has so many benefits.

That said, if a very large Fortune 500 customer asked for BYOC, they have done this in the past: they give them a number. The number is very high, and no one has said yes so far. At the right price he would still build it. Separately, BYOC might be a way to work with neolabs that already have large capacity reservations. How do you get them to run Modal when they already have GPUs? BYOC might be a solution: keep the control plane in Modal’s environment.

The flip side is they could maybe even run other people’s workloads on that capacity, aggregating it into one big pool. He is excited about some of those ideas. In general he does not know if they will ever do on-prem. BYOC, maybe — bring your own cloud.

Hypothetically, if they raise a lot more money soon, what would they use it for? The capacity situation. A lot of the cash flow required is capacity prepayments. That is the best use of cash today. They are ramping capacity very aggressively. Maybe they move into ownership or other deployment models later, but there will always be a cash component: a down payment or prepayment. That is the greatest use of cash right now. Salaries and office space exist, but in the grand scheme of the financials they are basically a rounding error.

Will that change? He does not think so. The operating leverage is kind of insane. They have 200 employees, who cost a lot of money, but the GPUs cost a lot more. If he had to name the three biggest focuses right now: first, capacity and the financial complexity — pricing and packaging, the tension between owning and renting, whether to go more neocloud or the opposite, the counter-position. That is probably the biggest thing he thinks about.

Second is product strategy: what are the next building blocks? If they do more agentic stuff, which storage or compute primitives does the world wake up and realize it needs over the next few years? Ideally Modal already has them. They have been pretty good at anticipating demand — serverless functions and then sandboxes — and building in advance. What is the next thing?

Third is culture and company building. They recently crossed 200 people and are growing extremely fast; they will probably hit 250 by year end. At that size everyone is trying to figure out how they can be most valuable to Modal. They need the right context, the right motivation, excitement to come in because they get to build something cool, and a clear sense of what makes Modal successful. Implicit trust to do a lot, plus accountability to deliver — and to be held to it if they do not.

At 10, 20, 30 people you can sit in a room and feel it. People can read Erik: they can tell when he is annoyed and when he is happy. At 250, people do not necessarily see it, and a lot of new hires are a little nervous to talk to the CEO. How does he remain approachable? Dunbar’s number is more fable than the story around it, but something around 150 to 200 is still real. Laying the foundation of the culture there is very important.

How many people are they trying to hire over the next year? They grew about 4× on an annualized basis in the last 12 months. That is unsustainable on a relative basis. You can double a team every year without it being super hard. At 4× you start to see cultural drift, islands of different working styles, people doing very speculative things. Erik watches for that in engineering: you talk to someone and it turns out they are building something fringe. Is that really what they should spend time on?

Growing fast has a cost, including ramping people up. By the end of this year, maybe 250; by the end of next year they will keep growing, maybe 2.5× or 3×. It is very likely they end up around 700 people. Most of those are engineers. They are ramping sales a lot too.

What kinds of engineers? Primarily systems engineers. They have always been deep in Linux, filesystems, scalable systems, networking and cloud. That is the core. They have also moved up the stack into the LLM-inference space, so now they deal with CUDA, GPUs, MFU, maxing out tokens per GPU. A lot of that work is still systems: KV caching and routing are systems problems. Plus more GPU-performance work.

They are starting to build a research team, moving into post-training, distillation and better quantization methods — for customers and internally, both. It is a bit like the FDEs. Initially they will do a lot with customers to optimize models; over time those things have to become products. If every optimization requires a human sitting with the customer, they have failed in a certain way. Same pattern: customer work first, then internal tooling and self-serve products. Lots of work in post-training and GPU performance right now.

Back to product strategy: what do we need over the next year or two? It is incredibly hard to say, and if he claimed to know exactly, he would be delusional. What he believes at a general level is that we need better building blocks. This is a new era of software, unlocking applications that could not be built a few years ago — LLMs that output code that has to run in isolation, new primitives for GPUs and for language models.

They have built some of those blocks, and that has unlocked applications the world has seen, not just on Modal. Over the next few years there will be many more. He is bullish on training, on fast jobs, on agents and low latency. In the end they consider themselves an infrastructure shop. They are not trying to build narrow things. They try to build general-purpose building blocks, put them in front of customers, and let those customers combine them into applications Modal could never have anticipated.

Vibhu wants to go through some of Erik’s tweets for the last part, and Erik can double-click on anything. First: a lot of long-running persistent sandbox usage today feels lift-and-shift. Erik suspects sandboxes will evolve from “dev box into the cloud” into something more ephemeral — one environment per command — with snapshotting so you can pause, resume and clone. They have already talked about this a lot.

The median sandbox on Modal today is about eight seconds long, which is kind of incredible. Over time it might get even shorter. They want to support long-running sandboxes too; those are valid. His vision is that the world also needs to massively parallelize, which means reducing the overhead of starting and stopping.

Vibhu reads another tweet about an arms race among competitors; Erik has no comment. Then the core one: the biggest downside of GPU scarcity is not that GPUs are hard to get. It is that almost all capacity is sold as a three-year rectangle starting three to six months in the future. That is not what demand looks like for most users.

One thing Modal has always been very good at is fully on-demand capacity. If a customer needs 100 GPUs, they can typically get them immediately, without talking to a salesperson. Real workloads are not flat rectangles. They scale up and down through the 24-hour cycle, they grow over time, they have burst periods. That is a real problem for the space: the prime way to get capacity is still those rectangular blocks. It is not great.

He is biased, but that is why he thinks Modal can play a large role in what he calls duration transformation: offering more flexible capacity. They can do it because they aggregate many thousands of customers and demand gets very smooth in aggregate. That is a lot of the value prop. Is it a differentiator? Massive, he says.

If you want to be reductive, Modal’s value prop is two things. One is developer experience: iterate and build without thinking about infrastructure. The other is capacity. The more he thinks about the long run, the more it is aggregating the world’s compute and slicing it very granularly — the work they have done to scale up and down extremely fast, GPU snapshotting, making that much faster. That model works really well at large scale because you can pool lots of customers.

Another tweet: every month he keeps saying sandbox revenue might plateau, and it keeps going up 30 percent. It is kind of nuts. Every month he thinks they grew a lot and it must level off, and then it grows more. VCs ask how he knew sandboxes would be big. The truth is he did not — he thought they would be bigger than they were in 2023, but a third of revenue is insane. They are happy about it and have a large team on it. He will give themselves credit for seeing something in 2023. He would never have bet the market would be this large.

Next tweet: there is a structural reason inference providers should exist separately from model builders. The best models keep changing, and inference providers can reallocate GPUs between them because overall demand is quite smooth. Every time a Chinese lab — and hopefully more US labs — drops a new open model, you see big shifts in demand. If each lab held its own capacity, demand would spike on launch and fall when the next lab wins. In the background they are already training the next version.

Demand is sticky — models, prompts, the rest — so it does not all move overnight. Still, Modal can be running thousands of GPUs on one model today and shift 50 percent to another tomorrow. Nobody has to tell the system. That is what it is built to handle. Vibhu asks if that means LLMs are commodities, and if so where the big labs’ business is. Erik is not sure that follows. Is Taylor Swift a commodity? She drops an album, tours, makes a lot of money, then has a down year recording, then drops another. She still has brand and margin.

As long as the labs can keep launching new models, and the margin on those models can pay off the amortized cost of training them, it can still be a decent business. Does he think intelligence is plateauing? No. It remains to be seen. Models are very good at things you can train efficiently with reinforcement learning: coding, from snippets to production-level code, math, cybersecurity — verifiable domains. It is less clear they will be great at cooking, or anything with diffuse targets and long feedback loops. Humans will stay competitive there for a long time. AI models are still getting very good, as everyone can agree.

Vibhu asks which applications that are not huge now will become huge, given how good the models already are at coding. Erik does not know for language models in general, but anything with a verifiable loop. Computational biology: not exactly verifiable, but you can have an LLM output predictions, then run molecular-dynamics simulations for feedback. Materials science is similar: synthesize, measure properties, put an agent in the loop. He has a physics background; he is excited about that. Drugs, materials — he is drawing a blank on what else, but he is sure there are many. That is all Vibhu had. Erik is good; it was a lot of fun. They thank each other.