AWS Is a $100 Billion Cloud. Matt Garman Still Sees a 10x Migration Ahead

Matt Garman took over as CEO of Amazon Web Services in May 2024. He is not an outsider. In 2005, while he was still in business school, he interned on the pre-launch AWS team under Andy Jassy, came back as the company’s first product manager, and has now been on the business for eighteen years. In that span AWS went from nothing to a $100 billion run-rate company. Guo and Gil note the more recent slice: roughly $500 million of revenue in 2010 to about $90 billion last year, nearly $89.5 billion of incremental revenue in fourteen years.

The number Garman wants people to sit with is not the $100 billion. By most industry estimates, he says, about 85 percent of workloads still run on-premises, somewhere in the 80-to-90 range depending on who is counting. That leftover estate, before anyone adds the generative-AI jobs being created every day, is a tenfold business. Cloud computing, in his view, is still one of the few opportunities large enough to transform every industry, and AWS is still in the early stages of what it can be.

Garman had worked at startups before business school and wanted to see how a large company did internal entrepreneurship. Jassy would not describe the new technology business until he started. He joined anyway, helped write the original plan for which companies might use the first services, and returned full-time as AWS’s first product manager. For years afterward he had to explain why a bookseller was offering compute and storage. Amazon was not yet seen as the deep technology company it already was, and the obvious founders of such a platform, in the market’s mind, were names like Google.

The product idea that won, he argues, was building blocks rather than a new religion. Google and later Microsoft, in his telling, came at the category as if developers should change how they built applications. AWS started from an internal Amazon mandate around 2003, when Jeff Bezos required the company to break a scaling-constrained monolith into services. Once that worked, the team guessed other companies would hit the same wall. They asked, from first principles, what you need to start a company: compute, storage, databases. Then they shipped those without forcing a new architecture.

An early EC2 instance was just a Linux server. It auto-scaled and came up in 30 seconds, in an era when a startup otherwise went to a colo, bought racks, and waited as long as six months for a machine. S3’s put, get, and delete were a little unusual, but the storage concept was not. That familiarity let people build immediately. Only later did AWS add stranger primitives such as Lambda and Bedrock. Elad Gil used the young platform for his first company in 2007 or 2008; after Twitter acquired it, data-center capacity planning became a real pain that a dedicated team had to live with. Cloud erased that class of headcount.

Large enterprises were not early believers. Guo remembers, as late as 2010, hearing from big financials that they would never use public cloud on security grounds, that AWS could not beat them on cost, that their own platforms were better, and that the upstart was unreliable. Garman dates a turning conversation to a 2008 or 2009 trip to New York. Goldman Sachs, JPMorgan, and peers were fact-finding so they could run internal IT more efficiently. They were polite and clear: their workloads would never run on AWS; maybe the website, someday.

AWS’s response was to ask why, write down the compliance, audit, and regulatory requirements, and spend the next decade checking them off. The strategy was to solve the hardest jobs first. If JPMorgan can run on AWS, if a U.S. intelligence agency can run on AWS, the objections from everyone else get small. Startups were never dropped; the company simply did both. A year into the job, Garman told a business-school classmate that AWS could be a billion-dollar business. The classmate asked if he knew how big a billion dollars was. It is now a $100 billion run rate.

The credibility event was a confidential U.S. intelligence-community cloud contract won against HP, IBM, Oracle, and other incumbents. Because it was secret, nobody could be told. IBM sued, arguing the process was unfair. That lawsuit made the award public, and the agencies then said out loud that AWS was the most technically sophisticated, most capable, and operationally strongest choice. Garman admits there was luck in the suit; without it the endorsement might still be classified. For enterprises, a government saying you are the best operator is worth more than any campaign.

The remaining 85 percent is stuck for several reasons. There is no easy button that turns a mainframe into microservices and containers, which is what customers actually want, and big SAP estates are tangled into surrounding systems. Outside core IT, telco 5G loads and factory-floor control have barely been modernized; AI could rewrite those workflows, but the market has hardly moved. On-prem depreciation is still being amortized, and there are still people whose job is to run those rooms. The original AWS thesis, Garman later says, was “we take care of the muck so you don’t have to.” He thinks generative AI can finally speed that muck-clearing.

AWS had been investing in AI for a decade and started its own processors five or six years ago because chips are a long-cycle bet and the team saw demand coming. OpenAI’s leap, Garman says, was in some sense a discovery of what was possible. The company then took half a step back and asked the platform question: if this technology keeps getting better, how does every company build with it? Not how does AWS ship a consumer app, but what building blocks do startups and enterprises need.

Three hypotheses followed. Customers will not suddenly stop caring about securing infrastructure. There would not be one model: there would be large ones, small ones, and new ways of combining them, a bet that was less obvious at ChatGPT’s launch and looks right two years later. And the interesting IP an enterprise brings is mostly its data, which must not leak back into a model or escape its environment. From those principles came Bedrock, the Trainium and Inferentia chips, and a shelf of proprietary and open-weight models. On Bedrock, users send tokens and get tokens; they never touch a GPU.

Asked how much first-party models matter, Garman says they are important but not critical. Google is the hyperscaler with a true first-party flagship in Gemini; Microsoft has co-opted OpenAI and recently listed it as a major competitor. AWS’s Titan embeddings model is already the most popular embedding model inside Bedrock for search indexes. The company is training larger first-party models too. He calls Anthropic’s Claude the best-performing models in the world right now, and AWS launched Llama 3.1 on the day Meta released it. Enterprises like open weights because they can fine-tune, distill, and add their own data. Adobe’s Firefly, built on AWS, is his example of a specialized model.

The open-source stance is old AWS doctrine. “I want my customers to want to run on us,” Garman says, not to be stuck on a Microsoft license or an old-school Oracle database. Aurora is 100 percent PostgreSQL-compatible; take the code elsewhere and it runs, just not as well. Gil quotes former AWS leader Alyssa Henry: AWS loves open source and makes more money on it than any open-source company. Around the models, Garman is platformizing knowledge bases for grounding and real-time truth, guardrails (a bank chatbot that starts giving investment advice can draw real fines), agentic workflows that actually do things rather than summarize, fine-tuning and distillation down to industrial edge devices, and evaluations. Scale AI labels data; LangChain helps with agents. Inference, he later argues, will become a default application building block, with flavors the way compute has Intel or Graviton and storage has block or object.

Capacity will stay constrained for a while. A semiconductor fab takes years; TSMC is racing, but memory vendors, data centers, and power are all chasing a demand curve that “exponential” may undershoot. Demand, Garman says, looks almost infinite. AWS’s answer is diversification and prep it already knew how to do: eighteen years of capital allocation, buying its own power, and insisting that power be green. The company has been the largest purchaser of new renewable-energy contracts each of the last four or five years. Custom chips sit beside NVIDIA, not instead of it. NVIDIA, he notes, runs its own AI training clusters on AWS because the infrastructure is the most stable and the performance is the best.

Trainium and Inferentia are that diversification, with some workloads expected to run better and cheaper on them over time. Training and inference are roughly 50/50 today, but “in order for the math to work out, inference workloads have to dominate,” or the investment in large models will not pay off. Some foundation-model customers talk about needing gigawatts. Capital outlays are in the tens to hundreds of billions of dollars. AWS hedges by buying fungible assets such as land and power that can be reused if the slope changes, while offering better rates to customers who make very large long-term commitments. Anything growing at multiple hundreds of percent a year will not be forecasted cleanly, so the plan is to keep buffer for upside.

Asked what he would tell AI founders who now have to think about infrastructure the way 2005 startups thought about racks, Garman is blunt. Have a monetization plan. Do not assume venture funding will always refill the tank. “The only reason that any startup goes out of business is because they run out of money.” His first company raised $27 million, then a large sum, burned it in about 18 months, and died when the early-2000s funding window shut. Hype cycles, he says, do not guarantee another billion six months later.

Guo observes that in her AI portfolio, spend and progress are not always correlated. Gil asks where value will accrue, compute vendor versus model vendor versus application, a slide-deck question from prior software waves. Garman says most profit currently goes to NVIDIA because the cycle is early, and that is fine given the technology. Longer term, value sits with whoever solves a real enterprise problem: faster sales, better manufacturing, something a company could not do before. Most firms will not train their own models any more than they write a CRM from scratch; they will use Salesforce-like applications on top of models they might tweak. The bottleneck he sees is proof-of-concept graveyards. Whoever turns a working pilot into production catches the demand.

Guo has déjà vu: large organizations insisting that no vendor meets their needs, so they will build data management, GPU scheduling, evals, and compliance themselves. Garman has heard the sequel many times. Customers later say that if the platform had already had the feature, they would not have built it. Managing GPUs is not what stockholders think an enterprise is good at. That layer belongs on SageMaker, on AWS, or on partners, integrated with the data lake. The next three to five years, for him, are about abstracting the muck so Bedrock stays tokens in and tokens out, and so inference becomes as ordinary a building block as compute, storage, and databases.

Guo’s last question is why a $100 billion franchise still spends so much time on startups, against the usual advice to pick an audience and march upmarket. Garman does not hesitate. Startups are the lifeblood of AWS’s growth. The company learns an enormous amount from them, they will remain incredibly important, and if anything AWS will lean further into supporting them.