Scaling Laws Aren't Done. Don't Lock Yourself Into Yesterday's Architecture.

The episode opens on a claim Kevin Scott will spend the next hour defending: the things that feel a little too expensive or a little too fragile to ship today get cheaper and sturdier as models scale, and then more complicated work becomes possible. That, he says, has been the story of every generation so far. Sequoia's Pat Grady introduces him against a blunt backdrop. On any given day Microsoft may be the most valuable company in the world, and "arguably no one has been more ambitious, more strategic, or more effective in its AI strategy."

The guest is Microsoft CTO Kevin Scott, whom Grady and Sequoia partner Bill Coughran have known for a couple of decades, back to overlapping years at Google. What follows is not a product tour. It is an argument about why Microsoft concentrated its AI spend on scale, why a billion dollars for a then-productless research lab was not actually a large check, and why every builder should refuse to freeze today's model into tomorrow's architecture.

Scott is 52. He grew up in rural Virginia, the first person in his family to finish a bachelor's degree—neither parent had gone to university. He majored in computer science and minored in English literature, and at graduation his two advisors argued over whether he should pursue a PhD in CS or in literature. He was seriously considering both. He was also, in his words, so broke and so tired of being broke that he took the pragmatic path.

For a while that path still pointed at a professorship. His graduate work was compiler optimization and programming languages, and near the end he flinched: six months of effort to write a paper that made a synthetic benchmark three percent better did not feel like impact, and he did not want to repeat that loop for twenty-five years. In 2003 he sent a cold resume to Google and got an email from Craig Nevill-Manning, who had just gone to New York to open Google's first remote engineering office. Every compiler person at Google seemed to be on his interview slate. He then worked on nothing even remotely close to compilers.

What he did work on was the next fast-growing thing. He left for AdMob when mobile was taking off and advertising infrastructure was the bottleneck, came back through Google, then ran engineering and operations at LinkedIn through its IPO and Microsoft's acquisition. None of it, he insists, was a plan—just right place, right time, and a habit of doing the most interesting work on whatever was growing quickly. The object lesson is the one he keeps repeating: if you happen to be interested in something that is growing really, really quickly, and you stay motivated to learn more, you will probably end up in a reasonable place.

Coughran asks how that history should shape hiring now that so many AI practitioners hold PhDs. Scott draws a line. For the platform layer—huge distributed systems for training and inference, networking, silicon, system software, algorithms—a doctorate is super helpful, because you need a skull full of prior art and proof that you can finish a very complicated project. Those projects even look a bit like AI platform work, except the platform is built by lots of people and a dissertation is often built in isolation. For applying the same platform to education, healthcare, or developer tools, a PhD helps and is certainly not necessary. Clever people, he notes, are not found only in doctoral programs.

Microsoft, Scott says, is a platform company. It was a pillar of the personal-computing revolution, contributed to the internet revolution, and largely missed mobile. The AI strategy is the same motion: build the technology platform of the era so other people can build useful things on top of it. That means frontier models and small language models, highly optimized inference, hyperscale training and inference, and a cheaper, more powerful platform with every turn of the crank—plus developer tools, safety infrastructure, and testing. The other half of the job is listening, as intently as possible, to the people hitting walls as they try to put this in front of users.

Asked to grade Microsoft, he first describes his own wiring. He is an engineer, and he thinks most engineers are short-term pessimists and long-term optimists: you walk in, everything looks broken, and you fix it anyway because you believe the problems are worth fixing. The highlight he will own is that Microsoft, alongside OpenAI, made very powerful AI dramatically more accessible, and customers who otherwise would not be building serious applications are now doing so. The lowlight is timing. Microsoft Research had been an early AI leader, but the company was a little late to the basic shift. Most of the important advances of the past twenty years, he argues, have been a function of scale: data scale and compute scale together unlock things that were impossible at lower scale points.

Past a certain point that scaling is so exponential that fragmented bets stop working. It becomes economically impossible to bet on ten different things that all need to scale exponentially at once. Microsoft was spending a lot on AI, but the spend was scattered, in part because the company "didn't want to hurt any of the feelings of smart people." They were not as quick as they should have been to say that scale is what matters and to focus investment on it in a principled way. When Scott became CTO in 2017, his job was to scan Microsoft and the industry for holes they would deeply regret by 2019 or 2020. The biggest item on the list was that the rate of progress on AI was not fast enough.

Google's BERT paper in 2018 crystallized the belief. Shortly afterward he restructured teams inside Microsoft to get more focused on AI; about a year later came the first OpenAI deal. What he saw in OpenAI was a shared platform bet. At Google he had run teams that trained outrageously effective but narrow models—click-through-rate prediction for ads, compute you could not amortize onto anything else, with new data, new PhDs, and new process every time you wanted a new application. Large language models broke that pattern: one model for translation, sentiment, and the rest of text, with transfer learning improving as scale increased. Everything these models can do, he still believes, gets better at the next scale point, and the capability set broadens. OpenAI had the same belief, a principled forecast, and experiments that kept landing on the line. A billion dollars felt like a big number; relative to what was happening, he says, it was not. GPT-3 was on forecast. GPT-4 was on forecast. Plenty of people disagreed with him. His conviction was high.

Coughran, speaking from a Google systems background, expects training to be dwarfed by inference soon—otherwise, Scott jokes, they are building models nobody knows what to do with. Each GPU generation's price-performance gain has usually exceeded what Moore's law used to deliver for general-purpose computing: the A100 was about three to three-and-a-half times better than the V100, the H100 was not quite that much but close, and on paper the next generation looks very good too. Some of that is process, some is architecture, and a lot is narrower word sizes—nobody needs 64-bit arithmetic for this work, and there is an embarrassing amount of parallelism. Networking is sprinting as well, because frontier training and inference have been multi-GPU, multi-node problems for years. For the nerds, he adds, effective Dennard scaling has been gone since around 2012: transistors keep arriving, they do not keep getting cooler, and power density is the constraint.

Training and inference data centers are already different buildings. Training environments are large capital projects that take a few years to stand up. Inference is more forgiving: a better silicon design, a better network, a better cooling trick, and you swap some racks—Scott admits his data-center team would yell at him for the simplification, but the direction is right. That should produce more diversity, more competition, and a faster rate of improvement on the inference side. Software already shows the pressure. Inference is a large fraction of the compute footprint, demand exceeds supply, and the incentive to squeeze the stack is very, very powerful.

Whether the demand-supply balance eases depends, in his view, on whether the platform keeps getting more useful. If it does, demand increases; the shape of that demand will move around. Building a frontier model is extraordinarily resource-intensive, and he is unsure how many you actually need if they sit in the same capability tier—that is an awful lot of money to spend on roughly equivalent things. His analogy for founders: believing you must build your own frontier model to ship an application is almost like saying you have to build your own smartphone hardware and operating system to ship a mobile app. Maybe you need it. Probably you don't. Lots of inference is a health signal, because it means products have found product-market fit and are scaling. Lots of speculative dollars into infrastructure R&D, he suspects, ends the way many speculative infrastructure booms have ended.

On training data, Coughran notes Microsoft's own finding that quality matters at least as much as volume, and the industry worry that high-quality sources are running out. Scott thinks quality-over-quantity is a feature: it funds the partnerships that feed models a real curriculum, which is the point of work such as "Textbooks Are All You Need." He then corrects a category error. A large language model is not a database of facts but "the world's crappiest database," with recall as imprecise as a person's, so capabilities belong in the weights and facts belong in retrieval. That split implies two businesses—training data, which is hard to price because a token's contribution is as diffuse as a word from *Moby-Dick*, and referral data that agents reason over, which can look like rev share, licensing, or a new ad unit. If he were in his twenties, he would go figure out the ad unit for agents and build the company; he would be shocked if the right mechanism were not an auction.

Where are we on the spectrum from copilot to autonomous agent? Scott says Microsoft chose the name Copilot deliberately, at least to push internal teams toward assistive rather than substitutive technology: augment someone doing cognitive work, do not replace them. Narrowing rough frontier capability into a domain is also the easier path to a useful tool. GitHub Copilot is the one that has most clearly hit the bullseye and is in daily use by people doing non-trivial cognitive work. The more general the copilot, the harder it is to let it take high-precision action autonomously, especially when it represents you and the mistakes have consequences. Microsoft would rather wait until a thing is very good than ship something whose first impression is "this doesn't work," after which users may not come back for a long while.

A recent partnership with Devin is the same idea in a more agentic wrapper: a frontier model plus a lot of other machinery, optimized to give people high-quality recommended actions they can accept with reasonable confidence they have not created a new mess. Grady reports a pattern from Sequoia's portfolio that he calls an 80/20 rule, or maybe a 98/2 rule. Companies can automate most of a task quickly. Getting the last couple of percent—end-to-end, autonomous, consistent enough to trust—has been elusive. The open question is whether that last mile is the foundation model's job or the software vendor's.

Scott thinks it will be both for a while, but he rejects the premise that scale-up is hitting diminishing returns. There is still an exponential; you only sample it every couple of years because supercomputers and training runs take time, and the next sample is already on the way. What feels too expensive or too fragile today gets cheaper and sturdier, after which more complicated work becomes possible—that has been every generation's story. The failure mode he watched inside Microsoft was teams demanding giant GPU pools to train a custom model because the frontier felt unusable, then watching that expensive special case get wiped out when the frontier snapped forward. Architect so you can plug in the next model when it arrives; keep your skepticism if you want, but leave yourself the option that the last six years will continue.

Grady asks the long-term optimist to name what he is excited to see. Scott's framework is to look for the gnarliest zero-sum problems in society—places where people fight, or where people are immiserated, because there does not appear to be enough of what they need. For a good number of those problems, the only thing that has ever reliably turned a zero-sum game into a non-zero-sum one is a technological breakthrough that creates more. More does not automatically get distributed equitably, and those distribution fights are real. He still wants the more, aimed at the places that are having a tough time right now.

He tells a story about his mother. She is 74, still in rural central Virginia, and has lived with Graves' disease for 26 years: a hyperactive thyroid, irradiation to knock the gland down, then lifelong hormone replacement. A doctor fiddled with her dosage, and she ended up in the local ER six times in a short span of weeks. The first visit she presented with cardiac symptoms, and it was pretty clear nobody had read the chart closely enough to notice Graves'. The move they needed was a TSH panel, then an adjustment to the replacement dose. Scott is careful not to ascribe ill intent. The system is egregiously overburdened, the population is aging, and there are not enough young doctors coming in.

He ran the experiment himself. Feed GPT-4 her symptoms and record, and it asks for a TSH test; feed it the result, and it flags the hormone-replacement dosage. This is not theoretical. It could have spared her a massive amount of suffering. The reason she got out was that he intervened and sent her to a specialist 400 miles away she could not have reached on her own. He cannot stop thinking about the 74-year-old women going through the same thing without someone who can force the system. That is the optimism: give kids a leg up in education, give scientists better tools for carbon-capture catalysts and safer transportation, get to a post-carbon economy faster. What he does not want is for the ecosystem to get sideways over a model saying something that hurt someone's feelings. Feelings matter. The cost of not deploying the good, he says, is a high, high cost.

In the lightning round, Grady asks whom Scott admires most in the world of AI. The answer is Ray Solomonoff, one of the people at the 1950s Dartmouth workshop where Marvin Minsky, Herbert Simon, and others coined the term artificial intelligence. From the beginning Solomonoff pushed the idea that probabilistic methods would matter. When Scott was in graduate school in the 1990s, the prevailing academic story was still a minimalist calculus of mind: rule-based systems, ontologies, symbolic reasoning, a physics-like hope that once you divined the simple rules you could emulate intelligence in software.

Solomonoff's position was the opposite. Intelligence is an extraordinarily complicated phenomenon, and the only way to get there is to model it with probabilistic methods. He was right. He was judged wrong for a very long time, and Scott admires the contrariness—the willingness to stick with the belief for an entire career, starting in the 1950s. Scott does not even know whether Solomonoff lived to see how right he was. It is a fitting close for a conversation that keeps returning to the same bet: the ugly, statistical, scale-driven approach keeps working, even when the commentariat is sure this time it will stop. The builders who will be glad they listened are the ones who left a socket open for the next sample of the exponential.