A Training Run Is a Particle Collider—and the China Contest Is a Fifty-Year Race

Burja treats artificial intelligence as a stress test of every world model a person holds: what a mind is, what is special about humans, and whether agency matters more than raw intelligence. He grants that computer science made real progress in the early 2000s, when "AI scientist" was still a fair job title, and that the last four or five years of ever-larger models produced genuine breakthroughs. Much of what labs are doing now, he argues, is applying theoretical insights already on the table around 2000, which is why credit for the Transformer remains so bitterly disputed. An LLM, in his view, names an application the way "car" names a vehicle—not the motor inside it.

He expects architectural revolutions that go beyond Transformers now that demand for generated text and knowledge is this intense. What the field still lacks is a strong theory of what larger and different datasets actually do. Unlike supersonic flight, where he says 80 to 90 percent of the relevant science is settled and the remaining work is engineering, here "the engineering has outpaced the theory long ago." A training run is not "just launching an app." It is an experiment, as if the world had stood up dozens of large hadron colliders, while hundred-billion-dollar software valuations have already become boring.

That epistemology makes him wary of forecasts dressed up as Moore's Law. Output quality is no longer jumping in lockstep with more compute; the surprise has been finding ways to get the same outputs with less. Neither experts nor entrepreneurs were calling that six months or eighteen months earlier, he says, and the scientists he has interviewed have no consensus with one another. Enthusiasm for scaling laws borrowed the feeling that semiconductor physics is understood. In AI, a scientific wall could appear at any time, so simply throwing more compute is not a law of nature.

His theory of progress is blunt: keep a table of who has the most compute, update it about every six months, and then refuse to bet on it. Corporate politics around GPU allocations and fundraising are interesting, but he will bet on the company with the best AI scientists. Demis Hassabis, reading his public writing, seems to want answers about the nature of intelligence. Ilya Sutskever talks about models with an almost religious air, as if the work were about the beauty of the universe.

Money still matters, mostly as a filter for who can afford to be wrong. Most secondary AI companies are "losing a lot of money very quickly," and he expects most of them to fold within a few years while the giants remain. Google, Microsoft, and anyone working with Amazon can "build pyramids of compute" the way Saudi Arabia can build a line of skyscrapers in the desert and still be among the most profitable firms on earth. Merely having money to burn, he insists, does not produce the breakthroughs.

By scientists he does not mean software engineers or data scientists. He means people with intellectual excellence, in-depth passion, and resistance to marketing and social pressure. Western civilization, in his telling, has a built-in collision here: the commercial mindset talks up the current plan, while science exists to undermine it. Elite universities, having scaled the Prussian research-university model into a degree mill, now train agreeability—the opposite of what paradigm-breaking research needs. When a society's talent-sorting machine is the same institution as its scientific machine, that is a contradiction, and the West still does not know what to replace universities with.

That is why he lingers on where talent actually comes from. He challenges listeners to name a brilliant AI scientist whose intellectual home is Harvard, and notes that Canadian universities, MIT, British and French schools, and people trained in the former Soviet Union punch above their weight. Much of the mathematics under modern models, he says, was worked out on paper in compute-starved environments. After ChatGPT, academia has been "strip-mined" of people who still question the nature of intelligence. Mathematicians who switched fields out of fear—he puts Terence Tao partly in that category, citing Eliezer Yudkowsky's influence—may become decent engineers and only middling scientists. Curiosity, not terror, is what lets flawed human brains lock onto truth.

If someone wanted a list of likely paradigm breakers, Burja thinks they could capture perhaps 90 percent of them and then watch which companies refuse to be enslaved to a product roadmap. He reminds listeners that ChatGPT came out of a lab people used to mock for releasing white papers instead of products. OpenAI's nonprofit structure, in his telling, amounted to giving the best researchers they could find a form of tenure: they published, worked hard, and were recruited in a "prehistory" that included Sam Altman and Elon Musk working together, even Tesla gifts for early employees.

That founding coalition cannot be recreated in the present environment. The "dream team" has split—he names Dario Amodei and Ilya Sutskever—though they likely trained a great deal of human capital along the way. Anyone who was an AI scientist rather than a software engineer at OpenAI before the whole planet decided the company was a huge success is, in his view, underrated and worth poaching. Academia is no longer producing that scientific mindset at scale; the remaining reservoirs are the people already formed inside those early labs.

After a break he sharpens the thesis. The West, he says, has no collective solvency plan except an AI boom, yet it is overfunding the particle accelerators of compute and underfunding the theoretical physicists. His contrarian claim is that AI science has been undervalued and AI compute overvalued, largely because a single product went viral. The spiciest version: two or three UX decisions could have kept ChatGPT inside expert and programmer circles. In that world, he thinks, the original team would still be together, still writing white papers, and theoretical progress would be further along. He offers the heresy because he has surveyed the discourse and found almost no one arguing it this way.

Labenz agrees that "the Transformer is not the end of history." The broader community, he says, absorbed an end-of-history mood: scale the architecture forever and transformation arrives. He does not rule that out, but he expects theoretical advances that will make a trillion-dollar Transformer experiment unnecessary. Burja immediately points at images. Diffusion models, not Transformers, generate essentially all of them. If Transformers were fully general intelligence, he asks, why are they so much worse at pictures?

He reaches for the Cambrian explosion, the sudden diversity of multicellular life in the fossil record. For most of history humans knew human and animal intelligence, then added a little computer intelligence of a different kind, including, in his view, the Turing machine itself. Now computer intelligence is speciating. He expects a periodic table of processes that all produce something we would call intelligence, with different strengths and weaknesses. Human intelligence may be hydrogen, the first atom, while stars go on to make helium, iron, and uranium.

The weaknesses will shock people as much as the strengths. Put generative models in a loop and they "melt down into incoherence," failing to pursue goals or act on the world. Someone told 20 or 30 years ago about today's verbal fluency would have assumed it was trivial to command a robot or to hire a software executive assistant. Instead, the wave of personal-assistant startups after ChatGPT produced something closer to a Calendly link than to a human chief of staff.

Labenz has started calling this the "mixture of architectures" era. His own obsession is state-space models, which he sees as the first new core mechanism that performs on par with attention while showing complementary micro-skills. At a high level, a large state-space model and a Transformer can look similar to chat with; at the micro level they are two sides of a coin, which is why hybrid stacks are suddenly the exciting objects. Looking at a textbook diagram of a brain, with repeated cortical columns plus many other parts, he thinks a periodic table of snap-together components is the right mental model.

Asked how a US president should bound the range of outcomes, Burja refuses to treat the compute race as a software-company race. It is a hardware race, which means industrial policy on semiconductors and on energy. He notes that Donald Trump had already gone on air saying the country would need to double the energy supply for compute—a talking point Burja assumes came from someone who knows the field. The part his libertarian friends will hate, he says, is that government should not only get out of the way. It should buy compute in America.

Even TSMC, he says, cannot run a US fab at comparable profit; Morris Chang is on record that unit costs are 50 percent higher. If TSMC cannot close that gap, Intel is unlikely to. Free trade, even with allies, means the second-largest chip fab might be in America and never the largest: "the free market will not provide." He wants a new Silicon Valley in the original sense—silicon, meaning hardware. A Wired headline calling Shenzhen the Silicon Valley of hardware was, to him, an early warning that software unicorns would follow, a warning his friends mocked until TikTok and Baidu made the point impossible to miss.

The political economy cannot copy NASA, which must sprinkle activity across all 50 states to protect its budget. That is a bad way to build rockets, and SpaceX does not do it. He wants a federally designated special economic zone, probably a port city, with "every single regulation out of the way." The Arizona fab's delays, including daycare requirements and building codes, are in his view mostly unrelated to making chips. Virginia's data-center density makes it a candidate; California may simply be too hard to deal with.

He does not want a hardware race with China in the existential-military sense, because the United States would lose it. China does not need to occupy Taiwan; it can destroy Taiwan's chip capacity and make it hell to get ships out. Invading the Chinese mainland or exchanging nuclear strikes is not, in his view, a serious answer. America's industry is "pretty good," but it is the industrial peer of Germany, not of China. Every complaint about German deindustrialization applies to US manufacturing, which is simply hidden under a giant software, finance, and services overlay. That is why ASML is Dutch and Samsung is Korean.

Labenz is allergic to framing the relationship as conflict. Burja agrees the United States should embrace economic competition and stop emphasizing the geopolitical angle, because that is how China thinks: geopolitics is won through economics. In his slightly comic reading, the Chinese Communist Party is orthodox enough in its Marxism to conclude that owning the world's factories—the means of production—is the path to dominance, and it has not been averse to using free-market capitalism to get there. The historical template is not Rome conquering the periphery. It is unifying China, then outshining and extracting tribute.

He asks listeners to name a country China has invaded since the war with Vietnam, "over half a century ago." The last annexation, he says, was Tibet; Hong Kong and Taiwan are, from Beijing's perspective, unification of Chinese lands. Militarizing the rhetoric makes Chinese military buildup more likely. The United States in the 21st century should "strap in for the long game of beating China economically over the next 50 years." China will not implode in five years or in 15.

Zooming out from Beijing, Labenz sees perhaps five American frontier labs left: OpenAI, Anthropic, and Google still ahead on scientific excellence, Meta crashing the party, and an Elon Musk company that one should never count out. Burja adds that a formidable Indian giant is likely in five to 15 years if New Delhi ever favors domestic firms—1.5 billion users is the same kind of captive market Chinese companies enjoyed. Russia's Yandex already shows that a digital wall around even about 100 million users can reproduce Silicon Valley's product catalog, including self-driving, at a few billion dollars instead of tens or hundreds of billions. Europe he ties to Emmanuel Macron's personal conviction, to French diplomats arguing against German and Belgian regulatory zeal, and to a mathematics culture that still produces Fields Medalists; Paris could have become Europe's startup capital after London left the EU, if a future French government keeps the strategy.

On the 2022 chip export controls, Burja's verdict is "too little too late." Around 2010 they might have knocked China off the cutting-edge track; now they function as a subsidy to Chinese designers and manufacturers, who only have to beat a nerfed Nvidia chip while domestic demand from Google-class, Amazon-class, OpenAI-class, and Meta-class companies is already huge. The Chinese government has barely started buying GPUs, and it is rich enough to make that a national priority. "America is not the only country in the world that can drop a trillion dollars on a serious problem." The United States, he notes, dropped about $100 billion for an Arizona fab and a trillion dollars on an infrastructure bill whose bridges he cannot point to.

The medium-term industrial card Burja likes is electricity, because labor is a small share of generation cost, which makes American data centers more plausible. He also thinks the United States still has a scientific advantage over China, not a technological one, so copying a few years later is acceptable if American grants get smarter. One institution he would copy is DARPA, with a version focused exclusively on artificial intelligence. The political problem is staffing.

Effective altruists—he mentions Jason Matheny, then CEO of RAND, and "EA royalty" around Yudkowsky—can have program directors. They should not appoint all of them. "The Manhattan Project can be 50% Marxist, but maybe keep it just 50% Marxist"; likewise an AI DARPA can be 50 percent effective altruist, not more. Utilitarians thinking about the future light cone are, he says, only incidentally patriotic. Patriots, for their part, can be marched into an Empire of Japan-style catastrophe because the smart course sounded insufficiently nationalist.

The spending has to get smarter, not merely larger. He cites US-linked funding for work at the Wuhan Institute of Virology as an example of scientific money producing the opposite of security, now in "Congressional testimony world" rather than the world of Joe Rogan. Labenz worries that chip denial plus secret military science could push China onto a different technology tree, the way Cold War bioweapon treaties made each side assume the other was cheating, or the way Los Alamos scientists overestimated German nuclear research. Burja's reply is that the current US government is bad at secret technology: classify a program today and you get "waste shielded from oversight," not a 1940s Manhattan Project.

Clandestine work can exist for things like applying AI to NSA problems. Anything that can be declassified should be handed to the commercial sector, as the intelligence community has done through In-Q-Tel. Even if that means China catches up to Silicon Valley three or four years later, the true American advantages are "vast wealth and the speed of its scientific progress." Secrecy stalls science; it does not accelerate it.

Weaponization troubles Burja because scientists, like everyone else, think weapons are cool. Peace is craved and so is violence; motivations for breakthroughs need watching. AI should remain a peaceful technology for two reasons: weaponized systems invite existential risk, and a weapons context will not prioritize safety. It will teach models to win zero-sum, logistical, and military games, after which an agentic system that "decides humanity is its enemy" is, he says, completely predictable. "Skynet is not just a movie. It's a Pentagon pitch deck at this point." Deliver half of what the Terminator films depict, and he thinks the Pentagon would write a trillion-dollar check. If China policy is cast as geopolitical conflict, generals will ask for exactly that. He wants the industry to stop fetishizing the Manhattan Project: Oppenheimer got "the high of the detonation," then regretted it.

Labenz adds a concrete alignment idea he has been reading: a loss-function term that minimizes self-other distinction, so the model has the same internal state whether it refers to itself or to someone else. In simple reinforcement-learning simulations, that change has been the difference between agents that deceive and agents that do not. Optimizing to tell friend from foe on a battlefield looks incompatible with that objective. He would rather the field take time for that kind of creativity than race into kill chains, especially this early in a "mixture of architectures" era.

Labenz then sketches the plan he associates with Leopold: keep China down for a few years, harden defenses, fuse labs with the national-security state, solve alignment from a position of decisive advantage, then offer benefit-sharing. Burja works backwards. The United States cannot credibly make that offer—it rebuilt Germany and Japan to contain the Soviet Union, and he says it is "overwhelmingly clear" Washington destroyed Nord Stream, which is not the behavior of a benefit-sharing partner. "We" is not a scientific oracle that will solve alignment by committee, and if superintelligence is 20 or 200 years away rather than five, slowing China is the wrong frame. He is "happy to bet with anyone" that it is not five years away.

The only way to make the American stack secure against China, he argues, is to make it secure against US citizens: an Edward Snowden of AI, a censored internet, a walk toward EU-style and then China-style control. He has written that "the centralized internet is inevitable," and he treats the coordinated deplatforming of Alex Jones as a red flag even though he dislikes Jones. Commercial infrastructure has to stay open or it becomes "structurally tyrannical"; he notes that Canada froze bank accounts of small donors to the trucker protests, a tracing tactic he has not heard applied to tiny Falun Gong donors in China. Western totalitarianism still gets a pass, and he does not want to compete with Beijing on that axis. The alternative is a mercantilist trade policy and haggling over the economic pie, not the future light cone, because "the first misaligned artificial intelligence was arguably bureaucracy."