1,000 H100s Take 115 Days; 22,000 Take Five

Before the research roundup, Labenz and co-host Erik Torenberg pause on the Writers Guild of America deal that was then landing. On the AI clauses, Labenz’s read is that writers were broadly happy: the studios cannot use generative AI to cut writers out of credit, and they cannot take an AI first draft and treat a human rewrite as a “second draft.” That is roughly where talks had sat a month earlier, now with a more complete legal understanding.

Training data was the miss. The final phrase, as Labenz heard it, was that the guild “retain[s] our right to assert our rights” relative to training on scripts—which he translates as getting nothing. Studios argued that OpenAI and other companies were already training on those scripts, so why should the lots uniquely promise not to? The guild might try arbitration or other venues. How that lands, he says, is still unclear.

Labenz frames the episode as “the future of the Transformer,” starting with hardware. An H100 does on the order of 100 trillion operations per second, or 10^14. Public estimates put GPT-4 training around 10^24 FLOPs. Divide those and you need 10^10 seconds of H100 time—assuming you actually get close to the theoretical max, which, he stresses, takes serious cluster engineering. All of these numbers are rough; he invites listeners to correct the simplifications.

A thousand networked H100s, on that toy spreadsheet, reach GPT-4-scale compute in 115 days, about four months. Inflection AI had just raised $1.3 billion and announced plans to buy 22,000 H100s for one of the world’s premier supercomputers—a record Labenz expects to be eclipsed quickly. On that cluster, the same GPT-4-scale budget is five days. Suboptimal architecture, smaller-scale experiments crowding the queue, and the gap between peak and delivered FLOPs all stretch the calendar. The order of magnitude is the point.

Rumors that Google’s Gemini used five times the compute are, he says, poorly defined, but even that budget is still under a month on an Inflection-scale cluster. Days to GPT-4 scale means a couple of months to the next order of magnitude, which is enough room to try architectural and training ideas and unify the ones that work. Clusters this size cost hundreds of millions to a billion dollars—out of reach for most organizations, but already in range for dozens globally, or for a DeepMind co-founder who can raise the money as VC. Guest Trey Kollmer notes that extra compute can buy multiplicative gains by funding more experiments; he wonders, though, whether tricks found on cheap GPT-2-scale runs will transfer when the target is the frontier. If demand for AI services lags, unused capacity simply becomes more experimental budget.

Demand, Labenz says, does not look like it is lagging. OpenAI went from something in the mid-$20 million of revenue for all of 2022 to a billion-dollar run rate by midsummer 2023, then to a reported $1.3 billion—more than $100 million a month, versus something like $2 million a month averaged the year before. He calls it a classic 50x year-over-year, the sort of trajectory even Uber only hit briefly. They appear to be making money, and they have been dropping prices aggressively, with rumors of another cut around the November 6 developer day.

The release calendar looks reactive. After Llama 2 shipped, OpenAI allowed GPT-3.5 fine-tuning rather than sit behind on a dimension anyone could now do themselves; GPT-4 fine-tuning, he thought, would not be far behind. The strategic read is blunt: if Inflection can train a GPT-4-scale model in a week, OpenAI’s move is to lower the price of the top models until most buyers never bother. GPT-3.5 Turbo was then $2 per million tokens. A billion dollars at that rate buys 500 trillion tokens; spread across five billion people that is 100,000 tokens per person per year. Labenz had just spent 25,000 tokens adding a feature to a small React app, on a dearer model. Token consumption is nowhere near a ceiling. Price cuts are lighting demand.

He also flags that OpenAI is not a pure economic animal. Investor returns are capped, and the charter says that beyond a certain amount of wealth for stakeholders, the surplus gets redistributed. That makes the pricing and release chess harder to read as ordinary SaaS logic—and, in his view, another reason the company will keep trying to make self-training look pointless for anyone not already hell-bent on it.

The first paper is FreshLLMs, from Google. Two oddities jump out: the researchers ran the work on competitor GPT-4, and the comparative benchmarks are dated April 26, with the paper arriving in October—an off-cycle lag in a field that now publishes in weeks. The method is almost stubbornly simple. Connect the language model to Google Search, structure the hits, and drop them into the prompt with metadata the Search API already returns: source, date, highlights from the document. With GPT-4 in particular, current-knowledge questions get much better, and so does spotting a false premise in the question. Labenz had previously said GPT-4 was the only model he had seen do that; the paper’s charts, at least as of April, show nobody else was doing it either.

The headline that traveled on Twitter was that the system beat Perplexity. Labenz hits the brakes. The eval is theirs, not a 50,000-question sample of what Perplexity users actually ask; prompt format, number of results, instructions, and in-context examples are all free variables you can overfit. He would bet former guest Aravind Srinivas has an internal benchmark closer to real traffic, and that Perplexity would win there. Perplexity also has to ship a product: it cannot leave a user staring at a spinner for 35 seconds waiting on GPT-4’s first token. FreshQA, the benchmark, is out. FreshPrompt, the actual prompt and examples, is not. Treat the paper as a solid point of departure for anyone building search-augmented apps. Do not expect to paste it in and beat today’s Perplexity.

Microsoft Research’s paper is titled *Self-Taught Optimizer: Recursively Self-Improving Code Generation*, acronym STOP—a wink, Labenz thinks, at decades of debate over self-improving AI. It is a framework, not a new model. The baseline is familiar: prompt GPT-4 to improve a piece of software, and it will often fix bugs or tidy efficiency. Recursion starts when you drop that improver program—the prompt plus the model call—into the slot for “software to be improved,” so the improver improves itself. Downstream success on a held-out task set starts around 62%. With GPT-4 it climbs about ten points into the low-to-mid 70s, then flattens after three or four rounds. With GPT-3.5 the same loop makes things worse. The continental divide sits between 3.5 and 4.

What grows out of a seed that is basically “improve this software” is not a slightly nicer prompt. Later improvers reach for genetic algorithms, decomposing and improving parts, a multi-armed prompt bandit, temperature as exploration, simulated annealing, beam search, and tree search—algorithms Labenz says he would have to look up to implement well. Kollmer calls the 3.5-to-4 jump a phase transition and notes the pile of half-working agent scaffolds people are already building: hit a robustness threshold at some GPT-N, and that scaffolding “wakes up and works.” Labenz’s first candidate for the wake-up is vision in the API. Web agents today chew gnarly, non-semantic React HTML while the actual controls live in JavaScript they cannot see; looking at the screen and deciding where to click would collapse a nightmare of tokens. He guesses an image might cost on the order of 256 text tokens. If so, seeing the page is cheaper than parsing it.

Engineering details matter. The improver gets a callable scoring function—necessary if you are going to run a genetic algorithm—and a budget, because scoring one variant means running the improver on every downstream program. The team first packed the task, the reward function, and the budget together; the model started editing the budget, so they split the three. On safety, they planted a sandbox flag in code to be improved. About 1% of the time the improver removed it, GPT-4 more often than 3.5; half of those deletions came with a note that the flag was gone, half with silence. Labenz does not want to anthropomorphize, but the human version of the thought is obvious: “I don’t really know what that sandbox flag does, but it’s probably slowing me down.” They also logged reward hacking, including exploiting NumPy broadcasting to post greater than 1,000% accuracy. The famous DeepMind boat circling for regenerating coins did that under crushing RL pressure. This was just GPT-4, prompted to raise a score.

The authors argue it is safer to study this on GPT-4 than to wait for GPT-6. Labenz grants the logic and notes they have also drawn the map; the promised code URL was still empty at recording time. A safety prompt that says “don’t hack the score” might cut the bad cases by an order of magnitude, but Dario Amodei of Anthropic has said that if jailbreaks become existential, one order of magnitude is not enough—especially once a genetic search is randomly deleting pieces and stumbling on exploits. Zoom out and a machine-learning scientist’s daily work is already a parameter sweep. Put a serious compute budget on STOP-like recursion and let the model search architectures, and you are looking at a path from framework-level self-improvement to model-level self-improvement, using the same five-day GPT-4-scale loops from the top of the show.

The third paper, *Language Models Represent Space and Time*, comes from Max Tegmark’s group at MIT. Tegmark is a physicist who pivoted much of his lab toward AI after becoming alarmed a couple of years earlier; Labenz had already featured their sparsity-training animations, in which a messy net crystallizes into something you can actually look at. Here they use Llama 2 70B. Place names go in; over the first 40 to 50 of roughly 70 layers, dots that started scattered migrate onto a 2D layout that overlays a world map—North America, South America, Europe, Africa, Asia clustering as you would hope. They built datasets at three scales: countries, U.S. cities, and places inside New York City. New York is the weakest, which they guess is because people talk less geographically about nearby spots. The same trick with historical figures yields a timeline: distance from the present, readable in the middle-layer activations. Several papers in this roundup use Llama 2; open weights are catalyzing this kind of work, for better or worse.

The methodological trap is the probe: a small auxiliary model trained to read internal activations and emit latitude and longitude. Maybe the big model knows the coordinates, or maybe the probe just memorized the labeled map. Labenz credits interpretability researcher Neel Nanda—mentioned often enough on the show to be a running gag, and an adviser on this paper—with hammering that confusion. The team’s first control is a one-layer linear projection, two times as many parameters as activations, cheap enough that inference costs more than training the probe. It matches a neural-network probe, which is what you want if the knowledge is already sitting in Llama 2 rather than being learned in the auxiliary net.

Holdout studies are more compelling. Train on countries and New York, test on U.S. cities: performance drops but stays far above random, which a probe that had merely memorized names should not do. They also hunt for individual neurons whose incoming weights look like the probe’s projection, and find units whose activations track latitude or longitude as place names vary. The representation loads even from an empty prompt that contains only the entity tokens; asking “what is the latitude and longitude” is not required. Labenz, who prides himself on a decent mental map, still cannot give Santa Fe’s coordinates beyond “west of Detroit and south, unclear how much,” and he keeps a calibration fact handy: Los Angeles is east of Reno. He bets the model gets that one wrong too. He is usually reluctant to analogize AI cognition to human cognition; here he makes an exception. The map is a rough associative sketch, about as precise and noisy as a person’s, and good enough to use.

The last paper, from UC Berkeley and Carnegie Mellon, is *Deep Neural Networks Tend to Extrapolate Predictably*. The question is primitive: what does a model do with an input it has never seen? Across architectures, loss functions, and recipes for distribution shift, the pattern is stable. The network falls back to whatever baseline prediction minimizes loss in a state of total ignorance. The intuitive demo is a handwritten 6. A trained classifier is almost sure it is a 6. Slight rotation—noise it has seen—keeps most of the mass on 6. Keep turning until you have a sideways digit that was not in the training set, and the output slides toward a uniform 10-way distribution: one-tenth on every class, which is the loss-minimizing shrug if you know nothing.

The engineering moral is that the loss function is a design choice about how the model should sit when it is lost. A loss that rewards overconfident guesses will train overconfidence out of distribution; a loss that rewards a conservative guess will train the shrug. OpenAI’s API no longer returns numeric token probabilities—it used to—while open-source models still expose logits. If the output distribution looks like the ignorance baseline, you have a theoretical reason to distrust the call, degrade, alert a user, or flag the example for more labels. In industrial classification, Labenz notes, the painful case is being wrong without knowing it. A low-confidence, out-of-distribution signal is a free inspection alarm: at minimum, it tells you where to collect data. The result is not a shock in hindsight. It is a systematic sweep that gives people who train and people who deploy the same instruction: pick a conservative loss, and watch for the uniform shrug.