A Trillion Tokens Later, the Concept Finally Lands: Łukasz Kaiser on What Transformers Still Miss

Jacob Efron opens Unsupervised Learning with a guest who helped write the stack the field now stands on. Łukasz Kaiser is one of the eight authors of *Attention Is All You Need*, the 2017 paper that introduced the transformer. He built on that work at Google, later joined OpenAI for the turn toward reasoning models, and still talks about the shock of discussing research-level mathematics with a computer — something he never expected in his lifetime, having started as a mathematician.

Efron goes straight to the question Kaiser had posed about six months earlier: is reasoning enough to get to generalization, or is another method needed? Kaiser first says yes, in the practical sense. Current transformers with reasoning, agents, and a shell already do amazing things. Two years ago, before this stack, he would never have believed that a next-word predictor plus chain of thought, RL, and tools would spend hours with him on hard problems at work and actually implement them.

The turn comes immediately. It does not feel quite like us. Humans generalize from less data and make bigger leaps. Kaiser has a line for it: people say Americans will do the right thing after exhausting all other options, and LLMs are the same. They will learn a concept, but only after burning through a trillion tokens of surface-level patterns. Only when those patterns fail to explain something do they finally learn the concept itself. That is not how people learn. People sometimes invent a concept from a few examples, even if the invention is not great.

That missing piece, he says, feels like a different form of understanding — more long-term. It is a feeling, and every time researchers put a thumb on it, the transformer seems to catch up. Both sides of the argument have grown: transformers have gotten better, and the case for something else has gotten better too. A number of labs now pursue post-transformer architectures and are seeing interesting results. Who wins, Kaiser says honestly, he still does not know.

The “whiff in the air” Efron heard at NeurIPS is partly intuition circulating at San Francisco parties and on podcasts, so it may be self-fueled. Kaiser still thinks there is something fundamental underneath. Yann LeCun has been saying a version of this for years: the networks are called neural because they were meant to imitate the brain, and they do not, even if they share some similarities. Humans, as learning machines, do much more from much less data. The counterargument is equally hard. These models have always had a trillion tokens; nobody optimized them for scarce data. Give them the same compute with a data limit and you might tweak transformers into something much stronger.

The comparison is messy because people also get torrents of visual and motor data by moving through the world. Kaiser’s practical point is that the problems that are *not* data-limited are being solved very fast, so the bottlenecks that remain will be. The physical world is already one of them: train on one robot body and the data does not scale the way internet text does. He is a huge fan of Waymo — “people ask where the self-driving cars are; I drive them” — and then notes that the company cancelled highway driving because it still could not handle some construction zones. Years of work, millions of simulated miles, real driving, and it still fails a transfer no teenager would miss. A construction zone is a construction zone.

Whether transformers can close that gap is an open research question: architecture, data, loss, and optimizer may all need tweaks, because those pieces only work when they click together. Attention will probably survive. Recurrence sits deep in Kaiser’s heart from his RNN days, and reasoning has brought a version of it back — same weights, new token — but other attempts at recurrence still have not clicked. Tiny models such as TRM and HRM, with a bit of recurrence and a different loss, crush pure transformers on Sudoku and ARC-AGI. The live question is whether those tricks scale to language. Several labs are trying to merge them with LLMs.

Then Kaiser changes register. This year, agents are the biggest change in how he works as an ML researcher in perhaps the last twenty years. He can quantify it. On a private machine he tried to reproduce old papers he had always been interested in, including some of his own whose code he had lost. At least one of them he had already reproduced the hard way: about three weeks to a runnable state. With Codex he got there in two days. Call it 5x or 10x; the rhythm is what changed. He now starts three things in parallel. Before, he usually did one.

On private projects he has basically stopped looking at the code. A friend asked if that made him less sharp. He thought about it and decided the opposite. He no longer tracks every class name and small function, but he knows agents go off the rails. Once, while he was away, a run invented an auxiliary loss that was totally out of place and blew the training. You need full control in your head of what the system is doing: the loss, the batches, the training dynamics. You do not need the exact words in the function.

That split, he says, actually gives him more mental control than writing the code himself. Previously he would implement, forget the big picture while debugging the little things, then remember a detail only when it was already wrong. Now he stays in flow at the machine-learning level: think what is supposed to happen, tell the agent, verify, watch it happen. It is not just time saved. It makes the work nice. Among researchers he calls it a mild psychosis. They just cannot stop.

OpenAI has said publicly that it wants a research-intern-level agent by November. Kaiser feels close to an intern, with the caveat that you have to check carefully — interns do not usually add a loss you did not ask for, though creative ones might. Leave it overnight with the goal of a better model and lower perplexity, and it never works. It makes trivial tweaks that are not interesting. It is certainly not at the level of a researcher.

Kaiser has worked on long context and memory since before transformers, and a million tokens already looked huge given what attention does. The agent-era solution is almost embarrassing: write things into files, give the model grep, tell it to build index files like a little library. Five years ago he would have called that a hack, not a solution. He does not talk that way now. Machine learning is a collection of hacks. Dropout is a hack. We do not judge solutions by how they look. We judge them by whether they work, and this works amazingly — especially once you add a little RL.

Compaction is the concrete reason he prefers Codex to Claude Code: the thread can keep going. There is nothing mysterious about why. People prompted it well and put some RL on it. Years ago he would have called “RL the model to use tools, find stuff in files, and summarize” a band-aid. He repeats the rule: judge by how it works.

Could that stack become a researcher without a new architecture? Some people say you need post-transformer concepts and longer-horizon goals. Others say you will talk to Codex for a month, prompt it to find meta-patterns, write those to files, and, with data over a thousand people plus RL, it will start behaving like a researcher — which is roughly how human researchers learn. Kaiser does not think people have tried that very hard yet. Current RL wants all the rollouts in the loop; if a rollout is weeks long, training becomes months, which is impractical. Humans spend twenty years on one problem without two hundred prior twenty-year problems. That is a fascinating question, and also an argument that the human side of learning still has something machines lack.

For him the Codex era started around Christmas. Codex and Claude Code existed before, and people used them. Around Christmas it suddenly felt completely different. It has been barely half a year, and outside the San Francisco bubble many people still do not get it. Unlike the RNN-to-transformer shift, or the later reasoning shift, this jump is hard to attribute. The harness moved, post-training moved, new pre-trained models arrived, and together they leapt. Kaiser does not know which piece did it, and he is not sure it is clear to anyone. That makes it confusing to know what to scale next.

Efron asks the standard RL questions: will it work beyond verifiable domains, and will it generalize without a new dataset for every field? Kaiser thinks calling law and medicine “non-verifiable” is slightly unfair. They are not as clean as programming competitions, but large parts of them *are* checkable, and products such as Harvey plus benchmarks such as GDPval show real progress. He also thinks people overstate how verifiable math is. Most GPT-era proofs are not formalized in Lean. Frontend code is not a contest problem. The picture is a spectrum, from contest tasks down to “hard to say.”

His pet project is translating poetry into Polish, which sounds impossible to score. Run models as verifiers and they catch a fair bit: rhyme, cultural references, the sorts of checks people have always used. You can verify a pile of metrics and still have no taste. Taste is not easy to put into words; if it were, it would already be verified. That does not mean the missing quality is not real. You read the output and something in your brain insists they missed it.

Reinforcement learning, he reminds Efron, is a teacher saying good or bad. Drive into that hole on purpose and you can nail whatever you can point at. Image generation already does this: during training, people click beautiful or not, a sparse signal, and the pictures get more beautiful. Every hole can be plugged by hammering on it. Every hole you plug stops being the bottleneck, and the next unplugged hole becomes one. It is whack-a-mole. A more brain-like method with fewer holes to plan around would be much nicer. It does not feel as if any domain is impossible under current RL — but you need a strong, expensive, usually closed starting model, substantial data, and a company. Talking to the model and having it just work would be better.

Generalization is already happening. Kaiser says law is simply not in the RL pipeline, and Harvey’s experience is that capability either emerges or catches with a few touches. The shape of that generalization is strange. For a long time IMO geometry would not crack even while models solved very hard problems in other areas. People blamed missing spatial understanding. Then the models just saw more geometry problems — not spatial or physical data — and started cracking it. They generalize in a weird alien way: from here to there, but not across two points that look close to us. That jaggedness is why trust is hard. There may always be a spike you have not hit. Using the tools as an ML researcher keeps him honest. It is not good for capabilities. You hope the sharp edges go away. For now they do not.

For application companies, Kaiser’s advice starts with scale. The bigger and better the pretrained model, the fewer sharp edges you get, whether you RL or fine-tune. A year or two ago people said LLMs were dead and small models were the future. Gemma-class models of a few billion parameters now do things GPT-3 was not supposed to do under a hundred billion. That is amazing. If you actually want to solve big problems and adjust to your data, there is still nothing like the elephant — and those models are expensive, hard to use, and even harder to train.

Hardware is the quieter half of that story. Kaiser bought an RTX 5090 for a personal tower under his desk. On BF16 it is about 200 teraflops (the box says 400; some units are turned off). The GPUs used for the original transformer research were 9 teraflops, eight to a machine, maybe 70 or 80 teraflops in total. The card under his desk is five of those machines. A few thousand dollars, a normal little tower you could put in the kitchen, and you could redo the transformer paper’s compute. He remembers joining Google when people estimated that something like a brain would take one to 100 petaflops and assumed that was decades away. A single consumer GPU now touches the low end. In principle you can run a year of human processing in a day for hundreds to thousands of dollars. If someone at a university actually had an idea for how the brain learns, they could run a childhood in a few days and see if it works.

Agents also remove a different bottleneck. RNNs are painfully slow in PyTorch because they are sequential; to go fast you write a CUDA kernel, which is awful — unless an agent writes it and a unit test checks it against the slow version. They are not amazing at it yet. Bigger models will be. The hardware is still hardware: it needs to be parallel. The range of ideas you can try has changed. Kaiser is bullish on academia. Five years ago a single machine meant tiny tweaks. Now one box reaches nanochat scale: Karpathy’s open project trains a GPT-2-level model in a few hours. At that size you are doing real experiments, not toy ones. He wants more wild papers. His own embarrassing history is a NeurIPS paper the year before *Attention Is All You Need* arguing that you do not need attention, and should replace it with active memory. The conclusion was not quite right. Exploring the wrong direction is how humans twist toward the right one — and it is still what models are worst at.

Multimodal is the other place the architecture feels wrong. Humans take in an insane stream of sense data without predicting every pixel autoregressively; neurons are slow, the world is parallel and huge. Transformers are too serial. A high-resolution image every millisecond does not fit, because the model splits it into tiny patches and chews them in sequence. Thinking Machines’ recent multi-stream transformer is a small tweak that may be the right direction: several streams attending at once. Kaiser’s daily irritation is Codex executing a bash command while he wants to steer it, then waiting three minutes. People see, hear, and speak at the same time. Models should too.

Looking back at OpenAI, the defining decision of Kaiser’s time there was the plunge into reasoning: treating it as important as pretraining, launching reasoning models even when they were not chatty, personality was hard, and they were slow. Two product lines at once was awful; unification took a long time because everything was moving. He thinks the company would not have today’s results without that bet, and some larger labs still have not caught the RL quality. His worry is whether a company that has grown perhaps twenty times can still make a wild bet. He had watched Google fail to throw things away, get criticized for missing its own inventions, and then catch up fast because the team was already there.

After Google I/O he opened Antigravity 2 and could not tell which window was Codex and which was Google’s. He then tried his Codex-style work on the new Gemini 3.5 Flash and it did not work; it has not crossed the Christmas barrier, which he guesses is because it is distilled from Pro. Breadth makes catch-up safer. It may also mean you never get the immediate win of being first, the way Anthropic was first to nail coding. Kaiser’s account of that win is blunt: Anthropic focused on coding partly because it could not compete on ChatGPT. OpenAI had a billion users and plenty to do. OpenAI was not absent from coding, which is why it could catch up reasonably fast, but coding was not the focus. AI, he says, goes through upheavals. You have to bet on something that is not what is winning today. ChatGPT was the most amazing AI of 2025. Clearly not of 2026. Maybe 2027 will be something else again.

The coding market, he thinks, is big enough for two serious products. The harder question is carrying the power to other occupations. Codex used to open by asking for a GitHub repo, which cuts off a lot of people, and it is still named Codex — an accountant does not hear “this is your tool.” Claude Cowork is Anthropic wrapping the same core more friendly, which he thinks is the right direction. The abilities are there, including Excel. Getting them still takes a learnable skill, a hump of trust, and enough watching that you would not want to let it run unattended even if it were superhuman at first. He calls these coding agents AGI in the same spirit as the Turing test: one day we may pass the word the way we passed the test, and stop arguing. Step outside San Francisco and many people still treat AI as last year, before Codex, as if it will never change. That, he says, is the wrong way to treat it.

On open versus closed, he expects a gap that persists without exploding. Distilled models are never quite the big one. Mini models are fine until they trip and cost so much time that you go back to the large one — his Gemini 3.5 Flash versus 5.5 experience in miniature. Labs will try not to be fully distilled and will not fight open source to the death. Countries will want sovereign models for hospitals and police stations that cannot depend on one company’s outage. Both incentives are real, so the pattern should last, famous last words in a field that eats predictions.

In the quickfire, Kaiser admits he was wrong about research interns arriving this fast. He used to barely talk to ChatGPT — one query yesterday, one three days ago — and assumed he would not talk to a computer much. Now he talks to it about work, and he no longer opens a programming editor. He just tells the model to change the code. On existential risk his stance has not moved: not too worried, not complacent. The risks he would actually watch are the small ones, models hacking systems or taking down the grid, plus the ability to turn the data centers off if we decide to. He does not feel a threat from the models even as they have gotten much better.

Karpathy heading to Anthropic to work on recursive self-improvement gets a dose of cold water. Kaiser is part of the same “psychosis” about research assistants, and they will make many parts of current systems better faster. The space of post-transformer ideas is vast, and most of them are wrong, which is why it is called research. Even a 10x researcher does not guarantee you hit the right idea. A huge community of very smart people spent years on these questions and did not crack them. Efficient search through a vast space can still miss. He is not existentially worried about that loop.

Asked why he never started a company, unlike his transformer co-authors, he says he is happy he has not so far. Google and OpenAI were a privilege, and he loves technical work. Everyone who starts a company thinks they will not have to spend so much time on company work, and it seems they have to. He leaves listeners with the same charge he has been circling all episode: this is the most exciting time to be an ML researcher. There is a strong GPU under the desk, an agent that can push it, big labs driving transformers, and a whiff of something else in the air. Publish wild things. You do not need another paper that takes a pretrained model and RLs it slightly differently. Explore the wrong direction. That is still what humans can do and models cannot. With agents, failing is cheaper than it used to be.