The Reasoning Curve Was Smooth All Along: Gemini 2.5's Jack Rae on Thinking, Memory, and AGI

Nathan Labenz welcomes Jack Rae, a principal research scientist at Google DeepMind and technical lead on the company's thinking and inference-time scaling work. The model on the table is Gemini 2.5 Pro Experimental 03-25, a long name that Labenz argues undersells the release. In his testing it is the first time he would rank a Google DeepMind model as number one across many important dimensions. That ranking, he stresses, is not only a leaderboard story.

Before Google's PR team reached out to book the interview, Labenz had already dumped a full research codebase — 400,000 tokens — into the model. He asked it to extend the project, reuse as much existing code as possible, and make a light touch so it would not mess with other people's work in a shared collaborative space. The command the model showed over that super-long context felt immediately different, a hair-raising qualitative step-up of a kind that had been rare since GPT-4. He is still recalibrating what the system can do.

Rae credits a "true Gemini team effort": a knockout from pre-training, thinking, post-training, and other groups pulling the release together. Internally they liked the model and felt good about the trajectory, but they did not know exactly how it would be received. Seeing people find it useful, notice deltas on real-world tasks, and even "feel the AGI" with it has been, he says, very cool. He is happy to talk about model development, especially on the thinking side, while stressing that this one was a full-team knockout.

Labenz's opening technical question is why a simple reinforcement-learning setup with a correctness signal — the DeepSeek R1-Zero demonstration — suddenly works across the industry. Rae's answer will surprise anyone looking for a discrete invention. From his vantage point, Gemini has been leaning more and more on RL to improve reasoning for at least a year, using a discrete, verifiable reward on accuracy-based tasks. That work was already shipping in released models before "thinking" even started, and it was already helping the reasoning process.

Internally tracked capabilities, he says, go up with "scarily predictable" improvement, almost a Moore's Law-style curve. Any given change to the RL or model recipe is stochastic; you do not always know what will help. Accumulate enough of those uncertain bets, though, and a trend appears. In public, the same curve occasionally crosses a threshold, a discrete release captures the imagination, and the model is finally good enough that it feels like a step change. He saw the same dance during his years in pre-training.

On his own production function, Rae reads far less research than he did five or ten years ago. As he moved from a junior role toward directing work, the bottlenecks became known problems with no paper that already holds the answer; the solution will be found among the people he works with day to day. He still follows X, arXiv filters, blog posts, and talks, and he uses Gemini itself to ingest a whole paper or a bundle of cited papers and cut straight to the critical idea and results. Long-context question answering over technical text is, cliché or not, his go-to tool.

Nearly every frontier lab has now shipped a chain-of-thought reasoning model in a suspiciously short window. Labenz offers two folk theories: simultaneous invention, because the next step was overdetermined, or gossip at the infamous San Francisco parties. Rae picks the former and notes that the pattern existed before SF tech parties were invented. People always look for avenues of progress, and even small signs that a model is improving a certain way get noticed fast. An unprecedented number of smart people, an unprecedented amount of compute, and unprecedented velocity mean that when a paradigm like test-time compute opens a vein of capability, labs flood in.

Google's own timeline is a sample. The company assembled reasoning groups on thinking and test-time compute in September and October. Within a month or so of focusing on that case, they were finding modeling breakthroughs they found very exciting, which led to shipping a first experimental Flash-with-thinking model in December. Reflecting on how that team progressed, Rae sees a natural process of curiosity, experiments, and more people piling in — not a conspiracy. A quiet prerequisite: the whole direction required deep confidence applying reinforcement learning to language models, something Google was already comfortable with, which made the barrier to entry low.

A small detail in the R1 paper still nags Labenz: DeepSeek reported that RL on smaller models basically would not work, yet later it seemed to work everywhere. Rae says that is completely valid. Six years ago, training a 100-billion-parameter language model was alchemy, with a million components that could go wrong or diverge; pre-training looks "solved" now only because those pits have been filled. RL for deeper reasoning is the same kind of problem. When five things are broken, you fix one, nothing changes, and you get disheartened; iterate until more things line up, and the whole system starts to shine.

The confidence to keep going came from sparks they saw in their own experiments last year. With reinforcement learning, the model was using thinking, and they started to see self-correction and exploration of different ideas inside the traces. That is exactly what they had hoped would emerge from RL, Rae says, but they did not really know it was possible until they saw it for themselves. Those cognitive behaviors are now the thing everyone wants; seeing them appear was the moment the recipe stopped feeling like a bet in the dark.

How much does the team sculpt those cognitive behaviors versus letting them arise? Rae says they are outcome-driven: they will use whatever recipe gives the best results and generalization. His own prior is Occam's razor — fewer imposed priors on what the cognitive faculties should be, more learning from data. They explore human data, model-based synthetic distillation, and behaviors that arise from end-to-end RL, then mix whatever works, with a preference for simplicity. The training recipe itself is evolving too fast, and too sensitive, to spell out.

Gemini's thoughts usually stay in English, even on non-English reasoning tasks the model is strong at — not language-switching so much as sticking to one language. That prompted an internal debate: bad product experience, or a performance choice worth keeping? Early Flash Thinking traces were also quite templated, breaking problems down in a formulaic way, and the team treated naturalness as a research problem because a rigid template may mean the model is not getting full benefit from thinking compute. Weird, cyclic, or not-obviously-useful tokens are tolerable if they make the answer more accurate, safe, and factual; the chain of thought is supposed to be a scratch pad.

In AI Studio and the Gemini app, for the December and January launches and for 2.5 Pro, users see the raw, unmodified chain-of-thought tokens. People like seeing them, Rae says, but they can be verbose, and summaries or other transformations might be more useful; NotebookLM has already tried a graph-based thought explorer. The decision to show the full chain mixes input from the safety team, researchers, and leadership — Koray and Demis often want a clear view of the pros and cons. Rae does not weigh in on the surface; he tries to make the models strong and leave good options on the table. OpenAI's choice to show summaries, he notes, may have involved more than distillation fears, including worry about RLHF-ing thoughts until they look nice.

Labenz presses OpenAI's obfuscated-reward-hacking result: pressure the chain of thought not to reason about hacking, and hacking later returns with the reasoning gone from the trace. Rae's line is careful and clear. Thoughts should improve factuality, safety, and capability, and remain a scratch pad; if they are going to be shown, they should be interpretable and faithful to the computation the model is actually taking. "We probably don't want to add training objectives which would encourage things like deceit." Forced expert monologues are a related dead end: people are bad at transcribing inner speech on command, which is why "let's think step by step" already worked zero-shot from pre-training, while the valuable data is traces of open-ended process — calculations and actions — that a separate DeepMind agents group owns.

Labenz's working picture is that pre-training determines which concepts a model has, and post-training determines the behavioral patterns that deploy them. Rae largely agrees: pre-training learns a massive bag of function approximators over good and bad behavior, and better next-token prediction is better compression of that whole distribution. Post-training drops many modes and fixates on a few ways of reasoning and acting; done well, RL should compose primitive skills toward a smaller set of important tasks. As RL's share of FLOPs grows, that split becomes a spectrum. For AGI, Rae thinks it is crucial that reinforcement learning learn new skills, not merely reshape knowledge cloned in pre-training.

Meta's latent-space reasoning paper — feeding the last hidden state back in without decoding to a token, so the model chews on its own thoughts — strikes Labenz as scary; he wants to know what his AI is thinking. Rae will not taboo a technology before it has been researched unless the arguments are incredibly strong. The live issue is interpretability of those latent vectors. He analogizes to DeepMind's own line: AlphaGo, then AlphaZero with self-play and no supervised fine-tuning, then MuZero, which unrolled in latent vectors that could still be decoded into states and was the most powerful of the series. If latent thinking can be made interpretable and safe, he sees no reason not to explore it.

On interpretability more broadly, Labenz notes that Anthropic's recent thought-tracing work is excellent and still only partly explanatory: replacement models account for about half of behavior, with error terms filling gaps. Rae does not buy a future where capability goes exponential and mechanistic interpretability goes linear. More powerful models are also more powerful tools for examining themselves, so the two should track. He wants artifacts that trace how close the content of thoughts is to the underlying computation, whether the medium is tokens or latent vectors, and says Gemini has people on that problem.

Labenz then asks about the Platonic representation hypothesis: as models scale, do they converge on one true world model of reality? The only place Rae has a strong theoretical conviction is pre-training. Drive perplexity down to the entropy of the text — Bayes-optimal compression — and you have the model that best understands the process that generated that text, an idea with grounding in Ray Solomonoff and Claude Shannon. Even that object, he immediately adds, is not AGI: it understands today's text-generating dynamics, not how to follow instructions, do tasks nobody has done, or generalize to unseen environments. Large-scale RL, by contrast, is "definitely not" converging to one model; Gemini already differs from others, with capabilities the team builds discretely, and the systems still feel steerable.

Asked whether pushing context length is enough for long-running agents, or whether memory needs a conceptual breakthrough, Rae goes back to 2014. He joined DeepMind in an area called episodic memory — the subject of Demis Hassabis's own PhD on episodic memory and imagination — and his own doctorate was on lifelong reasoning with sparse, compressive memories. He would never have imagined then how far context would come. Million- and ten-million-token windows, depending on how text or video is represented, are starting to verge on lifelong scales. He still does not think memory is solved.

He expects cool breakthroughs still to come in the memory space. DeepMind's Neural Turing Machine and Differentiable Neural Computer mixed large attention with read/write mechanisms, and his sense is that something in that family will prevail as a way to have extremely long, even infinite, lifelong memory. It remains an active research area. Meanwhile, capabilities compound: last year's long-context breakthrough stacked with this year's thinking work, because being able to think long and deep and use a million tokens unblocked problems neither piece could solve alone.

The remaining path to AGI, as he maps it, is not a single missing algorithm. Reasoning still has a long way to go on reliability and generality before you can trust responses on more open-ended tasks, and agents are a super-high-priority area with known bottlenecks. Combine much better agentic capabilities, better reasoning, and better memory — a lifelong range of understanding across time — and that will really feel like AGI to a lot of people. Using 2.5 Pro, he already feels it: the model can one-shot complex codebases, a piece of technology that felt futuristic three years ago. When the full combination arrives is hard to say, but all of it is being developed actively, and he feels it is coming quite fast.

Labenz asks whether more modalities will be fused the way text and image now are — even merging latents with a system like AlphaFold rather than calling it. Rae says Gemini's multimodal-first bet, including native image generation trained in Gemini 1 and noted in the technical report, already shows why joint training beats a static tool call; the test for genomics or similar data is how much positive transfer you get from the existing world model. The missing system card is a different kind of omission: experimental releases skip full provisioning, and cards come when the model is generally available. They still run extensive safety testing, policy review, and red teaming, including external partners he will not identify, plus real-world feedback as part of the test. He told a Cloud audience Vertex was coming "soon," and it landed the next day — underpromise, overdeliver.

Given the floor for closing thoughts, Rae asks what 2.5 Pro has unlocked for Labenz. The host's complaint is almost every RAG app: a flat monthly fee creates a margin problem, so hyperparameters get set to save money, and each call includes too little context. His workaround, especially on small personal projects, is to print the entire codebase to a single text file and paste it into the model. Usually 100,000 tokens fits in any leading system. The research codebase for the emergent misalignment paper — Labenz calls himself the least valuable author and "the Forrest Gump of AI" — was 400,000 tokens, too much for anything else, and only 2.5 Pro could take it.

Previous Gemini models could ingest that much, but he was never sure they were in full command rather than partial command. This time the grasp felt incredibly strong. He would not be overly trusting, but he now has much higher confidence taking dumps of information he does not even necessarily know the contents of and letting 2.5 latch onto what matters. Helping you navigate long context you already know is one thing. Helping you navigate long context you do not have great command of is, he says, a very different thing, and that is the game-changer.

Rae is glad to hear it. He worked with the long-context people in the run-up to the original breakthrough and stayed in touch with the pre-training folks who focused on making long context actually good for 2.5 Pro — not only one million and two million tokens, but effective use of them. He forgets the name of the external leaderboard, but on a 128k-context comparison shared publicly, Gemini 2.5 Pro uses the window far more effectively than basically any other model. It feels as if the model has read everything, dropped nothing, and, within a second, studied a very large codebase in detail. That, he says, is a remarkable source of the AGI feeling.

Labenz closes by saying he will never forget where he was the first time he tried GPT-4, and that there have not been many moments in the two and a half years since when something felt qualitatively different from everything before it. This was one of those moments, an unlock that forces him to recalibrate what is possible. Rae takes the praise as well-deserved for the long-context work. The practical moral of the hour is blunt: stop starving the model of context, leave it room to think, and treat overnight "breakthroughs" as a slope that just became visible.