500 Trillion Tokens a Month: How Google Grew 50x in a Year

Nathan Labenz joked that this episode could be titled "the decade of the week of May 15th to May 22nd, 2025." Google I/O and a flood of model and product launches had made even dedicated observers lose their grip on the news. Logan Kilpatrick, now at Google DeepMind after years in developer relations at OpenAI, was back for his fifth appearance on *The Cognitive Revolution*.

Labenz asked why leading labs keep shipping similar things on similar timelines—reasoning models, then coding agents, then connectors into Gmail and Docs. Kilpatrick said some of it is research: once a technique is shown to work at scale, others light the same path and fold it into their own independent bets. The rest is competition. AI, he said, is currently the most competitive ecosystem in the world for money, talent, and speed, and nobody wants to look behind in the present while still trying to make distinct long-term bets.

The harder number was not the launch calendar. At I/O, Google said it was processing 500 trillion tokens a month across its services—up from 10 trillion a year earlier, a 50x jump, or more than 50,000 tokens per month for every person on Earth. Labenz noted the curve was still climbing. How that happened at a company once described as a sleeping giant, Kilpatrick argued, is mostly a people-and-organization story that will never sell a front-page headline.

Historically Google ran several AI organizations with different goals, for reasons Kilpatrick thinks were largely the right ones. Google Brain pursued a wide research agenda—the Transformer came out of that breadth. Google Research did more applied work meant to flow into other products. DeepMind, under Demis Hassabis, bet on a specific path to AGI. When no short-term winner was obvious, spreading those bets made sense.

Once one approach started working in the near term, Google consolidated. In mid-2023, Brain, part of Research, and DeepMind merged. Kilpatrick called that the start of putting the company in position for the next decade with this technology. Large human systems are messy; training runs are slow; releasing models end-to-end is slow. OpenAI had been running that iteration loop longer. TPUs for research and inference also do not appear overnight, and they were not sitting idle waiting to be used.

What excites him now is the slope of improvement: teams working as one, breadth of research flowing back into the main Gemini models, infrastructure staying ahead, and a real release cadence with hard lessons baked in. DeepMind itself has shifted from a foundational-research lab to an organization that ships the Gemini app and developer tools, which then feed back into how models are trained. He expects the next three to five years to show the results of those hard decisions.

Incentives and DNA matter too. Sundar Pichai has called Google an AI company since he took over around 2016, and Kilpatrick pushed back on the joke that Google sat on the Transformer: it was already powering Search at multi-billion-user scale, just not in today's generative form. Better models help Docs, Sheets, Waymo, YouTube, and Cloud; they are not an add-on. Labenz contrasted Meta's still-fragmented setup. Kilpatrick did not comment on Meta, but stressed that Google's products all depend on the same model improvements.

Looking ahead, Labenz asked whether leaders will keep competing on the same bar charts or start to diverge in form factor and strengths. Kilpatrick would bet on more divergence. At a dinner a few nights earlier, many founders were betting on model convergence. His view depends on the level of abstraction, but the easy gains have been captured. What remains are structural advantages, and Google's infrastructure is one of them.

Getting to this point was not unexpected. Getting to the next level will not be easy for a lot of teams, and that is where world-class groups making an order-of-magnitude bet will show through. AI innovation, he said, does not get easier from here. Some labs might even choose a narrow lane—being the world's best coding model, for example—though he doubts Anthropic will do that, given its broad mission.

For companies training foundation models outside the top labs, finding something they are genuinely good at, and building a differentiated model or infrastructure perspective around it, "makes a lot of sense." Labenz said he would not want to be a tier-two foundation-model trainer today, especially if DeepMind is saying it only gets harder. Kilpatrick did not sign on to the claim that big tech will win everything it wants, including at the application layer, and turned instead to what builders can still do.

Google now has Jules, a coding agent, in the same market as products that use Gemini. Kilpatrick said Jules is still early. The team is strong, but the level of adoption at other AI coding products is on a different scale: Cursor had just announced $500 million in ARR. That did not dampen his view of the application layer.

"There's no better time in human history than right now to be building a startup," he said. If you are competing with the big labs on language models, you had better be very well capitalized, because that is a very difficult problem. If you are building on the application layer, the barrier has never been lower. Time-to-software, the chance to explore new ideas, and the ability to scale monetization and build retentive products have all gotten easier in this moment.

There are a million and one problems to solve. Large products solve some of them in a general way; a startup can go deep on one user segment. Big companies often cannot use the flood of new AI tools because of security, privacy, and enterprise constraints, so their tooling evolves more slowly. Labenz cited Shortwave founder Andrew Lee, who concluded after real soul-searching that "our only advantage is speed." Kilpatrick added a second word: focus. Google has a million innovations happening at once, which is a burden. A startup has one thing to do, and does not need to make a lot of decisions.

Labenz asked about Windsurf: after a deal with OpenAI, Anthropic cut off Claude. Should founders worry that Google might someday ship Gemini 3 into its own coding agent, Gmail, and Docs first, and only later into the API? Kilpatrick had empathy for Anthropic's line. Putting compute toward long-term partners, and not supporting a competitor, is ordinary business strategy, and he has spent a lot of time thinking about how to get compute to the right teams.

Google's situation is different. Google Cloud, he said, is the fifth-largest enterprise business in the world, with a mandate to bring Google-scale infrastructure to builders so they do not have to recreate it. That strategy is unlikely to reverse. The picture is often more extreme than Labenz sketched: external developers frequently get models *faster* than internal Google teams.

Workspace features Kilpatrick had barely thought about can have 150 million monthly active users. Those teams cannot flip models casually; behavior drift, evals, and responsibility to users all take time. A small product can switch overnight. Forcing internal adoption before an API release would push models out the door far later than today, which he has a hard time imagining making sense—and giving models to developers is a core part of the Cloud business.

On Daniel Kokotajlo's *AI 2027* scenario—labs widening the internal-external gap and keeping the best models for recursive self-improvement—Kilpatrick does not buy it for Google. Evals are hard; without a release you lack signal on whether a model is the right long-term bet. Momentum among developers is a real competitive pressure. Switching costs are high. Game theory, he said, points to models continuing to ship to the world much as they do now. The economic unit might someday stop being a token billed per million, but he still thinks there is a great business in giving capability to other people.

Labenz described dumping a 400,000-to-500,000-token research codebase into Gemini 2.5 Pro in AI Studio and iterating on bugs—something no other provider he could access even supported at that length. Kilpatrick said long-context command is currently one of the largest performance gaps. On OpenAI's MRCR eval, the eight-needle hard setting, the latest 2.5 Pro was on the order of 20% better. Gemini 1.5 Pro from a year earlier was already near 100% on single-needle retrieval; the failure mode is exponential decay as needles are added, and there had not been much long-context innovation in the intervening year.

Jack Rae, who leads Gemini reasoning and originally worked on long context, has been talking with Kilpatrick about the fusion of the two: reasoning is what lets you actually use the full window. Request-size distributions have shifted; with 2.5 Pro, long-context usage is dramatically higher than in the past. Kilpatrick thinks people will keep stuffing more into the window. RAG will still be needed in some cases, but the old dismissal of long context because the models were not good enough no longer holds.

For a qualitative feel, he built a Colab that pulls "from me" mail via the Gmail API—threads he has actually sent into, which still fit in a million tokens for many years of mail. The data never leaves Google; it goes from Gmail to the user's own Drive. Then you ask the model questions about yourself. He also invited builders still listening this far in to email him for early access: lkilpatrick@google.com. Veo 3's API was still being scaled. Consumer high-price-point usage and API-scale demand are different infrastructure problems, and he did not yet have an external onboard path.

Labenz offered a horseshoe theory of agents. On one end, turn-based tools like Claude Code or Jules feel like original ChatGPT with bigger, more valuable turns, and the human is still on the hook for judging each one. In the middle sit highly railed workflows—LangChain-style systems with eight prompts for eight steps—that people actually trust enough not to inspect every output. Open-ended "here are 50 tools, go" agents still produce magic moments and frequent misses.

Kilpatrick's take: with reasoning, a lot of scaffolding will move into the model. You send a request; in the reasoning step the model already has search, code execution, a sandbox, and function calling. Models are on a path to being agents out of the box. Scaffolding will not disappear, but products should avoid one-way doors. Many companies built complex stacks because models were bad at things; now those layers can come out, and ripping them out of a large product is hard. Reasoning is specifically what made that transition possible for a lot of teams.

Josh Woodward, who runs the Gemini app and Google Labs, told Kilpatrick the same story about NotebookLM. Audio Overviews started as a 14-step process with many Gemini handoffs; today it is about four steps. Fewer sequential calls made the product faster. Labenz had seen the cycle at Waymark: image understanding that once needed a scaffolded pipeline is now basically a prompt to Gemini Flash. On Agent2Agent, Kilpatrick's mental model is that MCP does not cover the whole production-agent story—authorization is one gap—and it is still open whether MCP will absorb those problems or leave room for other standards.

The underappreciated thread Kilpatrick wanted to flag was diffusion language models. Labenz had not used Gemini's yet; Kilpatrick said they are unbelievably fast, and the coarse-to-fine pattern matches how Labenz thinks better than left-to-right next-token prediction. Cost, quality, and performance tradeoffs are still open. If those land near parity, that speed could enable personal generative UI—tokens rendering in the blink of an eye—and editing tasks look like a natural fit. Even if the paradigm does not win, it is a reminder that next-token prediction is not the only bet worth making.

What is missing for AGI? Memory, for one. But Kilpatrick's real claim is that AGI will be a product experience, not a model drop everyone recognizes. Definitions already disagree. Someone will weave together a smarter model, better long context, better reasoning, and memory—the last of which is as much engineering and human psychology as ML, a problem of surfacing the right things at the right time—and people will *feel* that AGI has arrived. The model might only need to be 50% better on a few axes; the rest is product. That is not the model-driven AGI narrative, and he does not think that is how people will experience it.

He still writes his own emails and tweets; in about 95% of cases there is no AI in the loop, because he wants agency over what goes out under his name. A digital twin that makes loose approximations is, for him, a foreign idea. Labenz worried NotebookLM was starting to eat podcast time. Kilpatrick's reply, echoing Sundar on Search versus chat: Search query volume is still growing even after ChatGPT's hundreds of millions of users, because they solve different problems. People will still want Labenz's perspective. In a world where you could spin up a thousand similar shows, "people want the Nathan experience."