Uber Burned a Year of Tokens in 13 Weeks. Retrieval Is How Agents Win

Early in 2026, Uber burned through its entire yearly token budget in 13 weeks. The method had a name, token maxing: once million-token context windows arrived, some teams stuffed them on every call. Labenz's introduction sketches the whiplash that followed. Limited windows had forced everyone to build RAG pipelines. Giant windows produced a "RAG is dead" moment. Token maxing collapsed as fast as it arrived. Now data volumes are up, usage is scaling, and naively filling the window costs multiple dollars a shot. RAG is a priority again.

Pete Johnson is Field CTO of AI at MongoDB. His job is to travel; by the time of the recording he had already been to seven countries and spoken with about a hundred enterprises. The point he keeps returning to is that agent performance, especially cost-adjusted performance, depends heavily on effective retrieval. MongoDB has been shipping the pieces developers need for that: vector search, rank fusion and score fusion, a `$rerank` stage, and embedding models from the Voyage AI acquisition, including a shared embedding space and a Matryoshka structure.

He is also blunt about the maturity of the field. The industry has been building databases for sixty years and agents for about 18 months. Nobody has all the answers. Where enterprise AI goes wrong, he says, it is rarely the model. Bad data quality and a bad security posture are not problems AI solves. They are problems AI amplifies.

Johnson likes to start in June 1970, when IBM researcher E. F. Codd published the paper that gave birth to SQL. Johnson was born that February, a joke he enjoys: he is as old as SQL. Applications then were departmental. Not every company had a computer; the people who did often sat on the same floor, worked nine to five, and could live with weekend downtime. Of memory, compute, and storage, storage was by far the most expensive. So you normalized. First normal form and third normal form were ways to lay data on disk without storing any piece more than once. Two people at one address meant a people table, an addresses table, and a third table to link them.

MongoDB's first commit was in October 2007, after 47 years of Moore's Law, the internet, the cloud, and the iPhone. The scarce resource had flipped from disk to time. Apps had to stay up on weekends, serve users anywhere, and return results fast. Mongo's flavor of NoSQL stores JSON, on disk as BSON, so the format stays roughly the same from disk through the server to the client. Take the address example and denormalize it: two documents, each with a name and the same address. You duplicate the address. You get one disk read instead of three. Constraints changed, so the optimum changed. That does not make every problem a Mongo problem, Johnson says, and SQL still solves plenty of the old ones.

A common misunderstanding is that MongoDB is schemaless. It is schema-flexible. Documents in a collection need not share a shape; adding a field does not require the production surgery of altering a SQL table and watching the cascade. Labenz recalls first seeing Mongo about 15 years earlier, when a young Guillermo Rauch, now CEO of Vercel, was showing it off with then-futuristic Node.js apps. Today Mongo is in about 75 percent of the Fortune 500. Fiscal 2026 revenue was about $2.5 billion, which Johnson pegs at roughly 3 percent of a $100 to $110 billion database market. Enterprises pick it for speed, scale, and, increasingly, retrieval quality. Some choose workload by workload; others go all in. There is no single right forest path.

Mongo's search story began in an odd place: lexical search. In 2020 the company noticed customers standing up their own Apache Lucene servers next to Mongo clusters so they could run keyword retrieval on text fields. That year Atlas Search put lexical search into the managed service. Vector search was the next logical step. A vector, Johnson says, is an array of floats. Send text, an image, audio, or video through an embedding model and you get that array back. In a document model it is just another attribute, plus an index. Replication, sharding, and encryption were already there; vector search inherited them.

Lexical plus vector plus a pre-filter — only books published after 2000, in his running example — is hybrid search. In 2025 MongoDB bought embedding-model company Voyage AI for $220 million, which Labenz notes is less than 1 percent of Mongo's market cap. Why buy a model company rather than a vector-database startup? Johnson's answer is that vector search was straightforward on JSON. The market treated embeddings as a commodity, and he did not. Voyage's models sit near the top of Hugging Face's RTEB retrieval benchmark and can beat common cloud embeddings by as much as 14 percent. Anthropic does not ship an embedding model; it recommends Voyage. Pick the wrong embedder, and the gap between a hallucination and a correct answer can be that 14 percent.

Voyage's unusual features are aimed at a lower learning curve. Contextualized chunking sends the sentence you care about and its surrounding context as two strings and returns one vector, so small chunks can still retrieve well. Version 3 shipped last summer; version 4 landed in the six weeks before the interview. Matryoshka embeddings nest like Russian dolls: embed the corpus at 1,024 dimensions, then lop off the last 512 floats to try 512, without rerunning the data. The January v4 text family — large down to an open-weight, free Nano — shares one embedding space, so vectors are compatible across sizes. You can embed the corpus with large, run development queries on Nano on a laptop, and pay no query tokens until production. Auto-embeddings watch a collection, field, model, and dimension; when a document changes, the vector and the index update themselves.

The query side is being collapsed into one round trip. Rank fusion and score fusion combine lexical and vector search in a single API call, merging by rank or by score. A recently released `$rerank` stage does the same for reranking. Johnson says a reranker alone can lift retrieval quality 5 to 10 percent. His rough threshold: for a demo or POC, Postgres, pgvector, and the cloud vendor's embedding model are fine. Around 100,000 vectors, or whenever retrieval quality decides whether the answer is right, it is time to get serious. His joke is that there are two kinds of developers: those who love MongoDB, and those who have not tried it.

Asked what a good memory system looks like today, Johnson starts with the architecture timeline. Late 2022, ChatGPT was query into a context window, model, result. RAG arrived because models only know public data. In 2025, knowledge-cutoff problems were patched with tools and MCP: ask for MongoDB's stock price and the agent can go look it up. Then came looping. One pass's output became the next pass's input, but the context window is still dumb — fresh every call — so memory became necessary. Early memory was crude. Short-term dumped the whole session into context. Long-term dumped every session from the last three days in there too.

Two side effects followed. One was money; Uber is the extreme case. The other is subtler. Academic work Johnson cites finds that the first roughly 7,000 tokens and the last roughly 7,000 are the important ones, and the middle can confuse the model. So the question flipped from "how do I cram a million tokens in every time" to "can I pick the right 200,000 for this loop?" Enterprises are growing finer memory types, including taxonomic memory: a vertical or a company may have a hundred internal terms, but this loop only needs the five that apply, and the next loop picks again.

A coworker had written him a line the day before, which he quotes on air: write, change, recall, forget. "We've been building databases for sixty years. We've been building agents for, like, eighteen months, man. Nobody knows all the answers yet." Forgetting is the hard part. Memories have a half-life; last week beats three months ago. Labenz has felt that viscerally. His own system holds about a gigabyte of five years of email, Slack, DMs, and podcast transcripts, rolled into monthly and yearly summaries under an entity wiki. The model kept a project he had merely tried, then dropped, as an open thread for months. He has come to treat forgetting as a virtue.

Mature systems, Johnson says, give the application author two jobs. First, call memory with a token budget — "give me the best 50" — and let the system mix memory types. Second, send the answer back so memory can curate and write, with RBAC so people in the same role can share what they learned. For huge corpora, customers mix a graph of two to six layers, sometimes a dozen, then do vector search at the leaves. Retailers segment by product category, then search inside the node. Because the store is JSON, graph, document, vector, embeddings, and reranking can live on one platform. The anti-pattern he flags is using extra LLM passes to classify and shrink the corpus. That burns tokens. Embedders and rerankers are cheaper and, at this point, lower latency. ElevenLabs, Mongo's biggest customer in this telling, runs multiple smaller agents per customer for audio editing and did not take that expensive detour.

Johnson sorts the hundred companies he has talked to this year into three camps. Camp one bought a license for one tool and declared the AI strategy done, which is almost never specific enough. Camp two has run POCs and a few production deployments and cannot make the ROI math work. His diagnosis is blunt: teams stuck in POC purgatory usually picked the wrong problem, before the technology was even applied. Camp three is further along, buying platforms or building in-house around more sophisticated memory. There is still no LAMP stack for agents, no React and Angular. Buy the infrastructure you can, and spend the time on business logic.

His project filter is concrete. List the ten to 15 most painful business problems. Keep the ones with good data. Keep the ones that already have metrics. That last cut is the one people skip, and without a baseline you cannot know if AI helped. Call centers are the classic win because cost per call and volume already sit on the bonus sheet; when the numbers jump, you can attribute them. Software delivery went the other way. Early in the year people bragged that Claude Code or Codex let them write five times as much code. Johnson is unsparing: lines of code is one of the worst productivity metrics you can use. The number that matters is how fast an idea reaches production.

Security draws a bright line. Most Fortune 500 work he sees is employee-facing, with a human in the loop. If two employees leak a paycheck, it is embarrassing. If two customers see each other's data, a VP has to take the blame. Customer-facing autonomy is higher risk for the same reason Labenz found when he ordered pizza the previous Friday from Greg's Pizza, a one-location shop in Detroit. An agent answered the phone. The happy path was smooth; when Labenz asked which company powered the agent, it thought he wanted to order pizza for his company. Even small businesses are plugging in AI support. Meanwhile Mongo itself is rewriting docs for agents: an MCP server so agents can talk to Mongo instances, and a handful of agent skills — six to eight markdown playbooks for data modeling and operations that used to live as human documentation.

Labenz presses on the Voyage deal as a macro signal. Mongo built its own vector search instead of buying a vector-database startup, then spent $220 million on a model company. Johnson's lesson is developer time, not a story about models getting commoditized. Adding an array-of-floats attribute and an index was straightforward on the document model, and it stood on a decade of Atlas replication, sharding, and encryption. What was not straightforward was the market treating embeddings as interchangeable. Voyage stood out. The "better together" pitch is auto-embeddings and a single console for data, vectors, keys, and rerankers so developers are not hopping across four dashboards.

A vector-database founder once told Labenz that most of the data landing in vector indexes had never been in a database at all — it sat in lakes, warehouses, or piles of documents. Johnson says that is true. Vector search maps a blob into n-dimensional space and asks what is nearby. Video, audio, and SharePoint PDFs are suddenly findable in a way lexical search never allowed. Mongo is seeing kinds of data in documents that were not there in previous eras, and hybrid search — pre-filter, lexical, vector — is how you pull the levers.

The travel notes are the surprise. Johnson lives in Cincinnati. This year he had already been through Amsterdam and London, then Toronto, Bengaluru, Mexico City, and São Paulo. The two most sophisticated customers he spoke with were in Mexico City and São Paulo. Both assumed American competitors were ahead. The opposite was true. He does not blame American conservatism. In the cloud and mobile waves, geography was a real barrier: no hyperscaler data center in your country, no luck. Almost every country now has at least one, and with it access to models, vector databases, embeddings, and rerankers. Infrastructure that used to take years to roll out is already in place. For the first time, being ahead is less tied to where you sit.

Johnson's leftover technical note is shared embedding spaces. In January, Voyage released four versions of the v4 text model, including Nano, open-weight and free on Hugging Face. All four share an embedding space, so a corpus embedded with large can be queried with Nano during development. You take a retrieval-quality hit, but you can drive query token cost to zero until you switch back. It is another instance of the same philosophy: give developers time back for business logic.

The practical checklist from the conversation is blunt. Give the context window a budget instead of filling it; remember the first-and-last 7,000-token pattern, and put only what this loop needs in the middle. Check the embedding model before you blame the LLM — it is not commoditized, a better one can be worth 14 percent, and a reranker can add 5 to 10. Audit memory with write, change, recall, forget, and add a half-life so abandoned projects do not haunt the agent. Choose work by pain, data, and a metric you already have. And take Labenz's bitter-lesson pass: walk the stack, find the handmade patches you added for a dumber model, and delete the ones you no longer need.

The pace, Johnson says, will only get faster. MCP shipped on the Monday before Thanksgiving two years earlier; by March every competitor had embraced it. Token maxing was named in early March and was over as a conversation by April. Databases took sixty years to learn. Agents have had 18 to 24 months. The fun part, he says, is that the industry still has a lot to figure out together, and you will keep seeing more agents in ordinary life as a result.