Use Retrieval for a Wi-Fi Password. Use Experts for a Company's Private Language.

Ask a large language model which number is larger, 26 or 21, and it may get the comparison wrong. Worse, ask it to count down from 21 to 26 and it will keep counting, because it assumes it will eventually reach 26. That example comes from Kelvin Guu, a senior staff research scientist at Google, talking with investors Sarah Guo and Elad Gil on the *No Priors* podcast in May 2023. The episode is about how to personalize large models without dumping every fact into pretraining or fine-tuning.

Guu's path into the problem started in math. He wanted tools that help people learn more easily and find information faster, so he used mathematics as a foundation, then did a PhD in Stanford's statistics department and migrated into the NLP group. He joined Google in 2018, in a group that was then exploring early ideas on pretraining language models, work that became BERT. Playing with BERT made something obvious that prior generations of models had not: a large amount of world knowledge had landed in the weights without anyone deliberately encoding it in the fine-tuning data.

That observation pushed Guu and his colleagues toward retrieval-augmented models, a way to get even more knowledge into the system without pretending the parameters can hold everything. REALM, on which he is a lead author, became one of the early landmarks of that line. The rest of the conversation runs from retrieval to instruction tuning, from editing a model's beliefs to the missing instincts of autonomous agents, and it is dense enough to take in pieces.

People arrive at retrieval-augmented modeling for three reasons, Guu says. The first is memorization capacity: parameters cannot hold every fact, so an external corpus is a better place to put them. The second is modularity. Data sources should swap in and out the way a database does, which matters in a long list of business settings. The third is timeliness. Sports scores and other daily events should enter the system without a full retrain. Those original promises, he notes, are still only partly met by today's tool-using models that issue searches or attach citations.

The architecture is straightforward at inference time. The model embeds the user's input as a dense vector, searches a vector space of embedded documents with nearest neighbors, then reads the retrieved passages with cross-attention and makes a prediction. Training is the interesting part. REALM was built in the masked-language-modeling era of BERT: blank out a few words and ask the model to fill them in. The model is allowed to retrieve first. If the documents help it fill the blank, it learns those retrievals were useful. If they do not help, it learns not to retrieve them again. Nobody wrote a retrieval policy by hand.

When should you retrieve at all? Guu's answer is somewhat counterintuitive. Common knowledge on the web needs retrieval less and less. If a dense model sees a fact often enough, scaling puts that knowledge into the weights. Memorization tracks frequency in the corpus; a model is like a student who remembers more detail the more times it rereads a document. The remaining case for retrieval, he argues, is modularity, personal information you would not want in the main model, and the special data of an enterprise customer.

Elad Gil asks how this relates to mixture-of-experts models. Guu points to a paper called Branch-Train-Merge: split the pretraining corpus, train an expert on each slice, then route or vote at inference. Retrieval and experts are trying to do the same job at different grain. Retrieval goes all the way down to a single document. An expert owns a broad region of the corpus. If you want a precise factoid, such as a Wi-Fi password, retrieval is excellent. If you are adapting a model to a company whose codebase uses a specialized language nobody else speaks, retrieving one snippet will not teach the language, while an expert trained on that corpus will adapt faster. Modularity also matters because most organizations cannot train a large model from scratch. They need to spend the least possible effort adapting something that already exists.

Guu sketches three generations of adaptation. As late as 2017, people designed a special network for each task, including custom architectures just for sentiment classification. Then came the pretrained-model era: everyone shared one architecture and swapped fine-tuning data. The latest turn is instruction following, work he was involved in at Google. In the FLAN paper, one of the early results showing that large models can follow instructions, the recipe is simple. Start with a pretrained model, collect about a hundred tasks (sentiment, question answering, summarization), annotate each example with an instruction, and train across all of them.

The result that mattered was transfer. After training on a hundred tasks, the model could adapt to a 101st task it had never seen. At the time many people thought you would need thousands of tasks for that to happen. Because it worked, InstructGPT followed, then ChatGPT, and suddenly you no longer needed specialized fine-tuning data to use a large model. Guu is careful about the boundary. Hallucination is the usual example: you cannot write "please do not make things up" into a prompt and expect the problem to vanish. Many other instructions will not be followed either. Fine-tuning and reinforcement learning are still the tools for personalization and for soft traits that are hard to put into words.

He had expected continuous learning, updating weights in production in real time, to become popular. It did not, for a practical reason: if the weights keep moving, you never get a clean chance to validate that the model has not gone off the rails. Weekly or monthly releases turned out to be enough, which surprised him. A compromise he likes is prompt tuning. Freeze the large model, treat the prompt as a trainable vector, and run gradient descent on that vector. This week's prompt can be swapped for next week's without touching the base weights. His group had also just published Simfluence, in the emerging area of training-data attribution: which examples taught the model to do a particular thing? The gold-standard check is to delete an example and retrain the whole model, which almost no one can afford. Simfluence trains a lightweight simulator of the training run to estimate each example's effect. If you know which data is valuable, you can go get more of it. Longer term, it is how you would find out whether a model that simulates a Linux terminal or draws a unicorn in LaTeX is generalizing or cleverly patching together something that was already in the training set.

Asked which ideas about memory he finds most exciting, Guu points to a line of work he calls model surgery. The representative paper is ROME, from a lab at MIT. The demonstration sounds like science fiction. The model believes, correctly, that the Eiffel Tower is in Paris. Researchers edit certain weights so that it believes the tower is in Rome. Related facts update with the change. Ask what the tallest structure in Rome is, and the model answers Eiffel Tower. The edit does not sit as an isolated fact. It propagates through the network.

The theoretical picture underneath is that the many matrix-vector multiplications inside a large model can be read as lookups. The matrix is the table; the vector is the query. Updating the matrix updates the knowledge. Guu sees this as a third kind of modularity. Retrieval augmentation and mixture of experts hang extra modules outside the model. Model surgery cuts inside the parameters and still keeps some generalization. That is why he would tell a student interested in knowledge representation to work with dense models rather than chase a fully canonical knowledge base.

He had started his research career in knowledge bases, which look perfect on paper: one canonical fact, reasoning over a graph, a clean source of ground truth. In practice it is very hard to get a representation that is a hundred percent canonicalized. In text, a single fact lives in many places, so an edit has to find every mention. There is a standing trade-off between centralization and coverage. Dense models keep expanding into more applications, so anyone who can make those dense representations more manageable and more controllable is, in his view, standing on the right side of the momentum.

Guu flags that what follows is personal opinion, drawn from public discussion, not from his work at Google. The going approach to autonomous agents is inspired by the Turing machine. Models operate over a small promptable context, so people chop long-horizon tasks into smaller promptable ones and bridge them with a memory mechanism, storing intermediate results as text or vectors. Sarah Guo makes it concrete: plan a family trip to Hawaii, find kid-friendly hotels, book activities, remember that you like the aisle seat. The current hack is a checklist the model ticks off.

The failure mode is a sticky note. The agent decomposes a task into subgoals and writes one down. If that subgoal was wrong from the start, it becomes a canonical goal, and the model has no good way to reverse the decision. More prompting, so the model considers every angle, is the usual patch. Humans do prompt themselves, Guu says, but a lot of what we do is instinctive, and those instincts are trained over time. Psychology's chunking result is his analogy: a person cannot easily hold more than about seven unique items, but once those items are packed into meaningful chunks, much longer sequences become rememberable. Humans start with explicit reasoning and distill it into intuition. Current agents never take that second step. Everything stays explicit, and at some point that will not scale, or it will become brittle.

A real travel planner would also learn your preferences from feedback rather than from a fully specified instruction list, and that learning loop does not yet exist in the methods he sees. The gap is even clearer with complex tools. Issuing a search query is almost stateless: something in, something out. Operating a spreadsheet an organization has used for years is different. You have to implicitly understand how it is used, what is allowed, and what is not. He also points at two farther-out ideas. Reinforcement-learning papers have proposed a concept they call fear, teaching a model to recognize states it cannot back out of, the analog of human emotions that keep us from irreversible moves. Human memory consolidation also prunes what you have thought about, and early agent systems have no analog of that either.

Then comes the counting example. Ask whether 26 or 21 is larger and the model may get it wrong. Ask it to count down from 21 to 26 and it counts infinitely, an extreme version of human cognitive dissonance. A person eventually inserts an introspection step: wait, I have been counting for ten hours. People try to patch that with chain-of-thought, reason, then ask whether the reasoning is right, then loop. Guu's question is whether introspection has to be that explicit. Humans have associative mechanisms that ping them after a while. The alarm does not have to be another prompt.

Guu is most interested in recipes that let people adapt models easily, and the right interface is not settled. Training is hard largely because of data quality. Curating examples at the scale of ten or thirty is tractable, and that range may be the sweet spot for many users. Imagine, someday, a language model in the home, which he is careful to say does not exist yet. You want it to follow your family's values. Do you write those values into an instruction prompt and demand that they apply everywhere, or do you shape the model through repeated feedback: no, I do not think you should say that? He is not sure. "Do as I say, not as I do" seems easier in this setting, he jokes.

Elad Gil worries that training on how people actually behave would teach the wrong social behavior, and he brings up Anthropic's constitutional AI. Guu's read is practical. Model providers have a strong incentive to put as much as possible into prompting, because that is the interface with the broadest reach. They do not want to tell every user to fine-tune. Instruction tuning can also be pushed further: if you can anticipate that someone will say "follow these family values," you can prepare the model at training time to be more responsive to that kind of request. It is a game of how much of the user's adaptation you can anticipate. Sarah Guo likes the phrase family values, because it sounds like something a household is in together, rather than a framework being enforced from above.

He also has a product observation. The current paradigm parks a person in front of a chat box and asks them to boil every relevant context into a wonderful prompt. That takes a lot of brainpower, so plenty of people never think to use the model. The public ambition he sees is to make models more contextually available. You have been in the browser for twenty minutes, you ask why you cannot get this done, and the model looks at everything you have been doing, infers your intent, and helps. Lowering that context barrier, he implies, is more important than polishing the prompt.

Guu's advice to future researchers starts with a split he keeps seeing. One camp feels that even if they sit and do nothing, the work will get done. The other feels there are too many things to finish in time. His answer is to move the goalposts. Goals that felt ambitious a short while ago are no longer ambitious enough. Think farther out and the unsolved list is long: agents that are more autonomous and safer, training-data attribution that actually explains a behavior. Paying attention to the product world, he adds, will throw use cases at you.

Technical knowledge still matters. Building valuable applications with a simple skill set is great, and there is real creativity in that. If you want a technical challenge, though, understanding how these models work is still wide open, and the field needs that understanding for safety and predictability. Elad Gil asks the sharper career question. A year ago he hoped his kids would go into computer science. Fifteen or twenty years out, machines may write better code and do better CS research than people. What then? Guu says the valuable skill has already started to shift. Technical proficiency used to be a hard gate: without it you could not even test an idea. Models now help you write in languages you do not know. Differentiation moves to problem formulation, to noticing a problem and framing it so it can be reduced to a technical one.

Even if a model autopilots a lot of the work, someone still has to validate the result, and that may still take technical skill, unless you prompt another model to validate, in which case who validates the validator? Gil pushes one step further. Chess, Go, poker, then diplomacy: each time machines crossed a line, people retreated to the next supposedly human preserve. Noam Brown's work on diplomacy is the latest example. Nobody can say where the human advantage actually ends. Sarah Guo closes on a lighter note. Robotics still lags, for data reasons, so maybe Elad's kids should be woodworkers. Guu hopes they can still be knowledge workers, then adds that physical work is great too, even if that future would say something uncomfortable about humanity's place.