One Model Inside 38 Products: Paige Bailey on How Google Trains and Ships PaLM 2

Paige Bailey, lead product manager for PaLM 2 at Google DeepMind, joined Nathan Labenz on *The Cognitive Revolution* a few months after Google I/O. The model had been announced in May 2023, and unlike previous research unveilings it was already in the product stack. Bailey said she had never been prouder of a team effort: getting the model ready, tested, instruction-tuned, and then folded into Alphabet's surface. She pointed listeners to g.co/palm2 for the technical report and the announcement video.

By the time of the conversation, PaLM 2 was in "dozens" of products and features across Alphabet. Shipping a model that has already survived dozens of product teams' scrutiny is a different problem from shipping a research checkpoint. Bailey called that combination exhilarating, and later in the interview she would put the live count at more than 38 products and features, at sizes ranging from small enough for a phone to large enough to need a cluster.

The conversation also landed in the middle of a corporate rewrite. Google Brain, where Bailey had been until recently, was being merged with DeepMind under Demis Hassabis into Google DeepMind — a decision she attributed to Sergey Brin and Alphabet leadership. She had already watched dozens of collaborations spring up across groups that used to sit in different product areas. From her seat, it was the best thing Alphabet could have done for its research teams.

Bailey is not a classic "where does the button go" product manager. She implemented her first machine learning model around 2009 or 2010, spent years as an engineer and data scientist — including developer-programs work at Alphabet — and only moved into product a few years ago. She still programs every day; if she could not, she says she would be "a very sad person." That engineering empathy, she argues, is what makes it possible to product-manage a research model at all.

Asked what it means to product-manage something with this much surface area, Bailey walked through the full pipeline. Pre-training data mixture comes first: compliance matters, but so does anticipating how people will actually query the model, then folding data that can answer those questions back into pre-training. That takes either strong intuition about downstream use or rigorous analysis of every surface the model is already deployed on.

Evals sit next to the data work. The team tests responsible-AI behavior and capabilities in the same breath: does the model generate code well, explain code well, detect security vulnerabilities, solve math problems. After pre-training comes instruction-tuning so it can give zero-shot answers instead of few-shot ones. Deployment constraints — especially latency at different model sizes — then shape how product teams can use each variant. RLHF and RLAIF become the levers for incremental improvement over time.

Doing this well, she said, means being "really, really invested" in every use case the model is deployed into. She is impressed by PMs who live in finance, visual design, or button placement; she is not one of them. The skill set she built as an engineer is what makes a research PM role work, and she does not think that combination is for everybody.

PaLM 2 was a "very, very significant" compute allocation, which meant Bailey's product work included convincing many Alphabet teams to approve TPU use. The pitch only works if the resulting model can be used by as many teams as possible, with no compliance landmines and no "stickiness around data." Google had been announcing generative models at I/O for years. This was the first time, she said, that they announced one with over 25 deployments already in place.

The lineage she sketched is BERT, original PaLM, PaLM 2, and then Gemini. The team stood on aggregated datasets, open-source corpora, preprocessing recipes, and architectures other groups had already found impactful. Then they had to mobilize researchers across the organization to train the model, choose a tokenizer, and "babysit" the run — monitoring, restarting, rewinding when training went sideways — before a final polish so it actually worked in products.

"It feels an awful lot like working on an Apollo program," Bailey said. Outsiders see the compute and the data; they underweight the need for a strong team with deep expertise in many different areas. She would not call herself a manager in the people-ops sense. Technical leads own each workstream, define the work, hold people accountable, and review changelists — though she reviews CLs too.

Her day job is coordination: keeping every team pointed in the same direction, making sure a newly designed eval has not leaked into the pre-training mix, and staying "tied at the hip" to the latest checkpoint so they can test whether the metrics they care about are actually going up. The original PaLM paper came out in April 2022; PaLM 2 was announced in May 2023. In that window the team had to design the next generation, grow the dataset, prove the giant run would work, lock compute, pick sizes, instruction-tune, and help product areas deploy.

Labenz asked how the team even decided how they wanted PaLM 2 to behave, given that you could write almost infinite instructions for a system this large. Bailey's answer was layered. Downstream teams define personas: Bard has one for how the model is presented to a user; a code assistant in an IDE has another; Med-PaLM and other domain models have their own. The foundation team tried to create a general-purpose instruction-tuned model that would be "generally useful" across as broad a variety of use cases as they could.

Each product team then owns making that model a helpful tool for its users. Immense work goes into that process, and into instruction tuning that nudges the model toward desired behaviors. Alignment work, she said, is now one of the most interesting things foundation-model teams do. She singled out Anthropic's papers on AI alignment as work she particularly loves reading.

Compressed into a year, the job included deciding how much bigger the dataset should be, running experiments to convince themselves the giant training run would work, wrangling people, choosing model sizes and domain-specific variants, instruction-tuning, and helping product areas deploy. It was "an awful lot of work" but also energizing. The payoff was Sundar Pichai on stage saying the model exists, it is already in over two dozen products and features, and congratulations — you are using it. Alphabet had never been able to do that before.

Labenz put the public critique on the table: after ChatGPT, the story was that Google had lost the ability to ship, that bureaucracy had frozen it. Bailey said she has always been impressed by how many features Google regularly ships — inside Photos, Gmail, search-algorithm updates — even when they are not immediately obvious to an end user. Labenz noted the contradiction in the discourse: Google does not ship, except for the three chat apps. His own view was that the narrative was overstated, and that pre-ChatGPT models such as GPT-2 were not actually useful enough to put in products without looking strange.

Google has been a machine-learning-first company for a long time, Bailey said, with AI in nearly everything it does, but those models used to be simpler and easier to serve. Part of why PaLM 2 and the GPT-3-and-later variants saw such adoption is that serving got much more efficient, and distillation can now take performance from a model with hundreds of billions of parameters and fit it into one with "a couple dozen." She does not want a world where only people who can pay a premium for accelerators get these tools. The goal is powerful models with a small enough footprint to run on a CPU or a single accelerator — something that would not have been realistic until recently.

Her personal eureka was not a PaLM 2 eval spike. It was GitHub, testing the very first instances of Copilot. She already knew IntelliCode, which did well on C# and .NET, and Tabnine, which used GPT-2 under the hood; those tools could produce a decent line or two, sometimes three. Copilot, out of the box, produced five, six, seven generally good lines related to what she was asking — and could even plan a weekend itinerary in San Francisco from text. She decided on the spot that building models, especially making them better at generating code and math, was the only thing she wanted to do with her life.

On GPT-4, Claude 2, and Llama 2, she refused a horse-race frame. Everyone is doing wonderful work; she is especially excited about Llama 2 because continued pre-training and "into the weeds" optimizations are possible in a way a commodity REST API does not allow. Google is focused on improving datasets, exploring RL techniques, and building bigger models like Gemini. PaLM 2 was a text-and-code model; Gemini is multimodal from day zero, a superset of PaLM 2's capabilities. Inside Alphabet it can feel like "living in the future": a checkpoint beats human-level on a task the team thought was eighteen months away, an eval jumps ten points from one approach, RLHF moves a number 25 percent. It has been a sprint since December 2022. She does not feel tired. She feels excited.

Labenz wanted to know how to tell real generalization — grokking — from stochastic-parrot mode, given that loss curves are smooth but capabilities appear in spikes. Bailey pointed to a recent ICML workshop on reasoning, including a panel with Samy Bengio. Papers have proliferated claiming GPT-4 can reason. Probe a little deeper, she said, and the story changes. The canonical puzzle is five blocks on a shelf. Change the blocks to cheese and an apple, tweak the language of "moving," and "the model isn't reasoning at all." It has learned pattern matching from web pages it saw in training.

The other side of the coin is capabilities nobody enumerated in advance. Put source code, diffs, and pull-request commit messages into pre-training, and you can later ask the model how to apply a performance fix or migrate an API — tasks no one sat down and listed before release. When Bard gained Lens, a user sketched a website, photographed it, and asked Bard to generate the code; it worked. The team had not imagined "take in images and generate websites" when they shipped Lens. Rohan Anil, an engineer on the team, had the model generate code and then explain each line in Malayalam. Multilingual ability was a PaLM 2 feature; that particular trick was not in the spec.

That is why Bailey issued an open invitation. Creative people testing the limits of these models are accelerating research: they show labs what to evaluate, how to improve instruction tuning, and what data to generate next. "Every single researcher in the world is tight at the hip to social media just kind of seeing all the cool stuff" users do. Labenz seconded the call for citizen scientists. With a chat interface and a little code, he said, an amateur can make discoveries that contribute to global understanding.

For anyone trying to tell reasoning from recitation, she recommended a talk from the same workshop, "Avenging Polanyi's Revenge." Reasoning items are formatted like SAT or ACT questions — same structure, same entities — so models get good at looking like they reason, the way some people get good at standardized tests. Change blocks to something else, the shelf to the floor, "move" to "place in different locations in space," and everything falls apart. Another finding from that talk: some papers only got the right answer after graduate students repeatedly corrected the model, which led the speaker to argue that the model's ceiling was what the grad students knew. Course-correcting until the destination is not the model reasoning on its own. As for internals, Bailey said she has little domain expertise; Anthropic's recent work can highlight which words in the corpus or context window matter most, but production scale is very hard. Dario Amodei, she noted, still has more questions than answers. "If anybody tells you that they've got everything figured out, then they probably haven't looked in enough locations yet."

Bailey did not have an exact count on hand, but PaLM 2 was in more than 38 products and features, at sizes from mobile-small to cluster-scale. The project she is driving at Google DeepMind is Code AI: 150-plus researchers and engineers on code generation, explanation, security-vulnerability detection, automatic performance fixes, source-code migration, and generating code that calls tools and APIs dynamically. The long-term shape is a general-purpose autonomous agent: a high-level task, decomposed into steps, stitched together in code, able to assign itself new tasks.

Products people can try now include Codey — PaLM 2 with continued pre-training on a lot of source code, offered as a GCP API for completion, explanation, and related tasks — and Duet AI for developers, announced at I/O, which spreads models across GCP: IDE generation, vulnerability detection, log analysis, DevOps help, console autocomplete and Q&A, even automatic architecture patterns. She also pointed to Med-PaLM and the biosciences opportunity, and to Colab, where the Codey API is already in Colab Pro+ for generation, completions, and code Q&A, with error fixing on the way. Labenz's theory: bloated, hard-to-navigate clouds are exactly the products that benefit most from an AI layer that gently guides you.

The next leap she is most excited about is execution feedback. Give a model a high-level question, generate code, check validity with static analysis or another model, actually run it, and if the output did not accomplish the task, recursively feed the failure back until it does. The feedback is not "just keep going" in natural language; it comes from compilers and execution, not only from humans. Accumulate enough of those traces and they become the next round of training data. "That's the only way to get truly powerful systems."

Continued pre-training of PaLM 2 on code produced gains not just on code tasks but on math, logic, and even natural language — enough that more source code and more math-style data in the pre-training mix appears to make the model better at "all the things." She would encourage anyone doing continued pre-training to heavily weight data types that move downstream evals. As for a successor to the Transformer, she would not be surprised to see one. Plenty of "very brainy people" are thinking about it inside Alphabet and out, including Noam Shazeer and the Character.AI team.