Ten Percent of Search Is Health — and the Doctor Is Googling Too

Sarah Guo and Elad Gil introduce Karan Singhal as a Google researcher leading work on medical AI, specifically Med-PaLM 2, just as the company has announced PaLM 2. The timing is not accidental. PaLM 2 is the base model behind Med-PaLM 2, with stronger multilingual, reasoning, and coding skills than its predecessor, and the question hanging over the episode is whether a language model can be made trustworthy enough to talk about health.

Singhal's first AI project, at nineteen, was a fake-news detector built around the 2016 U.S. election. He wanted socially responsible uses of the technology. He now calls that project a little naive. The longer-run question that replaced it is how to shape AI's trajectory so it produces less harm, and medicine became the setting where that question is unavoidable.

He had not spent his whole Google career in healthcare. For the first years he worked on representation learning — the layer under models such as GPT-3 and GPT-4, going back to convnets, word2vec, and GloVe — and on federated learning, so models could improve without centralizing data. About a year and a half before this conversation, he and a few colleagues pitched a Brain Moonshot, Google's internal incubator for ambitious research. It was funded, they got compute and leadership cover, and the first thing the team built together was Med-PaLM.

PaLM, the Pathways Language Model, is Google's company-wide stack for training large language models. The 2022 original was a 540-billion-parameter decoder-only Transformer, then the largest densely activated model, with jumps in code, multilingual work, and reasoning. A few months later, the Minerva project showed what Singhal calls a zero-to-sixty effect on STEM benchmarks: from random chance to solid scores. That result fed Jason Wei's work on emergent abilities, and it is part of why the Med-PaLM team started with multiple-choice medical tests.

Out of the box, PaLM already looked decent on medical multiple choice. The instruction-tuned Flan-PaLM variant did better still. It was the first model to clear the pass mark on MedQA, a set of USMLE-style questions. Then the team asked it to write long-form medical answers and compared those answers with clinicians. The gap was obvious. Multiple-choice competence did not transfer to a reply you would want a patient to read.

The Med-PaLM-specific alignment was instruction prompt tuning, a data-efficient method they explored in the first paper. Doctor labels are expensive, so they collected expert demonstrations of good behavior and used those to tune parameters — more learned than a prompt, cheaper than full fine-tuning. That was the attempt to close the long-form gap without a full second pretraining run.

Moving from PaLM to PaLM 2, and from Med-PaLM to Med-PaLM 2, was not a copy-paste of the first paper. Singhal credits a large group, then names three changes. Pretraining used a mixture of objectives, in the spirit of the UL2 paper. The team empirically checked scaling laws for data versus compute and landed close to DeepMind's Chinchilla result: earlier models had been over-parameterized relative to the data, and feeding more tokens produced a better model for the same compute. The third change was the data mix itself — more multilingual text, and code in more programming languages.

Gil asks for a framework: when do you pretrain from scratch, fine-tune, prompt-tune, or just write a prompt? Singhal's answer is almost entirely about how much relevant data you have. If the supply is effectively infinite and compute is no object, pretrain end to end. If web-scale pretraining already confers the abilities you need — medical reasoning, he says, leans on general reasoning — then start from an existing model, especially if cost matters.

Once you have a pretrained model, the rest is a sample count. Three to five examples: prompt it. Ten to fifty: prompt tuning and fine-tuning perform similarly, and he would take prompt tuning if compute or cost is a constraint. More than a hundred examples and a need for the best score: fine-tune. That is still far cheaper than pretraining. Most teams, he implies, should stop arguing about building their own giant base model and spend the effort on adaptation and evaluation.

The landscape the team walked into was not empty. Galactica and BioGPT had already pushed large language models into science and biomedicine, and Singhal is careful to credit them. What was missing, in his view, was a systematic way to evaluate clinical knowledge. Papers hopped between benchmarks; some of those benchmarks did not look high-quality; and detailed human evaluation — with both physicians and lay people, on a use case such as medical question answering — was rare. That is why the Med-PaLM work was designed to be evaluation-forward.

Gil has a medical-company past: he co-founded Color, and the team agonized over every number it returned to patients. He then tells the counter-story. He took his son to an emergency room; the doctor said he needed to research the case; Gil walked around the partition and found the physician in a cubicle, Googling the child's symptoms, clicking through ordinary search results. There was no deep, accurate source. There was a query.

Singhal treats that as two competing forces. Stakes in medicine are high, so any informational product has to beat the information a person would have gotten otherwise. At the same time, "indecision is a decision as well." Roughly ten percent of internet searches are for health information, and some of those searchers are physicians. Shepherding the technology safely into that world, he says, is a responsibility, not an optional extra. What their own work still lacks is grounded evaluation inside a real workflow, showing short-term safety *and* longer-term patient benefit, before any real-world use.

Guo later puts the same point more sharply. A patient with a complex, uncommon condition does not know the error matrix of their doctor. The status quo is not a gold standard. The safety case should be reasonable, not unlimited — unlimited bars, she argues, are exactly what buried other mission-critical AI applications. Gil adds that healthcare debates have almost stopped doing cost-benefit, even as people such as Jensen Huang keep asking the field to count the benefit side, including global health equity.

Guo notices a pattern: biomedical AI companies keep building another protein-folding or small-molecule shop. Why, when health information looks like the bigger driver of care? Singhal's answer is commercial. Drug discovery already has a playbook a new company can follow for near-term revenue. Existing pipelines predict absorption or toxicity from small-molecule chemistry; drop in a stronger model and it will probably beat the last one. That is a relatively safe bet, which is why the category is crowded.

Health information is, in his phrase, going for the jugular. The organizations that already control that access tend to think in risks, not in the counterfactual of better information. That caution has slowed both large companies and startups. The group that gets it right, he argues, also gets a seat when policy and regulation are written. Gil does the arithmetic: healthcare is about twenty percent of U.S. GDP, pharmaceuticals about twenty percent of that, and drug development only a fraction of pharma. Models like Med-PaLM are aimed at something like sixteen percent of GDP, before you even count clinical decisions about who receives a drug.

Asked where the technology shows up first, Singhal does not start with autonomous diagnosis. Documentation and billing workflows are already being tried, with teams stuffing models such as GPT-4 into those lower-stakes jobs. Players like Epic, he thinks, can partner with existing models and actually deliver value, and general-domain models may be good enough there. Higher-stakes clinical work is where specialized models earn their keep, and in the short term that looks less like a replacement physician than like an assistant.

His concrete picture is radiology. An agent sits with the radiologist, helps interpret a scan, and pulls in the patient's record, prior scans, and different angles so the report is more accurate. That workflow is feasible today, it uses extra context and potential multimodality, and it sits in a regulatory sweet spot. It is assistance with a paper trail, not an unsupervised opinion handed to a patient.

Privacy is still improvisation. Some people paste patient information into GPT-4; some redact first. Singhal wants the stack more privacy-forward, starting with the low-hanging fruit: models that automatically strip sensitive details before anything moves down a pipeline. Organizations can also push for HIPAA compliance internally. Further out he is less sure — software of unknown provenance may be hard to scale into high-stakes settings — but he expects a moment when improving outcomes makes a serious conversation about regulation and privacy unavoidable.

Gil is harsher on HIPAA itself. The law is nearly thirty years old, well-intentioned, and in his view it sometimes backfires on patients. He describes an MIT researcher with glioblastoma who joined a small trial and then could not obtain his own data, so he could not try to find drugs against his own tumor. A pre-internet statute is governing a world of liquid data and patients who want to drive their own diagnoses.

Singhal spent his early career on privacy-preserving machine learning and federated learning, and the fit with healthcare looks obvious. His hesitation is practical. For the next few years, the most capable centralized models will dominate, and those models are already doing surprisingly well *without* patient records. Med-PaLM and Med-PaLM 2, he says, were trained with no patient health information: they take PaLM and PaLM 2 and align how the model behaves. Federated learning on health data has so much compliance friction that he doubts the biggest advances in the next few years will come from it. Intermediate steps — trusted execution environments that keep engineers away from the data — are more likely.

Guo asks him to unpack an idea he has been carrying: a medical assistant as a laboratory for safety and alignment research. Singhal did not come in as a clinician. He came in looking for a place to nudge AI toward being more beneficial. Google, DeepMind, Anthropic, and OpenAI are all using general chat assistants as the testbed for alignment. He has become convinced that medical question answering may be a better scenario for technical safety, misalignment, and hallucination.

The reasons stack. Incentives: anyone who wants to ship a product here has to work these problems harder than a ChatGPT team does. Stakes: nobody debates whether they matter. The sharp technical point is scalable oversight. Once AI capabilities reach human level, human feedback becomes poorly informed, and methods such as RLHF start to fail. Medicine already lives there. You need experts to grade answers. As Med-PaLM 2 approaches physician-level performance, it is hard to tell models apart, and hard to tell models from physicians. Uninformed oversight makes "aligning to human values" a slogan with no evaluator.

Gil reaches for Go and chess: once machines were better, humans learned new ideas from them. Medical advice that people cannot interpret is not like that — you cannot study a move you do not understand. Singhal does not claim to have the answer. The lines of work he names are self-critique guided by human feedback, debate (easier to judge a contest between two answers than the answer itself), and weaker AIs supervising stronger ones, which is already the logic of a reward model and of Anthropic's constitutional AI. Ask researchers across labs whether this is solved, he says, and most would say no.

A side effect, Gil notes, may be the data. A physician assistant whose suggestions can be joined to what actually happened to the patient is a closed-loop retrospective set that an ordinary EMR does not give you. Singhal agrees: that kind of feedback could be one of the most important training signals, actual data-driven medicine rather than the version that only sometimes happens.

Gil's cautionary tale is MYCIN, a 1970s Stanford expert system that outperformed the medical staff on predicting infectious disease and was never adopted. Medical technology, he says, sometimes has an anti-adoption curve for the things that would matter most. Singhal's answer is that catalyzing both the medical-AI and the broader AI communities was the point of the Brain Moonshot: the models already existed; the remaining limitations were what blocked uptake. Conversation has exploded since GPT-4. He sees wide optimism and a quieter fear, the same feeling programmers get watching GPT-4 — not necessarily that jobs vanish next quarter, but that Med-PaLM 1, GPT-4, and Med-PaLM 2 arrived about three months apart. The useful response is to pick the important problems instead of riding the hype.

Five years out, he splits the hope into two buckets. The first is raising the standard of care: broader access to health information, physician assistance, clinical decision support, quality checks on reports. He gives a small example: a radiologist dictates "no pleural effusion seen," the transcript writes the opposite, and a model flags the mismatch. Augmenting telemedicine, he thinks, is achievable in that window. The second bucket is scientific workflows — finding novel biomarkers across modalities, using language models as research assistants and for literature search. That may run longer than five years, and it is where he thinks AI's real upside lives: helping scientists see what they are missing.

For real-world uptake he names three capabilities that still have to be done right. Multimodality, which Google had just previewed at I/O. Grounding in authoritative sources, including Toolformer-style tool use to query medical information and attribute it — Google's own health-search progress, he says, exists largely because it can point to the Mayo Clinic. And human feedback: the field is still arguing pairwise comparison versus rewrites, and how to use that feedback under scalable oversight is a significant limitation of Med-PaLM as it stands. Those look like foundational AI questions that simply become more acute when the topic is someone's health.