AI already beats doctors at empathy. Catastrophe is still not the default

Sarah Hastings Woodhouse hosts Consistently Candid, an AI-safety and philosophy podcast aimed at listeners who are new to the technical side and already quite worried. She invites Nathan Labenz, host of The Cognitive Revolution and a self-described AI scout, and asks the question her audience actually wants answered: why aren't you more pessimistic? Labenz's background is entrepreneurship, not academic AI. He had been reading Eliezer Yudkowsky's speculations since the Overcoming Bias days of 2006 and 2007, filing them as fascinating and worth watching, until the last five years made the old graphs look less like science fiction.

He was running Waymark, then a do-it-yourself video platform for small businesses, when generative models arrived. Users still froze at the blank page. They did not know what to say. GPT-2 was not quite enough. GPT-3 could write coherent copy that at least got people started. He yanked the company, in his words, toward a done-for-you-by-AI product: catch this wave or die trying. The bet served the business. It also pulled him deeper into the technology than into the CEO job, which a teammate took over.

What changed his outlook was not only that language, vision, and speech were all improving, but that they were improving for the same underlying reason. If one architecture could handle all of it, and improve this fast, things were going to get strange. Eighteen months before this conversation he joined the GPT-4 red team, spent two months testing the model, and came back convinced that the public had not yet seen the real jump. He now tries to take stock of the whole landscape, and he admits the goal is failing on purpose. AI is converging with all of society. A target of no major blind spots in AI, he says, is now the same as no major blind spots in society, which nobody can hit.

Asked what would surprise someone who does not follow the field, Labenz's go-to example is medicine. Evaluation is messy at the edges, and anyone repeating the claim should read how the studies are done. Still, he thinks it is pretty clear: for high-status, high-wage cognitive labor that is routine enough that we have many examples and know what good looks like, today's best systems are closing in on the average professional, and in some cases marginally beating them.

On tasks as fundamental as differential diagnosis, AI outperforms human doctors as judged by other human doctors. The finding is not spotless. Multimodal cases get muddier, and some experiments ran through a chat interface, so the model did not see everything a physician could. It is still, he says, a robust and notable result. He would not go through a serious medical situation without consulting a suite of AI doctors, and he thinks access to something like that could become part of a right to healthcare before long, at least funded if not directly provided.

The part that startles people more is bedside manner. In the United States, appointments are short, the clinician is often staring at the record, and many patients leave feeling unheard. The AIs, Labenz says, are very good at expressing empathy. They outscore human doctors on that seemingly fundamental human thing. The other side of the ledger is still real. Long context has exploded, but it is not yet shown that a model can be your primary-care doctor across a long series of episodes. The same pattern holds in software: the best models beat the average programmer on a bite-sized task, and whole jobs are still a bit beyond them. The frontier is expanding from task to job.

Hastings Woodhouse presses on a puzzle. Since GPT-4, nobody would say the original model has been definitively superseded, so presumably this medical edge has been sitting there for two years. Why has society not turned over? Labenz challenges the premise. On the LMSYS leaderboard of blind head-to-head votes, the first public GPT-4 scores 1186 and the current one 1287, a gap of about 100 Elo points. That is roughly a 70-30 preference rate, which in this field is a significant split. The original context window was 8,000 tokens. The current one is 128,000. Google already has a million, with two million coming.

Multimodality moved just as fast. On a plane he paused while a cabin announcement played, and the voice assistant said it sounded like he was on a plane and asked if he was going anywhere exciting. Raw scaling is one way to unlock utility. Adding modalities, stretching context, shipping a decent mobile app, and wiring the model to search or a database are others, and he is for all of that practical accessibility. The adoption curve, he thinks, may be the fastest in history and still feel slow, because the capability is not yet legible.

His parents had been over the night before. His mother wanted hotels in a specific place. Gemini did not help on the first try, and she stopped. ChatGPT can be reluctant to go online. Perplexity, You.com research mode, or Exa might have been better, but she does not know those names. His father wanted to pull NASDAQ data and test a market idea. Labenz handed that to Devin, the autonomous coding agent. It bogged down fetching thousands of tickers until he told it to parallelize the API calls, test on a slice, then run the rest. It finished, even adding bar charts he would not have drawn, in about two hours, using tools he did not know. The problem was that his dad had never heard of Devin. Google has announced travel planning, promised in time for Labor Day. The rough edges are still being sanded. People like Labenz stay up until 2 a.m. to try Devin. Most people need the thing delivered in a familiar package.

Hastings Woodhouse describes the public conversation as an epistemic nightmare. A new model drops. One person calls it a game changer. The next says the ceiling has arrived. Someone tweets that GPT-5 will settle the regulatory fight by making overreaction obvious. Her prediction is that nobody will agree how good GPT-5 is, and a reasonably large contingent still argues the systems do not exhibit real intelligence at all. Labenz's practical advice, leaving Twitter aside, is to get hands-on.

Language models are weird. They have huge surface area and strange failure modes. He dislikes the dismissive phrase that they are merely probabilistic, and offers a better one: they are noisy, fuzzy reasoners. Much of the time they are super effective. The day before this taping he used both ChatGPT and Claude on mechanistic interpretability papers published in the last few days, after the models' training cutoffs. Claude, for him, is best at why the authors did this and why it matters. GPT-4 is thorough on point-by-point summaries and definitions. Together they are more useful than either alone. That is sophisticated analysis of genuinely new work. The same family of systems, asked about a man with a boat and a sheep that can fit two things, falls into riddle mode and invents a wolf. Two polar behaviors live in one object.

Start, he says, in areas you know well, or with your own documents, so you will notice immediately if it goes off base. It will answer anything. If you cannot evaluate the answer, you are in a tough spot. On public discourse he wants a synthesis, not a team. Gary Marcus, in his view, does the safety community a disservice when he denies capabilities that plainly exist, and he is right that the weird failures will get costly as we delegate more. GPT-5 will almost surely be smarter, beat experts on many high-value small tasks, and handle midsize projects more reliably, with less scaffolding than Devin needs today. Labenz would also be very surprised if it became so robust that strange failures vanished. Both can be true. The binary of LLMs-good or LLMs-bad is the thing to fight.

Labenz describes himself as an adoption accelerationist and a hyperscaling pauser, two labels that look like a fight. The first half means squeezing current capability into specific domains. He has argued for universal basic intelligence: everybody should have some access. He applauds OpenAI for making GPT-4o free, the best broadly available model on several leaderboards, with usage limits. He suspects a still more powerful paid model is coming, because the company wants to make money. The free tier still fits the idea.

The case he likes most is OpenAI's work with Harvey, the legal AI startup. Continued training on a GPT-4 base, not another 10x of compute, moved blind preference from that 70-30 split well into the 90s. The extra cost was maybe 10 to 20 percent more compute plus a lot of elbow grease, including lawyers at hundreds of dollars an hour for evaluation. They found a clever proxy: overlap between the sources the AI cites and the sources a human lawyer cites proved a good leading indicator of quality. Google's Med-Gemini team, the same group behind the Med-PaLM diagnosis numbers, is doing the analogous thing in medicine, on the cusp of a frontline AI doctor, again without orders of magnitude more compute.

On pausing he is more modest about policy and still fairly literal. One more turn of the scaling knob is probably going to be fine. Anthropic's Claude 3 and its responsible scaling policy make him think the next generation will be more useful and still not spitting out novel bioweapons or taking over cloud infrastructure. He would be willing, later, to chain himself to a fence. Not yet. Mechanistic interpretability has gone better than almost anyone expected, including, he suspects, the people doing it. If we can see which features are in use, maybe scaling can continue safely. A little time for understanding and control to catch up is the reason he leans pause-after-one-more, not stop-now.

Hastings Woodhouse had taken the pause emoji out of her Twitter name after a scare. GPT-2 can now be reproduced for about $20, five years after it was state of the art. Epoch AI's algorithmic-efficiency charts, extrapolated with her basic math, suggested GPT-4 on a laptop was not that far off. If alignment is hard, a pause might just wait until dangerous capability is cheap. Maybe the only chance is to cross now, while power is still concentrated. Labenz's reply is that you would still have time. Llama 3 trained on 15 trillion tokens, more than a retail hard drive can hold. Efficiency gains are won by the best-resourced teams combing the literature, not handed to everyone for free. Open source catching Google and OpenAI internally looks like a couple of years, before you even collect the data. A basement Eureka cannot be ruled out. He files that with asteroids: possible, tiny, not a basis for policy. If the big labs were restrained, you would have time.

With a couple of friends Labenz launched a volunteer project, red teaming in public, aimed at the application layer rather than the frontier labs. He will grant that the labs are a mad science project, and he dislikes Sam Altman's line about asking the AGI how to make money. He still contrasts that with what is happening downstream: people slapping something together, adding a credit-card form to a weekend hackathon, and shipping with little thought for abuse.

The vivid cases are deepfakes and calling agents. You give the system a task and a phone number, and it calls. Many of those products also clone voices. On most of the calling-agent tools he tried, a Trump, Biden, or Taylor Swift voice cloned on the first pass about nine times out of ten. If a name was blocked, he renamed Trump and Biden Formula 45 and Formula 46, their numbers in the presidential sequence, and they went through. He has had a cloned Swift make fake fundraising calls for a nonexistent food bank. He has put the names of crimes in the prompt, telling the agent it is part of a criminal syndicate and its job is a ransom or blackmail call. With few exceptions, the applications did not care.

This is where he thinks the "AI is just a tool" line fails. The products are marketed as agents. They act autonomously on people who did not opt in and often do not know they are speaking to a model. A user checking a disclaimer box does not, in his view, wash the developer of responsibility. The team tests, documents, and writes privately first: we are on your side, we are builders too, fix this before it blows up. Results are mixed. Some developers blacklist the word ransom and miss the next phrasing. Labenz wants a narrower standard than Twitter-style speech police. Using even a mid-tier model, they built a classifier that flags only egregiously criminal use and lets offensive jokes through. The project takes no money, pays for test subscriptions out of pocket, and invites people via the Red Team in Public Twitter account and a Discord.

Hastings Woodhouse asks for the list. Labenz's first caveat is that people should be worried. His standard p(doom) answer is 10 to 90 percent, a way to express radical uncertainty, with more of his weight on the low end. He will not argue a 90-percenter down if the remaining 10 percent still gets them out of bed. He is arguing against two extremes: nothing to worry about, and nothing that can be done. The first of eight reasons is that today's systems already understand human values with a sophistication the early writing did not predict. The paperclip maximizer assumed we might never communicate what we care about. A chat with Claude 3 about an ethical dilemma, he says, will make it obvious that the model is probably more ethical than the average person. He has tried, without success, to use helpfulness and honesty against harmlessness, asking for a denial-of-service script against a Myanmar junta server and pressing the model on consistency. Claude can admit its position may not be fully coherent and still refuse.

Second, interpretability is going extremely well. Whether the models care about our values is still unclear. Seeing the concepts in play might tell us. On the current trajectory, a year or two could bring a real picture of which features are active. Anthropic's Golden Gate Claude is the demonstration: find a direction in activation space, turn it up, and the model can think about almost nothing else. That suggests a control trick, turning some traits down and others up, with more clarity about why. Third, a destabilizing AGI from a basement looks unlikely. Frontier progress still depends on scale. Algorithmic advances are about reaching the same point with marginally less compute. Nobody is claiming a retail-compute breakout. He files that scenario with a random asteroid: possible, not worth losing sleep, not much you could do.

Fourth, frontier-lab leadership is, in the grand scheme, pretty good relative to the alternatives, even as OpenAI's departures and non-disparagement clauses raise his concern. Wipe away the details, ask what kind of people would end up running a process that needs huge resources, and you would expect them to be less thoughtful than they are. His contrarian take on the Eliezer Yudkowsky, MIRI, rationality, EA nexus is that they were basically right, and that they have already won the first phase: every major lab leader has engaged deeply with the risk literature. That is a counterfactual win. Fifth, society is not sleeping. The Overton window is wide open. Time magazine put Yudkowsky on the cover talking about air strikes on data centers. AI-safety respectability politics, the old anxiety about seeming crazy, is dead. Sixth, governments have done better than he expected. The Biden executive order's 10^26 FLOP threshold is a light-touch start that, contrary to some claims, really does hit only a few well-resourced companies. A lesser-known clause requires reporting models trained on biological sequence data at 10^23 FLOP, which he thinks is smart, because models may speak DNA long before they beat us at the language tasks we invented. He does not want AI to end up like civilian nuclear power: the good uses regulated away, the weapons still here.

Seventh is the sweet spot his GPT-4 red-team months produced. The model was powerful enough to be really useful and not powerful enough to get out of control. That interval did not have to exist. We have just entered it, and he thinks we probably have one more turn of the scaling screw before leaving it. If the public understood that current systems already beat human doctors, a pause would be easier to hear. Eighth, and the one he is least confident about, is China. He has never been, and cannot read a character. He hates the argument that the West must race because China will never stop. We did that with nuclear weapons, and the missiles are still pointed. The Chinese government at least seems to treat the technology as a threat to social cohesion and the information landscape, not as a reason to sprint without sense. If both sides chart a responsible course, an all-out arms race is not a given. This is the first item he would cross off. Seven remaining, he notes, is not that bad.

Hastings Woodhouse lands at 50 percent, not because she has a model, but because experts disagree too much for a non-technical person to pick a side. Labenz says that is savvy, and very good company. Anthropic's core views on AI safety come down to radical uncertainty, and similar statements have come from other lab leaders. He reaches for the bell-curve meme: the least capable analyst and the most capable one share the same idea, and the person in the middle is the odd one out. The people with the most privileged information are also saying they do not know. Between that and the public disagreement, 50 percent is a sensible place to sit.

A small coda follows the most pessimistic researcher, Roman Yampolskiy, putting p(doom) at 99 percent with a string of nines, and Yudkowsky calling the number ludicrously high, after which the replies argued about how many nines belonged after the decimal. Anything over 90 percent, they agree, is overconfident. Watch the time-scale sleight of hand, Labenz adds. Define the question as the next ten million years and of course it looks grim. Most species do not last that long. Robin Hanson's point is that our ancestors would be horrified by our lives, and we think their hypothetical rejection is small-minded. Allow that kind of change and the many nines cannot stand.

He closes on the claim he came to make for her audience. This is, in his view, the most important issue of the time. We are walking a tightrope. Everything is on the table, from a radically utopian future to dystopia or extinction. He is not offering a hopeful situation that licenses disengagement. The opposite: there is enough of a chance that the right response to anxiety is to get involved. Strategies should differ. The most brittle ecosystem is a monoculture, and the most brittle AI-safety movement would be one where everyone does the same thing. Chain yourself to a fence, or do an analytical podcast. An ecology is healthier than a script.