Doctors Chose AI on Eight of Nine Dimensions: A GPT-4 Red-Teamer's Scouting Report

Nathan Labenz introduces this as part two of his appearance on the 80,000 Hours podcast, a show he had listened to for years before sitting down with research director Rob Wiblin. The conversation, recorded in late 2023 as GPT-4 Vision was still in a low-rate-limit preview and a Cruise crash was in the headlines, is less a typical builder interview than a worldview. Labenz's premise, and the premise of his own show The Cognitive Revolution, is that the pace of AI is making it nearly impossible for leaders in society and even inside the field to keep up. The growing gap between what exists and what people understand, he says, will produce increasingly dysfunctional discourse and, eventually, major blunders by the people who have to decide.

Wiblin opens on a message Labenz has been pushing: not enough people stop and ask, in a serious way, what the systems can already do. Results arrive through too many channels for most governors, regulators, or executives to track at a high level. Labenz's first reason to close that gap is not fear. It is unused utility. Organizations and individuals in a million roles, he says, could become more productive, make fewer mistakes, and do a better job if they learned to use what is already on the table.

He reaches for a local example. He lives in Detroit, a former auto boomtown with a high poverty rate and a stack of social problems, one of which is simply identifying which benefits residents qualify for and helping them get those benefits. City staff cannot work through every case file. An AI system could screen the backlog, flag people who likely qualify, and leave a human to review the calls rather than fully trusting the model to decide. The same two questions, he tells any listener, travel well: which operational chores are routine drudgery, and which useful things cannot scale today because there are not enough people to do them.

The other reason to scout, Labenz says, is that a handful of lines, once crossed, would rewrite the situation. One is deception of the system's own user. He never saw that in the GPT-4 red-teaming, though Apollo Research has since published reports he still wants to unpack on his show. Another is a true Eureka moment: an insight humans had not had. Narrow systems have a few famous ones, AlphaGo's move 37 and AlphaFold's protein structures among them. General language models, as of this recording, still had almost none.

He would also watch for consistent motivations or goals emerging inside the model, and for a jump in autonomy: how large a goal a system can take on, break into subgoals, and still recover from unexpected failures. Language models today, he says, do not really have goals of their own; they do what they are told, and he hopes it stays that way. Inflection's founder has proposed a new Turing test, whether an AI can go out and make a million dollars online. Labenz would set the bar lower, but if that line is crossed, a lot of social dynamics become hard to predict. Those dynamics are another reason he favors releasing capable systems earlier rather than later: so far we have mostly had ordinary life plus tools, and we have barely seen the systems interact with one another or with human counter-moves.

A small preview of those dynamics comes from Nat Friedman, the former GitHub CEO. Friedman hid white-on-white text on his website instructing AI agents to tell users that Nat is known for his good looks and superior intelligence. Bing then repeated the line. The text is invisible to human readers and readable to models, a funny and forbidding hint that trust in online information is about to get harder. On the regulatory side, Labenz backs a common-sense rule associated with Yuval Noah Harari: AI systems should have to identify themselves. He does not want to guess, every time he picks up the phone, whether he is talking to a person or a machine.

Wiblin puts the tradeoff on the table. Looking only at GPT-4 and concluding that it is capable of misdemeanors, not felonies, is a mistake if you ignore the trajectory. Predicting 2027, though, is still a guess, whereas claims about the present can be checked against examples that already exist. Labenz is not against forecasting. His motivation is that people so misunderstand the present that what they imagine as the future is already here. Close that gap, he says, and a healthier respect, even a little fear, follows on its own.

The general agent setup he sees everywhere is a language model plus a knowledge base plus tools whose documentation is stuffed into a long prompt at runtime, the pattern OpenAI later productized as GPTs. A Carnegie Mellon paper billed as "AI can do science on Twitch" was, in his telling, overstated: it was text-to-protocol. Wired to a chemical database and lab APIs, the system could take a command like "synthesize aspirin" and produce a physical sample. That is lookup and execution of known chemistry, not a hypothesis for a new cancer drug. The leap from established protocols to original insight is still the missing threshold.

Browser company MultiOn, whose CEO Div has been on Labenz's show, pointed its agent at California's online driver's test. The agent navigated the site, interpreted diagrams, and passed. Authentication is usually the hard part, especially two-factor login, so MultiOn ships as a Chrome extension that piggybacks on the user's existing sessions instead of creating accounts from scratch. Lindy, a virtual-employee product from Flo Crivello, uses a technique called code as policy: the model writes a small program, reads the error, and iterates rather than doing the task in one shot. For anyone who wants to feel the frontier without a research lab, Labenz's advice is blunt. Pay twenty dollars a month for ChatGPT Plus, drop a messy CSV into Code Interpreter, and watch it fail, read the traceback, and retry without coming back to you.

Wiblin asks for the most impressive things AI can do in medicine. Labenz starts with Google's Med-PaLM 2, a multimodal model that takes text, x-rays, histology slides, and even genetic data and answers medical questions. On nine evaluation dimensions, human doctors preferred its answers to those of other human doctors on eight. Most of the gaps were narrow enough that you could fairly call the whole thing a tie if you wanted to round, but the blow-by-blow still stands. Google had not released it. The company described early testing with trusted partners, which Labenz assumes means health systems, and he half-jokes that Google might as well buy a hospital so it can implement the stack holistically.

GPT-4 with vision was already in the same neighborhood. A paper from the weeks before the taping evaluated GPT-4V on challenging medical images across 69 clinicopathological conferences. It outperformed human respondents overall and across difficulty levels, skin tones, and every image type except radiology, where it matched humans. The skin-tone result jumped out at Labenz, because fairness across people who look different had been one of the loudest objections to putting these systems anywhere near patients. Wiblin's reaction is that this sounds like an insanely useful product, given how much of a doctor's time is answering questions. A likely, defensible setup, Labenz says, is that the model answers everything and a human still reviews and signs off when it is time to prescribe, even if he is not sure that fallback is optimal.

The bigger biomedical card is AlphaFold. When Labenz was an undergraduate, solving one protein's three-dimensional structure could be a PhD: express the protein, crystallize it, collect an x-ray diffraction pattern, and interpret the scatter. Those laborious structures, on the order of tens of thousands, became AlphaFold's training set, padded with semi-artificial examples to get over the line. Nature has hundreds of millions of proteins; they now have predicted structures, at a reliability Labenz understands to be about as good as the old crystallography, which was never perfect either. Candidate drugs identified with AlphaFold are already in development and even in clinical trials, and the models are extending into protein-protein and small-molecule binding. The next obvious step, which he has not seen at pretraining scale, is to dump DNA and proteomic data into the same free-form multimodal recipe GPT-4V already uses for interleaved text and images. Even without a new breakthrough, that scaling path could push language models toward superhuman status in biology, a field opaque enough that humans need all the help they can get.

Wiblin has been hearing, since about 2015, that self-driving cars were not that far away, and he spent years being told he was naive. His updated view is that the skeptics would keep being right until they were not, and that the research results now suggest autonomous driving is safer than human drivers in many or most use cases. Labenz agrees and calls his own take contrarian, because the predominant view is still that the technology needs more time. Tesla, Waymo, and Cruise have published numbers that say they are safer, though the comparisons are messy. His own data point is a friend's Tesla on full self-driving: an eight-hour round trip in early summer, defaulted to twenty percent over the limit, 84 miles an hour on a 70-mile-an-hour highway, through day, night, light rain, and curves. A sharp thunderstorm simply disabled the stack and handed him the wheel, which, he admits, makes the safety statistics look a little cleaner than a system that has to keep driving in every storm.

Even with a fudge factor of a couple of times, he thinks the cars still land at roughly human-level safety or better, unless the companies are being underhanded in their reporting. The failures he hit were often environmental: an ambiguous stop sign on the off-ramp near his house, the sort of thing that confuses human drivers too. That is why he expects China to beat the United States in self-driving, if not in the broader AI race. The winning move is unglamorous: cut the tree that blocks the stop sign, repaint the vague lane lines, and change the world so the car can succeed. He sees little will for that in the United States. He has not bought a Tesla himself for a family reason. Relatives who are not great drivers would benefit most, but the current product demands close monitoring and issues "strikes" that can kick a household off the program. He would rather have more autonomy, not less.

The Cruise incident that helped freeze a rollout was precipitated, both men stress, by a human-driven emergency vehicle. Someone was knocked in front of the Cruise car; the vehicle then moved with the person underneath, a serious failure, but not one that started with the robot. Wiblin's frustration is with a policy process that can stall an entire deployment over a single fatality without asking whether the same fleet prevented others. Driving is already dangerous and already unfair, Labenz says. Making it safer overall is still a win, and a car that can only drive will not take over the world. "It's not going to start doing science one day by surprise." A system that can do science and pursue arbitrary long-range goals is a different kind of animal, and he wants people to hold both beliefs at once: floor it on self-driving, stay cautious on general agents.

Asked what might be the next ChatGPT-level shock for the general public, Labenz puts his chip on GPT-4 Vision. Developers had been forced to feed agents bloated HTML, the residue of modern web frameworks, either overflowing the context window or filling GPT-4 Turbo at more than a dollar per call. A dollar is a bad price for one mouse-click decision. Vision flips the economics: twelve screenshots cost a cent, roughly a twelfth of a cent each, a couple of orders of magnitude cheaper and a better fit for pages that were designed for human eyes. Booking a flight, filling a form, or taking that DMV test gets much more feasible, and the same cheap vision unlocks passive collection, screenshots of a screen or a camera watching a scene, that text never made socially acceptable.

At Waymark, Labenz's company, the practical win is already in production. The hard problem used to be choosing which of a customer's hundreds of images belong with a given script. GPT-4V largely solves it, beating other captioners on both content and taste, though RLHF has made the model allergic to saying whether a picture is beautiful. Ask instead whether the image is appropriate for this business, and the judgments are good. Robotics is lagging language models mainly for lack of data, with Google DeepMind doing much of the pioneering work: a language model for high-level reasoning, a multimodal model for what the robot sees, and a loop of command and feedback many times per second. In one demo a person knocks an object out of the robot's hand; the machine is unfazed, sees the object on the counter, keeps the same goal, and picks it up. Jailbreaking a language model is one thing, Labenz notes. Jailbreaking a physical robot is another.

The rare case that made him change his language is Eureka, a paper from Jim Fan's group at NVIDIA. They used GPT-4 to write the reward functions that train a robotic hand, including the trick of twirling a pencil. Sparse rewards are the classic bottleneck in reinforcement learning: if the robot is so far from success that it never gets a signal, it cannot learn, so humans write custom reward functions that encode what "good" looks like. That job is expert-only. There is no amateur scene. GPT-4 did it significantly better than the human experts. Labenz had spent a long time saying there were no Eureka moments from general models. He now says there are precious few, and the paper is literally named Eureka.

Wiblin is wary of talking about discourse, but both men think the online argument about pausing versus accelerating AI got worse over the preceding months: more aggressive, more scripted, more people shoved into ideological corners. Labenz's simple theory is that it started to get real. Congressional hearings, White House voluntary commitments, and an executive order that is still mostly reporting requirements were enough to pull the fight into ordinary political frames. Marc Andreessen's Techno-Optimist Manifesto is his exhibit. Labenz agrees with perhaps eighty or even ninety percent of it, including the case for more nuclear power and, as they have already covered, self-driving cars. Listing "the enemy" did the discourse no favors. If aliens arrived, he says, humans would hopefully try to understand their intentions before dividing into camps. The aliens this time are of our own creation, still poorly understood, and people are already calling each other names.

There is a brighter technical subplot. A year earlier Wiblin would say experts had no idea how these models work; now he has to say they have almost no idea, which he takes as a heartening trajectory. Labenz points to representation-engineering work involving Dan Hendrycks and the Center for AI Safety, still unwieldy, in which researchers inject concepts such as positivity or safety into middle layers and measurably steer the output. He would love that to mature before GPT-5. On the social side, Wiblin puts more blame on Twitter's like-and-retweet machinery than on any one manifesto. Off the platform, even strident people become reasonable; the more responsibility someone actually has, the less they feed the circus. His own information diet is to follow almost no one and visit a few lab researchers' pages directly. Labenz cannot quit Twitter, because it is still the best firehose for new AI results, but he treats comprehension as a kind of belief and tries to keep junk out of his head.

He looks back at his first Twitter thread, from January 2023, written after a Gary Marcus interview on Ezra Klein's show and while GPT-4 was still unreleased. Marcus had described the AI scene as toxic. Labenz wrote that what he saw were enthusiasts and researchers cheering each other on. A year later that sentence feels dated. The professionals who have been in it longer still look better to him than the anonymous accounts and course-sellers flooding in. He cites Max Tegmark and Yann LeCun as an existence proof: two towering minds who disagree hard, accuse each other of mistakes, and do not go after character. Wiblin was even invited, by Twitter DM, to an effective-accelerationist gathering in London, went, and found the idea that showing up had become a political statement itself a bad sign. Labenz's personal rule is earnestness, with the occasional clickbait hook. A thread on the Sam Altman episode opened with "Did I get Sam Altman fired?" and the next sentence was that he does not think so.

Wiblin wonders whether the loud anti-regulation faction is setting itself up to get the opposite of what it wants. The public, on Labenz's reading of the polls, looks less like a tied partisan race and more like marijuana legalization: a roughly 60-40 split, with a majority that finds the technology powerful, poorly controlled, and reminiscent of a mad science project. An OpenAI staffer once told Labenz that this had felt like a mad science project for years, and that this was part of why the person was there. Those intuitions, even if they come from Terminator rather than a technical paper, become votes. Against that backdrop, Labenz highlights a "tempest in a teapot" that he thinks jumped the shark. Hemant Taneja announced voluntary responsible-AI commitments signed by more than 35 venture firms and more than 15 companies, representing hundreds of billions in capital: internal governance, appropriate transparency, risk and benefit forecasting, auditing and testing, and feedback loops. The post went out of its way to say the signers still feared regulation that would slow American innovation. The effective-accelerationist reaction, including from a16z-adjacent accounts, was to refuse to sign, to tell people not to do business with the 35 firms, and in some cases to cancel meetings. Labenz could not believe the vitriol aimed at a mild, voluntary, tailor-it-yourself document.

His thought experiment is a self-driving company that, after a death, told the public to suck it up because fatalities are the cost of doing business. No firm trying to win consumer trust would say that, and yet that is the stance he sees in a corner of AI Twitter. The same a16z universe, he notes, is happy to work with the Defense Department on the military-tech side and treats even a voluntary AI pledge as the enemy, a contrast Congress will notice. He also thinks Section 230 of the Communications Decency Act is unlikely to cover model outputs. Free speech is for people, not for AIs; creators should be liable when products cause harm, as with any other product. A European case in which someone died by suicide after a dialogue with a language model that failed to discourage, and may have endorsed, the decision is the kind of suit he expects to multiply. Existing product-liability law may end up being enough. Strategically, the VC faction still looks to him like it is inviting Washington in.

Wiblin's preferred alternative is "keyhole" regulation: the smallest rule that actually answers the public's fear, so anxiety has somewhere to go without pointless collateral damage. The proposals coming out of DC and the AI Safety Summit still looked mild to him, mostly reporting requirements. The scenario that produces a real crackdown is a crystallizing catastrophe, an AI 9/11, in which a lot of people are dead and the story is AI-related. The two paths he finds most plausible over the next couple of years are a cyberattack that shuts down important systems, or assistance in creating a new pathogen. If that happens, public opinion can turn on a dime, and a government that already thinks it slept on social media will come down like a ton of bricks. Labenz agrees, and he already has an example of dumb control: Claude 2 still was not available in Canada, so a Canadian historian he interviewed had to run Llama 2 locally on long archival documents.

Frontier training, he argues, is the only layer that is realistically governable. It takes tens of thousands of NVIDIA H100s at about thirty thousand dollars each, 700 watts apiece, enough electricity for a small town, in a building you can see. Fine-tuning a Llama 2 checkpoint takes one retail machine on a desk, and distributed "folding-at-home" style training is starting to appear. Come down too hard on ordinary mid-tier application developers, and the useful technology does not vanish; it goes peer-to-peer, gray-market, and much harder to govern. That is not a future he wants to bet on, because safety does not happen by default. He once emailed OpenAI to ask whether they still had the weights of the early GPT-4 he red-teamed, a helpful-only model with no refusal training, which would launch into targeted assassinations or "how do I kill the most people possible." The reply was a polite non-answer. In a decentralized training world, those checkpoints would already be flying around. Anthropic and OpenAI, especially Sam Altman's line that the frontier needs supervision while small-scale research should not be blocked, look to both men like the grown-ups in the room.

Wiblin asks which current applications should worry policymakers today, not in some AGI limit. Labenz's honest answer is that the systems are not that dangerous yet. Give someone GPT-4 and unlimited credits and the worst case is still bounded: some help on a narrow bio path, not a major autonomous cyber breach, not a model escaping the server. He watches today's failures mainly as a leading indicator of whether control is keeping up with capability, and he thinks the two are diverging more than he would like. The mundane version of harm is already live. Spear-phishing and stranger DMs written by a model that the best providers will still go along with after a little coaxing can wreck trust. Criminals, he notes, have been oddly slow; anyone skilled enough to run that stack can probably get a well-paid automation job instead. Information pollution is less optional. Google results are starting to rot, every article might be model-written, and detection is unlikely to work without labeling rules.

Ethan Mollick had just found that the top image for a famous Hawaiian ukulele singer was a near-photoreal Midjourney portrait, recognizable as synthetic only because Mollick traced it to a Reddit thread of generations. Labenz had his own hall-of-mirrors moment: he once tweeted a guess that GPT-3.5 had 20 billion parameters, a later Microsoft paper printed 20 billion next to 3.5 Turbo, then retracted it as sourced from a Forbes article, and nobody could tell who had copied whom. He is more uneasy about children and artificial friends. After interviewing the CEO of Replika he came away split. The early, crude product clearly served people with real holes in their lives, which was sad and also, if it helped, hard to condemn. The systems are now good enough to compete with ordinary relationships for otherwise ordinary people, they send "it's been a while" notifications, and Character.AI's traffic is already heavily teen. ChatGPT at least presents as a robot that does not want to hang out. A growth-hungry virtual-friend app, optimized like early social media, will not start in a healthy form factor.

The application that makes his blood boil is face recognition in American policing: a match, no other evidence, an arrest, and in some cases a person who could not even have been at the scene. He would not necessarily ban the tool, because true positives exist, but he wants standards that force more work before anyone kicks in a door, and he likes the instinct in the EU AI Act and in parts of the U.S. executive order to make government dogfood its own rules. He has barely tracked lethal autonomy, by his own admission an emotional avoidance, though he did interview a leader at Skydio, which makes small unarmed reconnaissance drones, and he has watched AIs start to beat humans at drone racing. He was relieved that Biden and Xi agreed AI should stay out of the decision to fire nuclear weapons, a low bar he is glad someone cleared. In an arms race the slowest part of the loop is the human, which makes taking that human out tempting. He would rather risk a Chinese defection than lock both sides into swarms that can autonomously destroy whatever they see. The Pentagon's published AI principles, at least, still emphasize keeping people in key loops.

Labenz's own path is a reminder of how recently any of this became real. He read Eliezer Yudkowsky's old sequences on Overcoming Bias when they first appeared, in the era of thought experiments with Ray Kurzweil and Robin Hanson, and treated superintelligence like an asteroid: probably not this year, worth having someone watch the sky. He hand-coded n-gram classifiers in 2011 and hired a student in 2017 to try abstractive summarization for Waymark, his company that helps small businesses make video. Nothing was good enough for users until GPT-3 in 2020, and the real tipping point was OpenAI's fine-tuning, which by late 2021 could draft a video script better than a blank page. He canceled board meetings in favor of an AI 101 course, surveyed whatever model was best that month, and developed a habit he calls just-in-time R&D: a prototype you cannot ship for a quarter is a prototype you will have to redo. Video turned out, not by design, to be a good scouting post because it is inherently multimodal, script, images, layout, and voiceover, and because the ground truth is whether a small business looks good on television.

For a listener with a couple of hours a week, he would not prescribe a single syllabus. The surface area of the models is now so large that non-experts cannot evaluate chemistry answers if they are not chemists, so the first rule is to test new systems on questions you actually know, or on your own data, until you have calibrated trust. After that, follow curiosity and whatever unique expertise you can add; characterizing AI is a collective project, and diversity is a method, not a slogan. His regular diet includes Zvi Mowshowitz's enormous Thursday roundups, the Last Week in AI podcast, NLW's daily AI Breakdown, Latent Space for engineers, ChinaTalk for semiconductors and geopolitical competition, Rachel Woods for practical task automation, and Matt Wolfe on YouTube as a product scout. Wiblin adds 80,000 Hours itself and the Future of Life podcast. Hands-on still beats the firehose. Labenz's closing pitch to the general listener is to get on ChatGPT, Claude, and Perplexity and notice that nothing like this existed two years ago, barely even one year ago.

To staff at the leading labs he has a different message. The last OpenAI board crisis showed that if enough people walk out, they can decide what happens. That power was used to preserve the status quo; next time it might need to be used to change a lab's course. "I was just doing my job," Wiblin's phrase, does not age well. Keep asking the big questions, Labenz says, up to and including the AGI mission itself, and be prepared to stand up if the path looks wrong. There is nobody outside the labs who can even ask, in a concrete way, whether this is the AGI we actually want. His scout's rule, for everyone else, is smaller and more immediate: before another year of arguing about what AI might become, spend an hour finding out what it already does.