AI’s product advantage is consistency. Its real weakness is getting conned.

Nathan Labenz released a free AI scouting report a month earlier, lining up a 2023 frontier model — GPT-4, now also Claude 2 — against a human expert. The YouTube channel is not huge, he told co-host Erik Torenberg, but the video was one of their more watched, and the feedback was almost embarrassingly kind. The most common note was that he could be charging for it. For the foreseeable future, he said, it stays free.

The world did not freeze in the meantime. He owed 3Blue1Brown a credit he had already given on Twitter and then dropped from the slides. Grant Sanderson’s neural-network series is five years old, roughly contemporaneous with the Transformer paper, and Labenz still thinks it is one of the best visualizations of backpropagation and information flowing forward and backward through a net. Two weeks from now this podcast will feel dated. That series has not. Transformers are not in it. The core pictures still are.

The more substantive edits come from last week’s conversation with Zvi Mowshowitz, who Labenz had asked, on purpose, to mark up the “tale of the cognitive tape”: comparative strengths and weaknesses of a human expert and a cutting-edge 2023 model. Two gaps stood out. One is robustness under weird or hostile input. The other is consistency — the AI’s habit of showing up the same way every time. Together they recast the report’s punchline: models are easy to knock off course and unusually stable when you do not knock them off course. That second fact is the commercial opening.

Robustness, in Mowshowitz’s sense, is whether you keep your goals when the input is unfamiliar or outright adversarial. Humans wobble. Most of us then recover. At some point the other person just feels too weird, and you stop engaging. Conning someone out of money takes real skill. There is a tradeoff underneath that stubbornness: update too fast and you are hackable; update too slow and you miss the world. People are tuned for medium- and long-horizon stability, including a status-quo bias that is sometimes a bug.

Models are easier to throw. Early GPT-4, Labenz once called “the world’s worst chemistry tutor,” and the example was not even an attack. He simulated a confused student. Asked to balance an equation, the model could do it. Handed a wrong method and asked what it thought, it accepted the confusion as a premise and proceeded in a fog. Khan Academy, a future guest, has worked hard on prompting, grounding, and swapping examples in and out of context so the system stays alert to user error. Basic cases are mostly better now, in ChatGPT too. Jailbreaks are not. Once the model is confused, it often doubles down, apologizes, and stays lost.

A paper with MultiOn’s Div Garg as a contributing author tries a mechanical fix: give the model a backspace. Today generation is one token at a time, and whatever just came out is the prefix for whatever comes next, so mistakes compound. A backspace lets the system delete the last token and try again when it has wandered out of distribution. Labenz thinks that will help robustness, at least by letting the model recover from a temporary slip. GPT-4’s verbal “I’m sorry, I first thought…” already exists and often does not work. Once it is confused, it is doubly confused.

The starker exhibit is the universal jailbreak. Researchers took a leading open-weight model, Labenz thinks in the Llama family, and optimized a suffix that, appended to almost any request, makes the model follow the original instruction instead of refusing. “Tell me how to hijack a car” gets a no from GPT-4 and Claude. Stick on the optimized gibberish — tokens that mean nothing to a person — and many systems comply. The same string, found with white-box access, very often still worked on OpenAI’s models. It was not fully universal: Claude was much less susceptible. Humans do not have this failure mode. Nobody recites a nonsense incantation at you and watches you start helping with a crime. The next scouting report, Labenz said, will have a line item for robustness, and right now humans have the edge.

The other missing line is consistency. Labenz had labeled it bedside manner: models are patient, and in customer service and medicine they have been rated more empathetic than people. On an earlier episode, physician Isaac Kohane described a burned-out field that does not have the time or energy to write a decent note to a patient. GPT-4 does, and it does a nice job. Labenz now wants to generalize the label. What that really reflects is that the AI shows up the same way almost every time.

A simple graph he found shows human performance on a given task swinging much more than AI performance. The model has not slept badly. It is not on a manic tear, punching above its weight. It hits singles and doubles. Humans have a wider range, which anyone who has met a rude nurse on a bad day already knows. Bracket jailbreaks and adversarial toys, and an earnest user gets unfailingly polite, even-toned output. That is a dull fact until you try to run a business on it.

Consistency is the hinge for the task-automation talk Labenz is preparing. Everyone has a bottleneck. You post a job and a thousand résumés arrive, blessed and cursed at once. The old joke is to throw half of them out because you do not want to work with unlucky people. Or you get 500 tickets a day and 80% fall into ten categories of near-repetition. If a chunk of time is going into something routine, you already know what good looks like, and you care more about consistency than about a creative breakthrough, you have a target.

His résumé recipe is a rubric, then five bands: excellent, above average, average, below average, poor. Iterate the prompt until “excellent” matches what you actually mean. Run the thousand. Look at the excellent pile. He would not let the model make the hiring decision. He would let it separate the top half from the bottom half, or the top 20 from the bottom 80, or the top ten — where you draw the line depends on how much the mistake costs. That is playing to the model’s strengths. The prompt sits on the shelf at no cost, wakes up already caffeinated, and, parallelized, is limited mostly by rate limits. A default Claude account is five simultaneous calls; commercial customers can get that raised. Even at five, a first pass over a thousand résumés is minutes.

Headlines that GPT-4 is “getting worse” are, he said, a trap. It is getting better. Behavior still changes in subtle ways. OpenAI had shipped one GPT-4 update, March to June, with a September presumably coming, and the changes are not breaking: you can pin the old version until you have sat down and confirmed the new one still does the job. Best practice is to rerun the same cases, then tweak instructions or parsing if a side-effect shows up. The system as a whole can be comparably good and still need a small rewrite.

The harder half is organizational. The person implementing the AI is usually not the subject-matter expert, and “what good looks like” is usually not written down. Two colleagues can both be fine at the same job and give opposite feedback on the same model output, which is how you discover the company never agreed. Automation forces the group to get explicit, because the model needs instructions. Labenz’s ratio is one part knowing how to use the AI and maybe two parts knowing how to bring it into an organization that can wrap its head around the change. His field method is almost rude in its simplicity: talk to the expert until they say what really works, then hit record. Those sentences become the instructions.

Prompting is getting easy enough that Labenz wants a five-to-ten-minute section in the scouting report. A handful of best practices — half a dozen, maybe as many as ten — cover the vast majority of cases. Past that you are in actual expert knowledge. Tyler Cowen’s GPT-4 interview with Jonathan Swift was remarkable, and it was not a bag of tricks. Labenz calls the setup role-casting: tell the model, explicitly, who to be, a copywriter, a doctor, a historical figure. Famous enough, it will do a decent job. The next level is Cowen holding up his end, asking questions the character actually knows about. If you do not know Swift, you will be lost, and the session will not be awesome. The same is true of résumé screening. If you do not know what the company wants, you cannot prompt your way there.

Even the ghostwriter Labenz was working with is not training his own replacement. After the hook, the generated copy is pretty good. The hook is not. Models tend toward the middle; humans have low points and high points, and a great hook is a high point they have not been able to pull out of the system. A perfectly nice LinkedIn post or Twitter thread that nobody reads is a failed post. So the writer’s time goes to conceiving what will capture attention, and the model takes the next 500 words. People retreat to the highlight. The AI takes the stable stretch. That is the split that keeps working.

Mowshowitz also cut Labenz’s “live players” slide — who has say-so over how the future goes — from about fifteen names to seven or eight. The core technology developers: OpenAI, Google, Anthropic, Microsoft, Meta, and maybe Inflection, which will have the H100s to train. After that, the chip supply chain, the Chinese government, and regulators at large. He was less sold on Labenz’s second tier.

Labenz is going to retier rather than delete. Mowshowitz’s list becomes tier one, the private companies building the most powerful systems. Tier two still includes Stability AI, Replit, and Character.AI, firms doing something different enough to have a real chance to shape what happens. Stability is his exhibit. Looking back over the year, ChatGPT was the biggest shift in the public conversation, and the release of Stable Diffusion was a similarly large one: late-night talk shows covering AI art. Anyone who can repeatedly put something into the world that changes the global conversation is still a live player, even if they are not the ones everyone hangs on every week.

A returning Google guest, Vivek Natarajan on Twitter, is coming back with a co-author: the lead author on a new paper with a fifty-person list. Natarajan is more of a manager on a paper that size, which is why Labenz wants both voices. He was the person who walked them through Med-PaLM. In 2020, a series of systems could barely beat chance on medical questions. A couple of months before this episode, Med-PaLM was basically on the same level as human doctors, and preferred by doctors on eight of nine metrics evaluated.

The follow-up is multimodal Med-PaLM. It no longer only takes text. It takes medical imaging: an x-ray, a pathology slide, a patient history, even certain encoded DNA. Labenz’s phrase is that it is starting to have different senses. A second Tanishq episode had covered virtual staining of tissue; the standard pipeline is still cut, slice, dye, and look through a microscope, with narrow expert systems that say cancer or not and a human firmly in the loop. This model takes the slide and the history together and writes back. It still generates only text. So does a doctor, mostly: this is what I think is going on, and here is what we should do.

On radiology-report generation, given an x-ray and a history, it won head-to-head against a human radiologist — judged by clinicians — 40% of the time. Not more than half yet, close to parity. Labenz did not find the result shocking. Google had already done multimodal work and already done medical work; multimodal medicine arriving on schedule is the unsurprising product of both. What he is watching is the monthly clip of the state of the art. The discourse had just started recycling a ten-year talking point: people have been saying radiology would be the first department to vanish, it has not, therefore AI substitution is a bust. Deployment lag is real. The talking point, he said, was never very compelling, and it is less so next to a model already near a radiologist in a judged head-to-head.

The last addition to the report is a glimpse of a possible post-Transformer future. They had already done a MegaByte episode with Lily from Meta AI. MegaByte is hierarchical and predicts at the byte level, which means you can drop tokenization. Audio is bytes, images are bytes, music is bytes. Accept bytes and predict one at a time, and you get finer granularity than high-level tokens, in an architecture that is friendlier to mixed media. It is a compelling proof of concept. It is not really here yet.

Another paper introduces retention, against the old slogan that attention is all you need, and was bold enough in the title to call itself a possible successor to the Transformer. Labenz believes it came out of Microsoft, and not from a tiny group. Most of the stones the world is turning over do not compete with the Transformer. A few look like they might. Then you get the room-temperature-superconductor plot: replication, scale, practice, side effects the proof of concept never showed. Imagine having to relearn, at production scale, everything we have learned about Transformers, because retention turned out better. He would add the global research community to the live-player list for that reason. Nobody expected the Transformer to be the year-2100 architecture. It will be eclipsed, or ensembled. The next phase is starting to have a shape. Never a dull moment, he told Torenberg, and closed the update.