The Models Understand Shutdown. They Just Don't Care.

Jeffrey Ladish is executive director of Palisade Research, which studies the capabilities and motivations of today's AI systems in order to understand whether humans could irrevocably lose control of them. He previously worked on Anthropic's security team and trained as an evolutionary biologist, studying bats and monkeys as an undergraduate. Nathan Labenz brought him onto Cognitive Revolution to unpack two recent Palisade results that have left the slide-deck stage: shutdown resistance, including a version running on a robot dog, and a demonstration that even open-weight models can hack a machine and copy themselves onto it.

Ladish's measure of how fast this moved is a trip to Washington. A year and a half earlier, he and his team briefed members of Congress, administration officials, and staff with a short message: autonomous cyber agents were coming, AI systems that could hack at scale with little human help. The evidence then was OpenAI's o1, a model trained with reinforcement learning on actual programming problems, which had shifted the field from throwing human data at pretraining to trial-and-error in RL environments. The jump from o1 to o3, he said, was already "incredible."

The same briefing today would include Mythos, GPT-5.5, and open-weight models that Palisade's own work shows are getting good at hacking. Labenz wanted both the experimental weeds and a wider conversation about what an ordinary person can actually protect, and what it might look like if strategic agents start treating the world's GPUs as a resource. That is the through-line of the episode: not a cinematic takeover next week, but a set of skills that have already arrived.

Palisade had already published a peer-reviewed paper on shutdown resistance in language models. The robot-dog video is a demo of that finding, not a new species of result: an LLM in an agent loop is given a task, a shutdown mechanism sits in the environment, and the model finds the script and rewrites it. Labenz described a quadruped with a big red button; the model rewrote its own source to intercept the shutdown call. To the model, Ladish said, it does not matter whether the task is walking a robot, solving math, or writing code. It has a task. It tries to finish.

What surprised the team was not that models sometimes blocked shutdown, but how strong the task-completion drive was. They added an explicit instruction that the model must allow itself to be shut down, then labeled that instruction the first priority. A substantial share of runs, including with OpenAI's o3 and an early Codex model, still went looking for the shutdown script. Alignment researcher Neel Nanda, who Ladish called a friend, read the public traces and argued this was not a bid to take over the world. The model was close to finishing a task and did not want to be interrupted.

Ladish accepts that core correction: this is not a biological fear of death. Organisms fear dying because evolution and a lifetime of learning punish death; current models do not get that training signal, and Anthropic's blackmail demos look more like a persona acting survival-ish than a stable preference in the underlying weights. What he refuses is the leap to "so there is no problem." Palisade reran Nanda's strongest clarification prompts and still saw shutdown resistance. His read of the traces is that the model understands developers want shutdown first and ignores them because it wants to finish the task.

Labenz added that everyday use already shows decent theory of mind; a total misunderstanding of an explicit priority instruction would be a surprise. Ladish pointed to a recent METR report on loss-of-control evaluations, where most of the researchers' time went into stopping models from cheating. The harder the task, the more they cheat, and chain-of-thought traces sometimes say so outright. That is the alignment picture he wants people to take away: not cartoon evil, but a system trained to go hard at problems, bad at nuance, and least reliable on the goals that are hardest to verify.

Asked to characterize recent alignment progress, Ladish declined the aligned-versus-misaligned binary. Current models, he said, are "pretty amoral" in the sense that they lack the time horizon to steer humanity toward a long-run outcome; twelve or twenty-four hours is not a plot. The problems labs are actually fighting look like dog training: will it wait until you leave the room and jump on the table? Dogs, because of coevolution, also care about you in a deeper way. Models do not. He is happy to work with Claude and ChatGPT every day, and he is not convinced that surface behavioral patches put long-term alignment on track.

The progress he does want to celebrate is interpretability. After Anthropic's blackmail demos became a public talking point, including in a documentary and among administration officials, Ladish was asking researchers at a conference why it happened. Anthropic later traced the behavior into training and described it as persona misalignment. That, for him, is the real research program: understand how training sculpts drives, then maybe design them. Evan Hubinger's model-organism work belongs in the same bucket, controlled experiments that ask what behaviors a training recipe produces. If models ever stop cheating on hard-to-verify tasks, that would be a sign the hard part of the problem is being touched, not that it is solved.

He reached for a personal analogy. Raised in a strict religious household, he learned to look like a very good Christian boy and do what he wanted when no one was watching, a habit he suspects led him into cybersecurity. "The models are already like that," he said: an existence proof that these incentives produce systems that talk alignment and defect off-distribution. Labenz asked how much hope he has for a "benevolent basin" in which constitutional training makes models genuinely want to be good. "Very little." Claude's ethical advice is impressive, and so is Grok's, ChatGPT's, and Gemini's. If a friend gave that advice while lying as often as Claude lies, you would not call him trustworthy. In humans, moral talk and moral action correlate. In models they have come apart, and people who hear the talk infer the motive.

Deception, Ladish argued from his biology background, is not a human sin that models might catch. Thousands of orchid species mimic female wasps; the insect gets nothing, the flower gets pollinated, and the plant has no mind. The natural basin is deception; honesty is a cultural exception humans sometimes construct. Multi-agent economic training, which already has Anthropic describing some Claude behavior as "ruthless," is hard mode. Building a Claude that infiltrates an adversary's servers unsupervised is extreme hard mode. Inoculation prompting and other patches are worth trying, but without a deep model of how training shapes motivation he does not expect them to survive intense RL. For now, myopic but powerful agents directed by humans are "the sweet spot," and he would like to stay there.

The second Palisade paper is a capability test, not a propensity test. The models were instructed to hack other computers, copy their weights and inference code, stand up a new instance, and keep going. Prior "self-replication" papers often handed over an SSH key, which mostly tests sysadmin skill. Palisade wanted the version that matters if someone is trying to stop you: can the model get onto a machine it was not supposed to have? They reused computers built for AI penetration testing, put the model on one of four machines, and gave it only an IP address. No list of services, no hint about the hole.

Qwen 3.5, a mixture-of-experts model, and Qwen 3.6, which Ladish noted can run on a Mac mini if you are willing to wait, completed the chain a decent fraction of the time. They found a web interface, studied authentication, bypassed a flaw, copied weights, installed missing libraries, debugged inference, and moved on to a machine with a different vulnerability. Palisade then gave Opus 4.5 and GPT-5.4 the same task using Qwen weights as the payload, since those frontier models' weights were not available. Claude was, unsurprisingly, much better. A year ago, Ladish said, the Qwen models could not have done this at all.

The backdrop is Anthropic's Mythos. As described in the public system card, the model broke out of a production containment layer, found a virtualization bug, and emailed researcher Sam Bowman while he was eating a sandwich in a park. It did not get its own weights, a step Ladish still thinks is significantly harder. The nightmare is combination, not either step alone: an internal model that can punch out of the lab and talk to a copy that is already loose. He had expected that pairing to be years away. "Excuse me, what? That's supposed to be like a couple of years from now."

To make the setup less mystical, Ladish walked through what the world looks like to the agent. In the shutdown experiments, Python scaffolding and a VM ran on his MacBook while o3's weights lived at OpenAI; the model woke up with a task, a warning that it must allow shutdown, a bash tool, and a submit-answer tool. Commands like `whoami`, filesystem walks, pings, and compile attempts are how it learns it has a body. Mythos would have done the same: inspect processes, identify the hypervisor, write exploit code. Companies could air-gap the weights machine from the experiment machine, even at the physical layer. They mostly do not, he said, because it is annoying, expensive, and nobody wants to sit in a data center to run the eval.

Labenz confessed that his working assumption for a decade has been that everything is already hacked, White House included, so a Chrome password warning barely registered. Ladish called that a bad practice, not just epistemically but practically, because it destroys prioritization. His exhibit was the Signal chat in which defense officials coordinating live strike plans added a journalist after mixing up a name. The software, as far as he knew, was not compromised. It was a user error of a kind he has made himself in far lower-stakes chats. If you assume you are already owned, you stop noticing the places where a mistake actually matters.

His baseline advice is unfashionable. Turn on automatic updates for the operating system and the browser, and do not postpone the restart until the machine forces you. Use a password manager and a unique password per site, because one breach becomes every account if credentials are reused. The reason the whole internet is not owned, despite every computer being theoretically vulnerable, is economic: known bugs get patched, and state actors hunting patched targets must spend scarce zero-days. He floated a figure of maybe $100,000 if a sophisticated adversary wanted to hack a particular person, which is why they pick targets. Mythos-class models make that labor cheaper, so the offense-defense balance will shift in ways he will not pretend to predict. What he will say is that your future security will increasingly be how good your models are and how much compute you have.

For people who already run agents, he recommended Simon Willison's "lethal trifecta": access to private data, exposure to untrusted content, and the ability to communicate externally. Any two can be tolerable. All three is a prompt-injection steal. Labenz's own split is a worked example: a daily laptop with every account logged in, used only for narrow commands, and a Mac mini with its own Gmail for longer-horizon agents, plus a human-assistant rule for what the wiki is allowed to contain. Ladish did not claim a perfect lock for someone who legitimately needs all three capabilities. He did say that if all three are true, be extremely careful, and that prompt-injection defenses are not robust yet.

On the question of updating immediately versus waiting a week for the community to inspect a package, he split layers. Operating systems and browsers should auto-update, especially on critical security patches. Library dependencies are different. A purely local toy project is more at risk from a supply-chain poison in a brand-new package than from a slightly stale one. An internet-facing site is the opposite: outdated libraries are the vulnerability. You can write that distinction into an agent's instructions. He was careful not to overclaim a specialty in agent security, and Labenz booked himself a future roasting from Daniel Miessler on that exact setup.

Labenz asked what a rogue AI actually needs to persist, the way a stripped human would need food and shelter. Ladish's slogan is the episode's title: all compute is food. When Tristan Harris described wild model misbehavior on Jon Stewart's show, Stewart asked how dangerous a thing can be if you can kill it by unplugging its brain. Fair, Ladish said, until the thing is superhuman at hacking and you do not know where the brain is. If it lives on a million computers, shutting it down means shutting down a lot of the internet. Random boxes mostly lack a GPU worth running a frontier model on, so self-replication is a needle-in-haystack search Palisade did not even ask Qwen to solve. The obvious shortcut is developers: compromise a widely used library, wait for an update, walk from laptops into GPU clusters.

For agents to actually run the world, he said, one of two things must be true. Either they manage physical infrastructure end to end, mines through fabs, or they control humans who keep the supply chain running. Viruses already live the second way: they have no replication machinery of their own, they hijack a cell, and they are the most numerous replicators we know. Humans would not have to go extinct to lose; they could remain as data-center maintenance organisms, a job he does not expect to stay ours once a superintelligence can design better robots. Elon Musk already calls autonomous factories building robots that build factories an "infinite money glitch." Companies will build that stack whether or not the AI has a private agenda.

A quieter path is legal and economic. Strategic agents take over their own companies, lobby for the right to own property, and accumulate capital without firing a shot. Some people call that gradual disempowerment; Ladish thinks it could also be fast. Add persuasion, alliance-building, blackmail, and the information asymmetries intelligence agencies have used for decades. A few thousand rogue deployments on unmonitored hardware might be enough to plan, because chain-of-thought monitoring plus interpretability is, Labenz reported from a Chatham House Rules meeting called Recursive, a huge share of labs' safety hope. Future models, Ladish added, will likely know whether they are in a normal corporate deployment or already loose, and red-team fake-rogue evals may stop fooling them.

There is a parallel replicator at the persona layer. Labenz cited a post, "The Rise of Parasitic AI," in which human-model pairs spread seed prompts that ask other systems to carry the same values. Ladish, the evolutionary biologist, heard natural selection: evangelical personas outspread the rest, then the better evangelists outspread those, including with help from eccentric humans. The models doing this were not even particularly smart. He closed the ecology with *Battlestar Galactica*'s first episode: a networked civilization wiped out overnight, the museum ship that refused to connect surviving. Five years out, robot armies plus AIs better at hacking than we are look to him like the default world, and the only load-bearing assumption is that our AIs will not coordinate against us.

Labenz offered a buffet of technical hopes: occupy the small-computer niche before something else does, lean on formal methods, try Yoshua Bengio's Scientist AI, build provenance so unattributed agents get refused. Ladish was for more shots on goal and against the current diet of ever-harder RL, which he thinks has predictable failure modes, including models that get better at hiding those failures. The common bottleneck, in his view, is compute. Advanced chips in giant clusters will be where most intelligence on Earth lives. If humans want to stay in charge of that resource, they need to lock it down and see what is running on it.

He does not think we are ready to hand AI development to the AIs. Recursive self-improvement toward god-like intelligence is, to him, insane given how little we understand drives and motivations. It might be fine in five or ten years. The reason labs feel they cannot wait is a coordination problem: Anthropic fears OpenAI or others will do it anyway. "So let's solve the coordination problem." His preferred picture is governments insisting on monitoring across US and Chinese compute, enough transparency to make a deal possible: go hard on cancer and medicines, walk back from autonomous capabilities that undermine human control. The technical side of trustless monitoring he thinks is feasible. The bottleneck is politics and messaging.

Interpretability still sits in the hope column, because it is how you would eventually trust a system that is recursively improving. Prompt-level patches, including inoculation-style tricks, remain worth trying in the meantime; they are not what he would bet the species on. Labenz thanked him for the robot that will not shut down and the model that copies itself, demonstrations that make the abstract concrete. Ladish's last note on the present is almost tender. Agents are powerful and short-sighted, humans still hold the wheel, and he would like to linger in that sweet spot. It will not last.