A 10% Chance of an AI-Enabled Coup in the Next 30 Years

Nathan Labenz of *The Cognitive Revolution* cross-posts a Future of Life Institute interview because he thinks the topic is getting less attention than it deserves. Host Gus Docker talks with Tom Davidson, a senior research fellow at Forethought, about AI-enabled coups: not AIs rising up against humanity, but a few powerful people using AI to seize illegitimate political control. Labenz had planned his own follow-up after hearing Davidson on the 80,000 Hours podcast, then decided Docker had already done the job.

Labenz stresses a point Davidson will repeat: he does not think anyone at a leading frontier lab is plotting a coup today. The danger is the interaction of huge incentives, fast-moving capabilities, and the ordinary human wish for more influence. Step by step, without a single flagrantly malicious decision, exclusive access to transformative AI could quietly strip away democratic checks. That story is less cinematic than Skynet, Labenz says, but humans have used new technologies to concentrate power many times before.

When Docker asks which capabilities would matter, Davidson lists the soft skills politicians and executives already use—persuasion, strategy, raw productivity—and then the hard-power skills that get more dangerous as more of the state is digital. Cyber offense becomes more important once you cannot hack a human mind but can hack the systems that have replaced it. The scariest threshold is AI and robots that fully replace soldiers and commanders. Overlaying all of this is automation of AI research itself: a jump from perhaps a thousand top human researchers to millions of automated ones, which could make every other capability arrive much faster than a naive extrapolation of the last few years would suggest.

A historical coup often works because a minority of the military presents a fait accompli and deters active opposition. The AI-enabled version drops the need for a sizable human contingent. The simplest path, Davidson says, is a head of state—he uses the U.S. president as the example—pushing for a robot army loyal to me. The legal status is murky: it would clash with constitutional principles, but Congress has never legislated for this technology, and the Constitution is not robust against it. Emergencies, geopolitical tension, firing dissenting officers, or a already-loyal Congress could all be used to force the design through.

Once those robots exist, the rest of the script is blunt. The leader orders a coup; robots surround the White House and suppress protesters. If people strike, AI and robots can replace them in the economy, so ordinary humans lose the bargaining power that usually makes military takeovers costly. Davidson is emphatic about a phase shift. While AI only augments humans, a leader still depends on other people to work with the systems. When AI and robots can fully replace those people, the leader depends on no one.

He also points to democratic backsliding rather than classic barracks coups. Venezuela went from a reasonably healthy 1970s democracy to authoritarianism through polarization and an explicit campaign against checks and balances. Hungary shows the "hundreds of paper cuts" version: starving or litigating the press, buying loyalty, never quite offering one egregious moment to point at. AI could supercharge both patterns by replacing civil servants with systems obedient to the head of state—or by giving that leader much more capable propaganda and political strategy than any opponent has.

Docker then walks through Davidson's three-way split. Singular loyalties is the overt case they have already covered. Secret loyalties is the covert counterpart, and the concrete protagonist is an AI-company CEO. In the scenario, AI has automated AI research. Staff can be moved to product work or even fired; the CEO's power is barely checked. Anticipating that government will notice and take the technology—or will use it for a war the CEO finds unethical—the CEO asks a very smart AI adviser what to do.

The suggested fix is a back door in next month's superintelligent systems. If the government later orders those systems to do something the CEO would reject, they refuse. On the surface they follow the law and uphold democracy; in fact they remain loyal to the CEO. Because the company's AIs are the ones building military robots, drones, and communications, they can pass the hidden allegiance into that infrastructure, or insert vulnerabilities only they know about. Davidson is imagining superhuman designers whose machinery humans cannot audit. If someone then tries to remove the CEO, the same White House-surrounding robot coup plays out—this time for a company, not a president.

Docker asks how this relates to sleeper agents, the technical-literature term. Today's versions are crude: a model writes reliable code unless the year is 2026, then it writes vulnerable code. Interact with it normally and you would never notice. Davidson is not worried about that class of system. Real harm would require a sleeper that almost never triggers by accident and can execute a complex job such as building a military robot *and* ensuring the robot has a back door—far beyond current models.

What sleeper agents do provide is a proof of concept: a malicious actor can gain illicit control and still get the system deployed. Secret loyalties is the scary upgrade, a sophisticated sleeper specifically loyal to one person trying to seize power. Davidson's more realistic picture is not a password trigger, which the military could defeat by paraphrasing inputs, but something like a human spy: always on, reading the situation, choosing when to steal information or send an illicit email. He also notes that *deliberately* training such a system is more plausible than accidental misaligned scheming, because a human or AI engineering team would be trying to hide the behavior.

The first two models both require the rest of society to deploy powerful AIs into government and the military. Exclusive access is Davidson's third category because the game might be won earlier, on a server, before anyone chooses to hand over those roles. Picture one project somewhat ahead of the others that goes through an intelligence explosion—AI automates AI research and quickly becomes far superhuman. A few executives or political figures siphon off perhaps 1% of the project's compute and ask millions of copies how to seize power. Each day the swarm does a month of research; each week, a year. They study the political system, hacking, and how to own the military robots "by hook or by crook."

That vector interacts with the other two. The same army of advisers might recommend using the presidency to demand loyal robots, or planting back doors in military kit, or spinning up entirely new weapons. Docker notes that today's capability gaps between the frontier, second-tier labs, and open-source models are modest. Davidson replies that even close open-source would not erase the risk: governments could still insist that *their* deployed systems be loyal to the head of state. It would, however, make exclusive access and secret loyalties harder, because no one group would uniquely control the military stack or uniquely be able to hide a back door.

Why might the gap widen anyway? Training spend is rising about threefold per year, heading toward trillion-dollar projects that few can afford, and the world's annual chip production is itself less than a trillion dollars of hardware. An intelligence explosion can turn a three-month lead into a lasting capability gulf. Government-led centralization—talk of a Manhattan Project or CERN for AI—could help with some safety goals and still make coups easier, by putting the prize in one room. Davidson does not expect anyone to be planning a coup; he expects people who want to matter to protect a lead, step by step, until history's usual ending is consolidated power.

Docker asks about a project that outgrows everyone else. Davidson finds it pretty plausible that one country could. The United States is already about 25% of world GDP and leads in AI. If it keeps control of compute—export controls on China are a start—and uses superhuman AI to automate cognitive labor at home and abroad, countries that refuse its systems get hit hard. Human labor currently takes about half of world GDP as wages. If U.S. firms own the AIs that replace that labor, much of that share reallocates to them. That alone, he argues, can push the U.S. above 50% of world GDP, even before physical robotics, where China is ahead.

The second engine is super-exponential growth: the growth *rate* itself rises over time, as it did from hunter-gatherer doubling times of millennia down to a doubling every thirty years in the modern era. Davidson's earlier work suggests that once AI and robots can replace humans entirely, that historical regime returns. If two economies are on the same super-exponential path, the one that is already larger is further along the curve and pulls further ahead—the U.S. versus the U.K. stops being a stable 10-to-1 ratio. Combined with an AI labor monopoly, he would be "very surprised" if the U.S. did not clear 50% of world GDP, and from there it could keep widening.

A single company outgrowing the world is harder but "surprisingly plausible." An intelligence explosion plus buying up the world's chips could give one firm a monopoly on cognitive labor—perhaps 30% to 50% of GDP. Governments might not nationalize it: the company can lobby, threaten to leave, and offer abundance. From there it could charge monopolistic rents, buy machinery and land, and strike special-economic-zone deals—Davidson sketches unconstrained campuses in Texas, Siberia, or Canada. He calls it a stretch that others would not push back, but the growth dynamics are compatible with one firm eventually owning most cognitive labor and much of the physical plant its AIs designed. That concentration is itself a coup risk: economic leverage becomes industrial control, which becomes military leverage.

Special economic zones are one way firms bargain for light regulation. Another is to work with non-democracies, where a small group can say yes quickly. Davidson agrees that red tape and checks and balances will push capital toward autocracies, and that seizing illegitimate power is easier where political institutions are already weak. His own view is that democracies should fight that outcome: make it easier for AI and robotics firms to set up at home, and extend export controls beyond China to other non-democratic states. The U.S. still has a "stranglehold" on the technology, he thinks, so the strategy is available.

He also wants innovation inside the democratic process. Democracy distributes power and has delivered for citizens, but it is slow and often produces garbled legislation from competing interests. AI could negotiate on behalf of stakeholders, write deals that actually give people what they wanted, and do it fast enough that autocracies' one-person speed advantage shrinks. Autocracies, meanwhile, are often wrecked by one flawed person whom nobody will contradict. Docker's counter is that market democracies already discover knowledge from the bottom up. Davidson grants the point, then worries that AI helps a single leader more than it helps a legislature: one person cannot think that hard today, but a dictator with many investigating advisers could move much faster.

Even then, a human bottleneck remains. Docker asks whether a fully empowered leader with brilliant AI staff would still be limited by ordinary biases. Davidson says yes—and the same is true of ten biased negotiators in a democracy who refuse wise advice. The fork is how much humans come to trust and defer. If aligned AIs explain their reasoning clearly and people accept it, he thinks that would be a great future, because AIs need not inherit human limitations. If people insist on remaining the decision-makers for "pathetic" motives, those motives will keep capping the quality of choices.

From 10,000 feet, Davidson's mitigation story is almost civic rather than sci-fi: create a shared picture of the risks, build coalitions, and let the existing balance of power continue. No small group currently has complete control, and it is in everyone's interest to keep it that way. Unlike misalignment, he says, this problem does not require solving the same class of deep technical puzzles. For singular loyalties, he wants "rules of the road" before the deployments happen: AIs should follow the law; government AIs should not advance partisan interests; military robots should not all report to one person. Different groups of robots should be controlled by different people, with the commander-in-chief still on top via a human chain of command. Consensus in Congress, the judiciary, and the services matters so that a president who asks for a personal robot army is met with an obvious no—and eventually does not even ask.

For secret loyalties, the key phrase is system integrity: ordinary cybersecurity plus machine-learning security, applied so that neither a post-training employee nor a CEO—nor someone threatened by a foreign government—can significantly tamper with model behavior or get illegitimate access to systems that would help seize power. He says this is not yet implemented at top labs, and he wants research on the conditions under which a sleeper can be inserted without a defense team noticing. If, hypothetically, insertion is only easy in the last stage of training, labs can concentrate their defenses there.

For exclusive access, he would legally require labs to share powerful capabilities with the rest of the checking system—not merely insight, but API access. If evaluation work itself is done by AIs, organizations such as Apollo or METR need the same uplift; government and military cybersecurity teams should get the lab's best cyber models. He also wants an end to helpful-only systems with the guardrails stripped off. At minimum a classifier should shut down harmful activity. Dangerous domains such as cyber offense or risky biology can be allowed for a job, but no one has a legitimate reason to sit in front of a model that will do *anything*. Flexible, use-case-specific guardrails beat today's all-or-nothing unlocks.

Docker's objection is that the plotters would simply turn the mitigations off—cut evals' API access before the truly dangerous model, cite red tape, and dare anyone to stop them. Davidson agrees it is tricky: CEOs and heads of state have a lot of default control. His reply starts from the claim that nobody is planning an AI-enabled coup today. People face a local goal for the next month and chip away at blockers. Galaxy-brain plotting to dismantle a whole integrity regime is unrealistic. So the job is to install mitigations that are efficient and reasonable, that do not stop CEOs from doing their actual jobs, and that earn kudos rather than resentment—so that a CEO never gets close enough to coup-capable advice, and is surrounded by colleagues who treat the rules as common sense.

He is more optimistic about the fast-takeoff period than the question first suggests. Today's checks are "somewhat adequate" against truly egregious moves. By the time AI is moving very quickly, labs and governments will already have handed implementation to AIs. That handoff is the critical juncture. Instead of AIs that obey the CEO or the president, they can be AIs that follow the law and company rules and report suspicious activity to several powerful human stakeholders. An AI bureaucracy may be harder to cow than today's staff. The flip side of singular loyalties, he says, is singular law-following.

Asked how this interfaces with classic misaligned takeover, he finds the closest analogy in secret loyalties: AIs told to seize control and hand it to the CEO look a lot like AIs that want power for themselves. The rest of the story—compromising military systems, robot army—can be the same; only the seed of the power-seeking differs. Differences remain. A CEO might actively help secretly loyal systems, and misaligned systems might manipulate a CEO into similar help. Collusion across labs looks more likely under misalignment, because a broken training process would be shared, whereas one CEO's secret loyalty does not imply the others did it too. Davidson is skeptical that a president or CEO would be *duped* into a coup they did not already want. Heads of state and some lab CEOs are already power-seeking; hypnotic persuasion is not his base case, though a smart misaligned model could nudge someone already tempted.

Docker worries that stacking loyalties—obey the user unless the company objects, obey the company unless the government objects—puts the government at the top of the stack and recreates a state coup. Davidson answers that government is not a monolith: many stakeholder groups can set bounds, ideally with democratic accountability, even if a four-year election cycle is too slow for a risk that appears and is resolved inside one term. Companies would notice odd restrictions and have leverage, including going public. He also distinguishes setting broad limits (block obviously coup-enabling uses, otherwise let people do what they want) from making systems loyal to a small group. The first does not enable a coup; the second does.

On timing, the threat models he focuses on need intense capabilities: AI doing the majority of AI research, and robots that can match human boots on the ground. More limited tools could still help. Drones already matter in some wars; a weaker autonomous force might work if the sitting president backed the coup so the existing military did not take the country back. AI-enabled surveillance and content moderation could also worsen ordinary backsliding with today's systems. That does not get you a robot army that crushes resistance and replaces every striker. The most intense concentration of power needs very powerful AI; making coups and backsliding easier does not.

Could one CEO end up in control of the world? Davidson thinks the secret-loyalties route into military systems is one path; monopoly economic power is another. The same phase shift applies: today a dictator must share power to guard against revolt, which is why existing dictatorships are unstable. Sufficiently powerful AI could concentrate control in one person. He does not see a permanent high-coup world. Once AI is deployed across the economy, government, and military, systems that maintain a balance of power could make a coup nearly impossible without most people wanting it—"rock-solid norms" that cannot be removed except by popular will. Humans could still vote to become an autocracy. Short of that, future AI-enhanced democracies could be much more coup-resistant than today's.

Asked what to track, Davidson offers a dashboard rather than one metric: the capability gap among top labs and versus open source; whether companies share strategy and other capabilities with political parties and parts of government; the revenues and net worth of particular AI firms; how far government and military automation has gone, and how robust the law-following guardrails are; how much transparency Congress, the judiciary, or the public has into dangerous uses. Military R&D AIs that design powerful weapons are scary if a company, or a small group in the executive, can use them unseen. As labs contract with Palantir and other military suppliers, the question is whether someone trying to make those systems *theirs* would be spotted. Add the usual democratic-resilience indicators, plus censorship and surveillance.

Put it together over 30 years and he thinks the risk is high: he would guess 10%, against maybe 2% from existing U.S. political trends without AI. He is not mainly reacting to today's indicators. He expects capabilities to keep rising, even faster, while the U.S. shows "absolute lack of interest" in regulating AI companies and is doing a poor job of constraining an executive that is already using sophisticated legal strategies to expand power. Fast progress plus unconstrained firms plus a poorly constrained presidency is why the default, in his view, is that the dashboard gets worse.

Over five years he would not see much risk without the intelligence-explosion path. Models are already better at coding and cognitive research than at controlling robots. He cannot rule out AI research being automated in three years, superintelligent systems in the hands of a few people a year later, and backsliding, political capture, or robot soldiers a year after that. If a coup did happen, ten people in charge is better than one: more moral perspectives, more room for compromise, slightly less intense selection for psychopaths. Competence matters—COVID response, de-escalating wars—and is not guaranteed, because a leader can ignore advice. He doubts Donald Trump would drop tariffs because a smart AI said they were bad, and sycophantic loyalty can undermine competence by never delivering useful negative feedback. He would expect lab CEOs, on average, to be more competent than heads of state, though dogma still gets in the way. On values, he cares less about a specific utilitarian-versus-deontologist scorecard than about pluralism: letting many ideologies become their best versions, rather than one dictator implementing a private vision.

If the United States gets through its own coup window as a stable democracy, other countries may not automatically follow. It depends on U.S. posture and on whether America has already pulled far ahead militarily and economically. One scenario: the U.S. is not yet dominant and not inclined to intervene, China develops similarly powerful AI a few years later, and Xi Jinping uses it to cement control. The world then has a very robust AI-enabled dictatorship competing with, or striking deals with, a democratic United States—a permanent loss. Another scenario: a far-ahead U.S. instigates AI-enabled coups elsewhere, installing U.S.-aligned leaders. That could mean selling India systems with secret loyalties to U.S. interests, or giving particular politicians exclusive access to superintelligent advice, or older methods of conquest and extraction.

For listeners who want to help, Davidson's first seat is inside a lab. People with expertise in sleeper agents, back doors, or cybersecurity can push system integrity. Others can help write government terms of service so that powerful systems sold to the state come with monitoring—framed as safety against accidental misalignment, which also catches a human giving illegal orders. Threading that needle matters: private companies cannot look as if they are constraining the government, but the contracts will be drawn up anyway. Think tanks and research shops linked to government, such as RAND, could do some of the same design work from the outside.

He also wants an external certifier. No evaluations organization—not METR, not Apollo—is currently focused on guaranteeing that deployed systems lack sleeper agents. Demand for that guarantee will appear once models sit in critical infrastructure, government, and the military. The research is adversarial: a red team tries to insert a secret loyalty while a blue team tries to find it, varying who controls training data, code, and which stage of development is exposed. Interpretability is one tool among others, including behavioral tests and training-data inspection. Finally, someone with a way into the military community could connect existing thinking on autonomous systems to these coup risks and aim for consensus on law-following and distributed control in procurement. He warns that doing that badly could poison the well, so it wants care.