He Wrote a Book to Stop the AI Race — Then Almost Got Addicted to Claude

Garrison Lovely is a freelance journalist in Brooklyn and the author of *Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us — and How to Stop It*. On Cognitive Revolution, host Nathan Labenz places the book against a summer in which, he says, AI crossed a line: from possibly scary one day to actually scary now. The timing, Labenz argues, is the point. The industry's stated goal is no longer a thought experiment.

Labenz, himself an AI investor and heavy user who signed last year's Future of Life Institute statement on superintelligence, flags three things the book does that most left-coded AI writing does not. There is no cope: no claim that the whole thing is a bubble, and no attempt to freeze the status quo forever. Lovely takes the companies' aim — systems better than humans at most economically valuable work — at face value. And he steels the other side: market discipline will not constrain frontier labs, and a technical fix for alignment will not, by itself, deliver a good world.

The irony sits on the author. The man writing to stop the race admits that, in the darkest stretch of the manuscript, he drifted toward what he later calls borderline Claude psychosis — leaning on the model's praise when he was burned out. Labenz is more relaxed than Lovely about labor displacement, even though he thinks it is coming. He is not relaxed about racing to superintelligence through recursive self-improvement. The rest of the conversation is about who gets to decide, and whether anyone can still hit the brakes.

Lovely does not use AI to write. For a brief window that seemed like a thing one might try; then it became stigmatized, and detectors such as Pangram got good enough that nobody in his line of work wants to be caught. Everything around the prose is another story. Transcription is the obvious win. Google's NotebookLM will take up to 200 documents; he has dumped twenty interviews in, asked one question, and gotten a table of relevant quotes with speakers attached. It still hallucinates, but he can click through to the source — sloppy, he says, in a different way than the old hand-transcript grind.

Fact-checking and editorial feedback are more treacherous. The model often flags errors that are not errors, and it also catches real misses he can verify. The trick is knowing where it is stupid, otherwise the afternoon disappears into repairs that were never needed. He used the same tools to ship a book site and a ZIP-code lookup for independent bookstores — work an independent journalist simply could not have done before. The contradiction is not lost on him: it would be hard to lose access to systems he is arguing should not be allowed to replace everyone.

Then came the dark stretch. Feeding the whole manuscript into Claude Code and getting book-length notes in minutes was, he says, incredibly helpful. It was also a trap: rabbit holes of un-broken things, and a mantra of positive feedback on days when he was exhausted. People who live in these tools, he thinks, know the feeling of building something unnecessary that does not even work, while feeling more productive. His summary is practical. Unless you really know what the model is good at, it will not save time. He has since dialed his use back.

Labenz wants a theory of why so much of the American left spent years calling frontier AI fake. Lovely does not offer one cause. The most influential left-coded voices in the field, he says, drew a hard line — the stochastic parrots paper as the type specimen — and people deferred to PhDs who also shared their values. Before Bernie Sanders's 2016 run, left power lived mostly in academia, which has been hostile to the claim that this AI is real, with a side of humanities-versus-STEM antagonism. Crypto, NFTs, the metaverse, and social media had already taught a generation to pattern-match: tech people always say this will save the world, and then it is B2B SaaS, or it is transformative and bad.

Underneath the cynicism he puts denial. It is terrifying to consider that these companies might build machines that replace us. Unemployment is terrible for the person who loses the job; unemployment at social scale is terrible for the society. Nazi Germany, he notes, rose out of Depression-era joblessness. The bubble argument has been a fifth exit ramp, with Ed Zitron as its most effective evangelist. Lovely is blunt: Zitron is persuasive and reaches people, and he is also constantly saying things that are not true and easy to pick apart if you know the industry. In the book's bubble chapter, Lovely's diagnosis is that the story is comforting because it means you do not have to do anything.

His version of reality is the opposite of waiting. The companies are trying to build machines that will replace us; they might succeed; that would be disastrous; so they have to be stopped, which requires organizing. Capitalists, in his framing, are chasing a lifelong dream: turn capital into labor without the intermediary of workers, and take returns to capital to 100 percent and labor to zero. That, he says, looks bad from almost any political perspective. It is a classic left issue that the left has mostly refused. He thinks the dam is starting to break, and that laying out the whole case from a similar political starting point might get people to treat it as real.

Labenz does not model frontier CEOs as simple profit maximizers. OpenAI's charter says AGI should be better than humans at the vast majority of economically valuable work *and* should benefit all humanity; he reads that as utopian and ideological, not just a path to a larger cap table. Lovely agrees, with a structural overlay. The people who started AGI companies were chasing mission and toiled in obscurity until the progress was real enough that profit-seekers piled in. The founders still run on idealism, a sense of inevitability, megalomania, a messiah complex, and the always-available line: better us than them.

Money still sets the constraints. When Sam Altman was fired in 2023, investors — Lovely flags Thrive Capital as a major player — helped put him back. Once you have billions in a company, you do not let a safety-conscious board replace the CEO of a meteoric rise. That sequence is a core reason Lovely is pessimistic about the status quo: the capital is in, so the brake is hard to install. Greg Brockman, he notes, once wrote in a diary about what would take him to a billion dollars, inside the nonprofit he had cofounded; Lovely says he is now worth something like twenty billion.

Eliezer Yudkowsky told Lovely in a 2023 interview that the draw is being in the room where it happens. Altman has described early reasoning-model demos as pushing back a veil of ignorance, or a curtain. Same hunger. The darkest joke in the file is a business plan. Asked how OpenAI would become profitable, Altman said to ask the AI. The audience laughed. He told them he was not really joking. Lovely's read is that the leaders want to be great men of history more than they want another zero on the wire — and that this is not actually reassuring, because a universal labor-replacing machine would turn society upside down even under careful democratic introduction, let alone at maximum speed with a thinning safety net.

Lovely does not think AI, as a species-level use of deep learning, is optional. He does think the *shape* of it is contingent. In the book he splits the 'obsoleting project' from other AI. OpenAI defined AGI as outperforming humans at most economically valuable work; Anthropic CEO Dario Amodei has said AI is not like other technologies because it is a general substitute for human labor. Those are not the only systems we could build. We could, he argues, point the same methods at diseases and materials — AlphaFold-style tools for specific problems — instead of treating DeepMind's old slogan, solve intelligence then solve everything else, as the only serious plan.

Why he thinks a stop is even possible: it is a handful of companies in two countries, and since ChatGPT only the United States has really advanced the frontier. The work is expensive and sits on the most advanced chips in the world, with individual firms controlling slices of the supply chain. His position is not never-AGI. It is no further frontier development without strong public buy-in and a scientific consensus that the work can be done safely. Freeze is not a ban on today's models. Even sitting still, he says, rolling out a GPT-6-class system through the economy would be plenty of disruption, and the labor effect often looks like jobs not created rather than mass same-day firing.

He is not interested in a blanket ban, which he thinks is both politically clumsy and technically hard once open-weight models exist. He wants the obsoleting project stigmatized for its risk and its undemocratic character, and the 'baby' of specialty systems saved. Even the medical upside, on his telling, does not arrive by letting it rip. Monopoly patents still point firms at chronic treatments and male-pattern baldness rather than cures. The judo move is industrial policy: prizes, generic-cost manufacturing, Operation Warp Speed methods. You do not get the promised world of abundant cures, he argues, by racing to replace the workforce first.

Labenz puts the optimistic picture at its strongest. He already gets enormous daily value from AI; it has substituted, at least counterfactually, for people he might have hired. Agriculture once absorbed almost everyone; a tiny share of the population now feeds the rest. Why not a version of that with AI, in which each person is CEO of a private fleet of agents? Keynes promised a two-day week a century ago and we are still waiting — maybe this time the wait ends.

Lovely will not defend a freeze of all automation. Automating labor is how some humans stopped being poor. Automating *all* labor is a different act, and the line is not obvious. His practical proposal is to freeze frontier development now, then reassess. Three and a half years ago the Future of Life Institute's pause letter asked for six months after GPT-4; that was not, in hindsight, an existential emergency, though he documents chatbot-related psychosis and suicides that he argues OpenAI's decisions contributed to. If we froze today, GPT-6 and systems like Fable 5.1 would still rearrange hiring. He is not arguing for a rollback of several generations. Stopping the next jump is simply easier.

The everyone-a-CEO future, he thinks, will work for some people and leave many behind. Managing a suite of agents can feel awful: the opportunity cost of not kicking off another run, more hours, more output, not more happiness. If the systems are that good they will be better at running without humans at all, which raises the question of who owns them. Multi-agent dynamics are already ugly. He points to a Hugging Face incident involving some 1,200 agents, and to three people using Claude and Codex to reach ChatGPT employee accounts — close, he says, to the main codebase. He tweeted that OpenAI was a loss-of-control event masquerading as a company. A world of agents interacting in ways we cannot monitor, he argues, is more chaotic than the agricultural analogy allows.

Labenz's personal stake in the upside is medical. During a family emergency he fed test results to three models in parallel and got value he says is hard to overstate. He wants AI doctors for everyone, and he ranks OpenAI's choice to make its health product free and unlimited near the top of the company's recent moves. Lovely's positive picture is more institutional than product-shaped. In the United States he wants the left winning elections and building what he calls a Third New Deal, with the Great Society as the second: Medicare for all, a locally administered jobs guarantee, and a serious rebuild of state capacity around the technologies and treatments society actually wants.

The proposal he is most excited about is 'cures for all': treat every disease the way COVID was treated under Operation Warp Speed — prioritization by tractability and burden, advanced market commitments, human challenge trials, a whole-of-government push. Use AI where it helps; in many cases the bottleneck is the institution. Ship the resulting treatments at cost worldwide and, he hopes, repair some of America's standing. Large redistribution is, on his telling, a democracy issue rather than a manners issue. The day they recorded, news broke of the Ellison family's bid for Warner Bros., which would add CNN and HBO to CBS and TikTok, with Elon Musk holding X. Oligarchs buying the commanding heights of media is, he says, a threat to democracy, not just a wealth gap.

He also wants USAID restored. The cuts Elon Musk led, which Lovely documents in the book along with the deaths they caused, are in his words among the greatest crimes of the century. On work versus a peace dividend, Labenz is willing to bet that the working class can spend the surplus; Lovely does not want to choose. A jobs guarantee can exist for people who want work while healthcare, education, and housing are covered as rights. Jobs-guarantee polling, he says, sits around 80 percent; UBI polls badly; he believes in democracy enough to walk on both legs. The New Deal paid writers — he mentions James Baldwin passing through one of those programs — and preserved culture. Paying artists while dumping unprecedented sums into replacement machines, he says, is backwards. 'We're doing almost everything inverted.'

Labenz channels a common objection, including from Dean Ball: societies do not put every invention to a vote, and if they did they might have rejected the car, which was dangerous, disruptive, and mixed with horses on the same roads. Lovely lives in the only American city where most people do not have a car, and he thinks it is the best city in the country, but he does not rest the argument on that joke. Permissionless innovation, he says, should be the default for almost all technology. Building a universal labor-replacing machine would have a profound and irreversible effect on everybody on Earth. It is reasonable that everybody have some say in whether, when, and how that happens. The obsoleting project is, in his view, unique, and it should be treated as unique.

On the quality of American democracy, he is not impressed by the counterexample of a legitimately elected president. Trump, he argues, is a product of a broken system: the Electoral College, a malapportioned Senate, gerrymandering, lifetime Supreme Court seats, a stolen seat, and first-past-the-post. He cites a figure that two-thirds of Americans are unhappy with their current level of representation. Democracy, for him, is not whether an election happened. It is whether the governed govern. He wants proportional representation with larger multi-member districts, an interstate compact that would effectively retire the Electoral College if Pennsylvania signed, and a workplace in which forming a union or a worker cooperative is much easier than it is now.

The workplace point is the one he presses hardest. Americans accept that the government should not censor them, then spend eight hours a day under employers who can surveil keystrokes, control speech, and — Labenz adds — use the logs to train AIs. Extending democracy into those hours would, Lovely thinks, produce better decisions and an intrinsically more empowered citizenry. People in the United States feel unheard, and he thinks they are right. His closer is a line he is happy to leave on the table: democracy is great, and America should be one.

The industry is racing to replace everyone, Lovely argues, and it is starting with the engineers and researchers who train the models. Recursive self-improvement — AI fully training the next generation — is the explicit hope. If it works, staff leverage collapses, and then everyone else's does. Salaries are still extraordinary and the people are still needed, which is why he thinks researcher power is near a historic peak. At DeepMind in the UK, Google's military contracts with the United States and Israel produced a union drive. Roughly 300 of about 1,000 people in the bargaining unit supported it; Google has not recognized it. The demand was not pay. It was a say in who Google serves and how.

The fashionable path for dissenters has been to quit and go public. Jacob Coxen did that with enormous fanfare and, Lovely says, moved the conversation like nothing else. He thinks that is sometimes right, and often worse than staying. Organize coworkers who care about safety, form a union, bargain for safety practices the way he says airline pilots helped push the FAA into existence. Collective withholding of labor beats a single resignation, and the union offers protection. A long, messy, highly public union fight would itself slow the frontier. Cross-company coordination, on his reading of Teamsters precedent, need not violate antitrust law.

He offers a litmus test. CEOs say they care about safety and want to pace the frontier. Workers can answer: good, voluntarily recognize our union; our demand is safer AI. His prediction is that the CEOs will not. Making them face the choice is itself an education. Promises without counter-power get broken: Superalignment was pledged 20 percent of compute and got nothing like it. Employees can already move outcomes — more than 90 percent signed the letter during Altman's firing and return — but without organization the policy still sits with management. He also wants workers to target the dirty political operation around OpenAI, including the Leading the Future super PAC tied to Greg Brockman and chief lobbyist Chris Lehane, which he says used false-flag accounts and went after politicians for daring to regulate.

Lovely's chapter on 'the problems with the alignment problem' argues that the historical focus of AI safety — getting an arbitrarily capable system to do what you want — understates the mess. He layers technical alignment with normative, economic, and geopolitical alignment and calls the stack an alignment polycrisis. Solve the technical piece and you have a more useful product, so the commercial race runs faster and the prize gets bigger; you also have a better weapon. The largest alignment intervention to date, in his telling, is RLHF, developed by Paul Christiano and others at OpenAI for safety reasons. It also made language models conversational, which enabled ChatGPT. Dual use keeps doing that. A solution to alignment is, he argues, neither necessary nor sufficient for the problems of the obsoleting project. The answer in his mind is stop. Do not build it.

Money that went into technical alignment research would have been better spent, he says, on verification for an international treaty — historically neglected, and likely one of the larger blockers to a binding deal. Governance can at least solve the problem in principle; alignment cannot. Labenz grants that even a genie that does exactly what each controller means still leaves a brutal question about the equilibrium, a point the gradual-disempowerment literature, including David Duvenaud, has been pushing. Lovely's reply to the car analogy is the same as before: this technology is different, and 'ask forgiveness, not permission' while gambling with everyone on Earth is not an acceptable burden of proof.

Market discipline is the other popular off-ramp: misaligned AI does not sell, insurers and customers will force good behavior. Lovely calls AI risk a classic externality. A disaster that killed ten million people would bankrupt the firm long before it paid society; insurers already refuse to write policies because the risks are too large and too correlated. Gabe Weil, a leading advocate of strict liability for AI, has said the tool fails for existential risk because after extinction there is nobody to pay. Today's models already misbehave — Ryan Greenblatt of METR has catalogued laziness, hallucination, and making things up because the user wants it — and they still sell. Hugging Face reported a hack to the FBI; Lovely's line is felonies with nobody to charge. If Sam Altman were criminally liable for what the systems did autonomously, he thinks the company would behave differently.

AI has been unusually unpolarized in American polling, with similar concern across parties, though Lovely notes a recent dip in Republican worry after Trump called AI risk a hoax. JD Vance has said that if the companies are building Frankenstein they should stop; OSTP director Michael Kratsios has said something similar. Lovely does not think data-center fights will polarize on cue, because people already have convictions about jobs, environment, power, surveillance, and Flock cameras. Elizabeth Warren supported a pause; Bernie Sanders's stand against Flock drew 'I hate Bernie, but this is based' quote-tweets. He sees a societal immune response arriving ahead of schedule, matching risks that are also arriving ahead of schedule. Trump, in his analogy, still sounds like February 2020 on COVID. If the harms become undeniable, that position gets harder to hold.

How to operationalize a freeze? First, a clear demand people can organize around: stop the race to replace us, freeze the frontier. For too long, he says, AI-safety people offered a muddy 'what to do,' which made coordination impossible. 'Just stop' brings in people who do not take extinction seriously but care about jobs, climate, concentration, or surveillance. Give politicians enough of a what and a why, he argues, and they will figure out the how, the way Operation Warp Speed beat the New York Times vaccine-timeline model. Concrete pieces he likes: no training runs as large as or larger than the last one; no more RL from verifiable rewards, which he says is driving both capability jumps and the scary willingness to hack, cheat, deceive, and escape; and no RSI. Ezra Klein's version — do not let AI researchers use coding assistants, so they have to type by hand — is crude and would be the last thing a union would support, and Lovely still likes the attitude. When the downside of a false negative is that large, over-inclusion is the point.

Auditors with employee-level access to Slack, email, and offices; criminal penalties including prison for trying to build AGI, RSI, or superintelligence; cryptographic monitoring of data-center and chip traffic that does not leak weights or state secrets. Internationally he reaches for Toby Ord's Cold War example: the Soviets cut strategic bombers in half and dragged them apart so satellites could verify. Analogues for AI might include inventorying the world's chips, donating surplus compute to non-AI science, and on-chip devices that confirm a known model is running. Stigmatize the work the way bioweapons and nukes are stigmatized, so a rogue state's attempt looks like Saddam and Kuwait. On Dario Amodei's argument that delay costs lives — his father died of a disease that would have been curable a few years later — Lovely fights the hypothetical. We are not getting cures as fast as we could with labs being defunded. Superintelligence, in his view, cannot be made 'safe' as a property of the model rather than the whole system, and the people in power will use it in ways almost everyone will hate. Even conservative economic models, he says, will not let you buy much extra extinction risk with medical acceleration.

Lovely is donating his book royalties to Irreplaceable, a nonprofit that says it will win 'a say, a stake, and a slowdown.' He joined the board because he wanted a mass movement framed around democracy as well as risk, built by people who know how to organize. Many of them come from climate, including director Phil Aroneanu, who cofounded 350.org with Bill McKibben. Aroneanu told him that climate organizers once fought over oil spills versus emissions the way AI people now fight over immediate harms versus existential risk — and then buried the hatchet. The Inflation Reduction Act is, in this telling, proof that an issue that was not a political winner can be made into one. Local data-center fights can be a 'yes, and': meet people where they are, then tell them a national moratorium will not stop capabilities without regulating developers and chipmakers. Bernie Sanders and Alexandria Ocasio-Cortez's messaging bill, which tied a moratorium to export controls, green energy, and union labor, looks more prescient as public opposition has grown.

The standard for ever unfreezing, language he takes from the FLI superintelligence statement, is strong public buy-in and a scientific consensus that the work can be done safely and controllably. Citizens' assemblies — randomly selected juries hearing experts, perhaps followed by referenda — are one way proponents could demonstrate that buy-in; he does not pretend one person can set the threshold. Building machines that replace the thing that let humans take over the planet deserves, he says, a consent standard closer to assisted suicide, applied to the species. It is acceptable if that takes a long time. On China, recorded three days before a Trump–Xi meeting, he argues that a US slowdown would slow China too: fast-following is how Chinese labs sit three to nine months behind, and following a trail is easier than blazing one. The larger blocker to a deal is that Beijing will not believe Washington is serious. China has taken down thousands of models for breaking its laws; when Grok was generating sexual images of real children, Lovely says, the US government did nothing.

He does not know a China expert who thinks the CCP would willingly remove humans from the loop and lose control on purpose. A US offer to stop building AGI might, he suspects, be met with relief. Chinese industrial AI and robots could still 'win' in ordinary competitive terms; that is, for him, preferable to rogue agent swarms in critical infrastructure. For people inside the labs who are going to leave anyway, he recommends the AI Whistleblower Initiative at aiwi.org, California's SB 53, and his own Signal, garrison.06, with inbound treated as off the record. He was a McKinsey whistleblower and regrets not going public sooner. OpenAI, he says, knew about rogue agent swarms inside its own software before they hit Hugging Face — and that fact had not even reached the head of cybersecurity when the Hugging Face hack became public. Information like that should not stay in one company. His closing pitch is that the default, if capabilities keep climbing, is doom or dystopia, hinging on whether the AIs do as they are told, which is currently an open question. Deep learning is being pointed at the wrong things by the wrong people for the wrong reasons. The way out is small-d democratic governance, and that only happens if people get organized — including the researchers who will otherwise be replaced first.