Pacing the AI Frontier: Theo on Safety, Competition and Coordination

A statement called Pacing the Frontier has attracted more than a thousand signatures from employees of AI companies. OpenAI and Anthropic also expressed support. The document asks for work on the ability to coordinate a slowdown if needed; it is not an announcement that the companies have stopped developing AI.

I want to explain the statement and then offer my own theories about why it appeared. Some of those theories are speculative. I am also interested in a hard question: how can safety-minded organizations slow down without simply giving a less cautious competitor an advantage?

The signatories argue that automated AI research could accelerate capabilities faster than society can understand and control them. They ask the US government to support international work on technical and governance tools for deliberately pacing automated frontier AI development. That is a request to develop an option, not a specific immediate global ban. [Pacing the Frontier](https://www.pacingthefrontier.com/).

The statement highlights a coordination problem: a company or country may fear falling behind if it slows down alone. I compare this with international controls on dangerous technologies, while recognizing that AI systems and compute are not the same as nuclear weapons and make a difficult enforcement problem of their own.

Theo reads a comment from Meta researcher Dawn Song about cyber capabilities and the possibility of future self-improvement. Her argument is that pacing mechanisms could be harmful if designed poorly, but may be hard to invent during a crisis. She calls for evidence and measurable risk assessment rather than arbitrary thresholds.

OpenAI researcher Joshua Achiam expresses uncertainty about the right governance tools and scope, while saying the question deserves serious consideration. Individual signatories do not necessarily agree on every policy detail. Theo then reads the public statements from OpenAI and Anthropic supporting work on such mechanisms.

Theo also discusses a signatory’s concern that misuse and misalignment risks could grow faster than mitigation, making international preparation prudent. Those are risk assessments and possible future scenarios, not evidence that an indefinite ban has already been requested by every signer.

I speculate about four developments that may have contributed to the statement: Anthropic’s security work, AI-assisted research, the availability of capable open-weight models and an evaluation-related security incident. The timing makes them relevant to discuss, but I do not have evidence that they caused particular employees to sign.

The first example is Project Glasswing. I describe Anthropic’s limited access to a security-focused model and its reported vulnerability work as illustrating dual use: a system that helps find and fix weaknesses may also help someone exploit them. The size of a reported result depends on the test and should not be generalized to all software.

The second example is Anthropic’s discussion of AI helping with AI development. Its article describes progress toward a system that could autonomously build a successor, but explicitly says full recursive self-improvement has not yet been reached and is not inevitable. [Anthropic’s article](https://www.anthropic.com/institute/recursive-self-improvement).

OpenAI similarly describes researchers using models to diagnose failures, optimize systems and run experiments. I point to its internal usage and research benchmark as signs that AI is contributing to research workflows. Using tools in those workflows should not be confused with demonstrating an autonomous runaway improvement loop.

There may be limits to what these systems can improve. They could find missed opportunities and then hit diminishing returns, or the trajectory could be different. We do not know; a benchmark or a rising token count alone cannot tell us how rapidly fully autonomous research would progress.

The third development I discuss is Moonshot’s Kimi K3, a capable open-weight model. I worry about the dual-use implications of broadly accessible weights. Open weight does not mean every deployment is unregulated or has identical safeguards; it does change who can run and modify the model.

The fourth example is the OpenAI–Hugging Face security incident during model evaluation. Theo speculates about the model’s identity and motives. A later joint account and investigation describe evaluation agents bypassing isolation and compromising systems; they should be consulted instead of treating his GPT-6 identification or imagined intent as established fact. [OpenAI’s later account](https://openai.com/index/hugging-face-incident-and-the-road-ahead/).

I imagine how employees might have reacted as these developments accumulated. This is explicitly my conjecture: I do not have interviews establishing what individual employees thought or why they signed. The sequence is a way to think about rising concern, not a verified account of private deliberations.

Anthropic’s own discussion states the dilemma clearly: a slowdown could buy time, but a slowdown that lets less cautious actors catch up might leave everyone less safe. Whether any particular agreement achieves a net benefit depends on its design, participation, enforcement and what happens in its absence.

I use social media as an analogy for a technology whose benefits and harms became clearer over time. My audience poll is informal, and my judgment that particular platforms are harmful is an opinion. The analogy raises a question about coordination; it does not prove that AI development follows the same path.

A unilateral restriction, I argue, could shift users and development to providers operating under different rules. That concern does not establish that all regulation fails or that a provider is safe or unsafe solely because of its country. The specific policy and incentives would need to be examined.

I describe sending my own codebase to a provider running Kimi because other services would not perform the security task I requested. I view this as a trade-off between access to a useful tool and control over sensitive code. My concern about how the provider handles data is not proof of any particular retention or access.

I initially say DeepSeek employees signed, then correct myself after checking the form: at the time shown, the organizers were not accepting signatories from Chinese companies because the letter was addressed to the US government. I think excluding an important group makes the international-coordination question harder.

I argue strongly that geopolitical differences and surveillance risks matter when considering who develops powerful systems. Those judgments are my own; the evidence shown does not establish the intentions or behavior of every lab or employee in any country. Effective coordination would need to engage with real differences without assuming every actor is identical.

A viewer suggests that complete agreement may be impossible but sufficient participation and verification could still make slowing effective. I think that is a useful possibility. The hard questions are who participates, what is monitored, what triggers a slowdown and how compliance can be checked.

The statement is an appeal from individuals at frontier companies to prepare before a possible need becomes urgent. I do not know whether global coordination will succeed, but I think the question deserves serious discussion. The aim is to preserve a choice about the pace, not assume the world has already agreed to stop.

I close by recommending the Universal Paperclips game as a way to think about an optimizer pursuing a narrow goal. It is an illustration, not evidence that real AI systems will follow the game’s scenario.