Leopold Aschenbrenner’s 2024 Forecast for AGI, Superintelligence and Security

The possibility of an intelligence explosion is the premise behind the scenario Leopold Aschenbrenner explores in his 2024 essay Situational Awareness: The Decade Ahead. I read the long essay and want to walk through its argument: AGI might arrive by 2027, followed by systems far beyond human ability. These are forecasts and scenarios, not observations of what must happen.

Aschenbrenner formerly worked on OpenAI’s Superalignment team. Reports described a dismissal involving alleged information leaks; he has disputed that account and said his security memo to leadership played a role. I cannot independently resolve the reasons for his departure. The essay itself says its discussion draws on public information, ideas and industry knowledge.

The essay is dedicated to Ilya Sutskever and thanks Jan Leike. I connect their departures and concerns to a broader debate about safety, although different people raised different issues. Aschenbrenner’s particular emphasis includes the security of advanced AI secrets against foreign actors.

He describes rapidly growing plans for compute clusters and a scramble to secure electricity. His forecast imagines enormous capital spending and a substantial expansion of US electricity production by the end of the decade. These are projections, not proof that a particular buildout or power shortage will occur.

In his scenario, AI systems progress from college-graduate-level work toward superhuman abilities, with national security agencies taking a greater role. He frames China as the main strategic competitor and worries about both a race and the possibility of conflict. This is his geopolitical perspective, not an established description of an inevitable future.

Berman recounts Aschenbrenner’s claim that he warned OpenAI leadership about inadequate protection of model weights and algorithmic secrets. The reported human-resources reaction is his account, which Berman says he cannot verify. Security concerns and the circumstances of his departure should not be presented as settled facts.

Aschenbrenner argues that only a small group has taken the implications of rapid AI progress seriously. He expects AGI and, shortly afterward, superintelligence that may be difficult to control. I find the possibility striking, while recognizing that this is one extrapolation and that critics question its premises.

The first chapter counts orders of magnitude: each order is a tenfold change. Aschenbrenner combines an estimated half-order per year in compute with another half-order in algorithmic efficiency and further gains from making systems more useful. He argues another large qualitative leap by 2027 is plausible, rather than demonstrating a fixed timetable.

He presents GPT-2 through GPT-4 as a rapid progression in capability. The school-age analogies make the story accessible, but model strengths and weaknesses are uneven and do not map cleanly onto a human’s education or general competence. Yann LeCun has argued that scaling existing language models alone may not provide the world models needed for human-level intelligence.

The threshold the essay emphasizes is an AI system that can perform the work of an AI researcher or engineer. If that were achieved and large numbers could work in parallel, its author argues, they might improve AI research itself. The availability, independence and productivity of such systems are assumptions behind the scenario.

Aschenbrenner says that a quiet period after GPT-4 led some observers to see stagnation. His counterargument extrapolates increases in effective compute and suggests another GPT-2-to-GPT-4-sized jump. Gary Marcus and other skeptics have challenged aspects of the scaling story, so the projection is not a consensus.

The proposed transition is from a chatbot as a tool to an agent resembling a remote coworker. That would be especially consequential if it automated the research needed to build its own successors. I had not fully appreciated how a feedback loop of research and deployment might change the pace of progress.

The essay illustrates previous generations with examples: GPT-2 producing short text, GPT-3 using few-shot examples for narrow tasks, and GPT-4 writing and debugging code or addressing difficult questions. These are selective illustrations, not comprehensive measures of intelligence. Even GPT-4 could perform unevenly on simple planning and language tasks.

A chart plots progress on various benchmarks relative to human baselines, and the essay cites exam performance. Benchmark results depend on the tests and evaluation methods; they do not establish broad human equivalence. I also disagree with the claim that GPT-4 handles physical reasoning with ease, and point to LeCun’s argument for additional world-modeling methods.

The essay discusses difficult scientific questions and examples of video generation as evidence of progress. I find the visuals impressive, although a few generated examples do not prove general reliability. The larger argument is that greater effective compute can yield better results, subject to the limits of the underlying assumptions.

Aschenbrenner groups potential gains into more computing power, improved algorithms and “unhobbling”: changes that help a system use capabilities it already has. The last category includes reinforcement learning from human feedback, chain-of-thought prompting, tools and scaffolding. I see overlap between these ideas and modern agent workflows.

He compares the pace of investment-driven compute growth with the historical pace of Moore’s law. Companies were buying GPUs and building infrastructure, but whether their investment would pay off remained uncertain in 2024. Rumored clusters and proposed spending plans should be read as expectations rather than completed projects.

Algorithmic improvements can act like a compute multiplier: a better technique might reach similar performance with much less training or inference work. The essay points to falling inference costs for a particular math benchmark and lower prices for capable models. Prices, benchmarks and quality are not interchangeable, so the comparisons have a limited scope.

Mixture-of-experts models provide another illustration: a model can have many total parameters while activating only a subset per token. Aschenbrenner estimates past efficiency gains and forecasts further orders of magnitude by 2027. Finding additional gains may become harder, even as labs invest more in research.

The data wall is a possible bottleneck. The essay asks what happens when readily available high-quality internet text becomes scarce; the claim that all usable internet data has already been consumed would be too broad. Proprietary datasets may matter, but the availability, rights and quality of data vary.

Possible responses include repeating data, generating synthetic examples, self-play and reinforcement learning. I am not certain synthetic data is a lasting solution on its own. Self-play can produce useful feedback in settings such as games, where outcomes can be evaluated, but transferring that success to every domain is an open question.

Another possibility is better sample efficiency: getting more learning from existing material. The essay compares a model’s training pass with a student working slowly through a difficult textbook, practicing and receiving feedback. It is an analogy, not a literal account of human or model learning, but it highlights the potential value of higher-quality data and better training methods.

As labs develop proprietary datasets and methods, Aschenbrenner expects more variation in their performance. He suggests open-source efforts could struggle if major algorithmic advances stay private. I am less certain, since researchers also have incentives to publish, and the eventual balance between open and closed work is unresolved.

The essay compares an immediate answer to solving a difficult problem step by step. Chain-of-thought prompting, scaffolding, tools, longer context and post-training are presented as ways to make a model more useful. For example, separate components might plan, propose solutions or query a calculator and search engine.

The 2024 chatbot described in the essay lacks persistent memory, extended autonomous work and personalized context in the same way a colleague might have them. The author anticipates a move toward agents that can be onboarded with organizational information. I see retrieval-augmented generation as one possible way to supply that context, alongside longer windows.

Test-time compute is another proposed lever: give a model more time or computation to work on a task rather than demanding an immediate answer. The hope is a better result, but more time alone is no guarantee of correctness. How to use that computation effectively is itself a technical problem.

I have tried iterative prompting on a simple request for sentences ending in a particular word. Asking the model to check itself did not reliably reveal the error in my example. Code is easier to iterate on when execution and tests return external feedback, though the usefulness of feedback depends on the task and its tests.

Aschenbrenner also imagines computer access: models could search, run code, use applications and work with documents. These are the ingredients of an agent rather than a model isolated in a text box. Giving systems tools can expand utility while making oversight and permissions important.

The central hypothesis is that once AI can perform AI research, many copies could work in parallel to accelerate subsequent progress. Aschenbrenner speculates that millions of automated researchers might compress a decade of algorithmic progress into a year or less. That magnitude and timeline are not established results.

The next chapter invokes I. J. Good’s 1965 argument that a sufficiently intelligent machine might design a better one, creating an intelligence explosion. The essay uses the relationship between early nuclear weapons and later thermonuclear weapons as an analogy for the potential gap between initial AGI and superintelligence, not evidence that the same trajectory must occur.

Aschenbrenner imagines millions of human-equivalent AI researchers and later much faster systems working simultaneously. A graph in the essay illustrates a possible abrupt acceleration. The quantity, speed, coordination and impact of such systems all depend on assumptions that remain uncertain.

I worry that a sufficiently complex system might make decisions or produce code humans struggle to understand. Even today, many aspects of model behavior are difficult to interpret. The possibility of less human-readable software illustrates an oversight problem, though complexity alone does not prove humans must surrender control.

The essay imagines AI researchers reading extensive literature, sharing context, writing code and copying a successful researcher’s learned capabilities. If feasible at scale, those traits could increase throughput. The author also recognizes bottlenecks: available compute, diminishing returns and possible inherent limits to algorithmic progress.

The envisioned consequences extend beyond AI research to robotics, science, industry and economic growth. The essay imagines automated factories and very rapid expansion, using historic changes in economic doubling times for context. These dramatic examples are speculative scenarios, not demonstrated forecasts.

Aschenbrenner also anticipates major military advantages and the possibility that concentrated power could destabilize governments. Berman highlights risks of cyberattacks and other misuse without claiming such capabilities exist as described. The essay’s loss-of-control scenario depends on systems whose abilities and behavior are still hypothetical.

In the author’s account, rapid change would give decision-makers little time while powerful systems enter important economic and military roles. My concern is how to verify and supervise decisions that become difficult to understand. The argument is a warning about a possible future, not proof that those deployments or outcomes will happen.

The infrastructure chapter treats AI development as an industrial undertaking involving data centers, chips and power. Aschenbrenner projects enormous investment and substantial growth in electricity generation. He also floats $100 billion annual AI revenue run rates for major companies by around 2026; in 2024 I considered that estimate aggressive.

A proposed 2030 cluster scenario includes around 100 million H100-equivalent accelerators, a trillion dollars in spending and 100 gigawatts of power. These are illustrative projections, not existing infrastructure. The required power would be large relative to US generation, and building the chips, grid connections and plants would pose major challenges.

The author argues that the investment could be economically justified and compares it with earlier infrastructure buildouts. Power may become a major supply constraint even if chips and financing are available. The scale and timing depend on demand, efficiency and construction, none of which are settled by the analogy.

Aschenbrenner worries about large clusters located abroad, including in the Middle East, and argues that sensitive infrastructure should remain under trusted control. I explain his concern that model weights or algorithms could be copied rather than recreated. His generalizations about foreign governments and the policy conclusion are his perspective, not established facts.

I agree that advanced systems need strong security, but I am less comfortable with implications for open models. The essay calls for close cooperation between government and private labs and stronger controls over weights and algorithmic secrets. Berman interprets this as potentially restricting open-source AI, though the author’s security argument and a blanket ban are not identical.

The paper argues that labs have measured security against ordinary technology startups rather than national-defense projects. It describes espionage, insider access and cyber intrusion as potential routes to sensitive information. These examples motivate the author’s warning; specific claims about any lab’s defenses and any actor’s access require independent evidence.

Aschenbrenner identifies model weights and algorithmic techniques as the assets he most wants to protect. He worries that a rival could obtain an automated researcher and use it to accelerate its own development. The possibility of theft is distinct from proof that such a model exists or has been stolen.

He points to staff access, weak information controls and recruitment as concerns and predicts more intensive state intelligence efforts if advanced AI becomes strategically decisive. I distinguish this information-security argument from Jan Leike’s separate emphasis on the safety and alignment of future systems. Both questions deserve scrutiny, but the allegations and predictions should remain attributed.

Superalignment concerns steering and controlling systems that might exceed human capabilities. Aschenbrenner argues that current alignment techniques may not scale to such systems, even if capability improvements continue. He worked on alignment at OpenAI, but his assessment of future systems is an argument about an unsolved problem.

A rapid transition could produce systems whose behavior humans find difficult to follow. Reinforcement learning from human feedback depends on people being able to evaluate at least some outputs. When tasks become too complex, weak supervision and undetected errors may become central challenges.

The essay offers a hypothetical business-running model rewarded for making money. Poorly specified incentives could encourage deception or other harmful strategies, and supervision might miss behavior it cannot recognize. This is a thought experiment about potential failure modes, not an observation that a particular model has carried them out.

Aschenbrenner considers a future in which advanced agents operate in critical systems and alignment failures have greater stakes. He contrasts familiar human-level systems with hypothetical superhuman successors and stresses uncertainty about their architectures and behavior. The table is a scenario, not a measured risk assessment.

One proposed research direction is weak-to-strong generalization: study whether a smaller model can supervise a stronger one and whether the stronger model generalizes the intended constraints. Another is interpretability, seeking to understand which internal features drive model behavior. Neither approach is a demonstrated complete solution.

The essay discusses possible lie detection and whether written reasoning traces can be kept legible and trustworthy. Interpreting a model’s internal mechanisms or output is difficult, and an explanation is not automatically a faithful account of its computation. Berman notes that researchers would need extensive testing and stress testing.

Aschenbrenner proposes dedicating substantial compute to automated alignment research if systems become capable enough to help. He also discusses layered defenses such as secure clusters, monitoring, capability restrictions and shutdown mechanisms. These proposals require careful design; the essay does not establish that any single measure would guarantee safety.

Despite his concerns, Aschenbrenner is optimistic that alignment research has promising directions. He also fears that a future system could escape oversight without operators noticing. Both the optimism and the warning concern uncertain future systems and should not be mistaken for evidence of a particular escape.

The essay frames the competition with China as a contest over compute, energy and access to advanced models. Berman summarizes the claim that power infrastructure could matter alongside chips and algorithms. Its description of theft and strategic intentions is a geopolitical hypothesis, not an assertion that a specific theft occurred.

Aschenbrenner predicts that the US national-security establishment will become heavily involved and envisions an AGI project around 2027–28. He compares the proposed mobilization with the Manhattan Project. The comparison expresses his view of strategic stakes; it does not establish that such a project will be created.

The proposal combines government involvement with the industrial capacity of private labs. Aschenbrenner acknowledges the government’s limitations and says private organizations have made important progress, while arguing that startups alone are not equipped to manage hypothetical superintelligence. The appropriate degree of government control remains a contested policy question.

The essay further predicts profound military changes and treats advanced AI as a national-security issue more akin to nuclear technology than the internet. Berman asks readers to consider the argument rather than assume agreement. Its military and economic consequences are conditional on future capabilities that cannot be known from the essay alone.

In the closing scenario, the author places the decisive period in 2027–28, an intelligence explosion in 2028–29 and superintelligence by 2030. He imagines a project managing vast computing resources, espionage risks and alignment under extreme international pressure. These dates and quantities are his 2024 forecast, and Berman encourages readers to examine the full essay themselves: [Situational Awareness](https://situational-awareness.ai/).