2,700 AI Authors, No Consensus: Even Odds of HLMI in 2047, 5% Extinction

A year before this survey, researchers publishing in top AI venues put a 50% chance of high-level machine intelligence in 2060. Run the same questionnaire in 2023 and that date is 2047. Thirteen years of expected delay vanished in twelve months. Katja Grace, cofounder and lead researcher at AI Impacts, brought the full results onto Nathan Labenz’s Cognitive Revolution. The sampling frame was everyone who published in 2022 in six elite venues—five conferences and one journal—about 20,000 people. More than 2,700 answered.

Labenz calls it the most comprehensive measurement yet of elite machine-learning researchers’ expectations. Then he undercuts the temptation to treat it as a prophecy. The biggest takeaway, he says in the open, is that there is no consensus. Most individuals express high uncertainty on the high-level questions. Only small minorities confidently predict a very positive or very negative future. Researchers do not dismiss existential risk: median estimates sit at a 5 to 10% chance that AI could lead to human extinction. Timelines for key capabilities have shortened since the survey a year earlier. There is strong agreement that AI safety should be prioritized more than it is today, perhaps because no major interpretability breakthrough is expected soon.

Grace has spent about ten years trying to figure out “what’s up with the future of AI,” sometimes skeptical that extinction is likely, always treating the question as decision-relevant. AI Impacts exists to answer the high-level prompts—when, what kind, extinction or not, what other social changes—and a lot of sub-questions that feed them, including case studies of whether other technologies have ever been slowed for ethical or risk reasons. Labenz’s charge to doomers and e/acc alike is to update their p(doom) toward these community estimates. A 5 to 10% chance of extinction is obviously critical. The shape of the other 90% of outcomes is also worth worrying about.

Labenz asks whether any other field has had to survey its own experts this way. Grace’s reference points are older, smaller AI surveys, some going back to the 1970s, each with a different definition of human-level AI, which makes them hard to compare. Climate science has something analogous. Manhattan Project scientists at Los Alamos bet on the first nuclear test’s yield; nobody ran an industry-wide census of expectations. Methodologically, she is not leaning hard on those ancestors. This is the third survey she has been involved with, scaled up.

The 2023 wave wrote to everyone who published in those six 2022 venues. Earlier rounds had used only NeurIPS and ICML; they expanded for a wider topical range and more of the top people. Names came off papers, emails off those papers and others on the internet. They found contact details for more than 90%. A pilot compared no incentive with $50 each; the paid group converted much better, so everyone got the card. About 2,700 of roughly 20,000 responded—around 15%. The 2016 round was small; 2022 had several hundred, on the order of a national political poll; this wave is several times larger.

The instrument is a 15-minute branching survey. Nobody sees every question. Some anchors go to all respondents; the rest is a randomly chosen path. Answers are mostly structured numbers and choices, plus a few short open fields—a late-to-automate occupation, how you interpreted the last item (about 10% of people after some questions), what confusions people have about AI risk. Grace has not yet mined those interpretation probes. Labenz notes that language models will make open-ended survey analysis much easier; for this project they needed quantitative estimates, so the design stays sparse and comparable.

Grace treats the two headline definitions as closely related; respondents do not. High-level machine intelligence, or HLMI, is when unaided machines—any collection of AIs, not necessarily one system—can accomplish every task better and more cheaply than human workers, ignoring advantages that are intrinsically about being human, such as sitting on a jury. Full automation of labor, or FAOL, swaps tasks for occupations, with the same unaided, better-and-cheaper, feasibility-not-adoption wording. Logically, if every task is automatable, occupations as bundles of tasks should be too. The dates respondents give for FAOL are much later.

She offers two guesses. People may hear occupation automation as actual adoption and HLMI as technical feasibility. And the FAOL path walks you through four specific jobs, then a particularly late occupation, then “everything,” which may push the final date out. Nobody answers both items, so you cannot confront a single person with the contradiction. A small, annoying difference is kept for year-to-year consistency: only the HLMI prompt tells people to assume scientific progress continues without a giant impediment.

Eliciting “when” is its own fork. They want a distribution over time from each person. Drawing a full curve is too heavy, so they collect three year–probability pairs and fit a gamma distribution, then average those curves into a community aggregate. Half the sample is given years and asked for probabilities; half is given probabilities (including a 90% year) and asked for dates. Since 2016, including on Mechanical Turk, the two framings disagree systematically: years-then-probabilities produce later distributions. Grace’s guess is that wild events invite low probabilities if you can keep saying “low,” whereas being forced to name a 90% year, once you admit the event happens, makes it awkward to shove the date to eternity. They refuse to pick a winner. They keep both, convert, and average.

The cost of asking similar questions four ways is selective reporting. Grace is in a “constant battle” to present HLMI and FAOL together. HLMI alone tells a story of timelines crashing. Put every answer on the table and the signal is more conservative: people still expect this in quite a long time. Her mitigation is to put all the data and figures online. The deeper caveat is blunter. Forecasting is hard. These people are not forecasting experts. They answer in about 15 minutes about the long-term consequences of a technology they have not seen. She does not expect accuracy. She does think the best available guesses are valuable because decisions will be made anyway, and that “a lot of these AI researchers put some probability on various things” is itself information: those things are not ruled out.

The 39 narrow-task forecasts share a shape Labenz has seen on Metaculus: a lump in the near-to-medium term and a long tail. Medians sit earlier than means because the past is closed and the future is not. The headline figure of 25th-to-75th percentile ranges often stretches from something like 10 years to 100. Grace’s correction matters: those bars are averages of individual distributions, not a map of interpersonal disagreement. Each person is often very uncertain; people also disagree with each other. Figure 3 shows the aggregate in a bright line and a random subset of individual curves behind it. Steep early curves are confident short timelines. Slow rises are high uncertainty. A few oddballs spike around 2120–2130 on FAOL and may have misunderstood the question.

From 2022 to 2023, HLMI’s 50% date dropped by about a decade; FAOL dropped by four or five decades. Between 2016 and 2022, HLMI had moved only about a year. Narrow tasks on average also came in by about a year—the predicted calendar year, not merely “one less year of waiting.” Thirty-five of 39 tasks have a 50% chance inside ten years. The four that do not: replicate a leading ML conference paper (12 years), physically install electrical wiring in a new house from a one-sentence brief (17), research and write an ML paper from scratch (19), prove theorems publishable in top math journals (22), and solve a Millennium Prize problem (27). Each prompt is the same high bar: give the system the information you would give a human, then no more input.

That is not how people use models now. Labenz splits current practice into ad hoc copilot work—edit this paragraph—and scaled jobs such as personalizing 100,000 emails, where you spend the first 10% of effort on scaffolding, context, and validation. The survey asks for teammate-style delegation: the AI owns the task breakdown. Grace’s picture of autonomy is fractal. First a system can do a second of useful work before you redirect it; then it can stitch a few steps; ChatGPT already chooses words, tone, and a list of points in a letter; a human still decides who quality-checks it and who it goes to. The “easiest” item, writing readable Python for algorithms like quicksort, is already something ChatGPT can do, yet the aggregate curve still puts substantial mass after 2029. Labenz thought a quarter of respondents had not heard of ChatGPT. Grace says that is not how averaged distributions work. People may treat a survey item as evidence the task is unsolved; the instrument has no “already exists” option, kept off since 2016 because verifying completion is surprisingly hard. ChatGPT is often great and then terrible at counting. Some respondents may hold a philosophical line that the model is imitating quicksort, not doing it.

Even pulled forward, the community is more conservative than lab leadership. Sam Altman has told YC founders to build with AGI in mind. A credibly sourced Anthropic pitch deck described a 2025–26 lead that nobody could catch, because current models would train the next ones. Shane Legg has carried a median around 2028–29 for something like 15 years. The survey’s HLMI numbers are a 10% chance by 2027 and 50% in 2047. Grace’s explanation is selection: people who think AI is soon work on AGI. That Legg’s number has barely moved is evidence the optimism is a long-standing stance, not a reaction to last quarter’s demos. All cleaned, anonymized data are public. Labenz could filter out anyone who thinks quicksort is 15 years away and rerun. Grace encourages that, with a warning: you would mix people who misunderstood, people who mis-clicked, and people with a complicated philosophy of what “doing quicksort” means. Respondents also self-rated how much they had thought about each topic. People who thought more are more concerned—and that, too, is tangled with selection.

On future system traits, something close to consensus appears. More than 80% think state-of-the-art systems in 20 years will find unexpected ways to achieve goals, and will be able to talk like a human expert on most topics. Just under 70% think they will frequently behave in ways that surprise humans. Grace immediately qualifies the language. “Unexpected ways to achieve goals” can sound like ordinary delegation—you hired someone and they found a method—or like a norm violation that should have had more barriers. She wishes those items had been sharper.

The long-term value question forces every respondent, as a gateway item, to spread 100% across five buckets: extremely good, good, neutral, bad, extremely bad, as a result of HLMI. Figure 10 sorts people as vertical pixels. Roughly the middle two-thirds put non-trivial weight in all five buckets. About a sixth on each end look like optimists and pessimists, and even most of the “extremely bad” cluster still keeps a lot of mass off that bucket; some of it is extremely good. Labenz spots a bimodal band and a 20/80 band. Grace likes the chart because it shows how unpolarized the thing is. Online it can feel like doomers versus everyone else. In this sample it is a huge quiet middle.

Labenz’s headline from the picture is radical uncertainty as the only real consensus. Grace takes the next step. From an action perspective, the question is whether this is a risk worth paying attention to. Uncertainty that includes a non-negligible extremely bad tail is a consensus for yes. You would need to be pretty confident to say it is not an issue. That is the move from “we don’t know” to “you do not get to shrug.”

Coverage of the survey often led with a slight majority: 51% believe there is at least a 10% chance AI could lead to human extinction or similarly severe disempowerment. Grace corrects in the room. They asked three similar extinction questions. Straight-up extinction or permanent severe disempowerment: median 5%. The same, as a result of inability to control future advanced AI systems: median 10%. Within the next 100 years: median 5%. Means were quite a bit higher. The 51% line cherry-picks the highest of those medians.

Labenz smells a conjunction fallacy: adding “inability to control” should not double the probability. Grace is not sure that is the mechanism. People are randomized across the three wordings, so they are not the same respondents. In practice almost nobody types a number between 5 and 10; they round themselves. A pile at 5% and a pile at 10% means the median flips when the middle person switches piles. Last year, 48% said 10% for the extremely-bad-style item, so the median stayed at 5%; two extra points would have made it 10%. The underlying distribution did not “double.”

Demis Hassabis, on the New York Times Hard Fork podcast, declined a p(doom) number: we do not even know what process we are in, so a figure is meaningless. Labenz often gives a 5–95% or 10–90% range for the same reason. Grace agrees it is unknown and still wants the number. Probabilities, she says, are what unknown is for. Even a guessed figure lets you compare it to your other guesses and make consistent decisions over time. She practices on lots of topics and then checks her calibration. Labenz’s complementary use is not “how accurate is my estimate today” but “what can we shift the true probability to.” The image he wants to kill is a roulette wheel already built, waiting to be spun. We are still building the wheel. Grace’s preferred framing is conditional: if humans build this particular system, I think it goes badly, so build something else. The number becomes an input to a path, not a fate.

Eighty percent of respondents consider it unlikely or very unlikely that users will be able to understand the true reasons behind AI decisions by 2028. In parallel, there is strong agreement that AI safety should be prioritized more than it is today; almost nobody wants less. Speed splits. Only 5% want progress much slower; 30% somewhat slower; 27% current; 23% somewhat faster; 15.6% much faster. Grace’s caveat on the middle option is that it is relative to today’s speed, and today’s speed is itself accelerating. Many people may mean “this trajectory is fine” after not thinking long. She would like related questions before anyone leans on that item.

Scenarios of concern, as opposed to forecasts of what will happen, produce more agreement. More than 80% say spreading false information—deepfakes is the example—is worth substantial or extreme concern. Manipulate large-scale public opinion, authoritarian control of populations, worse economic inequality, dangerous groups getting powerful tools such as engineered viruses, and biased systems making unjust situations worse all clear 60% at substantial or extreme concern, with fewer than 10% saying no concern. Grace’s gloss: the questions ask what could happen enough to worry, not what will. That this is the only really clear consensus, she and Labenz agree, supports the hypothesis that nobody knows what will happen. None of the tails are dismissed.

Her own high-level summary matches his. People do not dismiss existential risk; the point estimate is 5 to 10%. Timing estimates are wide, coming in, and still mostly “more likely than not inside ten years” for the narrow tasks. No interpretability breakthrough is expected. More safety work is wanted. Faster versus slower is disagreement or confusion. As a community they do not have a confident view of what happens, how long it takes, whether it is good or bad, whether systems will be controlled, or even whether we will be here. Those are live questions with non-trivial weight on all the options.

Grace is reasonably likely to run the survey again, and the most natural move is to run it identically so the time series holds. New questions—concern scenarios, future traits—were costly. Cramming more into a short randomized instrument means giving each item to fewer people. Policy items, she thinks, might want their own survey. Labenz makes a formal request from people who keep asking him. The last year poured money into the application layer: databases, APIs, memory, task planning, the whole “cognitive architectures for language agents” stack. That scaffolding is a bet that a core-model upgrade will drop a much stronger engine into a chassis already built to paper over today’s weaknesses. If many half-working systems flip to working at once, semi-autonomous agents meet each other in the real world, and nobody has practiced that scene. Default worldview right now is “I apply AI; the rest of the world stays put.” That assumption may fail on a single upgrade.

Grace’s analogy is that most technologies diffuse slowly after a breakthrough because you still have to build the complementary pieces. This time the pieces are being built in advance. Why didn’t GPT-3 to GPT-4 look like that? Labenz’s answer: there was not much to build around yet. ChatGPT and GPT-3.5 arrived together; GPT-4 followed in about three and a half months. The application gold rush is a last-12-to-14-month phenomenon. He wants “scaling laws for scaffolding” on the next questionnaire. She takes the request and still leans toward a frozen instrument plus a separate policy survey.

Chip-export controls are the other candidate add-on. Labenz has heard smart people swear the bans will open a multi-year US–China gap, and he has tried Baidu’s Ernie 4.0 head-to-head with GPT-4 on a few queries and not found it dramatically inferior. If elite researchers—many of them Chinese speakers at those conferences—think the policy will not even work, that might be information for an already escalatory politics. Grace hesitates. AI Impacts usually assumes that more people knowing the situation in more detail makes things better. Near adversarial topics, a result helps one side. She does not want to contribute to an arms-race primer, and she would rather ask chip experts than ML authors. She closes without a forecast: amid quite a lot of uncertainty, the chance that things stay similar and AI does not drastically change people’s lives looks quite low. Public opinion will shape policy that can change the rest of the future. Labenz adds that Silicon Valley under-counts how many people feel like a family friend of his: “It creeps me out. I don’t want anything to do with it.” Full stop. The raw answers are public. Calibrating against 2,700 of them beats refreshing a feed.