Deep Research, AI Takeoff and the Limits of Early Reactions

Matthew Berman surveys February 2025 reactions to OpenAI’s research agent and argues that reasoning and tools could accelerate AI progress.

SASam AltmanSpeaker · OpenAI
Channel
Matthew Berman
Published
Feb 4, 2025
Video
13:04
Read
4 min
Slides
1
Original
English
Open on YouTube
Scroll to read and the video shrinks to the bottom-right corner. Captions follow your reading language.
Reading languageAvailable in 2 languages
You're reading the original English.
SUMMARY

In 30 seconds

Matthew Berman reviews early reactions to OpenAI’s deep research, from research reports and benchmark results to anecdotes involving cancer cases. He argues that the combination of reasoning and agent tools points toward a rapid AI takeoff. That takeoff and recursive self-improvement are predictions, not established by the examples. Benchmark scores and individual medical anecdotes have narrower scope, and OpenAI acknowledges that the tool can make factual and sourcing errors.

  1. 1A takeoff hypothesisBerman links recent advances to recursive self-improvement; the cited examples do not establish an autonomous self-improvement cycle.
  2. 2Reasoning plus toolsBerman highlights Ethan Mollick’s argument that reasoners and agents can combine to research and synthesize information.
  3. 3Benchmark limitsA GPQA chart shows results on a defined science question set, not comprehensive professional research ability.
  4. 4Medical anecdotes need validationEarly users praise particular cancer-related reports, but those cases are not a clinical study and should not replace clinician review.
  5. 5Google’s earlier productGoogle staff point to Gemini’s existing Deep Research product; a shared name alone does not establish copying.

Takeaways

  1. Deep research combines multi-step reasoning with web research and other tools to produce reports.
  2. Benchmark progress measures performance on particular tests and does not establish general expert equivalence or recursive self-improvement.
  3. Claims of an imminent intelligence takeoff are forecasts rather than conclusions proven by the video’s examples.
  4. Favorable accounts of cancer-related reports are anecdotes, not clinical validation; treatment decisions require qualified clinical review.
  5. Google had an earlier product with the same Deep Research name, but the reactions do not prove copying.
FULL TEXT

Read the edited text by chapter

Transcript only
01

Reactions and the takeoff argument

Berman surveys early reactions and predicts fast AI progress. The discussed examples do not demonstrate an autonomous recursive improvement cycle.

In February 2025 I look at reactions to OpenAI’s newly released deep research. Researchers and commentators describe its ability to search and synthesize sources, while some see it as evidence of rapidly accelerating AI progress. These reactions range from personal experiences to predictions, and they do not by themselves establish a general intelligence breakthrough.

I cite Stability AI founder Emad Mostaque’s prediction that machines will soon perform most digital knowledge work faster, better and more cheaply. He describes an intelligence takeoff. That is a forecast about future capability and economic impact, not a measured result from the deep research launch.

I connect this view to Leopold Aschenbrenner’s scenario of recursive self-improvement: an AI discovers useful knowledge, applies it to improve itself and repeats the process. I argue that we are approaching such a scenario, but the examples discussed here do not demonstrate an autonomous, sustained cycle of AI improving its own successor.

I mention an example discussed by Simon Willison in which a model was prompted to suggest a code optimization that reportedly made a specific operation twice as fast. A human-directed optimization of one workload is interesting, but it is not proof that a model has autonomously improved its own overall capabilities.

Technology columnist Kevin Roose compares reported scores on Humanity’s Last Exam, with deep research reaching 26.6% on the cited configuration shortly after a lower leading result. These are benchmark-specific results with particular tools and testing conditions, not a score for all human knowledge or a reliable forecast of a takeoff.

02

Reasoning models and research agents

He discusses reinforcement learning and Ethan Mollick’s comparison of research outputs. He explains how models and tools can work together.

I credit DeepSeek’s R1 paper with making reasoning-model techniques more visible, and highlight Nathan Lambert’s observation that OpenAI trained deep research using reinforcement learning on browsing and reasoning tasks. I see a combination of reasoning and tools as important, although describing it as a proven recipe for AGI goes beyond what these examples establish.

Ethan Mollick’s essay [“The End of Search, the Beginning of Research”](https://www.oneusefulthing.org/p/the-end-of-search-the-beginning-of) argues that reasoning systems and autonomous research agents are converging. In his examples, he judges the resulting reports to be of different levels of quality. Those are his evaluations of particular outputs, not a guarantee that every report matches expert work.

I describe the model as the reasoning component and tools as the way an agent can act: it can search the web, write and execute code, or consult other data sources. The combination can produce a synthesized report, but the quality still depends on the sources, the task and the model’s interpretation.

03

Benchmarks and medical examples

Berman interprets a GPQA chart and recounts cancer-research anecdotes. Both kinds of evidence have important limits.

The video shows a GPQA Diamond chart comparing model scores over time with human expert reference lines. The chart illustrates progress on a defined set of difficult science questions. It does not establish that a model is a general-purpose PhD, can independently conduct all research, or has exceeded experts across their work.

GPQA Diamond performance over time
SLIDE 1 / 1GPQA Diamond performance over timeThe graph compares scores on a defined science question set with human reference lines; it is not a general measure of expert research ability.

I interpret the graph as evidence for a hard takeoff and imagine many agents discovering AI research and improving themselves in parallel. That is my extrapolation. A rising benchmark score alone cannot establish recursive self-improvement, its rate or whether a hard takeoff is occurring.

I mention Sam Altman’s suggestion that inexpensive compute could produce substantially more value for a user. That is an illustration of possible value rather than a controlled economic measurement. Actual value and cost will differ with the task, output quality and review required.

Biomedical scientist Derya Unutmaz describes favorable early experiences with deep research and says two reports on cancer cases were impressive. This is a small anecdote and his professional assessment, not a clinical validation study or proof of reliable medical advice.

I also recount an OpenAI employee’s account of using deep research to investigate a breast-cancer treatment question for his wife and checking the information independently. The report is an example of information gathering, not evidence that the tool should decide treatment. Patients should discuss clinical choices with qualified clinicians and verify cited sources.

04

Economic claims and industry responses

He discusses Sam Altman’s estimates, Google’s earlier product and an anticipated announcement, all in the context of February 2025.

In a launch clip, Altman estimates that the system can perform a single-digit percentage of economically valuable tasks. He explicitly presents that as his estimate. I discuss its possible economic scale, but the percentage is not established by a comprehensive measurement of work across the economy.

Altman distinguishes synthesizing existing information from a future system that invents new knowledge. I think the direction could be transformative; the statement is a prediction, not proof that the product has achieved autonomous scientific discovery or recursive improvement.

Google staff react to the launch by pointing out that Gemini had already offered a product called Deep Research. I find the name overlap striking. Their reactions support the observation that a similarly named product existed earlier; they do not establish that OpenAI copied Google’s implementation.

Finally, I note Altman’s contemporaneous comment that deep research was not the additional announcement he had teased for o3-mini. That was a February 2025 snapshot of product announcements, not a statement about current release plans.

OpenAI’s own [launch material](https://cdn.openai.com/API/docs/deep_research_blog.pdf) notes that deep research can hallucinate, make incorrect inferences and struggle to distinguish reliable sources from rumors. Those limits matter especially for medical and other high-stakes research, where claims and citations need independent review.

More from this channel

12 slides34:21EN
Matthew Berman· Oct 27, 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyEnterprise AI+2
0:26EN
Matthew Berman· Oct 25, 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
2 slides0:36EN
Matthew Berman· Oct 25, 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvalsCoding agentsOpen-weight models
2 slides0:21EN
Matthew Berman· Oct 24, 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.BenchmarksEvalsGenerative media
6 slides25:50EN
Matthew Berman· Oct 24, 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 slides32:20EN
Matthew Berman· Oct 23, 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1