Li Guangmi on Pretraining, Agents, and a Roadmap to AGI

Zhang Xiaojun asks Guangmi to review large-model developments in the first quarter of 2025. He hopes to distinguish the main path toward AGI from the developments worth watching along the way.

Guangmi’s contrarian view is that pretraining base models has substantial room to advance and the next generation could improve significantly. Slower progress at one leading company, he argues, does not prove the whole approach is exhausted.

He argues that pretraining determines a model’s intrinsic ceiling, while post-training and reinforcement learning mainly elicit or strengthen capabilities. He had previously considered pretraining potentially exhausted, and acknowledges there is no definitive answer about whether reasoning models can raise the ceiling.

Looking from outside, he offers strategic and organizational explanations for OpenAI’s relative emphasis on pretraining and reasoning models: reasoning may improve benchmark scores faster, ChatGPT growth draws management attention, and staff changes can affect teams.

Guangmi suggests feeding synthetic reasoning data produced through reinforcement learning back into pretraining, or exploring RL during pretraining itself. Integrating training with inference sampling, he adds, presents infrastructure challenges.

He expects new capabilities, especially the use of computer tools. He sees Manus as an example of a product exposing a model’s tool-use abilities, and argues that product teams must find effective ways to express those capabilities.

On the AGI timetable, Guangmi expresses exceptionally high confidence in early 2025 that it could arrive within two years. This is his prediction at the time, concerning agents performing many human actions in digital environments.

His reason is that coding supplies a general digital environment: code can express tasks, execute actions, and receive feedback. He likens code to a model’s hands and uses AlphaGo’s board to illustrate why an environment matters for learning.

A useful environment, he says, is dynamic and actionable, allowing a model to observe the results of its actions. Coding, games, and science have such properties, while tools and scaffolding can help a model understand and act in them.

Comparing coding capabilities, he expresses confidence in Anthropic’s base-model and tool-use approach while acknowledging that OpenAI could later show stronger capabilities. Training data, methods, and organizational priorities can all matter, he says.

Zhang asks whether coding is a technical path or the final product. Guangmi compares it to a search or recommendation engine: the underlying capability matters, but an interface for ordinary users need not expose the programming details.

From the outside, he compares OpenAI and Anthropic: in his view the former emphasized reasoning models and consumer products, while the latter stressed base models, coding, and agents. These are his assessments, not confirmation of either company’s internal plans.

He attributes differences in direction to research capabilities, leadership choices, and organization. A model company’s strategy, he argues, depends on what its teams can actually do and how it allocates talent, not just on a stated ambition.

He sees a disagreement over whether traffic or intelligence should take priority. He worries that emphasizing consumer-platform growth too early can divert research resources, and argues for keeping model intelligence central.

Discussing competition between technical approaches, Guangmi argues that public results can speed up catching up, although accumulated research and execution speed still matter. His comparison rests on public performance and his own observations at the time.

Zhang asks how many people are needed to push the frontier. Guangmi stresses individual contributions, imagination, and execution in small teams; exploratory research, he argues, does not necessarily improve linearly with headcount.

He likens AI development to climbing a mountain: ChatGPT is one stop near the foot, followed by possible peaks in coding, coding agents, general agents, science, and robotics. This sequence is his roadmap, not a guaranteed schedule.

In his framework, intelligence itself is the biggest application. Products can create environments that take advantage of model capabilities as those improve, and he sees agents as a stronger expression of this shift than single-turn chat.

On robotics, he argues that the hard problems extend beyond training a model to physical-world data, cost, and generalization. He suggests asking which tasks can first be done virtually and which genuinely require a physical body.

On science, Guangmi hopes models will help researchers form hypotheses and explore fields such as biology, while treating his proposed timing as a personal forecast. Feedback and experimental environments, he says, shape the pace of progress.

He notes models for objects such as proteins and RNA and thinks they may help with problems that previously required repeated expert trial and error. He is learning more science himself to understand where models could contribute.

If virtual agents can do much of the work, are physical robots still needed? Guangmi says the answer depends on the task and economics; physical form should not be equated directly with the value of intelligence.

Asked what intelligence is, Guangmi interprets human development through survival, exploration, and automation. Exploration accumulates knowledge and institutions, while automation passes on and standardizes useful practices.

Using search and deep research as examples, he discusses a shift from one answer to multi-step exploration. An agent can repeatedly ask questions, retrieve information, and synthesize results, although the workload and computation also grow.

Guangmi argues that progress should not be judged by a single benchmark alone, but also by whether a system can perform more complex actions, explore unknowns, and reveal new insights. He is optimistic about the pace of those capabilities.

He sees agents as a new product category spanning information research, coding, and action in an environment. Applications need different tools, but all must connect model capabilities with executable tasks.

Guangmi identifies three key agent capabilities: long-context reasoning, tool use, and instruction following. Beyond the model, teams must provide an environment such as a computer, virtual machine, or browser where an agent can act and observe results.

Long context matters because a system must connect a task, its history, and multiple actions. He cautions that fitting in more information does not automatically mean the system can understand and use it effectively.

Guangmi compares long context to power lines: once intelligence is available, it must reach longer tasks at acceptable cost. He also discusses compression and memory so a system need not always reread its entire history.

As a possible future paradigm, he proposes online learning: can a model learn continuously from interactions with users and its environment and update some parameters? He presents it as an open research direction rather than a solved problem.

One difficulty is deciding when to update memory or weights and how to define rewards and goals. Human goals are complex, and a single metric may not capture good behavior; the more autonomous an agent becomes, the more important these questions are.

Guangmi argues that compute demand could continue growing if pretraining, inference, and agent tasks all expand. He connects opportunities for hardware and infrastructure companies to that assumption rather than treating growth as certain.

Discussing model pricing, Zhang asks whether monthly subscriptions can cover increasing usage costs. Guangmi says software subscription conventions do not map neatly onto models that consume compute with each use.

He speculates that lasting advantages for model companies could come from infrastructure platforms or operating-system-like ecosystems rather than merely releasing a model. Both paths depend on sustained participation by developers, applications, and users.

When investing in applications, he values teams that understand research progress and can build quickly. Models change rapidly, so applications must keep learning and adapting rather than building features solely around one generation’s weaknesses.

Guangmi compares a product team to a container for intelligence. Execution and iteration speed matter, he says, while products can also face competition as model companies expand their own features.

Discussing personal agents, he imagines a system with long-term memory and continuous learning that could better understand a user. The scenario involves highly sensitive personal information and describes a hypothetical capability, not existing authorization to access it.

On the model-company landscape, Guangmi discusses research, products, and organizations at OpenAI, Anthropic, Google, and others. These are subjective judgments from early 2025, not a substitute for verifying each company’s actual progress.

On GPT products and the timing of next-generation models, he argues that releases depend on technical progress as well as product and organizational choices. Outsiders, he acknowledges, cannot see the full internal picture.

He continues comparing cooperation and competition among model companies, arguing that shared protocols or infrastructure do not eliminate competition in research and products. His explanations of team motives remain speculation.

For new model teams, Guangmi values the mix of talent and product direction, while distinguishing replication of existing capabilities from new research breakthroughs. His views about particular teams’ future choices are outside inferences.

Comparing Manus and Perplexity, he sees strong execution in both teams. Manus especially depends on underlying model capability: stronger models can give an agent product more to work with, although competition is also more intense.

Zhang asks where deep research will go. Guangmi thinks it could integrate browser operation and coding and develop toward a general agent, but is unsure whether the result will resemble better search or specialized professional assistants.

On Devin and Cursor, he says coding-agent value changes with model capability and existing interfaces may not be the final form. Product teams need to continue exploring interaction and training approaches.

Guangmi recognizes DeepSeek’s infrastructure experience while hoping to see further fundamental research breakthroughs. Among people advancing AGI, he especially praises Ilya Sutskever and other researchers for their contributions to direction and architecture.

Zhang asks about Sam Altman’s contribution. Guangmi credits his imagination and ability to mobilize resources with bringing more compute to researchers; his reading of Altman’s public statements remains personal opinion.

In a hypothetical portfolio with freely available access to deals, Guangmi favors foundation-model companies and considers Anthropic, ByteDance, OpenAI, new research teams, and application companies. His allocations express his personal judgments about teams, valuations, and research directions at the time.

He argues that existing revenue can fund long-term research, while large organizations and small teams face different constraints. He would allocate more to Anthropic because he has greater confidence in its team, base models, and coding-agent direction.

Guangmi imagines intelligence technology giving rise to the next generation of search, operating systems, office tools, and science companies. His proposed company sizes and timelines are highly uncertain forecasts, not investment conclusions.

He considers organization and culture important competitive assets alongside compute. With limited resources, a team must choose where to bet and what to give up; talent density and judgment in key roles matter more than simply increasing headcount.

How can one judge whether an organization truly centers AGI? He suggests looking at whether research is valued, whether strong researchers want to join, whether leaders understand the technology, and which contributions are rewarded.

Looking at global competition, Guangmi favors OpenAI and Anthropic at the time and expects multiple Chinese teams to participate. He suggests open models could cross geographic boundaries, but these are forward-looking judgments.

He argues that large-scale training and reinforcement learning require strong infrastructure: combining sampling with training creates debugging and efficiency challenges. He values Chinese teams’ talent and engineering experience while recognizing that catching up depends on the path taken.

He hopes large companies will support long-term fundamental research, build full infrastructure, and develop talent rather than focus solely on growth in existing products. Capital, willingness to take risks, infrastructure, and culture together support innovation in his view.

His confidence in Chinese teams achieving AGI comes with conditions: existing pretraining and reinforcement-learning approaches must prove sufficient, and required compute must be attainable. His GPU figures are rough guesses.

Comparing innovation in Silicon Valley and China, he says abundant capital and risk-taking support exploration from scratch, while scaling is also valuable. He hopes Chinese companies can combine resources with research culture to produce more original breakthroughs.

He believes DeepSeek’s emergence renewed attention to model intelligence. Asked whether product-growth spending should stop, Guangmi says it need not, but worries that growth can consume too much organizational attention.

On globalization, he imagines a Chinese team building a world-leading product and argues that technical and product strength can help a company cross geopolitical barriers. This is his optimistic view of a complex political and market problem, not a guarantee.

Asked about the ceiling for Chinese investors in Silicon Valley, Guangmi says networking alone is insufficient and cross-cultural trust requires real professional contribution. He encourages deep industry work and value creation, and sees entrepreneurship as another avenue.

He says his belief in AGI comes from the importance he places on scientific progress: new capabilities can change product and business narratives. He links changing demand to technological supply without claiming a particular scientific breakthrough has already happened.

Looking ahead one year, he expects base models to improve and suggests agents could change search and advertising. He also acknowledges that changes may arrive from unexpected directions and linear extrapolation is only a guide.

Finally, Guangmi describes an entrepreneurial quality he admires: continually recognizing and seizing new opportunities. The two close by reflecting on each other’s programs and future discussions.