Wu Yi on Operator: Reinforcement Learning, Web Interaction and Agents

Zhang Xiaojun invites reinforcement-learning researcher Wu Yi to discuss Operator, released in early 2025. They distinguish product demonstrations, known technical information and speculation about the future.

Wu introduces his research at Tsinghua’s Institute for Interdisciplinary Information Sciences and his background in reinforcement learning, the perspective from which he considers web-using agents.

He focuses on demonstration details: the system must handle pop-ups, observe page changes and continue a task after acting rather than merely produce an answer.

He sees continuous multimodal interaction as a significant change: the model observes images, understands instructions, acts and adjusts to feedback. A demonstration does not prove reliability across all settings.

The guest speculates that stronger internal systems may exist and discusses generalization. These judgments draw on public demonstrations and experience, not confirmation of internal models.

He compares backtracking in reasoning models with sequential web decisions: a system may need to check its state and next step repeatedly rather than generate an entire action sequence at once.

They discuss why the product had a separate entry point. Wu suggests a new interaction mode and existing chat habits may call for different design; this is speculation.

He outlines three research ingredients: a capable foundation model, high-quality interaction data and reinforcement-learning infrastructure that supports interaction at scale.

Interaction data includes not only tasks and answers but observed states, actions and subsequent feedback. Collecting, filtering and evaluating those trajectories matters.

They discuss the Computer-Using Agent. Wu describes a loop in which a model sees a screen, chooses an action, receives the new screen state and decides what to do next.

He says foundation-model capabilities and training for interaction can work together. The specific training data and internal implementation are not fully public, so his explanation is not a complete technical disclosure.

He discusses the models, data and training infrastructure other teams would need for similar capabilities. Resource requirements depend on methods and goals and cannot be inferred directly from a demo.

Wu sees the combination of multimodal models and reinforcement learning as important to agent development at the time. Technical progress and a reliable, broadly useful product are different stages.

He recalls early OpenAI exploration of web agents and says earlier reinforcement-learning work was constrained by the models and data available then. This reflects his experience and interpretation.

The interview uses a five-stage framework covering chat, reasoning, agents, innovation and organizations. The guest treats it as a way to discuss development, not a validated timetable.

He distinguishes reactive dialogue from systems that plan, then discusses agents interacting with external environments. Later stages involve more complex creation and coordination.

He says action-and-feedback loops let systems discover problems and revise plans during execution, while errors, permissions and reliability remain challenges in long tasks.

As tasks become more complex, how people set goals, supervise execution and take over key steps may change. This is a directional discussion, not a claim that supervision can be omitted.

The host asks about the significance of moving from abstract reasoning to environmental interaction. Wu says real tasks expose different uncertainties and require plans to connect with actual observations.

On autonomy and user instructions, he observes points in the demos that require human intervention. When to pause for a person to handle payments or sensitive information is a product-design question.

He sees language, vision and action as a combination of foundation-model capability and interaction training: a system must interpret the screen and translate intent into actions.

Asked about multi-agent collaboration, Wu says a task does not necessarily require several models, though new interactions may emerge among services and agents.

He imagines websites adding interfaces for agents and potentially enabling transactions between agents and services. These are future scenarios, not descriptions of existing features.

On exploration and correction, he says example data can guide initial behavior before reinforcement learning improves it. The interview does not publicly establish the precise training of a particular system.

The host asks whether interaction can help models learn from human activity. Wu thinks stronger capabilities may produce more long-tail task data, while data quality, permission and evaluation remain important.

Web operation is more complex and costly than a simple chat. It may generate feedback about task completion, but that feedback does not always capture what the user actually needs.

They discuss the division between research and product work. Researchers may focus on capabilities and training methods, while product teams must also understand users, interfaces, reliability and operations.

He argues benchmark tasks and actual user needs do not completely overlap. A method that works on mathematics evaluations has not necessarily solved every product task.

Commercialization can support continued investment, and real use can reveal needs; whether data can be used for training depends on product permissions and policies.

On the “year of agents,” Wu says progress in multimodality, model capability and reinforcement learning opened opportunities to build previously difficult products. His timing is a prediction from the interview.

He recalls the term agent in game theory and reinforcement learning: an agent observes, acts and receives feedback in an environment, without necessarily using a language model.

In the language-model context, the term often means a model can use external tools through APIs or screen actions. The definitions are related but not identical.

He makes optimistic predictions about capabilities not yet released. Coverage estimates and dates are subjective forecasts, not established subsequent outcomes.

On startup opportunities, he says many websites and vertical use cases remain unsolved, allowing small teams to build around specific customer needs and workflows.

He also notes that large companies’ foundational capabilities may expand, so small teams must consider differentiation, distribution and platform competition.

The mouse and keyboard are action interfaces for computer use. He compares them to hands, illustrating how a system can act on software through existing human interfaces.

He discusses staffing and compute for research and products, acknowledging that exact scale is hard to judge; estimates should not be treated as facts without public data.

He distinguishes text and visual information on web pages from the more complex perception of the physical world. He sees extending screen interaction to physical action as a long-term research direction.

The analogy does not mean a web agent can already safely control a robot: the physical world adds real-time control, contact, hardware and safety constraints.

For human–AI collaboration, he hopes future agents can learn individual preferences through longer interaction, reducing the need to explain everything repeatedly.

Personalization also raises questions about memory, permissions and privacy. He is discussing possibilities, not an already reliable capability.

If more people use software through agents, websites and product interfaces may need to adapt. The pace and form of that change remain uncertain.

Facing large companies’ resource advantage, he considers how academia and small teams might choose problems: find different goals, methods or collaboration structures rather than simply replicate a scale race.

He is interested in genuinely multi-agent problems, where participants have their own goals and must collaborate or compete; he distinguishes this from having multiple models click the same web page.

The conversation closes by inviting further discussion. The episode records observations and predictions about agent technology from early 2025.