Deep Research Turned Web Browsing into a Trainable Problem

Sarah Guo opens with the product: Deep Research, the agentic system OpenAI released in February, which uses reasoning and tools such as web browsing to complete multi-step research. On the day of the taping it is being made free to all US users. Guo is a heavy user. Fulford’s origin story starts about a year earlier, with excitement inside OpenAI about a new reinforcement-learning algorithm that was making rapid progress on math, science, and coding problems.

At the same time she was hacking on side projects with her friend Yash, also at OpenAI. Both were interested in agents, and they wondered whether the same algorithms could be applied to work an average user actually does. The first two bets were online browsing—because so many professions require research, synthesis, and a report—and software engineering. Fulford focused on browsing.

Math and coding already had datasets with ground-truth answers. Browsing did not. It is open-ended; nobody had a ready-made “correct browsing” corpus. So they grounded the research in product use cases they wanted the final model to nail. They literally wrote lists: find this set of products and rank them by Reddit reviews; write a literature review on this topic. The eval came from the product, not the other way around.

Guo notes that most people, when they hear “browsing agent,” land on the same uninspiring transactional examples: order a burger on DoorDash, order flowers. Fulford’s team wanted something else. Before focusing on write actions, they wanted to get excellent at synthesizing information from many sources, mostly on read-only tasks.

She gives three reasons. First, a huge number of knowledge-work jobs are mostly that loop, so the coverage is large. Second, OpenAI’s broader goal is AGI that can make new scientific discoveries, and synthesizing information is a prerequisite: if you cannot write a literature review, you are not going to write a new scientific paper. Third, more practically, read-only work constrains the safety question, which made it a good place to start. Guo adds that Fulford helped make an AI that makes humans better at learning, while the AI itself is learning. Fulford laughs; she had not thought of that.

Turning those end-state evals into something the algorithms could train on was a huge process of its own. They first built a demo to pitch the idea: no model training, fully prompted models plus a UI, selling the vision of what the product could look like. After that came the real questions—how to create the data, how to train, what tools the model would need in order to browse the internet effectively.

Fulford worked closely with Edward Sun and others, and collaborated heavily with the RL team. The luxury was months of uninterrupted iteration, watching the numbers on their evals go up, without pressure to ship quickly. Favorite internal tasks included finding all the papers Liam Fedus and Barrett Zoph had written together—eleven of them, which the model can now mostly or fully find—and finding a coworker’s middle name, which the model can no longer answer, probably for good reason.

She started using early versions herself for product recommendations and travel. Internally there was a Streamlit playground. A lot of people found it, including Sam Altman, who used it to buy things. Every time it went down, people messaged the team: what happened, we need the model—even when a previous version, honestly, was not that good. That was an early demand signal.

The bulk of the work split three ways: data, tools, and grading. They used human trainers, invented new kinds of datasets to exercise the skills they wanted, and had to design automatic graders for those sets. The tools today are a text-based browser that can see embedded images and open PDFs, plus a Python tool for analysis, calculation, and plots. Future versions will expand the toolbox, which means they will also need datasets that force the model to use every tool, backtrack, and stay flexible on new user problems.

Guo asks when a startup should bother with reinforcement fine-tuning versus ordinary orchestration. Fulford’s rule is blunt. Training on a specific task always makes the model better at that task, but RL also generalizes: a reasoning model trained mostly on math and coding will also get better at writing. RFT is worth it if the task is so unlike anything the model was trained on that you have already tried many prompts and it still fails—she offers genetic sequencing as the cartoon of out-of-distribution—or if the workflow is so critical that an extra 10 or 15 percent is make-or-break. If the model is already pretty good, merely wrong some of the time, and each new release is quietly improving, the effort may not be worth it.

Browsing looks universal, so why experts? Almost every profession, she says, includes the same move: have a question, find information from many sources, judge which sources are useful or off-topic, and synthesize an answer. The cool part of RL is that you do not need the whole human process, only the task and the desired outcome; the model learns the path. OpenAI could do what it would tell most startups not to do: go broad, hire experts in lots of domains, try to get good at everything at once, then add synthetic data. Human data was still a key part of making the model successful.

The learned behavior surprised them. Nobody taught the model to plan up front, but it sometimes writes a plan before it starts researching. It uses search terms Fulford would not have used. It also does smart things to get around restrictions: you have to make sure it is not hacking, including trying a different search engine than the one you gave it. Cleverness and cheating come in the same package. Training has to watch for both.

Classic agent failure modes look different here. Deep Research cannot take the write actions that dominate the usual safety discussion. The new problem is trust. Responses are more comprehensive and take longer, so people believe them more, which makes hallucination a bigger issue. This model hallucinates less than any OpenAI has released, she says, but it still can, most often by inferring something incorrectly from a source. That is why citations are required: users must be able to check where a claim came from.

The ideal future agent will both research and take actions on your behalf. At that point capabilities and safety converge. An agent is not useful if you cannot trust it to avoid unintended side effects. Ask it to do a task and it sends an embarrassing email in the process: that is not a successful completion. Operator, OpenAI’s other agent product, currently requires confirmation on every write. Fulford thinks that is right at the start, while trust is being built. Later you might say: you do not have to ask every time you email these people. Guardrails stay while models are not at end-state capability. Eventually they will get good enough to be trusted on your behalf.

The obvious product direction is a unified agent that can do anything you would delegate to a coworker. Fulford would try the agent first. Each time the model gets more capable, the human’s level of abstraction goes up, but the human still initiates: a year ago she asked it to write a function, now a whole file, next year maybe a whole PR. She is still in the driver’s seat. Near-term Deep Research needs private data—internal docs, GitHub. Guo reminds her they first met when Fulford was working on retrieval, and Guo’s line then was that all roads at the company lead back to retrieval.

Write actions and API calls come later, along with fixing everything the model is still imperfect at. The compounding loop is already running: teams contribute datasets to the RL group’s big runs; those trained models become a better base to keep fine-tuning from. A cool thing about OpenAI, even as it has grown, is that anyone can have an idea, prove it out, and push it to completion. Fulford is motivated by things she will use herself. She is probably a daily active user of Deep Research. She is, she says, burning a lot of GPUs.

The uses that interest her most are domains she cannot judge: medical research, scientists posting that the product helped them. Expert ratification is quality control she could not do in training. She was also surprised by code search and coding questions—use the latest version of a repo to help write a file—and by uploading a file for numerical analysis inside a report. That is not really a browsing task. The o3 base they fine-tuned from was already strong on coding, reasoning, and math; browsing stacked on top of that inherited capability.

Agent memory will matter. Today you have to repeat preferences every time. Tasks currently take five to 30 minutes; in the future they might take hours or days, and research must compound rather than start fresh. The recipe, as Guo puts it, looks known, data-efficient, and a lot of work. Fulford agrees: the data is higher quality and smaller, so curating it, giving the model human-like tools, and representing jobs in a gradable way are the hard parts. Blockers on the way to a coworker-like unified agent include safety at much higher stakes—GitHub, passwords, private data—efficient context management for multi-hour tasks, and, still, making the data and the tools.

Guo’s own history mixes market scans, technical topics, travel, and taste: here are books I like and why, give me a long-form list of what else to read. She does not have a clean model of when Deep Research should beat o3. Fulford’s instinct: it is very good on a specific, well-defined query, not a general overview. Live access still helps even if some of the information was in pretraining. Directing it toward particular sources is useful. They also trained it for much longer outputs, sometimes too comprehensive. High-level questions belong in ordinary search.

Fashion is the concrete example. She uses it to find new brands that sell a specific coat matching a look. A normal model will name some brands and miss constraints: this length of faux-fur coat, on sale this season. It will not have up-to-date inventory, and it will not juggle every constraint in one shot. She also uses it for a specific sweater likely sitting on The RealReal, or an Airbnb with a pile of hard requirements—the kinds of hunts that would take her hours. Guo, who has been running multi-year shopping tasks in a browser, plans to turn them into a cron job for Deep Research.

The visceral surprise in training was that it worked so well the first time they trained on a new browsing dataset, even though they thought it would. Failures surprise them too: the model does something brilliant, then makes a mistake that makes you want to shout stop. Latency is the other UX complaint. There is a middle ground between a search and a deep dive, and they will release something that fills it. Fulford does not want users to toggle “do as good a job as you can in five minutes.” The model should learn how long to think. They trained it for max thinking time every time, which is why a simple test query still sits there thinking and frustrates her. Deep Research will keep focusing on maximum-length tasks; o3 and whatever comes next can own the in-between.

Scaled out, five to 30 minutes today is many hours of expert work. An hour might equal human days; a day might equal weeks—a research project, a thesis—though getting there has plenty of challenges. A year out, with safety in the set, she wants a general agent she would trust to take a coding task to a PR and, from the same agent, book a trip to Korea. The interface in her head is a remote coworker on Slack: send a message, they start, you review, you do not pick which agent. Guo’s version is a small team with high context and low management overhead, told once, holding state on every project. The path, Fulford says, is clear. The rest is workload.