Don't draw the flowchart: OpenAI's Deep Research team on training agents end to end

Sonya Huang and Lauren Reeder of Sequoia Capital’s Training Data open with the product’s sudden gravity. Deep Research launched three weeks earlier and was already being used by people such as the Collison brothers for industry analysis, medical research, and even birthday-party planning. It is the second agent OpenAI has shipped, after Operator, and it was trained with end-to-end reinforcement learning on hard browsing and reasoning tasks. The guests are Isa Fulford and Josh Tobin, who lead the product.

Fulford’s description is simple. Deep Research is an agent inside ChatGPT that searches many websites and writes a comprehensive report, the kind of work that would take a human many hours. Ordinary ChatGPT answers in seconds; this one takes five to 30 minutes, and it comes back with more detail and specific sources. Operator came first. Deep Research is second. “We’ll release many more,” she says.

The origin sits about a year earlier, before Tobin was on the team. Internally, OpenAI was seeing success with a new reasoning paradigm — training models to think before they respond — mostly on math and science. Fulford’s group realized the same regime unlocked longer-horizon work: online research, discriminating among sources, adjusting strategy as you go. They decided to train models on browsing tasks with the same methods used for reasoning models, but on more real-world work. The original prototype came from Fulford and colleague Yash Patil, later joined by engineer Thomas Dimson, whom Fulford describes as the kind of person who will dive into anything and ship a pile of it. Tobin had been at OpenAI in the early days, left to start a company, and rejoined about six months before this conversation. Hunting around the agent efforts, he landed here.

Tobin says the audience is anyone who does knowledge work as part of a job or a life: markets, companies, real estate, scientific research, and a lot of medical examples. The same “I need to spend many hours searching and curating” pattern shows up in shopping and travel, which is why they are excited about a ChatGPT Plus launch that will let more people try it. Reeder has already made it a daily tool. She wanted to buy a car and could not tell when the next model would ship; the web was full of speculative posts. She asked Deep Research to break down the gossip and the manufacturer’s historical patterns. The report told her to wait a couple of months — it should come this year.

The use case that most surprised Fulford is coding. People on Twitter and in feedback channels are using it for code search, for the latest docs on a package, and to help write scripts. Tobin is almost embarrassed they did not think of it: for ChatGPT users it seems obvious, and it works surprisingly well. Asked whether this will tilt business or consumer over a year or two, Fulford wants both; the capability is general. Tobin leans toward business having more of these multi-hour tasks, but he expects it in people’s lives either way. The magic, he says, is time: something that took hours or, in some cases they have heard, days can be dropped in here and return about 90% of what you would have come up with on your own.

On the consumer side, Fulford used it for months before launch. The team was in Japan for the release, and she threw it specific restaurant requirements and found places she would not have found herself. Tobin uses it the way he used to spend hours reading every review and forum thread before an expensive purchase or a special trip. The model is unusually good at instruction following: stack product facts, competitor comparisons, and Reddit reviews into one query and it will do all of them. Ask for a table with citations. Under the hood it can already embed product images and create graphs; those have not fully landed in ChatGPT yet. Personalized education is another sleeper — brush up on biology or a world event, list what you do not understand, and get a custom report. A friend of Fulford’s who is considering a CPG company uses it for similar products, domain availability, and market sizing, then forwards the reports. It is also good at a single obscure fact. Fulford’s brother’s friend’s father was stuck on a niche question about an Austrian general and a death in a battle; ChatGPT had gotten it wrong, he checked a public library, and Deep Research later got it right. The family was thrilled.

Tobin’s mental model is specific. Deep Research excels when you have a detailed description of what you want and the best answer requires reading a lot of the web. Vague questions get a clarification loop, but the high-water mark is a concrete information need. It synthesizes what it finds, digs up hard-to-find facts, and can draw some new insights from the material. It is not making new scientific discoveries yet. For coding, Fulford usually stays on the o-series — o1 Pro, o1, or o3-mini high — because writing code typically does not need knowledge outside pretraining.

The model that powers the product, Fulford says, is a fine-tune of o3, OpenAI’s most advanced reasoning model at the time. They trained it on hard browsing tasks the team collected, plus other reasoning tasks, and gave it a browsing tool and a Python tool. Trained end to end, it learned strategies for those jobs. Tobin’s intuitive picture is the loop users see: a detailed request, hard thinking, search, read, relate the page to the goal, decide what to search next, and finally assemble a tidy report with citations back to the originals. What is new as an agent capability, Fulford argues, is that so much of research cannot be predicted in advance. She does not think you could write a language-model program or script as flexible as what the model learns — reacting to live pages, changing strategy, inventing the next query. Chain-of-thought summaries sometimes look “very smart” about what to look for next.

John Collison had tweeted a question that went somewhat viral: how much of Deep Research’s magic is live web access, and how much is chain of thought? Fulford says it is a combination. Other search products that were not trained end to end are less flexible when they hit unseen information and less creative about solving a specific problem. A lot of the analysis also comes from the underlying o3 training. Deep Research is that model, specialized.

Tobin had seen the other architecture up close. Before returning to OpenAI he was at a startup “dabbling” in agents the way most people describe them on the internet: you construct a graph of operations, some nodes are language models, the model can choose the next step, but a human defines the overarching sequence. That is a powerful way to get to a prototype. It falls down quickly in the real world. You cannot anticipate every scenario or every branch, and the models at those nodes are often not good decision-makers, because they were never trained to make those decisions — only to do things that look similar. Deep Research is trained directly end to end on the tasks users actually run. There is no graph. It is all driven by the model.

Huang presses the implication. Many companies are prompting OpenAI’s API to solve specific workflows. Should those applications move to end-to-end trained models? Fulford draws a line: if the workflow is specific and quite predictable, a graph is fine; if there are many edge cases and you need flexibility, something like Deep Research is the better approach. Tobin’s addendum is the hard-rule exception. You do not want to bake “do not touch this database” into the model; encode that in human-written logic. The lesson he has watched people relearn is that we think we can do things smarter than the models by writing the logic ourselves, but as the field progresses the models come up with better solutions than humans do. Then the number-one lesson of machine learning: you get what you optimize for. Set the system up so you can optimize directly for the outcome, and the results beat gluing together models that were not optimized end to end. His long-term guidance: reinforcement-learning tuning on top of models is probably going to be a critical part of how the most powerful agents get built.

Asked about the biggest technical challenges, Tobin answers as an observer rather than a founding engineer. One of the hidden keys was making really high-quality datasets. It is another age-old machine-learning lesson people keep relearning: the quality of the data you put in is probably the biggest determining factor in the quality of the model you get out. On the team is Edward Sun, whom Fulford credits as someone who will optimize any dataset he touches. Reeder’s summary: find your Edward.

Trust is a product requirement, not a slogan. Users need to believe the outputs, so the model cites sources and training tries to make those citations correct. Fulford is candid that the model can still make mistakes, hallucinate, or trust a source that is not the most trustworthy. That is an active area of improvement. A less obvious design choice is the clarification flow: before research starts, the model asks questions. ChatGPT might ask something at the end of a reply; it usually does not interrogate you up front. The wait is five to 30 minutes, and the best reports come from detailed prompts, which are not natural first messages. People on Twitter have already built their own pipeline: talk to o1 or o1 Pro until the prompt is sharp, then send it to Deep Research.

How should people think about Deep Research together with o3 and Operator? Tobin says they are pretty disconnected today. The destination is not. The ultimate agent should not merely search the web or use a computer; it should fuse those actions the way a human assistant would. Naming, meanwhile, has not been a strength. Three products called Deep Research shipped in a few months — OpenAI, and by implication the others on the market. Tobin jokes there is not a lot of naming creativity in the field and tells people to try all of them. Quality differences will be clear, and they come down to the datasets and the o-series engine that lets OpenAI optimize models until they are really smart. Huang notes this is at least OpenAI’s best-named product: it describes what it does.

A year out, Fulford wants more data sources — private data on top of public web — and stronger browsing and analysis, fitted into a broader agent roadmap. Tobin’s recipe is the part he thinks will surprise people as it scales: take a state-of-the-art reasoning model, give it the same tools humans use to do their jobs, and optimize directly for the outcomes you want. There is really nothing stopping that formula from reaching more complex tasks. “AGI is like an operational problem now.”

Sam Altman had said Deep Research will take over a single-digit percentage of economically valuable tasks. Tobin unpacks it: the product cannot do all of what you do, but it can save hours or sometimes days at a time. The agents they build next should give back 1%, 5%, 10%, 25% of your time, depending on the work. Huang, whose own usage is heavy, jokes they just need to start writing checks. Are whole job categories in the strike zone — consulting, for example? Tobin used to be a consultant. He does not think any jobs are at risk, and he does not frame this as labor replacement. For knowledge work that is looking through information and drawing conclusions, it gives people superpowers. Fulford is especially excited about medicine: doctors have already posted about sweeping literature or finding a clinical trial for a patient. The deeper point, Tobin adds, is not 5% of the calendar. A task that took four or eight hours now costs a ChatGPT subscription and five minutes. What would you do if you had infinite time — research every startup you might invest in, not only the ones you can meet? Huang’s consumer version is the working parent too busy to plan a birthday party: now it is doable. Education is already a top use. Learning by talking to a system that personalizes to what you tell it, and someday to what it knows about you, feels more efficient and more engaging than textbooks.

In the lightning round, Fulford’s favorite Deep Research use case is personalized education; the stories that have moved her most are people finding information about a diagnosis they or a family member received. Asked which application category will break out this year after coding’s year, both guests say agents. “2025 is the year of the agent.” For reading, Tobin will not pretend anyone can keep up with the state of the art. Pick one or two subtopics you actually care about, curate a list of people saying interesting things, and — yes — use Deep Research to go deep. Fulford recommends a slightly old course she thinks is called Foundations of RL, from Pieter Abbeel; it is still a good introduction. Tobin seconds anything by Abbeel, his graduate-school adviser.

Huang’s last real question is whether reinforcement learning, after a peak and a lull, is speaking again. Tobin’s answer is two words: “It’s so back.” Why now? Because everything else is working. People who have followed the field will remember Yann LeCun’s cake analogy from around 2016: the cake is unsupervised learning, the frosting is supervised learning, and reinforcement learning is the cherries on top. When Tobin’s cohort was doing RL in 2015 and 2016, they were adding cherries before they had the cake — and in retrospect LeCun was probably right. Now language models are pretrained on massive data, supervised fine-tuning makes them follow instructions, and it is ripe to tune those models for any use case where you can define a reward function. From this conversation the hosts take two slogans: agents are the 2025 breakout, and reinforcement learning is so back.