Enterprise Coding Tools Flipped the 80/20. Reflection AI Built the Other Half

Reflection AI's mission, Misha Laskin tells Sarah Guo, is to build superintelligent autonomous systems. In an academic sense, he argues, superintelligence has already been achieved: AlphaGo was superintelligent inside Go, and other systems of that era were superintelligent in equally narrow domains. One path from here is to lock a lab away from customers, map every benchmark, and grow a very broad superintelligence. The other is to ask what the thing will look like in people's hands and build backwards. Reflection is on the second path: co-design product and research, stay focused, and pick a category large enough to be ASI-complete. Pick too small a category and you cannot draw a superintelligence out of it.

Laskin arrived at that bet from physics. A Russian Israeli American, he bounced through several countries and landed in a desert in Washington state, learning English with time on his hands and the Feynman Lectures on his parents' shelf. Transistors were invented by theoretical physicists; GPS needs special relativity. Physics looked like the root science. Then AlphaGo happened. The discoveries he was doing problem sets on were a hundred years old. AI, he decided, was going through physics' moment of a century ago—the root science of *this* time. He muscled his way into Peter Abbeel's reinforcement-learning lab at Berkeley, then to DeepMind.

At Gemini, co-founder Ioannis Antonoglou—the overall RL lead for Gemini 1.5, and earlier an AlphaGo contributor—ran the stack. Laskin built the reward models used to post-train Gemini 1 and 1.5. When that project landed, both of them reached the same conclusion: the last paradigm that has to be in place before what people used to call AGI, and now call ASI, is scaling reinforcement learning on top of large language models. The first instances have been arriving over the past year. Laskin thinks the field is still earlier than it believes. Things have started to work.

This week's product is Asimov, which Laskin calls the best code-research agent in the world—a comprehension system meant to feel like deep research for large codebases. The intended sensation is a principal engineer who deeply understands your organization, sitting at your fingertips. Every coding tool has some generation and some comprehension. After spending time with enterprise customers, Reflection's diagnosis is blunt. In large organizations, the actual productivity impact of coding tools is much lower than people expect. Sometimes it is negligible. Sometimes it is negative.

Guo asks whether he saw the recent METR report. He did, and it matched what engineering leaders have been telling him—not only in enterprises, but in any growth-stage startup whose codebase and team are large enough that no one engineer holds the whole thing in their head. Reflection is itself one of those places: an LLM-training codebase, a product codebase, and knowledge scattered through chats, project-management tools, and people's heads. The thing they are building toward is an omniscient oracle for the organization. Ask a question at any level of complexity and get an answer at principal-engineer quality. Once understanding is solved, more reliable acting agents fall out of it.

Today's market, he says, is focused 80% on action and 20% on understanding—80% code generation, 20% comprehension. Look at what an engineer actually does and the ratio flips. Eighty percent of the week is spent trying to comprehend complex systems and collaborating with teammates, and collaboration is usually someone asking someone else a question about a system they do not know. That gap is what would keep a superintelligence from working inside a company. Teaching agents to act is maybe 20% of the problem, and it is more or less solved. Guo recognizes the pattern: the stupid implementation you would never have written if you had reasoned with the system's context; the "works in my environment" bug.

What makes Asimov different is not one magic module. A researcher would say the answer is entirely in the agent or the model. A product person would point at drawing knowledge from outside the repo, or letting engineers dump tribal language into the system—"when we say environment jobs, we mean these Google Batch jobs"—so the next person who asks is understood. Laskin wants both: differentiated product almost independent of the models, *and* research on agent design and model training. That combination collapses into an evaluation problem, which he calls the least spoken-about and possibly most important work a frontier lab does. Claude feels magically better at code because someone did something right in the evals.

Reflection post-trains. Long-context reasoning, in their usage, is not a giant general model: it is small models with very long contexts that dive into huge codebases, suck up what they can, and reason out the relevant bits—almost neural retrieval. Tool use and multi-hop reasoning are different. One path trains the most general agent across thousands of environments, which Laskin analogizes to the pretraining of agents; that is what a frontier lab does, and what a release like Kimi K2 is for. It gives you a general base. Depth is the other path: train against a specific tool set so the reasoner knows when to read Jira, when to call the long-context model, when to hit another system. Grok 4 showed a visible gap between a general model and a tool-trained model on Humanity's Last Exam. Impressive, and weakly correlated with anything an end user needs. You still have to train for what users actually want.

A startup's only durable advantage against talented incumbents, he says, is focus and velocity. Look at the o1 model card: most of the people on that paper were doing evals. Inside a big lab you are one of hundreds of evals, and the resulting model is general but spread thin. In the AI age, Paul Graham's "talk to customers half the time" becomes develop your evals from what customers are saying and doing. Onboarding is the example he keeps using. A big-company engineer can take months to ramp. Build evals that compress that to a couple of weeks because every question can go to Asimov. Work backwards from the query to the capability: a 100-million-token codebase demands long-context reasoning. And you have to be able to change product, agent design, and model training. If you can only touch the first two, you are at the mercy of third-party general models.

Gemini was gratifying as frontier research and frustrating as a product loop. Information flowed through something like four people before a model reached a customer—a broken telephone. Best on academic benchmarks does not mean best product. The tight couplings he cites are ChatGPT, post-trained on the actual prompts coming in from chat (the first coding blog post it wrote for him felt insane), and now Claude Code. Excellence at both model and product, in his view, is the job.

File-level questions are the wrong demo. If you are staring at a file and want a snappy answer, you do not need a deep-research hammer that takes tens of seconds or a couple of minutes. Asimov is for semantic queries: you do not know the function name or the file. You know the test is flaky, and that is where your knowledge stops, so you used to shout into Slack. Inside Reflection, the example was infrastructure jobs running five times slower than usual. Two pull requests, one from infrastructure and one from research, both passed tests; together they created a race condition. That is the bug two or three engineers spend days on. It is the same split as ChatGPT's browser tool versus deep research: facts should be instant; compiling a lot of information around a nebulous question is where the product earns its keep.

Team-wide memory is one of the more fun—and more complex—features in the product. Who can edit it, who can view which slice, how you keep a repository of it, and how you encode authority, because people will be wrong. Early customers start with a gatekeeping group of trusted senior-staff-plus engineers, populate memory, then widen the circle. Permissions get org-wide and file-specific. Laskin describes it as GitHub++: you are no longer versioning code, you are versioning the meta-knowledge around it that helps language models understand the system. You are, in effect, building a new Git from scratch.

The review flow, he thinks, will look like a pull request, not static role-based access control. The person who actually understands that subsystem approves the change to the agent's memory. That is how Google's OWNERS files already work: if you are on the file, review has to go through you, and as people move teams the ownership files update. It is more nuanced than a personal `AGENTS.md` living on one laptop.

A common criticism of labs that pretrained for code is that they still needed general language and reasoning, so they paid for the whole stack without the general product. Guo asks whether capable open-weight models are what let Reflection chase superintelligent coding systems without the pretraining bill. Laskin says that is roughly the short-term logic. A year and a half ago they bet that pretraining was converging on a known paradigm: a giant, extremely noisy internet dataset, some algorithmic nips, and only so much signal left before the dollars got absurd. They expected decent open-weight models. The open-weight frontier, he admits, came out better than they thought.

The second bet is a brief period in history: RL flops are still manageable, about two orders of magnitude off pretraining. You still need a decent pile of GPUs, but if you are focused you can ship a best-in-class product without being coupled to a big cloud provider. Do it right and you generate enough revenue that you do not have to be acquired, or sign the strange deal where the cloud effectively owns you. That was the pre-LLM frontier-lab model. Anthropic's revenue growth, independent of Reflection, is the field-wide proof that a research lab can become a massive business. Capital requirements remain high—maybe an order of magnitude less than a full frontier lab today, and asymptotically they converge—but by then you are raising off a generational business, not a compute hostage situation.

If scaling RL on language models were simple, more people would be doing it more aggressively. Laskin splits the bottleneck into problem structure and algorithms, with structure as the larger one. His Gemini work was reward models. If you have a reward that accurately describes the outcome of any arbitrary task, even today's simple RL methods extract a lot, bounded only by exploration. We do not live in that world. LLM-as-judge with rubrics works to a point; a noisy, stochastic reward eventually gets hacked. Ground-truth rewards exist for only a few tasks, so you hope optimizing them generalizes. Every synthetic-data pipeline is a creative way to live with the rewards we have. He used to call the reward problem AGI-complete. Now he calls it ASI-complete: by the time a neural net can accurately verify any outcome, that net is probably already a superintelligence. Then you are back to evals. What are you training the reward model on?

The algorithms are also crude. Current RL is bad at exploration and credit assignment. The core update is: things that worked, do more often; things that did not, do less. It does not discern, along a reasoning chain, which step was right and which was wrong. That is why chains garden-path and meander through unnecessary branches and look nothing like a person's structured thinking. There is no atomic credit-assignment step. That, he files under algorithmic progress still owed.

Guo asks for a hot take on companies cloning Airbnb, Amazon, Salesforce, or Excel so they can train agents inside copycat software. Laskin is bullish, and then he goes further: "There's no such thing as generalization. There's just bring the test distribution into train." Users experience generalization because they never see how the model was made. If the synthetic environment was actually similar to the query, of course the model is good at it. Six months ago he said ASI, on his definition, was possible in a couple of years. He still believes that—not all of coding, but definitive superintelligence in meaningful slivers, perhaps a superintelligent frontend developer because the internet is full of that data and synthetic data is easy. Once you have the recipe, the rest is an economic decision: is it worth sinking X dollars to pull this category under distribution?

That script already ran before LLMs. Atari, AlphaGo, OpenAI Five, AlphaStar: the blueprint for superintelligent systems existed. If OpenAI and DeepMind had sunk more compute into Dota and StarCraft, those systems would have gone fully superintelligent. It just did not pay. Friends of Guo's at large labs now talk the same way: systems much more capable than humans at many kinds of knowledge work, less faith in free generalization, a resigned plan to bring everything under distribution. Laskin is in that camp. RL before language models had *zero* answer for generalization. Today's models feel general mostly because the training distributions are so broad. The future looks jagged: a handful of category superintelligences, maybe later subsumed into one model, with some domains so expensive to collect that craftspeople remain cheaper.

The glimpse we have is test-taking. AIME is saturated; Codeforces and other competitive-coding environments are almost best-in-the-world and probably will be within the year. Walk into a company and ask if these things have been helpful and the answer is uneven. The only GDP-scale signal Laskin will accept is walking into a firm and finding that *every* engineer is a double-digit percentage more productive. Benchmark-maxing still looks like training an RL agent on a board game, posting a landmark, and declaring superintelligence solved. Deployment is half the problem. That is why he keeps returning to customer evals and product-model coupling.

On Google DeepMind's Windsurf non-acquisition—OpenAI's deal collapsed; Google paid a large licensing fee and hired the founders into DeepMind—Laskin sees verticalization arriving in categories that matter to frontier intelligence. Search was first, through ChatGPT. Coding is second. The driver is not a tidy ROI model. If you want to remain a trillion-dollar company that can sustain frontier research, you have to own the categories that feed it. From the lab side, this is the Gemini lesson again: best model does not mean best product, which is why coding-tool adoption took off at startups. Claude Code is a successful verticalization. Buying an asset does not automatically fix a research team that is far from product, or a company with a hundred focus areas.

From the startup side the story is harsher. If you sit on a critical-path category like search or coding and you cannot build your own frontier models, you are in an existential place. Not every lab will verticalize well. One might be enough. You are subsidizing Anthropic's or Gemini's margin while they can discount their own products more than you can. Own the intelligence, or integrate so deeply with the customer that you are hard to remove. Otherwise the category's gravity will crush a great UX wrapped around someone else's model.

Guo, who works with robotics companies, doubts RL will dominate there versus imitation. Laskin trained robot RL in Abbeel's lab; reward modeling for Gemini grew out of that question. Language, he says, is a ready-made compressed representation of the world. Pixels and sensorimotor signal are raw noise, so vision-language rewards are infinitely more hackable and short-lived. That is why he left robotics for language. RL *does* work in locomotion, where the reward is clean: the quadruped moves at this velocity without damaging its body. Robotics also has teleoperation—humans are master manipulators. Language models need the chain of thought inside someone's head, which you cannot harvest. Early in Reflection they even considered having people talk into a microphone while they worked. Logistics killed it. If a company recorded a software team's Slack, docs, and conversations and sold the set, Laskin says they would pay, especially if the data were diverse.

Code is special because a language model will talk to almost any piece of software through function calls, which is code. A long-context organizational reasoner that can act on software therefore operationally generalizes to other knowledge work. Customers are already pulling them toward product managers and technical support; sales is on the list. Asimov can plug in as MCP to whatever IDE or coding agent the customer already likes. The startup risk is going diffuse before nailing a category. Their self-image is a contextual core for an engineering org, then adjacent expansion.

On deployment he is earlier than most people think. Within a couple of years the *blueprint* for building ASI in a particular category will mostly be set. Actually deploying it is a multi-decade project. Global GDP will not jump 10% two years from now. Pre-LLM DeepMind ran "strikes"—Go, StarCraft, Dota—each a snowflake at implementation time even when the concepts rhymed. Every large category will grow its own environment, tools, and post-training. OpenAI's Codex is already not the o-series; it is a model post-trained for that environment, the way deep research is. Coding, he thinks, will take longer than people hoped because enterprises have organizational problems the benchmarks do not capture—but it will still be one of the faster categories, on the order of dozens of months. "The next generational companies in coding are definitely being built today."