A Single Training Environment Can Cost Millions. Prime Intellect Wants a GitHub for RL

Sonya Huang introduces Will Brown and Johannes Hagemann of Prime Intellect as one of the sharper new labs in AI, with a mission to make frontier-lab training accessible to everyone. What differentiated them, she says, was the reinforcement learning Environments Hub: developer taste plus a community surface for the thing labs actually train on. The conversation that follows is about post-training, agent harnesses, and a bet that the infrastructure locked behind a handful of walls can be productized.

Midway through, Huang puts a number on the locked-in version. She has heard that foundational model companies are spending millions of dollars each on some of these environments. Brown does not dispute the figure. The other number in the same episode is almost comic by comparison: Wordle, from the New York Times, as a hello-world environment. A couple of GPUs, a one-hour run, a tiny model, and you can watch it learn to think about the game. The Hub exists to connect those two ends of the spectrum — the million-dollar sim and the toy that teaches people the feel of a model actually getting better.

The conceptual move that ties the numbers together is simple and, Brown argues, already happening. The evals that get reported at model release — SWE-bench, AIME, Terminal-Bench — are environments: a dataset of tasks, a harness the model sits in, and a rubric or reward function that grades the output. Used offline, that package is a test set. Hooked up to reinforcement learning, it is a training set. Companies already sitting on agent test suites are closer to a training flywheel than they think.

Prime Intellect's full-stack research platform is called Lab. The organizing idea is the environment. Brown's definition is blunt: a set of tasks, a harness — the shell around the model, including prompts, tools, and how it executes — and a rubric or reward function that scores the result. That is also what people mean when they publish SWE-bench numbers. The same object, used offline, is an eval. Connected to trial-and-error training, it is how a model improves interactively. Lab is built so people can use environments for post-training, evaluation, synthetic data, and RL, with a GitHub-like community layer of building blocks and examples.

Huang is not buying the merge at first. In her head an environment is old-school RL: state, observe, act, watch the world change, update a world model. An eval is a bank of questions with right answers. Those feel like different species. Brown's reply is that they have grown into the same animal. The environments you want for large-scale training do have complex state — they may be simulating a web app or a full coding platform — but Atari always had a reward and a goal too. You cannot just run the system and have a human vibe-check it. There has to be something that measures progress. Put state, interaction, and a score in one loop, and the eval/environment line dissolves.

Harness sits inside that loop, not beside it. For coding agents, Brown will call the CLI coding assistant the harness and the terminal the environment; in other agents the harness is a system prompt and some tools, or a parent that spawns sub-agents with harnesses of their own. He expects a Cambrian explosion of those patterns, which is why they picked a wider word. "Agent" is too narrow, "harness" is too narrow; any system-model interaction fits in an environment. The abstraction also refuses to bind you to RL. Distill by running a big model in the environment, filter with the same grader, and SFT on the best traces. Optimize prompts. A/B test models the way you already run evals. Brown's less controversial framing: every AI company should be optimizing its own AI system. RL is the big rib in the middle of that umbrella; prompt tuning, SFT, and online distillation are knobs around it.

Why give more people that training loop? Hagemann starts from open science: historically, big discoveries were things the world could talk about and build on. Brown adds the practical layer. Winning applications use AI for a specific agent or workflow, and they need to deliver it at scale with cost-effective performance. The way to optimize those systems end to end is to get the weights and craft the model for the problem, rather than doing all the crafting in a prompt. A general model plus a prompt cannot reach that depth. Huang asks whether the future is every company pretraining. Brown's version is slightly different: every company will be an AI company, and most AI companies will want an AI research lab. For some domains that means pretraining, especially if you do not want text-in, text-out. For most, it will be post-training agentic models on specific tasks, and that is getting productionizable and cost-effective for the right shape of problem.

Hagemann's chain is the one that stings. ChatGPT came from OpenAI and Claude Code came from Anthropic, he says, because those labs can optimize models for their own scaffolds — the models simply work better in their products. Huang has already flagged Claude Code as a harness-engineering showcase. The twist is incentive: the more popular those products get, the less reason a big lab has to make someone else's coding startup work as well. Claude Code is extremely popular right now. Anthropic's appetite for improving rival coding products is written on its face.

Cursor is the early bird that understood the loop. They built their own Composer 1 model with large-scale post-training, and the environment was Cursor itself: the model sits inside Cursor, with Cursor's tools, and gets optimized to be unusually good in that product. Hagemann expects more startups to follow — some to sharpen what they already ship, others to build products that are not possible today because the product-model optimization loop did not exist. That is the job he assigns Lab: pull what lives behind frontier-lab walls — compute orchestration, large-scale RL trainers, the Environments Hub, secure code-execution sandboxes, evals — into an end-to-end stack anyone can run.

Brown's rule of thumb for when to invent an environment versus when to train in the live product is both. If you want a great Rust coding model for many apps, you build domain-task environments, or a model that is excellent at your tools and DSL and sell that as a service. If the product is a UI where an end user talks to an agent, the product should become the environment. The companies who already care about Claude versus GPT-5 versus Gemini, who already evaluate system prompts and mini-model cost trade-offs, have been building the same infrastructure RL needs — they just have not been doing the training piece. RL, in that picture, is an extra tool in a system-optimization kit they already own.

RCI, a new lab on frontier open models, is the collaborator that forced the stack to exist. Prime Intellect's history is large-scale pretraining; with RCI they trained some of the largest mixture-of-experts models, open-sourced them, and continued into post-training. Hagemann describes working the entire stack with them. Brown's division of labor is that RCI aims at enterprises and end-to-end delivery of an artifact, while Prime Intellect stays on developer experience at the infra layer — making the path from idea to deployable model as short as it can be.

Medical AI labs are heavy users of the Environments product. Brown names Sophont and OpenMed as two close partners. They need benchmarks for diagnosis, patient interactions, medical-literature QA, and agentic search, and they need models that can deploy locally in a hospital with a traceable data provenance. They would rather spend energy on the domain than on generic LLM infrastructure; the platform is supposed to carry that headache. Grad students and early-career researchers use the Hub to learn. The harder environments come out of Prime Intellect's RL residency: it started at eight to ten people and is now fourteen to sixteen, a mix of grad students and full-timers building on the side. They have shipped verifiable software engineering, medical physics, and cybersecurity environments, and they now have the trainers to actually run them.

Wordle is the on-ramp. WikiSearch is the one people fork. It is agentic search over Wikipedia pages, designed as a template: swap the documents for a company's internal corpus and you have an enterprise knowledge-base search environment with the rest of the pipeline already standing. Cybersecurity, from the residency, is capture-the-flag brought over to LLMs. An agent lives in a terminal, with bash tools, maybe wrapped in Claude Code, rummaging through files. When it marks the rollout finished, other code inspects environment state and decides whether the flag was captured. Huang asks what actually goes into a million-dollar environment. Part of the answer is that pipeline: a full software world, a harness, and a grader that is itself code.

Why a Hub instead of a pile of GitHub repos? Hagemann says open source was already overflowing with environment implementations — Brown had previously written an open-source framework called verifiers — and the Hub is how you standardize plus add what a repo cannot. Environments plug into evals so you can test them across frontier models in one shot. They install into trainers: Prime Intellect's own large-scale trainer, and other people's, because teams have different trainer needs. Private environments are fine. The Hub still pays off because you can ablate a private task against a public cousin, or drop a known benchmark into a training run as a progress bar and watch whether gains on your task generalize or overfit.

Huang pushes the cybersecurity example into sim-to-real. CTF toys do not look like a real company's network stacked with security products. Is there a robotics-style gap, and is the only fix to train on the actual corporate environment? Brown's split is important. Complexity itself is not the barrier: anything that can exist on a computer or a network of computers can be an environment. The bottleneck is the cost of the simulator. The craft is finding clever ways to mock the right slice of the system instead of standing up the whole world.

His picture is a fog-of-war map. Most of the map is dark; light follows the player. If you already know which tiles a given task will walk, you do not render the rest. Tau2-bench, a popular customer-service-agent eval that involves a database, is the worked example. A real system might need millions of records. If the agent will only ask certain kinds of questions, an in-memory mock with the rows that task will touch is enough, and much cheaper to run. A large share of environment design, Brown says, is spotting which complexity is overkill. The platform's job is to keep doors open as those branches multiply — generic environments, then coding-agent environments, then a sandbox that is global across a run or one per rollout — and to put institutional knowledge into docs and skill files so coding agents can help people build the next one.

A quieter use of RL in the same stretch is quality control. If an environment has a backdoor, a model will exploit it and game the score. Running a small or medium training job in a single environment is how you poke for those bugs. Frontier labs put an environment through a lot of vetting before it lands in the next GPT or Claude foundation run; those medium-scale trials are part of the pipe. On whether a Fortune 500 can use any of this, Brown's observation is that those companies already have AI engineers who live in Claude Code and have opinions about models and prompts — that is the target user. Hagemann supplies the other half: many of them have nobody who can debug a TPU cluster, so the platform has to swallow orchestration, tool use, and code execution and leave people facing the environment.

Huang brings up Richard Sutton and *The Era of Experience* and asks whether constructing environments is the successor to the Scale AI era of data labeling. Brown says it already looks like it. Human data is not going away, because environments are by definition built for things models are not yet good at — which means humans are still better in some capacity. The move that matters is getting "what does a job well done look like" out of an expert's head and into the model. The emerging pattern is a rubric, often as yes/no questions an LLM judge scores, turned into a reward. Sometimes that flows through a reward model. RL, he says, is currently the most effective pipe for that transfer.

Andrej Karpathy, an angel investor in Prime Intellect, has a well-traveled complaint: RL is amazing and also like sucking bits through a straw. Brown grants it. RL burns a lot of compute for a small information signal. That is also the value. High-quality human data does not scale; RL is how you trade compute for data and squeeze more out of a small supervision signal than pretraining or SFT alone. The sharper case is when you are already at the biggest model you have: there is no larger teacher to distill from, no golden example to copy, and you have to explore untrodden territory. Exploration, Brown says, is the heart of RL. Hagemann would like to hear Karpathy's take two months after that Dwarkesh interview. Claude Code's last-month surge makes the usefulness of large-scale RL in coding harder to wave away.

They know the wall. Context length is a hard limit on how many tokens you can stuff into a window, so they have been watching recursive language models closely since original author Alex Zhang published the work. The harness gives a model a persistent Python REPL: data lives in a variable instead of in the prompt, and the model retrieves, transforms, and manages it, recursively calling sub-LLMs. Prime Intellect wrote a blog post a few weeks before the taping: give current frontier models that harness, train nothing, and long-horizon reasoning benches already move. The next few months of their research is training inside the harness so the model learns to use it. Brown is also betting on synthetic data — models curating their own training data and maybe their own environments, combined with prompt optimization and distillation, as a path to continual learning.

Closed models are not locked out of the abstraction. The trainer needs weights, so you cannot RL a closed model you do not have. Everything at the environment layer still works: evals, prompt tuning, model selection, harness evaluation, and using the environment as a data engine to distill traces from a large model. LoRA is what they recommend most people use for RL anyway; you still cannot upload your own LoRA onto GPT-5. The prompt-space analog Huang reaches for is DSPy; the algorithm that got attention last year is GEPA, and the Hub supports it on tool prompts, agent skills, and system prompts. Partial customization has a lot of rungs before you need a full fine-tune.

Huang closes on the optimistic picture. Hagemann does not want a world where the future value of AI in every vertical is owned by the big labs. Entrepreneurs and enterprises need tools to optimize their own products so they are not steamrolled. The payoff he wants is more Claude Code moments and more "Cursor for X" moments. When Huang asks whether every company becomes a NeoLab, he says, in some ways, yes.

Brown's version is the line the episode opened on. If data is the bottleneck, if real expertise is the bottleneck, would you rather have the smartest person in history at your company or someone who has been there for 30 years? Sometimes you really want the person who has been there for 30 years. Deep, long interaction with a problem produces expertise that a short prompt almost cannot copy. That is what training deposits. You want institutional knowledge to compound, and best practices to compound, because that is how companies get powerful: they stand on what they did before instead of resetting every day. Prime Intellect wants that compounding available to any company that wants it. As the barrier to coding falls, they expect the barrier to AI research to fall the same way.