Post-Training Wins Inference: Why Owning the Weights Beats Betting on One Lab

Jacob Effron opens with a phrase that has taken over the industry in the past few months: own your intelligence. The usual arguments for not depending on frontier labs are dark. They might pull the API, refuse a request on safety grounds, train on your data, or ship a competing product. Patil, who spent a couple of years inside OpenAI’s labs before founding Applied Compute, does not buy the malice frame.

These companies, he says, are made of great people, bound by contracts, and backed by real engineering to protect customer data. What he actually cares about is flexibility and control. The slogan only became practical when open-weight models got good enough to do real work. Control goes deeper than a fallback API: where the models run, how you deploy them, and what you optimize—cost, latency, or a specific domain.

To train models on your own data you need the weights, plus infrastructure to post-train, serve, and route among them. Single-vendor dependence is a real risk. Models have already been pulled from products more than once, especially in coding, and as labs move into verticals it is not obvious they will keep serving companies they compete with. The durable move, Patil argues, is investing in the full stack so the models that power your product are yours.

The timing of the taping made the point for him. OpenAI had just announced a partnership with inference platform Baseten in which customers can burn down OpenAI commit credits on open models. Patil calls the Baseten team, including the Two Sigma and Danny circle, a company he admires. He reads the deal as a clear signal: customers want choice, and OpenAI wants to look more like a platform and a gateway for enterprises using AI.

A few months earlier, he notes, the industry conversation was still about the biggest, strongest model. Now it is about the Pareto curve of cost versus performance. You do not need a mega-model for every task. Open weights take that further: you can slide along the curve by picking different sizes and families. Applied Compute’s thesis is that you can also *move* the curve, by training models to be better at particular jobs for less money.

Effron asks whether OpenAI can actually be a neutral partner that serves whatever model is best. Patil is unsure. A marketplace is a natural fit for Amazon, whose Bedrock already runs Anthropic, OpenAI, and open models. Whether OpenAI will take that to the extreme and let customers run Anthropic models next to its own is an open question.

The first post-training stories Effron kept hearing were about bills and latency: similar workloads at massive scale, or a product that simply feels better when it is faster. Patil agrees that cost and speed are tremendously valuable. Frontier models still lead open-weight models on base capability, but if you have proprietary data that sits outside the model’s distribution, post-training can lift capability too. The more out of distribution the data, the more likely that lift is real.

Even then, most people are post-training to optimize price-performance: take a cheaper model and train it to frontier parity at much better cost, then serve more users. Effron pushes on what stays out of distribution as labs soak up more domains, especially coding. Patil’s honest answer is that the *training paradigm* will change. Today’s default is offline RL: buy a high-quality dataset, hill-climb it, and get a model that is spiky in that domain.

Online RL methods already use production inference to improve models, which is how he thinks most enterprises will eventually encode judgment: the more they use the system, the better it gets. The data that remains proprietary is produced inside the company. He points to extreme cases like experimental science—shouting out former manager Liam Fedus—and, in the enterprise, judgment traces: different risk thresholds, operating models, expertise, and history that drive different decisions. Some of that you optimize by configuration, prompts, and context; the harder part is optimizing the policy itself, how the model reasons and judges.

Effron wonders whether companies even generate enough data for this loop. Patil treats it as greenfield research, all about data efficiency and learning from sparse rewards. Pre-training is not data-efficient but generalizes well. Supervised fine-tuning is more efficient and still not good enough. RL with verifiable rewards trades quantity for quality via replayable environments and a verifier, and it is still “pretty bad” on efficiency.

Newer work in online RL, on-policy self-distillation, and related techniques is pushing on that bottleneck. His one-line diagnosis: what blocks continual learning is extremely data-efficient training from sparse rewards, and that problem is unsolved. Applied Compute is not waiting for a paper. They already run online training on large inference deployments serving tens or hundreds of thousands of users, feeding production data back into the weights, with a research-systems team trying new methods in production.

The long-term picture, he is confident, is a non-static set of weights with a real-world feedback loop. That loop is most justified when the world changes over time, or when the work is judgment-oriented rather than a manual process the base model will eventually cover. One customer has an assistant product: users assign tasks and mark what the model got right or wrong. Building a giant skills file is impractical, so the team biases the model from that feedback with online training and sees satisfaction, completion rates, and tool-call failures all move the right way.

The practical question is total cost of ownership. Applied Compute is trying to make RL cheaper, more automatic, and more online, without hand-tuned fully replayable environments. Harness and context optimization still come first; once those are squeezed, you can keep squeezing the policy that uses the tools. A chip-design customer built specialized software that general models do not use well; RL moved the needle. Co-founder and chief architect Lyndon, Patil says, is the wizard on the infrastructure side. GPUs are surprisingly underutilized. The cost cuts will come from more data-efficient algorithms and better systems, more than from exploding tokens into synthetic data.

Satya Nadella’s line that there should be as many models as firms is, Effron notes, a convenient selling point for Applied Compute. Patil agrees with the spirit and not the literal version. He does not think every company needs to pretrain or midtrain its own foundation model. He does think every company has closed-loop feedback worth capturing. Low-hanging fruit is still harness and context work. The thesis underneath is that companies are not the same: they have different judgment structures, and if everyone uses the same base model, great models set the floor while optimization sets the ceiling.

Pharma is the easy extreme: firms focus on different diseases and generate very different experimental data, so of course they should have their own models. Banks look more alike, and a frontier lab post-training on general finance data might capture a lot. Patil still sees a long list of trade-offs each bank makes that another does not. Some fragmented work, like accounting, may consolidate into shared models that take the menial labor out. Competitive industries with real operating differences will not.

Capability is not the only reason to customize. The industry is in a compute supply crunch. Squeezing more out of the models and GPUs you already have is an advantage even if you are only moving the Pareto curve, not the frontier. Doing the same job 10x cheaper than a competitor is differentiation you can win on.

Labs started RL on math and coding partly because those domains are verifiable and partly, Patil jokes, because everyone at the lab loves math and coding. Non-verifiable work is surprisingly climbable if you turn it into a proxy-verifiable problem. Rubric-based RL works quite well: give the model an expert answer and grade against it. Different companies have different experts, so optimizing against two groups can produce very different models. Keep your former employees off the data-labeling vendors, Effron teases. Patil does not disagree.

What we have with RL, he says, is a hill-climbing machine. The hardest part is defining the hill. That is why evals are something companies should both build and safeguard. Public benchmarks will be benchmarked. If you have a private picture of what good looks like for your business, it is an advantage not to tell everyone else. Employees are not fungible; you would not be comfortable sending them to work at a competitor. The same is true of the models you have taught.

App companies live in a bind: sharing evals with labs expands what you can ship, and the same insights are hard-earned secrets. Patil’s answer is that every company should invest in open-model infrastructure and a multi-model future, because you are otherwise caught between a rock and a hard place. Applied Compute itself qualifies customers in two buckets. One is capability that is extremely valuable—pharma, cybersecurity, chips. The other is a huge inference workload, where a quality or efficiency gain amortized over a large user base or billions of tokens is worth the work. Large serving plus out-of-distribution data is the ideal customer.

Most of the dollars in models go to serving, not training, so inference is where value is captured. Patil’s view is that post-training actually wins inference. The pattern he sees: start on frontier models, experiment, mature the workload, then look at open weights to push the Pareto curve. The largest workloads, where you spend the most, are where you should post-train first. Gains concentrate there on a power law. Training and serving can also be co-optimized. How you train a long-horizon tool-using agent—including tool-call parallelization—changes whether you disaggregate prefill and decode onto different chips, versus a decode-heavy setup.

Kernels that match across training and inference help, and training-side efficiency work transfers. A lever he thinks is overlooked: token efficiency. You can cut an inference bill 10% with serving optimizations, or make the model 10% more token-efficient against the same eval. Those are two sides of one coin, which is why Applied Compute wants to be the provider for both training and serving. Getting production serving up is the easier half. You need GPU capacity; vLLM and SGLang are strong starting points, and most inference clouds have sunset custom engines in favor of those plus workload tuning. Echoing Dylan Patel, GPUs plus vLLM plus OpenRouter “kind of” gets you a production offering—though 1% savings across a huge fleet still matters.

The large-scale vision is a new AI hyperscaler. AWS, GCP, and Azure turned CPU compute into networking, storage, databases, and data lakes. The new primitives are GPUs and a pile of weights. An inference engine is the first software layer that has gotten real stickiness in the AI stack. Patil’s bet is that post-training is the overlooked piece: how you turn GPU compute into more intelligent tokens. Routing, security monitoring, identity, and agent sandboxing sit above that. He draws an inverted pyramid: training at the bottom, where few can compete, then inference, then routing, then context and harness. Applied Compute is 16 months old and has moved from training into inference; the plan is to widen from the high-leverage bottom rather than start at the easy top. They do not do harness work or zero-to-one agent builds. Customers already have mature products and product teams. Applied Compute optimizes the model behind them.

Building an RL environment is the same work as building a great eval: the right task, the right environment of tools and services, and the right verifier. Applied Compute’s Harvey review-table model is the concrete case. With Harvey they collected expert rubrics and answers, supplemented with synthetic data, gave the model tools and a document base, and graded extraction against expert answers with rubric-based RL. Synthetic tricks like back-translating a document into question-answer pairs can expand volume. When it involves real lawyers, they work with the customer’s experts directly. Data volume is modest—a few hundred to a couple of thousand examples per task—so quality assurance is immediate in the final model.

Effron asks whether lab-versus-app collisions—OpenAI shutting off Cursor, Anthropic and Windsurf—will keep happening. It has happened twice, Patil says. Competitive dynamics are real, and as labs move into the application layer and compete with their largest consumers, he would not be surprised if it happened more. On generalization, RL does bleed into adjacent skills like error recovery and tool use, but train on math and transfer to legal is weak. Intelligence is a sphere with jagged points: nearby regions generalize, distant ones do not, and it is nothing like pre-training’s breadth.

What he failed to fully appreciate this year is how real Jevons paradox is. OpenRouter numbers show that a price cut makes usage spike. The world is becoming extremely cost-optimized, and that pressure is a bigger driver than he expected relative to capability. Most open-source models can already do much of what people want; frontier models are still amazing, but people increasingly care about the right model for the right task. Cheap, fast specialist models also unlock new training jobs that used to be cost-prohibitive, such as billion-token-scale classification of rollouts and massive trace analysis for online training.

Hiring follows the same principle as post-training: do not outsource the thinking. Patil is not against AI coding—he worked on Codex—but there is a difference between executing a design and delegating judgment. Onsites let candidates use whatever tools they want; then the team asks why the system is structured that way. “Claude did it” is a miss. Finance and biz-dev candidates can use AI to build a model, then must explain every assumption. For RL-environment companies, the moat is the expert network: firms like Mercor and Handshake can run a campaign and narrow to the highest-quality points. Models, he says, find reward like water finds the path of least resistance—including exploiting bugs in packages to pass coding tasks. That is also why cybersecurity is such a natural RL problem: strong coding priors, and lots of optimization pressure on both sides of the fight.