They Refunded Nearly $2 Million. Then Factory’s CEO Explained the Dark Factory

Here is an unusual company decision. In its first two years Factory climbed to just under $2 million in revenue. Then the CEO decided to give the money back. Sales had closed the contracts; developers were not happy. Collecting the cash was a ticking time bomb: customers would churn, and the reputation would go with them. So they refunded everyone and promised to return in three months, this time with something that would knock their socks off.

The person telling the story is Matan Grinberg, co-founder and CEO of Factory, which builds autonomous agents for software development and calls them droids. This is his second appearance on Sequoia’s podcast Training Data. Host Pat Grady notes that second-timers are a small group, and that Factory is a dark horse in a market with a lead pack — Claude Code, Cognition, and others — that Factory is now coming up on. The conversation is dense: how to survive a desert when two years early equals two years wrong, why a multi-model harness can beat a lab’s own, how to allocate token spend, and a concrete forecast that in 12 to 24 months 90 percent of tokens will be asynchronous.

Factory was founded in April 2023. Enterprises were barely ready for GitHub Copilot, let alone fully autonomous agents. Grinberg calls the first two years a “journey in the desert”: the direction later proved right, but engineers were not ready and procurement teams were not ready either. Pat tries to give him credit for seeing the market two or three years ahead. Grinberg’s reply becomes the episode’s line. “Being two or three years early is the same as being wrong.” Nobody hands out bonus points for being early. There is no consolation prize. You either do the thing or you do not.

The explanation is personal. He dropped out of a PhD to start the company and had never held a job; enterprise sales does not come naturally to a physicist. Over those two years he convinced 20 of the smartest people he had ever met — people with families and kids — to quit and join, then walk customer after customer who did not get agents. Developers vote with their feet, he says. If you are giving them an NPS survey, they already dislike whatever you are measuring. The desert was not wasted. What they learned is the line enterprises will not cross: model independence. Nobody gets to be a single point of failure. Nobody gets to control their fate.

Enterprises still have scars from the cloud era. AWS and Azure signed three-year deals, subsidized the price, then 10x-ed at renewal. Want to leave? Start a two-year migration. Against Claude Code and Codex, products that arrive with enormous distribution, Factory’s pitch is that you still get frontier-model performance without putting your fate in one lab’s hands. A new model that is faster or cheaper can be hot-swapped in. Pat’s follow-up is the obvious one: are you just moving the lock-in to Factory? Grinberg’s answer is that automations, the skills registry, and the artifacts live in the customer’s own codebase. Factory is not sitting on tribal knowledge about the org. Cost savings from arbitraging models go to the customer, not into Factory’s pocket. The way you build trust is to make it easy to leave.

The refund traces to an operating principle: create obsessed customers. That is a flip of Bezos’s customer obsession. In Grinberg’s mind, being obsessed with the customer is an input metric, and input metrics are not worth measuring. “You could be customer obsessed and they file a restraining order against you,” he says, because they do not like what you are doing. The job is to build something so good that the customers themselves become obsessed. He reaches for basketball. A coach does not tell players to make sure they sweat. A coach says score points. Sweat is a byproduct. Measure the output, not the input.

Held to that ruler, the product failed. So came the refunds, and the all-hands that Grinberg calls one of the worst months of his life. Faces in the room went pale: is this the early sign that the ship is sinking? Telling investors was not easier. He remembers the conversation with Shaun Maguire at Sequoia: remember those updates where revenue was going up? It is about to go to zero. Internally they could already see the product scratching an agent-native itch. Customers were not there yet. Giving the money back was a way to keep the trust so they could return in three months and say this is the real deal.

The team did not scatter. Grinberg credits two things. Early hiring was ruthless: only people genuinely obsessed with bringing autonomy to software engineering. And they did not pretend. Competitors’ revenue was going crazy; Factory had just given all of its revenue back; they needed to get it together. The company has rowers, and rowing is a pure pain sport with one number that quantifies you. Shared rock bottom, he says, is what forges the deepest bonds. A background number: Factory started at a $5 million valuation. A lot of competitors, he notes, have never known what it is like not to be a unicorn on day one. His team has been to the bottom. Not a single person left.

The turn is a date. On September 26, 2025 — two years after founding — Factory shipped Droid CLI, a command-line agent that could use every major model and, at the time, sat at the frontier on the benches that mattered. The interaction they had been building for was too ambitious. Fully autonomous agents demand a complete change in developer behavior, and they tried it before many people were even using Copilot. By late 2025 most engineers had autocomplete, and many were starting to hand wholesale changes to a chat agent. Sonya Huang asks the natural question: was the biggest change that the models got better? Grinberg says not exactly. The biggest change was that developers, especially in the enterprise, lowered their guard and tried.

Thirty-year workflows make people stubborn. A lot of engineers believed their craft could never be done by an AI tool. Several forces had to line up: models strong enough that you needed fewer guardrails; more agent-native colleagues inside the org who could demonstrate the work; and, funnily enough, Andrej Karpathy. Early on he was not open to agentic work. When he started tweeting about it, a cohort of engineers changed their minds. You hear it from people you trust. Behavior change happens that way. Benchmarks, meanwhile, have a short half-life. A good one gets benchmaxed in three to six months and leaks into training data. Factory championed Terminal-Bench at launch because SWE-bench was too Python-and-single-file. Terminal-Bench covered real terminal work. It is now around 90 percent.

Then comes the most counterintuitive technical claim of the hour. The default assumption in the industry — Sonya shares it — is that if you train the model and write the harness, co-design makes both better. Grinberg says this will chagrin his friends at OpenAI and Anthropic, but it is not true. A harness that supports many models ends up stronger than a harness built for one. The analogy is pre-GPT-3 personal AI: people wanted to train a model on all of their own data so it would “know them.” The winning answer was train on the whole internet. What data is to a model, models are to a harness. The more models you feed it, the less it overfits to one lab’s quirks, and tricks learned on one improve the others. Before Terminal-Bench was fully saturated, every new Opus or GPT model scored higher inside Droid than inside Claude Code or Codex. Labs would prefer their model to be best only in their own harness. Reality did not cooperate.

Where the harness actually wins is unglamorous. Cached tokens cost about a tenth as much, so cache-hit rate is a performance number. Long sessions blow past the context window; the harness has to compact, and the test is needle-in-a-haystack: does the one critical fact survive summarization? Tool use and using the environment to validate work also matter. He illustrates with model personality. Opus is the friendly colleague who looks at your list of 20 tasks, drops five as unnecessary, and suggests a beer. GPT-5.6 is the OCD executor who will not sleep until all 20 are done. Early Claude Code had to nail the agent to a to-do list because the model liked to wander; Codex did not, because the model was already meticulous. Users still want the same experience when they switch models. Lose the list in compaction on step seven of a large migration, and the agent forgets what step seven was. That is a failure mode a multi-model harness is built to prevent.

Grinberg walks through enterprise AI buying as a cartoon in four frames. Phase zero: nobody believes in AI. Phase one: everybody does, and the board asks the CEO for a strategy, who yells at the CTO to make everyone use it. Phase two: the CTO puts AI use in performance reviews and publishes rankings of who burned the most tokens, because stubborn engineers will not move otherwise. Phase three is token maxing. People throw Opus at everything, including the weather in San Francisco. Factory works with banks spending hundreds of thousands of dollars a month on questions you could Google — what is the weather, tell me about Python. The overcorrection happened because everyone was terrified of non-adoption. He actually thinks the sequence is decent: prove it works, then mature the usage, rather than start limited.

Factory’s answer is the Router, which dynamically sends work to different models by task. Weather does not need the frontier of human intelligence. A COBOL codebase may want a fine-tune more than Opus. If you care about reliability, generate with OpenAI, test with Anthropic, review with Gemini. Routing rules can be written in natural language — even “don’t let Pat use open-weight models.” Sonya’s amendment: give Pat Gemini Flash. A blanket token cap for every person in a large bank, he argues, will look ridiculous a year from now. Product managers vibe-coding dashboards should not share a ceiling with engineers on critical infrastructure. Every CIO will have to answer, for every incremental token, where it goes.

Open-weight progress is the other data point. Grinberg calls GLM 5.2 “incredible.” Factory’s own engineers have no token limits, and half of internal tokens already go to open models — faster, cheaper, and comparable in performance. The comparison everyone gets wrong is GLM 5.2 versus the latest Opus or GPT. Open models arrive a generation behind; the fair match is the previous frontier, Opus 4.7 or GPT-5.5. Can open models match frontier-minus-one? “Unequivocally yes.” At the start of the year, less than 1 percent of customer tokens went open. In Q1 it became a single-digit percent. It has now crossed into double digits. His forecast is that token share will asymptote toward a vast open majority, because it is cheaper and more optional — but never 100 percent. Frontier intelligence stays valuable. The 1 percent of tokens that make the key decision have a different leverage.

He calls the idea intelligence allocation and gives it a household version. If you need an algebra tutor for your daughter, you do not hire Einstein. A high-school student will do, unless she is going to become a physicist. Map that onto an engineer’s day: a lot of it is low brainpower, and writing documentation is the classic example. Stripe extracted real advantage from great docs, but spending your best engineers’ hours on docs is a low-leverage use of them. Automate the low-leverage stretches so that eureka moments last hours, not minutes. Pricing follows the same logic. Usage-based is the alignment today; seats do not make sense for this product. He expects the 2030s to look more outcome-based, once enterprises are ready — they learned not to impose the future two years early. The sketch is a marketplace: an engineer wants a task done, labs bid a cost, validation loops decide success. Bid too high and you lose; too low and you lose money. Agents will already delete the code to “fix” it, so the incentive is to write brutally thorough tests.

The company was named Droid first, then Factory, before the phrase “software factory” was everywhere. One inspiration is Tesla’s plant: robotic arms doing the work, and Elon’s line that the factory is the machine that builds the machine. The end state is a dark factory — lights off, production still running. Grinberg’s read on the present is blunt. Every company already has a software factory. It is just extremely inefficient, like pre-industrial sewing. In an organization of more than 10,000 people, the path from deciding on a feature to releasing it involves hundreds or thousands of people, and most likely nobody can draw the process. It runs on the guru who has been there 30 years, a stack of approvals, and a checklist nobody remembers — tribal knowledge and human redundancy.

He points to a Jack Dorsey post: every company is like an AGI. If that is true, you should optimize the weights. Figure out which nodes are load-bearing, which need more tokens, where you need more nodes. You do not do backprop by vibes; you run the numbers. Real companies, by contrast, set token budgets by shooting from the hip. They lay off 20,000 people by shooting from the hip. There is no science that justifies the number 20,000. It is a chunk, to see what happens. Two years from now the CIO’s question will not be where each incremental token goes, but where each incremental dollar goes: headcount or tokens, and in which part of the org? Add tokens to a sales team and output probably does not move, because the alpha is sitting face-to-face with the customer. Add tokens to engineering and output actually rises. Those judgments need closed loops: the feature shipped, did retention move, did usage move. Ten years from now this era of software will look like ancient businesses that did not keep books. Pat’s add: like marketing in the *Mad Men* years, all creative, nobody knows what worked.

That forces a ruthless look at core competency. Grinberg has watched the counterexample: a consumer logistics company insisted on building its own software-development agents, then six months later admitted this was not the business. The internet-era analogy is a pizza shop that had to hire programmers to put up a website. Today no pizza shop runs an engineering department, and nobody mourns it. AI will force every company into a cold focus: double down on what is core, and buy the rest. His concrete prediction: in 12 to 24 months, 90 percent of tokens will be asynchronous. Claude Code usage looks ferocious now, but it is synchronous. If everyone woke up sick tomorrow, a lot of it would go to zero. The agent-native era is droids in the background — a customer signal appears, they go fix it, or they draft a first-pass solution. Lights off. Work continues.

He also throws cold water. Short term there will be turbulence. A lot of companies misallocated resources and got bloated, and the correction will be painful. Every AI CEO, he thinks, should bear more responsibility for that than they currently do. Long term he does not believe the line that engineers are going away. The world has a huge number of problems; a large subset can be solved with software; a small subset currently are. Freed engineers will move to problems that never got software talent — terrible government sites, pharmaceutical research that is as much a software problem as a biology one. The industry’s job is to do that reallocation in six months to a year, not ten.

Asked how customers actually get through the transformation, Grinberg’s answer is almost too plain. The companies doing best tend to do the same thing: a company-wide hackathon. Set aside a day. Everyone puts down their work and builds something with AI. If it fails miserably, that is fine. The point is to set the tone. EY is one of Factory’s largest customers and is not known as an AI frontier firm. Their stance is clear: they were late to other transformations, they will not be late to this one, they might mess up, they are going anyway — obviously respecting the things they are not allowed to mess up.

Two other splits show up among the orgs that succeed. They are willing to say there are no sacred cows: park the existing process, try something, and put the cow back if it does not work. And the change comes from inside — the tech team, the ICs, the leadership — not as a decree from the board. If it comes from the board, it probably will not go well. Put together, the list you can copy tomorrow is short: measure outputs, not AI-usage theater; decide each incremental dollar between headcount and tokens; refuse to bet the company on a single model; write validation so strict that deleting the code cannot pass; and if the team has not moved, run the hackathon and allow failure.