Don't Train a Model. Factory's Founders Say Go Dig for Gold

On Sequoia’s *Training Data*, hosts Sonya Huang and Pat Grady sit down with Matan Grinberg and Eno Reyes, co-founders of Factory, a thirteen-month-old startup selling autonomous software-engineering systems they call droids. Grady opens with a shared taste for a well-executed cold call. As a Princeton undergrad studying string theory, Grinberg emailed Juan Maldacena at the Institute for Advanced Study, an institution whose allure is that you do not have to take graduate students, much less undergraduates. Maldacena will meet almost anyone for two hours at the chalkboard, then quietly drop a problem you have twenty-four hours to solve. Grinberg had been warned by graduate students, stayed up all night, and later published a paper with him.

The second cold email went to Sequoia partner Shaun Maguire, another former string theorist. A year into a Berkeley PhD, Grinberg realized he was doing theoretical physics because it was hard, not because he loved it. His research had already slid from pure physics to ML-in-physics to mostly AI. He saw Maguire speak, recognized the name from old papers, and was shocked that a bona fide string theorist had started companies and joined Sequoia — most string theorists, he notes, are not the most socialized. A planned thirty-minute walk from Sequoia’s Menlo Park office to Stanford and back became three hours. Maguire missed a lot of meetings, told him to drop out, and left three options: join Twitter right after Elon Musk took over, because only the most badass people would; join Maguire’s company Foundry as 'glue'; or start something.

He picked the third, in parallel with a reunion. Reyes and Grinberg had maximal overlap of Princeton friends without a single one-to-one conversation — they were in eating clubs opposite each other. They reconnected at a LangChain hackathon after Reyes flew back from Atlanta, got coffee at noon, and have worked together almost every hour since. Reyes’s family story is the one that does not appear on LinkedIn: his father’s side came from Mexico to San Francisco in the late 1960s; his grandparents opened a Mexican restaurant in Los Altos in the 1970s, moved it to Haight and Cole in the 1980s, then left for Georgia, where he grew up. After a Princeton CS degree he trained, optimized, and deployed LLMs for enterprise customers at Hugging Face, including early LangChain integrations with Harrison Chase, and spent time at Microsoft.

The siren call for AI founders, Huang notes, is training a foundation model. Grinberg’s aesthetic from years of math and physics was fundamentality, and in AI research it was obvious that code is fundamental to machine intelligence. Reyes, watching open source move toward modeling human cognition with the LLM as only one piece of the system — Harrison Chase’s 'cognitive architectures' — saw that the most advanced players in code generation were still doing autocomplete. Software, they like to say, is Archimedes’s lever on the world; AI code generation is a lever on that lever, compounding.

Factory’s mission, Grinberg says, is to bring autonomy to software engineering. Concretely, that means automating tasks in the development lifecycle: code review, documentation, testing, debugging, refactoring. As he lists them, it becomes obvious these are the jobs engineers do not enjoy. That is intentional. Code generation matters, and inspirational demos matter, but in most organizations engineers are not spending their time on fun development. They do review, testing, and documentation too late, then suffer for missed deadlines. If the tools are going to be useful in the enterprise, they have to meet engineers where they are, on the work they are eager to automate away.

They call the systems droids: one for each category of task, one for each step in a clearly defined software development lifecycle. Framing the problems as games is convenient because the lifecycle already supplies the rules. Grady offers a gloss they accept: a droid is a job-specific autonomous agent that actually works. The name is a deliberate dodge. Thirteen months after founding, Grinberg says he would have thought the industry had moved on, but *agent* is still synonymous with unreliable, stochastic demoware and vaporware. They want systems that are valuable today, not just cool examples of what is to come, and not just for a hacker on a side project — valuable to enterprise engineers. Fun fact: they originally incorporated as the San Francisco Droid Company, then changed the name to Factory on legal advice, given how eagerly Lucasfilm pursues its trademarks.

Huang has just had Harrison Chase on the show talking about custom cognitive architectures, and she wants the Factory version. Reyes sees huge innovation on both the foundation-model layer and the orchestration layer. The technical analogy is DeepMind’s Alpha series: when a problem can be modeled as a game — rules, an environment, feedback loops — you can build systems that match or beat human reasoning, whether the game is Go, protein folding, or code. Most of Factory’s reasoning work is inference-time search through decisions, a mix of intuition and planning. They are not training foundation models yet. Labs will keep improving latency, context windows, and performance on some subset of tasks. Anytime you need action, environmental feedback, and long-term planning, a single foundation model will have a hard time doing it all. That is where the application layer gets to innovate.

Huang cites the Princeton SWE-agent paper as a proof point: small open-source models can show striking agentic reasoning on code if the architecture is right. Reyes likes that team, and likes SWE-bench, but stresses that building these systems is not one benchmark. It is collaboration across the model layer, the tasks, the evaluation data, and the overall architecture. Asked for a favorite cognitive architecture — tree of thought, chain of thought — he will not pick one. In game-like problem spaces you can search, and techniques such as Monte Carlo tree search and language-agent tree search are interesting. There is no singular architecture that makes sense for all tasks. Breaking the lifecycle into semantically meaningful segments lets developers define the boundaries of the game; Factory’s work is figuring out which design fits which task. Huang hears Rich Sutton’s Bitter Lesson: search and learning are the two techniques that scale. Reyes agrees you need both.

Grady then quotes a Grinberg line from a few months earlier: there are hundreds of engineers at OpenAI working on Factory’s margins. It is not meant as snark. Plenty of startups are training or fine-tuning models; OpenAI and Anthropic are pouring resources into making foundation models better, cheaper, and faster. Factory does not want to run a race unsuited to its abilities. Training foundation models is a battle they will not win, and at this stage it is not a particularly unique battle. What has not come to fruition is products people actually use. In the analogy VCs love, there are a ton of picks and shovels, and no one is going for gold. The thesis is to use the beautiful black boxes that OpenAI, Anthropic, and Google are spending billions of dollars and hundreds of engineers to make, build a product people use, and only then earn the right to do the fancier things. If you cannot build a used product with these models, fine-tuning and training will not save you. It is probably just not a good product.

Grinberg’s other analogy is Jeff Dean as a stand-in for a really good foundation model, a GPT-6 with incredible reasoning. Drop him into your engineering organization with all its nuances and best practices, and raw inference is not enough to contribute reliably. Give Jeff Dean a LeetCode problem and he will solve it. Give him a twenty-year-old legacy codebase, some of it dead, some of it last touched by someone who just retired, and you need a deep understanding of the engineering *system* — not just the code, but why decisions were made, how people communicate, and what is top of mind this week. Those less sexy details are what Factory is focused on for the enterprise.

Most AI coding companies optimize individual developer productivity. Reyes wants the system. A VP of engineering is not focused on whether one person finished one task an hour faster. That person may have produced code 40 percent worse, so others rewrite on top of it, or spent four hours planning and blocked five other engineers. First-order metrics — how quickly someone tabs to autocomplete a statement, how quickly they finish an isolated task — miss bottlenecks. Factory tries to look at what engineering leaders actually measure: code churn, end-to-end open-to-merge time, time to first answer in the eng org. Those, Reyes says, are more impactful to an organization’s speed of shipping.

The AI product that has penetrated the enterprise most, Grinberg notes, is Copilot. The success metric held up is autocomplete acceptance rate. If you are a CTO, how do you take that number to the executive team? They do not know what it means or how it translates into business objectives. There is also a hidden danger: orgs using tools like this have been found to increase code churn by 20 to 40 percent. The studies have problems, but the direction is clear — as the percentage of AI-generated code rises, and if you do nothing different in review, churn goes up. With engineering leaders, Factory talks about three axes: saving engineering time, increasing speed, and improving code quality, then rolls them into one number they can take to a CFO: engineering velocity. That is what the droids are aimed at.

They are not super public on case studies yet, but Reyes will share the across-the-board numbers. Average cycle time is up around 22 percent. Code churn is down 13 percent. The test droid saves engineers around forty minutes a day. The other dividend of having thousands of developers on the tools is a live benchmark. SWE-bench is a gift to the community, an open-source way to compare systems, but strategically, real-world deployment is what let them increase iteration speed on droid quality.

The big surprise from production, Reyes says, is that engineers love ownership. An autonomous product that takes over a task has to deal with developers who are fickle for good reason: they are constantly bombarded with tools and automations. Anything enforced from the top down needs to be flexible. Everyone has a different idea of code review. Some people want superhuman linters, some want deep analysis of the change, some are annoyed by review entirely. Grinberg’s internal line is that code review is like going to the DMV: no matter how clean the office or how short the line, nobody loves it, because at the end of the day someone is criticizing what you did.

That split is why they insist on a fleet. Review is the kind of thing an engineering leader loves to see moving org-wide metrics, and a developer finds not fun. The test droid is the reverse: incredible for a developer who no longer spends hours on unit tests, slightly less obvious to a leader staring at business metrics. They are not building only for the VP or only for the individual. They are building for the engineering organization as a whole. Grady’s summary: he does not have to go to the DMV anymore; they can mail him the license. Huang’s jab: given how Pat drives, they should not.

How far out are fully autonomous software engineers? Huang reaches for Waymo: it felt imminent, then came a valley of despair, then the future sped up again. Which inning is this, and when do we get fully autonomous Jeff Deans? Reyes wants to reframe the question. Technical progress has repeatedly changed labor dynamics by raising the abstraction at which people work. Letting people affect software with less domain knowledge has historically exploded demand. What Factory already sees is that when people are freed from secondary tasks — generating unit tests for a pull request, maintaining docs that 95 percent of the team already knows — they shift to orchestration, architecture, and what the PR is actually trying to do, not whether they followed the style guide. Over time, software engineers look more like architects or designers. In the future he expects ten times more people involved in software creation, each with the impact of maybe 100 or 1,000 people today. It just may not look like today’s lifecycle steps.

Grady reads a line from Factory’s site: they hope to be a beacon of the coming age of creativity and freedom that on-demand intelligence will unlock. Grinberg ties it back to product positioning. It is tempting to go after the sexiest part of development — building an app from scratch — but that is also what makes a developer defensive, because that is the part they enjoy. Automate development and the engineer is left reviewing, testing, and documenting, which any software engineer would call a depressing hellscape. Factory wants to be aligned with the developer, not antagonistic, by automating the work developers do not want. Five years from now, nobody really knows what the job will be called — curator, cultivator, orchestrator — but if they stay beside the user, they stay in the high-leverage seat instead of trying to replace them.

How far to reliable intern-level engineers — a year, a decade, already here? It depends on the task. For code review and testing, Reyes thinks they are there. One organization told them, early on, that the review droid was the best reviewer on the team. Decomposing a monorepo into microservices, the kind of work you give a staff engineer with a team, will not flip on in a binary moment. Responsibilities will decompose into planning and implementing the refactor one file at a time; as those subtasks get handed to AI, the role starts being called something else, because you are no longer focused on tomorrow’s line of code. They ate their own cooking on a GitLab integration they had dreaded for months. The code droid specced the steps, implemented the sub-tickets while they monitored, and they now have an integration none of them had built before. They had been considering an intern. Intern work, Grinberg says, they did today. The future is here; it is climbing the complexity ladder.

Huang’s competition question is not who else is in coding agents. It is how you stay sane when every rival has the same models. Grinberg says that problem defined the team. There are many ways to justify your existence against competitors. Their answer is obsession: he is willing to bet everything that the people they assembled are more obsessed than anyone else in the space. The corollary is that the only way you can win is by executing faster; everything else is sprinkles. You also want people who treat external pressure as fuel. If a competitor does something big and you deflate, you were not actually obsessed with the mission. If the goal is bringing autonomy to software engineering, that noise is irrelevant. Execute in the direction you set; the rest sorts itself out.

Lightning round: if they could not work on coding agents, Reyes would look at robotics — much of the team comes from autonomy, multimodal function-calling LLMs are here, hardware costs are falling, and yes, physical droids are on the roadmap, jokingly. Grinberg cannot fathom working on anything else; tunnel vision is a blind spot and a feature. Who do they admire? Grinberg says Jeff Dean, for the research, for TensorFlow and the DeepMind-adjacent work, and because he has heard Dean is a nice guy — responsible leadership matters in a community full of Twitter clashing. Reyes will not name a lone genius; AI is done by groups. The team they admire is Mistral, for bringing models to the cutting edge in open source in a short time. Advice to founders: Grinberg repeats the gold line. Reyes adds a product test: build something that gets 10x better if OpenAI releases GPT-6 or GPT-7. Internally they think of Factory as multiplying in value when new models drop. If you are stressing about someone else’s announcement, it might be time to adjust strategy.

The bonus is last week’s SWE-bench result. The benchmark is about 2,300 issues from twelve popular open-source Python projects, typically bug reports, with human ground-truth patches and tests. Can an AI system turn natural language into a change that addresses the issue? The previous high watermark was about 14 percent, from Cognition’s Devin. Factory put up 19 percent, in a field whose leaderboard includes Microsoft Research, Amazon, IBM, and ByteDance. Reyes’s explanation is almost the anti-benchmark story: they were not building the code droid for a benchmark, but to support real customers. Customers are the best benchmark. The technical report talks about planning and task decomposition, environmental grounding, and codebase understanding, but the north star of a real-world dataset is what matters. Each droid has its own cognitive architecture, modeled on the human process for that task. The meta-problem is balancing rigidity and flexibility: rigid entry points and consistent error recovery, then looseness during most of the problem-solving.

Nineteen percent is far from letting a code droid run wild. Reyes’s answer is that people are already using it in production, on tasks that do not fully overlap the benchmark. HumanEval tests a coding interview; SWE-bench is closer to real engineering, but mostly debugging and unexpected behavior, not a multi-change migration with humans collaborating. Internal evals on those customer tasks show much higher reliability. The other half of the design is admitting where the system falls short: mid-flight edits, some interpretability into why a droid decided something, a human who can step in. You do not need 100 percent on SWE-bench to get productive gains. As for 80 or 90 percent, the pace is fast, but he suspects we will see SWE-bench 2 and 3 first — benches that ask whether code is not only correct but ideal for a given codebase — because some current tasks are so string-match specific that getting them right almost looks like cheating. Two or three years, maybe. The SWE-bench authors are Princeton people too. They even shared a thesis advisor.