He Hasn’t Typed a Line of Code in Months. Here’s How Augment Reads a Million-Line Repo

Nathan Labenz opens this installment of The Cognitive Revolution’s software-supernova series with Guy Gur-Ari, co-founder and chief scientist at Augment. Earlier episodes looked at vibe-coding tools that take a prompt to a proof of concept. Augment, founded in 2022 when OpenAI’s Codex-style autocomplete was still mostly a hint of a different way to work and ChatGPT did not yet exist, is pointed at the other end of the industry. The bet was that professional engineers spend their days in vast, messy, long-lived codebases, and that a startup willing to solve context there could differentiate.

The company has since raised on the order of $250 million. Typical customers have hundreds of developers; some have thousands. Repository layout varies: some teams live in monorepos, the way Augment does internally, others keep one repo per microservice. Lines of code start in the millions and go up. In organizations of that size, zero-to-one greenfield work is the exception. Day-to-day engineering is feature work and maintenance that has to fit the code that is already there.

That is a different product than “prompt your way to an app.” It is also, Gur-Ari argues, where most professional developers actually live. Labenz flags the result up front: he used a preview of Augment’s coding agent, and Gur-Ari reports that he personally has not written a line of code in months. The rest of the hour is the stack that makes a claim like that even plausible inside a million-line repo, plus the builder habits Gur-Ari wants other AI teams to steal.

Labenz’s personal workflow for small projects is to have the model print the whole repo to a file and dump it into the prompt — o1 pro for planning, Claude, or Gemini 2.5, which can take him up to a million tokens. That trick dies in enterprise. Gur-Ari’s conversion is blunt: the ratio is roughly 10 tokens per line of code, so a million-token window holds about 100,000 lines. In industry that is still a small project. Stuffing even that much into the prompt has other downsides as the repo grows, but the arithmetic alone is enough to kill “just paste the codebase.”

The missing piece is not only volume. A developer who has been on a team for a while already knows which APIs to call, which parameters are correct, and which of several legal approaches matches local convention. A model that does not get that context produces bad completions and bad chat answers, no matter how clever it is. Augment’s premise from the start was to bake full codebase understanding into every feature: a completion should look up the function you are calling and other usages of it; a chat question like “where is the helper I used six months ago?” should search the whole repository.

Agents raise the stakes. As the model takes on more complicated work with less live supervision, every piece of context is doing judgment the developer used to do in real time. Labenz quotes economist Tyler Cowen: context is that which is scarce. Cowen said it before the LLM era. Labenz thinks it is ten times more true for the agents people are trying to make useful now.

Augment did not start with retrieval. The team tried several approaches to codebase understanding; the one they shipped was the third, and each attempt was a multi-month research project. What they landed on can be described as RAG. Code is uploaded, custom-trained retriever models built for this job index it — that is the “indexing your codebase” screen users see — and then every keystroke and every chat request hits the server. Part of serving that request is deciding which slices of the repo the generator needs in order to make a good prediction.

Background indexing is one problem. Interactive speed is another. A completion has to come back on the order of 300 milliseconds, covering both retrieval and the language-model call. Gur-Ari treats speed as a first-class product feature, not an optimization pass. Quality still matters: the retriever has to be good enough that the whole system *feels* like it understands the repo. Both constraints are in force at once.

He will not open the secret sauce the way some RAG vendors do. Getting retrieval right is hard; getting it right on code is harder than in other domains. Completions are passive. Unlike a chat question, the user often has not stated the task. The system has to infer what the developer is doing and then decide whether to pull a function signature, example call sites, or even counter-examples. The snapshot of the repo is also stale in places. An example usage might be the current idiom or leftover code the team no longer wants called that way.

The working system is a mix: RAG, some static analysis, multiple models assembling context, and a way for the user — especially in chat — to point at a directory or a file and steer the retriever. On the endlessly debated question of chunking, Gur-Ari is dry. There are better and worse ways to cut code, and code has structure you can hang on to. If the retrievers and the models are strong, chunking is not the blocker. Improving it mostly mops up the tail.

Labenz has been using Augment almost entirely through the chat panel, almost never typing functions by hand. Gur-Ari sees two camps, with overlap. Some developers want to stay close to the code: they barely use chat, they love completions, and they like Next Edit, which can suggest changes away from the cursor, including deletions and rewrites, not just insertions. Others live in chat. Agent mode is another step back: the model edits multiple files, runs tests, and the person supervises, diving into the code when needed. In a large codebase that dive-back is frequent, unlike a greenfield prompt-to-app session. He thinks most enterprise developers are already comfortable in chat a lot of the time. A default of fully autonomous agents will take longer, because the models still have to get better. He still thinks that is the direction of travel.

The go-to-market choice was an extension, not a VS Code fork. Augment ships a VS Code extension, a JetBrains extension, and Vim support. The first reason is meeting people where they are. Switching from VS Code to a VS Code fork is a small workflow change; asking a JetBrains user to move is not. Forks also mean tracking upstream updates, especially security patches, which is extra maintenance, and enterprise security reviews are shorter if you ride on Microsoft’s work. The cost is UI that is harder or sometimes impossible. Gur-Ari says the VS Code API has been enough, and that as the product moves toward agents — panels, web views, work that is not inside the text buffer — the fork-versus-extension distinction will matter less. He will not promise they will never fork.

They built their own vector database because nothing off the shelf met the product requirements. The model has to feel like it understands the *current* codebase: if chat just wrote a function, the next test-file completion has to know that function exists. That means near-real-time index updates, and a different view per developer, because feature branches must not mix — also a security requirement — without duplicating the whole database for every person on the team. Standard vector databases cluster embeddings to avoid a full scan. Updating those clusters is expensive, and a statistical query over a subset of files can miss the target files entirely. They are still iterating on the store as customer repos get larger. Could the database be a product on its own? It has come up. So have a lot of other lab-shaped side quests. They are trying to stay focused on the assistant.

For people standing up a RAG app, Gur-Ari would not start by building a vector database. Most systems do not have Augment’s latency constraints. Start with an off-the-shelf store and obsess over quality. Quality starts with an evaluation set you trust. It does not have to be huge. Most of their research projects begin with 10 to 20 samples labeled by hand. Run the easiest baseline — an off-the-shelf retriever — and see whether you are solving 20% of the set or 80%. Then hill-climb. When the set saturates, expand it: more samples, more diversity, more difficulty. Coming up with good evals and actually running them is, he says, one of the hardest things in research, not because the work is heavy but because it is tedious. It is also how you get good results. Pure vector search almost never carries a real retrieval system. Mix embeddings with structured queries and any other signal you have. With modern models, recall matters more than precision: get the right chunks into the context and let the model sort the noise.

An eval you trust is the ideal, and it will eventually diverge from real user experience. That has happened on every project they have run. You cannot boil “how people use an AI product” down to one number: the distribution of what they type and what they expect is too wide. Dogfooding and, later, user feedback have to sit beside the score. The destination he wants is turning philosophical arguments — vector or SQL, this model or that — into experimental questions. Labenz, whose company Waymark makes video for small businesses, notes that many tasks have no single right script. Even then, 10 examples of what “good” looks like get a team onto the same page, and that alignment is often the actual stumbling block. Gur-Ari agrees. With so few samples you can hold them in your head. You can run the old system and the new one and compare by hand. The number can come later. The 10 examples are the minimum.

Labenz’s other rule of thumb is to turn RAG hyperparameters up: more chunks, more expansion around a hit. Developers leave those knobs too low. Extra latency and cost almost always pay for themselves in time-to-something-that-works. Gur-Ari fully subscribes, with the usual three-way trade-off of latency, cost, and quality. Models of the last year or two were trained to deal with distracting retrieved context; attention is built to sift noise. Improving recall so the right chunk lands in the top 50 or 100 is doable. Requiring it in the top five or ten is exponentially harder. The exception is instructions. Flooding a prompt with hundreds of rules does not scale yet. The model starts ignoring them and skipping steps. That is a prompt-scaling problem, not a RAG problem.

When is it worth fine-tuning your own retriever instead of using OpenAI embeddings? Gur-Ari refuses a universal answer and pushes process instead. People reach for complicated solutions too fast. Bias toward simplicity. Optimize experiment time: taking a loop from hours to minutes changes not only how fast you find an answer but whether you find one at all. A hundred simple experiments — five open-source models, SQL-style queries, OpenAI embeddings plus extras — beat ten elaborate ones, for the same reason the right chunk is easier to catch in the top 100 than the top 10. Only when nothing off the shelf is close should you start thinking about fine-tuning. Let the eval tell you when that is.

Augment trains more than retrievers. Completions and Next Edit run on generation models of their own, improved with a method they call reinforcement learning from developer behaviors. The data split is sharp. Enterprise customer code is not looked at; it sits behind audit and access control. A free community tier is explicit: if you are comfortable with the company using your data to improve models, you can use the product for free, on open source or otherwise. That stream is how they see the real input distribution — what people are trying to do, and what they expect back — which is otherwise the universal hard problem in AI products.

Coding in an IDE is unusually generous compared with chat. In a chatbot you never really know the ground-truth answer. In an editor you eventually see what the developer wanted, because that is where they work. Reinforcement learning uses that. Each sample is an input plus a better output and a worse output; the model learns from the contrast. There is no single correct answer. The algorithm can be right and the style still wrong for this repo. They first applied the method to completions and got a better completion model out of it.

DeepSeek’s GRPO did not blow their minds as an algorithm; it felt like the next iterative fix to known RL problems. The chain-of-thought reasoning training in that work, he says, was remarkable, and he wishes the paper had shared more. The earlier bet that actually pays them every month is different: they do not train from scratch. They wagered, before Llama, when open-source models were still bad, that those models would get good fast. Gur-Ari had trained large models at Google and thought the playbook was already in the literature. They post-train the best available open-source bases for retrieval and generation, and they ride every new release.

Why is this pattern rare? Because most products cannot get ground truth. If a doctor or a lawyer is editing a document, you can, the way you can in an IDE. If they are in a chat box, you are guessing. As internet data runs out, he expects user data to get much more valuable. Everything so far is still human data — web text, RLHF, humans supervising at a higher level. Breaking past human level needs another reward. Code is first in line because you can execute it; DeepSeek already uses execution feedback. A story, a poem, or an essay has no such signal unless a stronger model is already judging. Science might get experimental validation. Reward hacking in their own developer-behavior RL? He cannot think of an example. The boring failure mode won: the model just did not understand what the user wanted.

Public numbers: on the order of $250 million raised, about 100 people on LinkedIn. In classical SaaS math that looks like a long runway. Gur-Ari’s correction is that AI is more capital-intensive than a basement Google Cloud app whose main cost is salaries. Training costs money. Inference costs money. Every keystroke is a retrieval plus a model call; chat is more; agents are a step-change. A single user instruction can spawn ten or more language-model calls, including large ones that edit files, run commands, and parse output — and that is still one agent in one IDE. They lease GPUs. They are not in the data-center business.

He has been using the agent himself. He has not written a line of code in several months. The agent has written a lot. The value is obvious, and it is early. He expects agent usage to explode over the next year, and cost with it. A given model gets cheaper, but people always want the latest one, and that does not cheapen as fast as usage grows. Model-price declines will not keep up. Unlike traditional SaaS, cost is a first-order problem.

List prices today are conventional, $30 and $60 a month, but some power users already cost far more than that to serve. Labenz’s rule of thumb is that companies should expect to spend about $1,000 a month on AI per employee. Gur-Ari does not have a definitive answer. There is a live question of fixed subscription versus consumption. Augment already sells credits: a developer who uses the product that month consumes a credit, and one who does not does not pay — unlike seat licenses you pay for idle or not. That was meant to align company and user. He guesses they will lean harder into consumption because of cost, but he is not sure. Usage is a wide distribution: some people already justify $1,000 a month, many do not.

The harder break is coming. Today the mental model is an interactive agent while the developer gets coffee and comes back in a few minutes. He is pretty sure the next year brings agents that run for hours, overnight, or over days, and agents on non-user triggers — API calls, automatic code review. In that world per-developer pricing does not even describe the unit of work. Until all of this is cheap, the pricing model will have to move. Labenz’s allergy is a product that sandbags quality to protect a fixed-price margin. Gur-Ari’s current stance is closer to accepting some $1,000-a-month users and figuring the rest out later.

Labenz floats continued pretraining on a company’s private data so the model “knows” the firm the way today’s models know the public web, and can tell whether retrieval actually found the right thing. Gur-Ari sees several problems. Training is sample-inefficient. A typical company knowledge base is too big for a context window and too small as a training set, so you would run multiple epochs and overfit into memorization. Keeping it current is worse: RAG latency can be pushed toward instant; retraining cannot. Per-user stores such as email make the logistics absurd. His advice is to try an agent that searches several ways until it thinks it has the answer, which is what the more advanced versions of their own retriever already do, and to reach for that before fine-tuning. He thinks you can get good enough performance without a model that has a human-like “I found it” feeling. That is how he reads their evidence.

Dario Amodei’s claim of superhuman coders this year or next gets a split verdict. If the question is “what generated the new line I am looking at,” Gur-Ari buys that it will be a model or an agent, not a person — not in three to six months given current adoption, but on a horizon shorter than three years. That is not the same as the model deciding what to do, supervising its own output, or running fully autonomously. The way he works, and the way he sees other people pick up agents, the code is almost entirely generated once they adopt the tool. If you took the human away, nothing useful would happen. The models are not good enough for “here are the product requirements, go build this.” Software developers are still needed this year.

Advice for people early in a software career is harder, and he is having the conversation at home. His children are seven and fourteen. For the older one he recommends a path tied to the physical world — mechanical engineering, robotics — because it will take longer to disrupt, and because he cannot see what software development looks like in three to six years. He still thinks we will need developers who understand the system. Vibe-coding enterprise software already goes wrong; he can see it in code he is “writing.” It will get better. He does not think it gets better at that scale. How many developers, and how much software, he does not know. Augment still hires juniors and seniors. They look for excellence. Getting value out of agents, especially inside a real codebase, has a learning curve, and less-experienced people are often fastest onto new tools. The on-ramp is still there. His closing pitch is simply to download Augment and feel an assistant that actually understands the repo you work in.