After a decade of code search, Sourcegraph’s CTO says the coding-AI fight is outside the model

Sarah Guo brings Beyang Liu onto No Priors as the cofounder and CTO of Sourcegraph, the company that has spent a decade building tools so developers can move faster. Their newest product is Cody, an AI coding assistant. Liu and Quinn Slack started the company in 2013; Guo thinks she met them at GopherCon that year or the next, already asking questions about developer productivity and code search. The through-line of the hour is not a new model. It is context: how you choose it, how you structure it, and why that work sits outside the language model itself.

Cody, launched in 2023, looks familiar on the surface—inline completions, codebase-aware Q&A, generate-a-unit-test, fix-this-compiler-error. It sits in VS Code and JetBrains, with experimental Neovim support, Emacs on the way, and an Ask Cody button on repository pages at Sourcegraph.com. Liu’s claim is that across that whole surface, the difference is the context they can pull in through Sourcegraph and through techniques refined while building code-understanding tools for human developers. Once models can see the right code, he argues, generation quality changes by a lot.

Liu and Slack are developers by background. They felt a gap between the promise of programming—flow, getting something new made, the joy of creation that got many people into the field—and the day-to-day of most professional engineers, which is toil and drudgery. Drill into why, and the time is spent reading and understanding existing code, a prerequisite for building new features quickly. They saw the same pain at Palantir, where they were dropped into large banks and Fortune 500 companies and embedded with those software teams. The struggle to understand legacy code and recover the context of a codebase was ten or a hundred times worse than what they felt internally. Sourcegraph was partly scratching their own itch and partly a bet that the pain showed up in every industry trying to build software.

Guo’s memory of the original idea is “super grep”: a version of search that works in real environments and is useful for getting back to flow. Liu’s first love in computer science was actually AI. At Stanford he concentrated in machine learning, worked in the Stanford AI Lab under Daphne Koller, and did computer vision in what he calls the dark ages of neural nets. After the late-80s and 90s OCR successes, the use cases petered out. Machine-learning 101 taught that neural nets had been tried a decade earlier and had not panned out; the work had moved to graphical models and statistical learning. Guo notes she has had Daphne and Lukas Biewald of Weights & Biases on the show, both lamenting that same period. Liu is still waiting for statistical learning and convex optimization to re-emerge. The pendulum, he says, always swings back, especially as people try to interpret why attention works as well as it does.

Sourcegraph started revisiting AI around 2017 or 2018, after the attention paper, when LLM applications in code began to appear. Tabnine was early to market with LLM-based autocomplete; Liu remembers chatting with someone who had implemented the idea on GPT-2, and it was nowhere near as good as it is now. They ran early experiments applying embeddings to code search. The results were interesting, not yet production-quality, but enough to keep going. Things really picked up in September or October 2023: internal work matured, Liu went on paternity leave and had room to experiment, and at the end of November ChatGPT landed and changed the game for everyone. That kick pushed them to explore in depth the efforts already underway.

Guo asks him to defend a view that has become more interesting as models get better: choosing and structuring large-repo context is the key unlock for code generation. Liu translates that into retrieval-augmented generation. Language models get a lot smarter when they are given a way to fetch context, most commonly a search engine. Bing Chat and Perplexity do this for the web; Sourcegraph does it for code, using code search plus something they call graph context to pull relevant snippets and documentation into the window so generation fits the codebase and Q&A hallucinates less. The human product already did this: search to find the needle, then walk the reference graph—go to definition, find references—without standing up a development environment or fighting a build system. The analogy is making exploring code as easy as exploring the web. Those same actions turn out to be useful for stuffing a model’s context window.

The retrieval backend is designed like a search engine, in two layers. At the bottom are retrievers. The base case is keyword search, or, in fashionable language, sparse vector search. Talk to RAG practitioners, Liu says, and the dirty secret is that keyword search can get you more than 90 percent of the way there. On code they combine classic keywords with regular expressions and string literals. How you index matters as much as how you query. It is essential to parse the code, extract functions, methods, and classes along with their docstrings, and treat those as separate documents. Indexing at the file level, or naive chunking, does not work for them. On top of that they run embeddings-based dense search over the same functions and symbols, plus helpers such as filename matching. The retrievers’ job is to preserve 100 percent recall while shrinking the candidate set enough that a slower, fancier method can run.

That second layer is the reranker. It would be too slow across the whole corpus; once the set is small, it bumps the most relevant results to the top and trades recall for precision. Human search is unforgiving—if the right result is not in the top three, people ignore it. Language models are more flexible, because even a 2,000-token window, let alone longer ones, can hold more candidates. Guo’s broader bet is that many domains will grow search pipelines attached to a language model, and that embeddings models and chunking strategy are data-specific. Domain-specific and even fine-tuned embeddings, from houses such as Voyage or built in-house, have a lot of headroom. Sourcegraph is collaborating with Voyage on a code embeddings model. Underneath all of it, Liu says, is a data-quality and data-processing pipeline. Model architecture only goes so far if the data is garbage.

Quality has a customer-facing meaning too. When Sourcegraph tells enterprises that Cody will search their codebase while generating, the first reaction is often: can I tell it to ignore large parts? Those directories are anti-patterns the team is trying to deprecate. The answer is yes, and it is easy at the search layer. RAG lets you be explicit about what you fetch. You can give someone a lever or a slider at query time. Sometimes you want the full codebase, especially for completions when you are just trying to get something out the door. Sometimes you want the model to attend to a golden copy somewhere in the tree, because shipping the feature is not the only goal—you also want to uplevel the quality of the code.

Liu splits the software lifecycle into two loops. The outer loop is the whole ring at team or organization level: plan a feature, implement it, write tests, submit to CI and code review, deploy, then observe and react. The inner loop is what a single developer iterates, often many times a day, on the way to a working patch. Unless you are a superstar who has already written the feature, the first attempt gets a lot wrong. You acquire more context, discover something that already exists, and learn. That learning process is what you want to accelerate. Every coding AI actually in production today, he says, is an inner-loop tool: editor completions and chat. Beyond those, Cody breaks specific forms of toil into commands—generate a unit test informed by the codebase, generate docstrings, explain code—laser beams aimed at the interruptions that pull people out of flow.

The longer vision is an AI engineer. The interface he describes is concrete: take an issue description, a bug report or a new-feature spec, and have the system generate a pull request that implements it without human intervention in the writing of the code. They are still a ways from that. The path is to automate the tedious, important busywork first—keeping dependencies up to date—then climb toward more sophisticated work. Guo offers migrating a database schema, and sketches a two-by-two of how tedious versus how high-stakes, with the product migrating toward the upper-right quadrant. She does not yet trust her own eye to put schema migration in the automatable box.

Getting there, he says, can be sliced in more than one way. One school invests in validation, test-time search, and sampling, hoping a feedback loop can make even small models work. Another school just wants smarter models that stop making stupid mistakes. Liu’s own cut is about issue-to-pull-request. You can decompose the task—which files to edit, which functions to add, which unit tests to write—until today’s models can solve each piece, then chain them. The limiter is per-step fidelity. If each step succeeds 90 percent of the time, the chain decays exponentially toward zero as it gets longer, and today’s systems are far below 90 percent. That compounding failure is what everyone building agents that way is hitting. The other path is AlphaCode-like: expand the context window, ask for a diff, roll the dice a hundred times, and keep the sample that comes with passing tests. It works on programming-competition problems. Zero-shot a new feature in a large codebase, and the number of rolls becomes cost- or time-prohibitive.

For both approaches, context quality is the lever. When Cody’s fetching engine works, the quality of generated code is night and day. Today’s LLMs are startlingly good at picking up patterns, the APIs in use, and the testing framework. That can lift reliability from a complete dice roll that needs a human in the loop to the point where two-, three-, or four-step workflows might chain. Guo is optimistic: it is an engineering pipeline with many inputs, each improvable, and for any given model quality there is some optimal pipeline they are far from today. All the dimensions are improving, so she still thinks the AI engineer arrives sooner rather than later. Liu is optimistic too when the context engine works. The open question is how much of that future system lives in a giant model’s parameters versus a traditional boxes-and-arrows software architecture.

Liu’s research-days principle is to do the simplest thing first. The fancy thing is sexier, and these days it is trendier: you can claim the mantle of having trained Beyang LLM on your own data. The lesson he took away is the importance of a baseline. Do the fancy thing first and you may have something that looks like a result, then someone else comes along with a dumber, cheaper, more iterable method that works as well or better. The example that stuck is a *Nature* paper in which a group trained a very large neural network for weather prediction and got good forecasts. A year later another *Nature* paper trained a network for the same task that was one neuron—literally a single aggregator—and it performed as well as the giant net.

That story is why they prioritized RAG over fine-tuning. It is not that they see no value in training at the model layer. RAG establishes a baseline; even after you have fine-tuned models in the mix, RAG is still last-mile data and context, so you want it anyway; and if you do it first, as Guo adds, you benefit from every improvement at the model layer, internal or external. Liu agrees completely.

He is also bullish on small models. Capable 7-billion-parameter and Mixtral 8x7B-class models surprised a lot of people; Cody had integrated Mixtral into chat the week before the taping. Small models are cheaper and faster, and if they approach larger-model quality for a specific use case, using them is a no-brainer. Inline completions already run on StarCoder 7B. With context, that matches larger proprietary models, and they are only scratching what context fetching can do. Faster inference and lower cost become a faster, cheaper product. The caveat is in-context learning. Some research suggests that ability emerges only at a certain model size or training-data volume, and fine-tuning a medium-sized model can spend parameter space on memorizing the training set and lose effective in-context learning. Cody’s chat still uses large models with more general reasoning, because those take better advantage of fetched context.

Asked what software development looks like in five years, Liu reaches for *The Mythical Man-Month*, the 1970s book that still rings true. Its thesis is that software is a strange, hard-to-measure kind of knowledge work, and the repeated mistake is to treat it as factory labor whose widgets are commits and lines of code. Anyone who has spent a month inside a real team knows the variance: some features eat thousands of lines and barely move the business; a one-line change can be a game-changer. The core problem, as he reads it, is coordination and visibility. Shipping the volume of software the world wants takes large teams, complex systems, and features that span many layers of the stack. A lot of the slowness and toil is coordinating people with different specialties and different incentives.

The real potential of LLMs, he thinks, is cohesion: getting a team to operate as if it were one extremely intelligent person, with a coherent vision, unified goals, and clear focus. One path is bottom-up. Make individual developers productive enough that more scope comes from a single human brain, which naturally carries a consistent experience and code quality. The other path is top-down visibility for whoever owns execution. Liu does not know a director or VP of engineering who reads the entire git commit log of their codebase; it would be so tedious they would have no time for the rest of the job. AI can read the code on their behalf, summarize the key bits, and finally give leaders transparency into how the system as a whole is evolving—and make that narrative visible to everyone else shipping day to day.

Guo takes Brooks’s law to its logical conclusion: the future is just her, a Jira or Linear interface, a really good spec, and one sprint later the AI engineer is done, because she did not have to communicate with a team. Liu grants half of it. If the goal is software as it exists today, a single human will be able to build applications that now require large numbers of people. The other half is demand. We are nowhere close to saturating the appetite for high-quality software, and humans stretch every technology they are given. Ambitions for complex systems will grow with capability. There will still be large teams; each person will own far more feature scope, the system will be more sophisticated, and people will still have to coordinate.

Guo closes with a hackathon she ran early in the year for talented undergrads building startups, ML demos, or research. This generation learned to code around AI tools; for some of them Cursor was the first IDE. A surprising point of view in March was that they just do not need to learn to code anymore. She sounds like the grumpy elder asking whether anyone still teaches garbage collection. Liu’s advice to a younger sibling or his own child is that coding remains an incredibly valuable skill. What will differentiate humans working with AI sits at two ends of a stack. At the top are product and user concerns: the right experience, the business objective. At the bottom are data structures, algorithms, and the specific machinery that rolls up to those goals. In the middle is a lot of mapping—abstractions, middleware, boilerplate—that is necessary, tedious, repetitive, and non-differentiating. AI, he thinks, will compress that middle.

The contrarian half of the take is that CS fundamentals will grow in importance. What you learn in a coding boot camp may get automated away. Knowing which data structures exist, what their properties are, and how to compose them creatively into solutions that meet high-level goals is the creative essence of software development. Humans will have more time to connect those dots because they will spend less time on the middleware. Domain expertise matters too: if you are building in a given field, understanding what actually moves the needle there is going to be really important.