Daksh Gupta on Building a Better AI Code Review Agent

I’m Daksh Gupta, co-founder of Greptile. We build agents that review and test pull requests with the full context of a codebase, looking for bugs, logic errors, and code-quality issues—especially in AI-generated code. Thousands of companies use Greptile, and Gupta says roughly 1% of GitHub pull requests pass through the service.

Some companies have needs so specialized that building an internal reviewer makes more sense. Greptile helps those teams because it has learned how to review codebases, run applications in sandboxes, and allocate inference efficiently. Gupta will explain that approach for teams that want to build a reviewer or understand what makes one good.

Gupta groups the product’s strengths into three areas. First is codebase context: Greptile began by teaching early language models to understand large repositories and built strong indexing capabilities. Second, it starts a customer’s stack in a dependency-rich sandbox and uses computer interaction to exercise the application and API endpoints.

The third ingredient is allocating inference carefully: choosing the model and inference budget that fit each review and validation task. Gupta says that combination can deliver more performance per dollar than an internal tool or competing product.

A team may want to build its own reviewer, or it may be choosing a review or testing product and trying to distinguish an adequate one from a great one. Gupta turns to the simplest possible design to show where it falls short.

Imagine you are on a platform team—or the person most eager to bring AI into the engineering workflow—and code is arriving faster than people can validate it. Engineers and even marketing teams are vibe-coding, creating a review bottleneck. The first step is to get a sandbox, either from a provider or from a tool such as Firecracker.

Put an agent in that sandbox, connect GitHub or GitLab, and use a pull-request webhook to send it each new change. Give the agent a simple instruction to review the pull request, then return its comments through GitHub’s review API. In principle, that is enough to produce feedback about five minutes after a pull request opens.

The trouble starts when teams rely on that bare-bones loop. First, it will miss bugs. Bugs can arise in many interacting ways, and a reviewer must at least catch the ones that could lead to serious outages or incidents.

Second, the agent may not understand how the codebase fits together, what direction the team is taking, or how a particular library works. In a company with thousands of repositories, a reviewer may need context from many of them to assess a change correctly.

Third, the cost can surprise teams: Gupta says companies have reported spending between $3 and $25 per pull request on coding-agent review. Token use accumulates quickly, and the expense can grow quietly.

Fourth, the agent may leave nitpicky comments: technically correct observations that are overly pedantic. Developers then have to sift through a flood of minor style complaints to find the serious issues. Worse, coding agents may obediently address every comment, turning a codebase into defensive, awkward code.

Repeatedly running the reviewer on the same pull request may surface a fresh set of bugs each time. That makes it difficult to tell when the review is complete and reinforces the need to focus the system on useful findings.

The easiest improvement is to make the prompt more specific. Instead of asking only for a review in GitHub’s comment format, direct the agent toward bug classes that matter to the company and tell it to skip nitpicks. Teams can study previous outages and incidents to identify the mistakes their codebase is most likely to produce.

The prompt should define nitpicks explicitly, such as demands for extra layers of defense without a concrete risk or complaints about a slightly imperfect error-handling style. Even a small prompt change can greatly alter the number of comments, which makes it hard to know whether the agent has actually improved.

A reviewer can show that it found a bug, but it cannot prove that no bugs remain. This is especially difficult when serious failures are rare: Gupta estimates that only about one in a thousand pull requests—or, he rephrases, perhaps one in a thousand lines of code—contains a catastrophic bug. An agent may miss one and appear successful for a long time.

The second improvement is a dataset of historical pull requests. If the company records outages and incidents, collect the pull requests that caused them and run each new version of the reviewer against those examples. That gives the team a clearer basis for tuning prompts and measuring whether the agent catches known failures.

The dataset needs regular updates because the patterns and kinds of bugs change. Many bugs repeat earlier mistakes, but increasingly capable coding agents also make novel errors. Without millions of pull requests, a company may struggle to cover every bug type; Greptile’s broader cross-company corpus gives it a larger evaluation set, while internal teams can still make useful progress with their own data.

The third improvement is model inversion: use a different model family to review the code than the one that wrote it. Gupta’s research suggests Claude models are more prone to one-off errors, while GPT models are more likely to introduce authorization bypasses. These differences can reflect how each family was post-trained.

If the same model writes and reviews a change, it may repeat the same blind spots. So, if Claude Code authored the code, try a GPT model for review, and vice versa. Identifying the author model is straightforward much of the time: Codex may leave a codex/ branch name, while Claude may appear in co-author metadata or the pull-request description.

The remaining cases are harder to classify, so teams may need heuristics, an internal record of which coding agents engineers use, or telemetry from developer machines. Gupta says Greptile observed a 10% higher bug-catch rate when models were flipped. He argues that even a seemingly small increase matters when the failures in question can cause incidents or outages.

The fourth improvement is better codebase context, an area Gupta says Greptile worked on before code validation. He argues that plain grep is both imprecise and costly: coding agents can spend a large share of their tokens searching files, reading them, and caching their contents, even when much of that information is irrelevant.

A more targeted method is to build an internal knowledge base that catalogs the system and indexes its components. For authentication, for example, it can explain how the flows work, diagram them, and list the files involved. That context matters when judging an authorization bypass, because the reviewer must know whether the code is already running inside an authenticated boundary.

Code can also be poor even when it has no obvious bug: it may fail to meet what the author intended. Gupta jokes that people increasingly use Slack as an IDE, but a short message asking an agent to fix something often leaves substantial ambiguity. The code can pass a narrow correctness check and still solve the wrong problem.

Give the agent more context to close that gap. Gupta highlights product documentation and tickets alongside the indexed codebase. Access to Jira or Linear, plus Confluence, Notion, or another documentation store, can supply the tribal knowledge that engineers use without realizing how much it shapes their decisions.

Greptile combines these sources into a living wiki—a changing model of how a company works. It can bring together codebase knowledge, application behavior from production telemetry and logs, and roadmap information from documentation and Linear tickets. The reviewer can use this context while also searching the code more precisely.

The fifth improvement is autonomous testing. A reviewer may inspect React changes and see no obvious issue, yet the rendered interface could hide a button on an iPhone. Greptile addresses this with configurable virtual machines that contain the application’s dependencies and can run a customer’s stack.

The agent then uses computer vision to test the application from end to end. Gupta says newer GPT and Opus models have made vision much stronger, bringing end-to-end quality assurance closer to practical use. Teams still need to document the important flows and make the build process simple enough for an agent to start the stack.

A useful setup needs a development box, the pull-request branch and its dependencies running in the app, plus a flow map—for example, which code handles authentication and how sign-in works. With that map, a browser agent can notice that a button technically renders but is hidden on an iPhone or falls below the initial screen.

Even with all these capabilities, one review can still cost around $3 to $20. Gupta’s sixth improvement is to use cheaper models for some subtasks. He says frontier models remain stronger overall, but his team reserves them for the most critical work—about 20% of its model usage—while smaller models handle exploratory work and file reads.

For example, a small model can gather relevant context, a lightweight computer-use model can test user flows, and a frontier model can analyze bugs and make the final review decisions. Greptile uses a carefully assigned model for each of roughly 20 subtasks, with a research team continually testing which model fits each job.

Gupta estimates teams may get 70–80% of the way to a strong system by swapping simple tasks to simple models. But a large evaluation dataset remains important: without it, a cheaper subagent could quietly make the reviewer worse at finding bugs, and the team might only notice after an incident.

A cheaper model can also filter the available context and decide which pieces are worth loading, reducing wasteful file reads. In theory, a decision model or API could rank context before it enters the frontier model’s cache, which Gupta describes as one of the system’s most costly and scarce resources.

With these improvements, teams can build a capable review agent. Gupta invites listeners to contact him at daksh@prod.com if they do, or to use Greptile instead. He closes by thanking the audience.