Theo Browne on How Claude Code Reviews Its Own Changes

Theo Browne asks how teams can review the volume of code produced with agents. He says the T3 Code project has received as many as 800 pull requests in a day, creating a triage burden for a small team.

He reads and responds to Anthropic’s account of the Claude Code team’s review process. The discussion combines the source article’s descriptions with his own observations about T3 Code; the two workflows are not identical.

Anthropic describes optimizing human review time while monitoring reverts and incidents. Theo agrees that reducing time spent reviewing only helps if the team does not spend more time repairing failures afterward.

The source reports that merged Claude Code pull requests more than doubled from June to September while the share reverted fell by over 40%. Theo cautions that model improvements may also contribute; the figures alone cannot isolate the effect of review-process changes.

Theo says code review has two jobs: detecting defects and building shared understanding of the codebase. As output accelerates, he worries about maintaining the second function even if machines read more diffs.

He questions the assumption that a human reading every line is a reliable way to catch every regression. His own view is that automated review can be valuable, but both people and models miss things and need other verification methods.

According to the article, many changes receive automated review and fixes before a person is asked to look. Theo prefers making this a standard step, reserving scarce human attention for decisions where it contributes most.

He distinguishes human energy from time: a workflow can be short yet exhausting. Waiting between a pull request, review, fixes and another review also adds latency beyond the time spent on each activity.

The Claude Code team’s stated goal is to merge most changes with at most one human involved without increasing reverts or incidents. The article treats incident and revert rates as checks against merely shifting work downstream.

Theo stresses giving agents ways to verify behavior before review. For user interfaces, he describes running the application, capturing screenshots or recordings and using predictable mock events so an agent can exercise the product without relying on a live model call.

He outlines Anthropic’s layers: required CI, automated bug review with multiple reviewers and verification, a separate security pass, risk-based human approval and a merge queue. After merge, internal use, staged releases and monitoring provide further checks.

Layers can catch different failures, but Theo warns they can also add unnecessary code if an agent blindly acts on every comment. His priority is keeping changes understandable and making bad releases fast to reverse.

Before a pull request, the source describes running tests, builds or visual checks and having a fresh-context agent review the diff. Some teams also use separate checks for bugs, simplification, end-to-end behavior and design compliance.

For repeated mistakes, Theo recommends first making the codebase easier to use, then adding deterministic checks such as types or lint rules, and only then adding instructions or skills where those approaches cannot address the problem.

The article describes an automated review that considers the diff, repository and stated intent. Its reported substantive-comment rate rose from 16% to 54% after a research preview, but more findings are useful only if they are accurate and actionable.

Theo questions the cost of the hosted review product for high-volume projects. He describes using reviewers from different model families and having the authoring agent assess feedback before escalating it to a human.

The article says multiple agents look for bugs and a separate verification step filters findings; accepted findings are ranked by severity and identified as important, minor or pre-existing. The author must fix a finding or explain why it is closed.

Security review is a separate blocking pass. Theo notes that a review thread can be resolved with a reason when a finding is false or not applicable; unresolved findings prevent a merge in the described workflow.

The source uses risk tiers to decide whether zero, one or two humans are required, with sensitive paths receiving special treatment. Theo favors a more flexible assessment but agrees that an agent’s approval should not count as a required human approval.

The author remains accountable for the change, regardless of who or what wrote the code. Approvals apply to particular commits, and later changes can require renewed review.

After merge, feature flags, gradual rollout, on-call monitoring and fast rollback limit the impact of mistakes. Theo favors improving detection and recovery rather than assuming that going slower alone solves the underlying problem.

To improve reviewers, the team builds evaluation cases from incidents and reverts, with engineers confirming the bug. It also measures noise by observing how people respond to comments and testing against known-clean changes.

Theo warns that a human can dismiss a minor suggestion quickly, whereas an agent may implement it, add a commit and trigger another review. Low-value findings can therefore consume time and inflate the diff.

He recounts leaving an agent review loop running for four days, after which the pull request had grown to roughly 80,000 lines. The example motivates stopping rules and escalation when repeated rounds stop making meaningful progress.

The Anthropic article describes a separate pull request with more than 50 overnight review rounds. It says many later findings concerned code already present at the first review, suggesting that repeated reviews were not always finding newly introduced bugs.

The team added a round cap, later diff-focused reviews, fixes for classes of bugs and suppression of repeated findings. It reports that mean rounds to pass fell from 2.08 to 1.32; these are internal results, not a universal expectation.

Theo experiments with a score threshold instead of requiring every comment to disappear. He also relays a colleague’s alternative: have an agent make a direct decision about whether a particular comment can be ignored.

Humans still decide direction, scope and whether work is worth doing. Theo highlights risky paths such as billing and CI, architecture decisions and review of actual evidence; authors remain responsible for outcomes.

The knowledge-sharing problem remains. Theo says he asks models for explanations of a module or a pull request, including what changed and where the risks lie, and values open team channels where people can see ongoing work.

The article’s ownership model emphasizes small reversible changes, telemetry and shared context. Ownership means being prepared to repair a change, rather than automatically assigning blame when something fails.

Because agents make starting work cheaper, Theo says teams can close or reshape pull requests when a better direction emerges. Raising alternatives early avoids investing review effort in work the team does not want to ship.

His practical checklist spans before, during and after a pull request: provide runnable checks, document testing and rollback, use CI and risk-appropriate review, stage releases and turn incidents into evaluation cases.

He returns to his main disagreement: instead of continually adding prose instructions for every failure, first improve the code and checks so that both people and agents are less likely to make that mistake.

Theo welcomes Anthropic’s detailed account and asks teams to share how they review agent-written code. The episode links to the source article: https://claude.dev/blog/solving-code-review-at-anthropic/