A passing PR check needs evidence tied to the submitted code

Cassandra Chin built Tracetoon to improve pull-request descriptions that may be inaccurate even when the underlying work is valuable. With Stephen Chin, she explains how it gathers coding-agent context, reviews claims and can add an optional comic.

The speakers use a fictional token factory to illustrate a real review problem: repeated drafting and review can consume many tokens while reviewers still lack a clear chain from the change to its rationale and checks.

A coding agent asked to write a PR may rely on its current session, whose context can be compressed. Tracetoon instead looks across sessions associated with a branch to reconstruct decisions and the work performed.

Cassandra traces the idea to an education project that needed to understand the work behind a submitted program. She applied the same idea to pull requests, where the final diff alone does not explain earlier decisions or command context.

The original Tracetoon flow researches traces, drafts a description and asks an independent reviewer for corrections. The loop can add missing context, but Cassandra says it can still introduce errors as it revises.

They describe a case where a passing-check claim was not supported by the recorded sequence: commands that needed to run sequentially appeared to run together. A fluent PR description is not evidence that the submitted change was actually tested.

Simply switching to a cheaper writing model reduced quality in their tests. The speakers instead kept a stronger writer and looked for ways to give it and the reviewer better evidence without repeated expensive searches.

They introduce Relate, an embedded graph database written in Rust. It stores relationships among changes, reasons and checks so the system can assemble a connected evidence packet, including sources, before an agent drafts the PR.

With that brief, reviewers can check specific claims against the same evidence. The team uses less expensive reviewers for the more mechanical cross-checking while retaining a stronger model for the writing itself. This is the arrangement they found useful for their task.

The resulting example explains the reason for logging the working directory, the exceptions, and which checks ran against the submitted content. The important detail is the connection between each claim and evidence from the relevant version of the change.

The speakers report comparisons using their own PR Bench setup, combining agent traces, code, model judges and human review. They report quality improvements and about half the token usage in their comparison; these are results from their project, not a general guarantee.

They conclude that provenance and relationships should be preserved outside a single model conversation. Tracetoon and Relate are presented as tools for giving writers and reviewers a common, inspectable basis for a PR.