Lauren Tan: Make AI-generated code trustworthy with enforceable checks

I shipped 3,000 pull requests to production last month, Lauren Tan says. Agents can make many more engineers highly productive; the challenge is maintaining or improving code quality as output increases. She calls the foundation for that trust in the agent environment, not blind trust in the agent.

Tan says it took her six months of investment to reach a point where she could ship thousands of changes, including while she slept. Her trust pyramid starts with code and architecture, then static analysis and tests, followed by runtime or formal verification. Rules, skills and AGENTS.md provide a smaller layer for work that needs judgment.

The codebase is one of an agent’s largest inputs. When correcting an agent, Tan recommends asking whether the correction can become an automatic check rather than another line of instructions: an executable rule is more reproducible and can support greater trust.

She compares the repository to a garden. When a few pull requests become hundreds, code arrives like a stream; architecture and guardrails are the fixed structures around which it grows. Workarounds, deprecated code and other bad examples can spread because agents copy what appears to be the established pattern.

Her team built Dune, a client framework for Grok Bot that she compares to Next.js for Electron applications. Its important feature, she says, is a collection of constraints and lint rules that encode lessons learned from building desktop apps.

One example is React’s useEffect: Tan says it has legitimate uses, but using it well requires nuance that agents often lack. The team bans it in product code and permits it in certain Dune APIs. That removes a recurring issue from code review rather than relying on reviewers to catch each instance.

Directory structure supplies another form of context. At Cursor, Tan saw agents import Electron main-thread code into the renderer, causing long tasks and jank. In Grok Bot and Dune applications, separate electron-main and electron-renderer directories make the boundary explicit; dependency analysis and lint rules forbid imports across it.

Tan has also seen agents use documentation and comments to justify shortcuts, or turn a minor correction into an inflexible rule. She describes imperative timers used as quick fixes for a mismatch between VS Code’s imperative architecture and React’s declarative style. Repeated timer creation and cleanup hurt performance, while comments explained why the workarounds seemed acceptable.

In Grok Bot, the team chose to ban about 99% of comments: Tan says comments become stale and cannot enforce their own instructions. Instead, lint errors should guide the agent toward fixing the underlying problem. Her general practice is to turn corrections into checks or eliminate the failure mode through API and architecture changes.

As those checks accumulate, Tan says there is less for a person to inspect in each review, and agents can merge many of their own changes. Her team increasingly reviews verification artifacts such as videos, screenshots, diagrams and formal checks.

Runtime verification gives an agent a way to run the software and use debugging tools such as the Chrome DevTools Protocol. A feature map describes the application from a user’s perspective, giving people and agents a shared vocabulary for features. Reusable command-line tools and drivers make the skills testable and reduce repeated token use.

The team has begun experimenting with formal verification. It uses TLA+ to model flows such as notifications across devices and Lean to check algorithms, including a custom virtualization library. Tan says these methods have already found bugs, but calls them an additional layer rather than a silver bullet.

The team is also trying property-based UI testing with Bombadil and has found UI bugs. Tan cites another engineer’s observation that 70% of daily tokens go to verification; she says most of her own token use also goes there. Agents can reduce the cost of writing and maintaining established testing techniques, she argues.

Formal models can also clarify intended behavior and whether agents understand it. Tan’s pstack plugin includes a statechart skill that visualizes state machines from the user’s perspective; she finds the result easier to review alongside screenshots and videos.

Tan reports that the Grok Bot team shipped about 11,000 pull requests to production in October and cuts a stable client release roughly daily. She says 227 people contributed at least one pull request that month, including occasional contributors with less context.

Despite the increased volume, Tan says the relative revert rate remained around 0.5%. She reports improvements in startup, typing, opening chats and switching between them since launch, while acknowledging that the Electron client has performance limits.

Before building a software factory, Tan recommends making the codebase better: add a lint rule against a repeated anti-pattern, clean up its existing occurrences and strengthen verification. If the inner coding loop produces poor work, automating it produces more of the same and consumes more tokens.

Once those foundations are in place, Tan says Grok Bot can orchestrate the outer loop through connectors, routines and projects. After a release, her team automatically launches a Cursor project to exercise the build, find bugs and open fixes; she says those fixes can then merge automatically because of the surrounding checks.