Theo on Coding Agent Loops: Review, Coordination, and Cost

Pete says developers should design loops that prompt coding agents. Early experiments such as the Ralph loop helped me imagine agents working longer, but they also greatly increased the error rate of my changes. I returned to prompting an agent, reading its plan, directing each step and manually passing review feedback back.

I have long admired how Pete experiments with agents, even when the results are rough. More recently I began using loops to review and revise code, monitor pull requests and issues, and bring context to me through Hermes Agent. I still write prompts, but I now think many individual agent runs need not start with one I wrote.

The post that made this click was Peter Steinberger describing an orchestrator that periodically wakes, checks repositories and directs work to threads. I had not realized one Codex thread could create another. The idea gave me a new way to think about parallel work and supervision.

We moved from asking a chatbot for code and copying the answer into a project to letting agents edit the code directly. Workflows and subagents add another step. I was skeptical of predefined personas for every worker because I value agents’ ability to construct the context and method a particular task needs.

Closing the loop goes further: the model can select work, audit the result, pass feedback back and repeat. I am not claiming that all code should be produced without oversight. I wanted to experiment with a limited version and see where it helped.

For a large refactor, I let the agent divide the work among subagents and asked it to monitor the pull request. Previously I had been copying comments from review tools into the agent myself. With a separate worktree for the PR, I could ask it to watch for comments and address them without blocking my main working directory.

The monitoring ran for more than six hours and made improvements, so I pushed further. I also tried asking one agent to run another reviewer and address its feedback. But a single PR is only one shape of work; some problems need several dependent changes.

While investigating Lakebed’s isolate layer, an analysis suggested that the data architecture might become a bottleneck first. Each mutation could cause many subscriptions to rerun. The proposed improvements included dependency-aware invalidation, batching, shared results and limits on refresh frequency.

I asked whether these changes belonged in one PR or several, and whether they should be parallel or stacked. The agent recommended multiple PRs, mostly stacked with some parallel opportunities. I asked for HTML plans, then had it start a new thread for the first change.

After the first PR went through review and I merged it, I asked for a new thread for the next part. Then I wondered if the process itself could be automated: one thread implements, another reviews, the implementation responds to comments, and the coordinator moves on after the PR is ready.

The proposed workflow used a periodic heartbeat to inspect implementation status, detect a PR, start a fresh review for a new head revision, return actionable findings and review fixes. It would check CI and required approvals before merging, then update the base before starting the next worktree.

I started it before bed and woke up to four stacked PRs that had been reviewed and merged. It also disrupted my editor. The result was exciting, but I would not recommend handing this level of autonomy to a production codebase serving millions of users yet.

What stood out was that the workflow created smaller loops suited to this particular problem, instead of applying one fixed reviewer pattern to every change. I see a contrast with static sprint structures: a process can be shaped around the work rather than forcing every task into the same shape.

The same thinking can be used to monitor PRs or surface useful updates on a schedule. I used a loop to watch for 5G hotspot options and it brought a new plan to my attention. In my development example, I still wrote the initial prompts, read the responses and decided to try the proposed process.

Look at what you do after an agent finishes. I used to run the app, check the result, commit, push, open a PR, collect review feedback, ask my team and then merge. Some of those steps can be delegated: ask the agent to run and check the app, or gather and address review comments.

I also suggest considering whether an automated review can catch obvious issues before a person spends time reading the code. The goal is to leave the human more attention for harder questions. This is my preferred workflow, not evidence that an agent review guarantees correctness.

A major catch is cost. Loops consume more tokens, and a wrong direction can run for longer before anyone notices. In one case, fewer than ten minutes of review produced a workflow that ran for eight hours and used over three million tokens to address three small comments.

My subscription usage happened to remain below its limit despite several loops, but that is a snapshot of my own accounts and plans at the time. Limits, prices and eligible use can differ. I cautioned that paying per API token could make this style of experimentation expensive.

A persistent goal in one thread is different from a dynamic workflow that creates separate pieces of work. I was also trying a long-running Rust rewrite of Hermes Agent to reduce resource use. It was an experiment; I did not expect it to be production-ready just because the loop ran for a long time.

I showed estimates of substantial inference use across several machines and subscriptions, and saw that as room to experiment. Those estimates are not the same as a cash saving or a prediction of anyone else’s costs. My practical takeaway is to examine your next manual step and ask whether an agent can help, then inspect what it actually does.

The value of loops is letting agents take on more of a process. Try asking for the next step and see how well it works in your own situation. My own experiments surprised me, but the result still needs to be evaluated.