Why Theo Values Reliable Coding Agents Over Peak Performance

David Cramer suggests that people using Fable or Astra try less expensive reasoning models such as Opus and Sol, because their tasks may perform no differently. I strongly disagree with that as a general conclusion. My argument is not that a cheaper model can never be sufficient, but that the way you define a task changes what differences you see.

I want to separate Astra from the rest of the comparison. In my experience it can do remarkable things but also make surprisingly poor decisions, so I understand why someone might prefer Sol in certain situations. My model preferences are based on my own work and do not establish a universal ranking.

I draw a rough graph of response quality to illustrate my impression: Astra has higher peaks and deeper dips, while Fable is more consistent. This is a subjective sketch, not benchmark data. I think people often notice a spectacular best result and overlook how much the worst outcomes affect practical use.

I have made both mistakes: overvaluing a model after seeing an impressive result, and dismissing one after an especially poor run. For everyday work, I often value a higher floor more than an extraordinary ceiling. A model that solves a hard problem but mishandles a basic instruction can still disrupt the workflow.

When a task is a precise ticket naming the files and desired change, multiple models may clear the bar. A bad failure can also make us lower our expectations and ask for smaller tasks. My point is to reconsider how much of the workflow we ask the agent to handle, not merely how difficult a single coding edit is.

I call this the width of a prompt: the distance from the initial problem to the desired end state and the number of steps in between. A wider task can encounter more opportunities for error. My diagram expresses a hypothesis about how consistency affects that experience, not measured probabilities for particular models.

When a frontier model changes, I retry tasks that other models failed and use the results to adjust my expectations. For a well-specified Jira ticket, I accept Cramer’s point that several models can work reliably. The difference I care about emerges when the agent takes on more of the surrounding work.

For example, a user sends me a screenshot of a T3 Code bug. I can give it to an agent on my phone and ask it to investigate, fix and test the problem, record a demonstration, file a PR and monitor the review. The important change is the end-to-end span and explicit outcome, not necessarily a harder individual edit.

Those runs can consume more tokens, but engineering time also costs money. I am willing to spend more if it allows a valuable engineer to ship more work. That is a business judgment rather than a measured productivity result: teams need to compare actual usage, quality, human review and the cost of mistakes.

Many developers give short, concentrated prompts and supervise the agent as it works. I encourage people to test longer workflows where appropriate. When an agent goes wrong, asking what files and tool outputs led it there can be informative, although a model’s own explanation is not necessarily the true cause of its behavior.

One developer describes overnight runs as a way to expose recurring friction in the environment, not merely produce code. A long run may hit a stumbling block that shorter runs would encounter less often. Looking at failures can reveal improvements to tools, instructions, tests or the codebase itself.

I argue that a codebase which consistently confuses a new agent may also be difficult for a new developer to enter. That is a useful question to investigate, not proof that all agent mistakes are the repository’s fault. I also recognize that my experience with T3 Code and other projects may not match every organization.

To widen a workflow, you might describe the problem and let the agent find relevant files, or ask it to run a simulator, verify the change and provide a video with the PR. The right permissions and checks depend on the task. A video can help a reviewer see a result, but it does not prove all behavior is correct.

I show a PR from Maria that included a visualization of its changes. For me, two useful questions are: how long can the agent work without human input, and how likely is the result to work when a human returns? They are related but not identical; a long run can still produce an unacceptable result.

If an agent repeatedly hits the same problems after half an hour, investigate and improve the workflow. I report runs that continued for hours and produced a PR ready for review, but these are personal examples, not proof that long autonomous work will be reliable in every repository.

My sketch also illustrates why a narrow task may show little difference between models: both can exceed the required bar over that short interval. As the workflow gets wider, there are more opportunities for mistakes. I prefer Fable because it has felt less likely to make disruptive mistakes in my work, not because every result is better.

Maria says she spent several days reviewing traces and refining skills before getting smoother results from a single prompt. This preparation matters: successful autonomy depends on the environment and workflow as well as the model. It should be counted when comparing approaches.

I discuss subscriptions costing hundreds of dollars a month and acknowledge that those prices are inaccessible to many students, people outside wealthy countries and other developers. My estimates of included token value depend on contemporary pricing and limits. A cheaper model can still be the right choice, and its total cost includes retries and human time.

My view is that a more capable model can reduce the work I do both before and after a run. David Khourshid and others argue that the most expensive models are unnecessary for many tasks; that may be true for their tasks. The practical question is whether extra model cost produces enough value for your own workflow.

After a viewer questions my use of the word exponential, I ask an agent to analyze my usage history. The data start in March because I moved computers. I report a median prompt duration rising from 53 seconds to 2 minutes 20 seconds and the 95th percentile from under seven minutes to over 16 minutes 20 seconds in the comparison I describe.

My logs also show longer upper-tail runs across months, including a jump from roughly 12 to 22 minutes in one monthly comparison. These are observations of my behavior and environment. Changing tasks, prompts, models, tools and willingness to wait can all affect duration, so the figures alone cannot establish that new models caused the increase.

I then use a simplified calculation: suppose each ten-minute interval independently has a 5% chance of failure. The chance of no failure over three intervals is 0.95 cubed, about 85.7%; over 24 intervals it is about 29%. This illustrates compounding under fixed, independent probabilities, not a measurement of real agent runs.

If the interval failure chance instead falls from 5% to 3%, the same independent model gives about 48% success over four hours, compared with about 29%. The roughly 19-percentage-point difference shows why small changes can matter over repeated steps. Correlated failures, recovery and changing conditions can make real outcomes very different.

I contrast current long runs with older workflows that required extensive specifications and could lose track during compaction. That is my experience rather than a controlled study. Specifications and clear verification can still be valuable; whether a longer run saves work depends on the environment and the result.

My final disagreement with Cramer and other experienced developers is about preparation and review: I think some people may still delegate only the coding step while retaining more surrounding work than necessary. That is a hypothesis about their workflows, not something I can prove from a post. Try varying the scope, measure outcomes and choose the level of oversight that fits the risk.