Why AI Agents Need Reliable Processes to Scale Across a Company

I am Mitch Troyanovsky, co-founder of Basis. We work on making agents reliable enough to complete production-scale work for accounting firms.

People at an AI conference may have dramatically increased their individual productivity with tools such as coding agents, yet the average company has not seen the same increase. Why does personal productivity not automatically scale across an organization?

Companies already coordinate nondeterministic systems: people. Leaders cannot observe every employee’s daily decisions, but they trust processes designed to produce consistently good outcomes, even when individuals are not deterministic.

Those processes exist at small scales too. For example, reconciling cash may require extracting bank data with approved software, rather than letting each employee pick an arbitrary tool found online.

Troyanovsky sees a competency gap in coding-agent-style products. Users must manage skills and memory, decide when to use the agent and develop a mental model of its behavior. He argues that many knowledge workers cannot be expected to manage all of that complexity.

The second issue is a process gap. If every employee configures an agent independently, the work varies by user rather than following an organization’s shared methods. In his view, senior leadership should establish the process.

For production agents to do work end to end, they should be simple for users and reliable in their behavior. The goal is not perfection: people are imperfect too. It is consistency and trustworthiness, including being predictable about what the agent cannot do.

A PDF can be read using a vision model, Python and OCR, a specialized provider, or combinations of those methods. Leaving the agent to choose freely may produce good individual outcomes but can make the process inconsistent. A production system needs an explicit position on how it should behave.

Some positions come from domain expertise. For example, he expects a tax agent to consult primary sources, and an agent creating a presentation to inspect an image for formatting errors. Such choices also have trade-offs: inspection adds latency and may not be worthwhile for every task.

Outcome evaluations compare work against known answers or rubrics, but a finite test set cannot cover every production trajectory. As agents perform longer tasks and make more decisions, it becomes harder to know whether the test distribution represents real use.

Troyanovsky suggests inspecting trajectories for desired behaviors even when no ground-truth outcome exists. That does not prove generalization, he cautions, but it provides additional evidence about whether a chosen process is being followed.

His accounting example is a bank reconciliation that does not balance. He prefers the agent to explain the difference rather than post a plug entry to force a match. Another organization might choose differently; the key is avoiding inconsistent behavior between customers.

Behavior comes from the system prompt, progressively disclosed context, tool descriptions and the model itself, not a single paragraph. He compares the context an agent sees at runtime to training data as a metaphor, and urges builders to care about how each piece influences behavior.

A behavior specification records the most important intended behaviors before the team writes or changes the context. It aligns builders and acts as a rubric: was a behavior relevant in this trajectory, and if so, did the agent exhibit it?

Behavior review can produce signals on trajectories that lack a known correct outcome. Outcome evaluations remain important, but he argues they are harder to scale at the rate of production activity.

Forking is another diagnostic tool. A team can take a trajectory that failed a behavior evaluation, restore the state near the decision, and branch it to investigate the decision or test a change in context.

Recreating a complete cloud environment, including databases and prompts, is difficult. But a fork helps connect the signal from a failure to a possible change in the agent harness, instead of searching blindly through a long run.

A cheaper, faster approximation is simulation. He asks a coding agent to create a sub-agent that pretends to be the production agent, providing the same rendered prompt and skills, and then presents a scenario to see what it would do.

The team can reproduce a situation and adjust context to see whether the simulated behavior changes. Troyanovsky says this is less faithful than a full fork, but useful for quickly iterating before paying the cost of recreating the environment.

Neither outcome tests nor behavior supervision proves that every production trajectory generalizes. His claim is that process supervision adds another signal, which can help a team identify problems and close the improvement loop.

As an agent starts with narrow work and consistently handles it well, people can develop trust in it like a colleague. The scrutiny applied to a pull request depends both on the risk of the change and on confidence in the person making it.

Someone may ask whether an unconstrained agent could discover a better way. Troyanovsky says it might, but argues that an accounting firm needs work done reliably at scale. His conclusion: simplicity encourages adoption, reliability earns trust, and explicit behavioral choices provide a basis for improving the system.