Jediah Katz: Why Agent Harnesses Still Matter as Models Improve

I joined Cursor in early 2025 and worked on its agent harness before moving to SpaceX as part of the acquisition. Today I want to ask whether smarter models have made the harness obsolete.

The terminology has changed rapidly, from prompt engineering to context engineering and then harness engineering. The underlying work is to place the right information and capabilities around a model.

I think of the harness as a model’s eyes and hands: it connects the model to applications, computers and external services, and shapes how the agent operates.

The runtime controls where the agent loop executes and how state is stored so work can resume. Building this foundation resembles conventional software engineering.

The context and orchestration layer chooses models, prompts, tools, memory and compaction. It also coordinates calls and controls what information the model sees.

Katz recalls a costly long-running Cursor experiment that coordinated multiple models, agents and machines. It illustrates how complex a harness can become when pursuing new tasks.

Later, teams reported removing prompts and guardrails that had compensated for model weaknesses. Cursor had previously used a small model to apply edits and provided explicit coding instructions.

Katz discusses studies comparing harnesses on coding and machine-learning benchmarks. He says newer models showed little score difference in those settings; this result should not be generalized to every task or deployment.

His interpretation is that stronger models have absorbed some skills previously supplied by scaffolding, such as exploring a repository, persisting and checking work. This is a conclusion about the tested setting, not proof that scaffolding never helps.

He calls the cycle a data flywheel: a harness enables harder tasks, more people use agents, data can reach model developers, and later models may learn skills that make old scaffolding unnecessary.

That does not make the whole harness disappear. Katz gives two reasons: it can open new task frontiers, and infrastructure, security and efficiency cannot simply be absorbed into model weights.

He describes Grokbot as a product using long-lived coordinators and worker agents for knowledge work. This is his employer’s product description, not an independent assessment of its performance.

For newer tasks, the team has added prompts and guardrails again. The appropriate scaffolding depends on what the model can reliably do in the relevant environment.

Examples include creating slide decks, travel workflows, communicating progress, deciding when to ask before acting and retaining useful preferences. External actions need appropriate user intent and permissions.

The flywheel can repeat: when a capability becomes more common in models, a harness may open another class of tasks and expose new limitations.

He describes an experimental software-factory workflow: triage a reported bug, start an agent to propose a fix, show evidence, and monitor deployment. He explicitly says the broader vision is not solved.

Long-running work needs infrastructure that survives interruptions and preserves state. Katz expects cloud execution to play a large role, though architecture depends on task and constraints.

Security includes scoping permissions to agents and people, protecting secrets and preventing unintended or costly actions. A model’s ability to use a shell is not a reason to give it every credential.

He notes that the studies he cited also found differences in cost per task even when benchmark scores were similar. Quality, latency and cost should be evaluated together.

His team says it improved cache hit rates, token efficiency and model selection to reduce cost and latency without an observed quality loss. These are company claims in the talk.

He expects harnesses to become easier to modify as models and products change. Hiding internal details can speed iteration, but it also reduces visibility for users.

Reusable primitives are composable building blocks. A subscription interface, for example, can wake an agent for a message, a CI event or a scheduled time without designing an unrelated mechanism for each.

The team tracks temporary patches, or shims, centrally and logs their use. If a patch stops being needed, an automated process can propose removing it; a human review can still assess the change.

He predicts agent work will increasingly run away from a user’s laptop and that agents themselves will become building blocks for higher-level systems. His six-month forecast is a prediction, not a measured outcome.

His conclusion is that some coding-specific scaffolding can be removed as models improve, while the harness remains important for new tasks, infrastructure, trust and efficiency.