Inside Anthropic’s Approach to Long-Running AI Agents

Anthropic product lead Jess Yan joins Peter Yang to discuss agents that can run longer tasks with access to tools and data. Jess presents Claude Managed Agents from the perspective of the team building it. The episode also contains a separate paid sponsor segment for Riverside.

Jess describes an evolution from simple prompt-and-response loops toward agents that can use tools, interact with external systems, and run longer. A harness is the surrounding scaffolding that gives a model access to tools and memory and determines when to ask a person for input. As capabilities and access expand, she says, permissions, observability, and steering become more important.

She characterizes Claude Managed Agents as a prebuilt harness and supporting infrastructure for delegating complex tasks. Rather than treating the model alone as the whole system, she argues that model behavior must be evaluated together with the environment and harness in which it operates. This is her view as the product lead.

In a demonstration, Jess opens a preconfigured data-analysis agent. It has a selected model, behavioral instructions, filesystem tools, and configurable permissions; skills can also be attached. For this example, she supplies the schema and instructions for analyzing a large grocery dataset, then lets the agent use its tools and produce HTML reports.

The output includes product patterns, shopper analysis, and an example predictive model for customer return. Jess says providing schema and a sequence of steps helps produce a more predictable result. The console exposes actions and session history, and she describes a debugging agent that can inspect the trace. A generated chart or prediction is not by itself proof of accuracy: data quality, methodology, and performance need separate validation.

Evaluation becomes harder as agent tasks grow more complex. Jess discusses fixed test cases, replaying multi-step interactions, comparing versions, binary checks, scored judgments, and checking whether skills or actions trigger when expected. She also describes agents reviewing outputs during a run. Automated review can be useful, but it does not guarantee a correct result or replace independent testing.

The speakers consider how instructions change when users specify a desired outcome instead of every intermediate format. Jess suggests giving the agent an example of quality or a target score and letting it iterate. Clear success criteria still matter, as do limits on what the agent may access and do.

For her own product-management work, Jess says codebase access helps her track merged and deployed pull requests, understand implementation, prototype, and support customers. She uses scheduled activity summaries and ad hoc research, and describes agents summarizing customer Slack channels. She sees current context and proactive triggers as important, while direct customer conversations remain part of her work.

Jess describes a temporary agent for a feature waitlist: it cleans invalid or duplicate entries and helps prioritize organizations for feedback and invitations. Because the need was expected to last only a few weeks, she preferred a task-specific agent to building a polished permanent product. Such workflows still require appropriate access controls and review of selection criteria.

For enterprise adoption, Jess recommends beginning with individual employees: give them examples or templates and let them adapt agents to extend their own capabilities. Organizations can then learn from that work before attempting large, cross-team automation. She and Peter discuss early user feedback and more formal evaluations as usage grows.

Looking ahead, Jess expects longer-running agents to increase the amount of work people can delegate and anticipates increasingly specialized products. Peter questions whether a standalone application can remain differentiated when users can create their own skills. They agree that agents need to fit the places and workflows where people already work. These are forecasts and product opinions, not guarantees.