Mikyo King on Code Mode, Sandboxes and Agent Evaluation

I’m Mikyo King, and I lead the open-source group at Arize AI. We work on an open-source observability and evaluation framework as well as our own agent. I want to explain code mode and how we have adopted it.

Traditionally, software for engineers and software for non-engineers have been different. A programmer asked to collect email invoices might write a script, while someone else would work within an existing interface. Coding agents can dynamically build some of that task-specific software, which changes the boundary.

A conventional agent may receive a large catalog of tools. As it repeatedly calls them, definitions and results accumulate in the context window. More tools, irrelevant outputs, pagination and long tasks can create context bloat, extra turns and eventually a need for compaction.

Code mode is an alternative architecture. Instead of presenting the entire catalog, the agent can search for a tool and execute it in a programming environment. It can sequence calls, run them in parallel or use conditionals, and return only the information needed in the model’s context.

Compared with accumulating each tool result in the context window, a well-implemented code mode can reduce overhead and turns. King emphasizes progressive disclosure: the model discovers and understands the tools needed for the current task rather than receiving every definition up front.

Composability lets an agent chain calls or fan them out. It can also trim or search tool output before returning it to the model. King sees a further benefit in emergent behavior: an agent may write a small program for a task for which no dedicated tool exists.

For example, a program could remove personally identifiable information from records before returning results, rather than placing sensitive values directly in the context. That capability depends on the implementation and the boundaries of the sandbox.

In Arize Phoenix, the team started by giving its agent Bash, so it could combine standard command-line tools with its own CLI and save intermediate results in files. It also added a Python sandbox exposing REST create, read, update and delete operations for traces and datasets.

The team added SQL because not every useful question fits a CRUD tool. An aggregate query can identify spans with a given latency characteristic without paging through all traces in the model’s context.

Arize also built a programming interface bound to its browser UI. Code running in a web worker can interact with application state, such as a time range or frontend data store. King describes this as enabling more complex UI interactions through code.

Code mode is not free: generated code needs resources. A hosted sandbox must manage CPU, memory and lifetime. Network egress deserves particular attention because an agent may need the web while encountering untrusted content.

Secrets and permissions are also hard. A sandbox may need credentials for a model or service, but those secrets should not return to the model’s context. Permissions that seem appropriate for individual operations can become too broad for a later operation or a composed workflow.

There is operational complexity as well. Sandboxes can fail to start, and debugging a program differs from debugging individual tool calls. Tool parameter accuracy becomes code accuracy, and a workflow can fail halfway through. Tool discovery still requires a way to find and understand available tools.

King also notes nested sandboxing when code mode is implemented both in MCP and in the agent harness. Results may be serialized and parsed between environments, adding work. Durability remains an open problem: a long program that fails midway may not resume at the failed step.

Language choice can matter for determinism or mathematics. Binary data such as images is another challenge: serializing an entire image into the context is undesirable, so a separate handoff may be needed. Smaller models may also struggle with code mode compared with frontier models.

To evaluate the approach, Arize uses its Phoenix platform and the Harbor framework. Agents run in sandbox environments on defined tasks with network access disabled. The team repeats trials, uses verifiers to score task completion and tracks pass-at-k across tasks to account for variable trajectories.

King shows a before-and-after comparison reporting 43% fewer tokens and 31% lower cost. The slide reports the same reward and a 0.6% latency increase, despite sandbox overhead. He treats this as evidence for the evaluated setup, not a universal guarantee.

Capturing both trajectories and evaluation results helps the team understand failures and improve instructions. A newly added tool can initially reduce efficiency; traces can reveal which instructions lead the agent in the wrong direction and guide further iterations.

King closes by emphasizing fewer turns, lower cost and the potential for agents to build longer-horizon behavior. He encourages teams to try a sandbox and evaluate their own agents rather than assuming the architecture is automatically better.