Nacho Martinez: Designing Memory-Aware Agent Harnesses for AML

Nacho Martinez introduces a hands-on workshop on agent harnesses. The example is a simulated bank whose agent examines transactions and flags cases for human review.

Martinez is an AI developer advocate and data scientist. He plans to cover the agent stack, memory, context engineering and how a harness surrounds a model.

He uses anti-money-laundering as a motivating problem. A bank may encounter large numbers of transactions that require investigation; a flag should prompt review, not be treated as proof of wrongdoing.

Long-running agents need more than a model call: the application must manage context, tools and stored information. Martinez argues that these surrounding components are a useful place for engineers to experiment.

His fictional example includes several suspicious transaction patterns. The aim is to surface possible issues to bank staff, who can then investigate them.

He divides the agent stack into an application, data, model, infrastructure and compute. This workshop focuses on the data and orchestration components that application builders can control.

The data layer includes an agent loop, connections to external sources, memory, context management, tools and skills. These components decide what information and capabilities are available at each step.

Martinez distinguishes chatbots, retrieval applications, fixed workflows and agents. He describes modern agentic systems as often combining automation with a degree of autonomy.

He uses a human analogy: a model performs reasoning, tools let the system act, and stored information supports recall. The analogy is a way to organize the components, not a claim that the system thinks like a person.

He calls the surrounding scaffolding the harness. Different choices for memory, inputs and tools can change how the same model performs on a task, so the surrounding system should be evaluated too.

Design choices include storage, retrieval, memory categories, the agent loop and context assembly. The right arrangement depends on what the application has to do and what evidence it must retain.

On files versus databases, Martinez recommends choosing according to the problem. Files are useful for unstructured material; databases can provide transactions and concurrency controls for shared state.

Several agents writing the same file can encounter conflicting updates. Worktrees isolate changes in some development workflows, while transactional storage offers another way to coordinate shared writes.

Memory design includes deciding what to save, how to refine it and when to retrieve it. Martinez distinguishes short-term context from information retained beyond a single conversation.

His proposed long-term categories include episodic history such as conversations and tool logs, procedural memory such as skills and workflows, and semantic information such as preferences and domain knowledge.

Context windows have limits, and relevant information can be harder to use amid long histories. The effect varies by model and task; a shorter context is not automatically better if it omits necessary evidence.

Summarization can condense a conversation before a later turn. Another approach stores material outside the immediate context and retrieves it when needed; both approaches can lose or fail to retrieve important details.

The agent loop assembles context from conversations, files, databases, tools and skills, calls the model, dispatches actions and incorporates results. The workshop's loop has bounded iterations and time.

The example uses a maximum of eight iterations and a time budget. The slide also shows a human approval gate for tools with side effects; actual controls need to be enforced by the application.

Loading every available skill and tool into every request can consume substantial context. Martinez proposes a toolbox and skillbox: index descriptions and retrieve relevant items for the current task.

Vector similarity can help find candidate tools or skills, after which the agent loads the selected material. Retrieval is imperfect, so the choice and results need testing for the application.

He calls turning successful runs into reusable instructions skill distillation. For example, a team can record a frontend workflow and its preferences in a skill file for later use.

Dynamic discovery can retrieve that skill when a related request arrives. This changes the information supplied to the model; it does not by itself change the model's weights.

Martinez contrasts external memory and retrieval with changing model parameters or embeddings. He focuses the workshop on the external-data approach and notes that the other techniques are separate choices.

He discusses instruction hierarchy as another part of context design: system and application instructions, user requests, and tool or retrieved content play different roles. Exact behavior depends on the model and platform.

The AML demo is designed to flag activity for later human inspection. Martinez stresses that it should not simply block accounts based on an automated suspicion, because an incorrect action could seriously affect a person.

The application displays a stream of synthetic transactions, some ordinary and some flagged, together with a view of the agent's context and stored data. The workshop is a demonstration, not evidence of production effectiveness.

The agent searches available knowledge and uses tools to assemble evidence for a suspicious-activity analysis. A real deployment would need validation, access controls and human review appropriate to its setting.

In the notebook, participants inspect the components and complete exercises. Martinez shows a model connection and database setup, then invites questions about the memory architecture.

Asked whether episodic memory could confuse an agent with old conversations, he says architecture and retrieval determine what is recalled. Separation helps organize information but does not eliminate conflicts automatically.

He discusses indexing third-party skills or tools by embedding their descriptions, so the loop can select candidates without loading every item. Access and trust still need to be considered for external capabilities.

He also discusses context and caching costs. Exact request sizes and caching behavior vary by provider, configuration and workload, so the numbers in his illustration should not be treated as universal.

In this workshop, context assembly retrieves a limited number of relevant tools and skills from separate vector stores. The example uses a default of three candidates.

A reranker could further order the candidates or help select fewer items when only one is needed. Martinez presents it as an optional refinement rather than a component already used in the demo.

For memory, he recommends starting simply: keep current context and a long-term store, then promote selected information when it is worth retaining. More elaborate categories can be added when justified.

The loop can retrieve stored memories before reasoning, using relevance or a similarity threshold. A threshold and index need evaluation because relevant context may be missed or irrelevant content returned.

He discusses database access through MCP and prebuilt queries as one way to expose known operations. Restricting the available operations can help, but does not remove the need for authorization and input validation.

Keyword and semantic retrieval have different strengths. Martinez describes hybrid search and combining rankings; its actual quality depends on the data, queries and evaluation.

He says a custom harness need not reproduce a full general-purpose coding agent. A smaller harness can be designed around the specific data, tools and constraints of a domain.

Different problems need different budgets and memory strategies. An unattended long-running research task might need context management, while this demonstration limits its loop to eight steps.

Martinez encourages participants to work through the notebook and ask questions. He says the workshop environment can also be explored after the event, subject to its actual availability and terms.