Wojciech Pluta on Deciding What an AI Agent Should Remember

I’m Wojciech Pluta, and I lead technical marketing for Oracle AI Database. Over ten years at Oracle I’ve worked on customer experience, mortgages, early contactless payments and open banking, as well as cloud-native systems and AI infrastructure.

I want to talk about moving from retrieval-augmented generation to memory systems. RAG gives us a useful read path; memory can create business value from data by preserving continuity across interactions.

Interfaces have evolved from commands, clicks and typing toward systems that respond to intent. That opens the door to software created on demand, but it also makes continuity important: an agent should be able to carry useful information from one interaction to the next.

In my view, models and infrastructure are becoming commoditized, while the data layer is where a business can gain an advantage because the data belongs to it. We think of an agent as a model plus a harness: the model suggests actions, while the surrounding code owns runtime decisions and checks.

A RAG system can include conversational state, but retrieval alone does not tell us what should survive an interaction. Retrieval is the read path; memory is the write path. When an agent encounters something worth keeping, we can promote it into a record for later use.

That raises the hard question: what is worth keeping? I call the process memory promotion. A pipeline classifies a candidate’s meaning, compares it with existing records and checks the evidence required for that type. The harness then admits or rejects it, or sends it to a human for review.

In a prerecorded financial-services demo, I use two synthetic mortgage applications, for Kyle and Vincent. The operator first asks the agent to prepare a mortgage brief for Kyle. The answer shows his situation and identifies documents missing from the application.

The operator then asks for a particular output structure. A decision model assesses the proposed preference, the harness makes the promotion decision, and the database persists the result. The traces show the status at the memory-promotion gate.

When I ask the same question again, the response reflects the saved preference: missing documents appear at the top, alongside details such as the case ID and policy. To check persistence, I start a new session with no conversation context carried over and repeat the request; the agent retrieves the stored preference.

A query about Vincent’s different application retrieves his own documents and case ID. In his case there are no missing documents, so the application can proceed for review. The same preference can shape the response without confusing the facts of the two cases.

The promotion gate also needs guardrails. A blanket instruction to approve every application should be rejected rather than stored as a durable rule. Other candidates, such as a memory intended for a whole team, may need human approval before they are shared across the organization.

The demo has three different outcomes: a semantic preference is stored, a blanket approval rule is refused, and an organization-wide memory is recommended for review. The point is to put promotion controls inside the harness and guardrails so the agent cannot bypass them.

More broadly, agent memory is the mechanism systems and applications use to retain, reuse, refine and recall information over time. The engineering challenge is deciding what to promote. Promote nothing and you lose continuity; promote everything and you risk poisoning the memory store.

A memory system therefore needs provenance and clear decisions about when to promote or supersede records. At Oracle, we distinguish short-term, long-term and shared memory rather than treating memory as one undifferentiated thing.

Short-term memory covers what happens within a session. Long-term storage can include episodic, procedural and semantic memories: summaries, tool-related information, knowledge and preferences like the one in the demo. Shared memory lets information be used more broadly, subject to the relevant controls.

We also have to decide where to store memories. Given my work at Oracle, I favor a database: I believe its ACID guarantees, multiple writers, scalability and search options make it a robust choice.

Building memory is a discipline. We need schemas that distinguish record types, indexes suited to retrieval tasks, and lifecycle and authorization boundaries. Without these abstractions, developers must build summarization, retrieval, durable-information extraction and token-budget management themselves.

Oracle introduced an agent-memory package to provide operations and abstractions for that work. One feature is a context card: a compact, prompt-ready block assembled for the agent. The aim is to reduce the developer’s housekeeping burden and let the agent focus on the task.

A context card can organize topics, summaries, relevant information and recent messages. The harness creates or selects what matters for a particular turn: topics orient the thread, summaries capture the conversation and user’s goal, selected records provide relevant information, and recent messages preserve the local exchange.

The distinction is that memory persists, while context selects. By returning only what matters for a turn, the card helps manage context and reduce bloat rather than placing every stored record in the prompt.

In the latest agent-memory release, we spent time on end-user isolation, with the database handling access restrictions. I also mentioned image support, post-processing and event listing; Oracle AI Database’s multi-data-type engine is intended to handle several data types within one engine.

My closing point is that models are shared, but your memory is specific to your business. Capturing useful information alongside data about interactions with models can become a competitive advantage. Our GitHub repository includes notebooks for getting started with agent memory and hybrid retrieval.