Manage AI sources: retrieve selectively and withhold context when useful

Today we are looking at managing sources and the information that goes into an AI model's context window. Preferences, memories, uploaded documents and retrieved material can all influence a response, including information that persists beyond a single conversation.

A simple place to start is saved preferences. If you want shorter answers, fewer emoji or less of a recognizable AI writing style, you can record that preference once instead of repeating it in every prompt. Major AI tools provide some mechanism for doing this across sessions.

In ChatGPT's personalization settings, for example, you can adjust characteristics such as warmth, enthusiasm and formatting, and add free-form instructions. I use that space to ask the model to challenge my ideas instead of simply agreeing with me.

Claude has similar instructions. Its published system prompt encourages minimal formatting and prose rather than excessive bold and lists. I can add my own preferences in Claude's settings, including an instruction to use more formatting while still challenging me. The point is to save the combination that works for you.

A second kind of personalization happens automatically. These products can retain information inferred from previous conversations, such as where you live or what you do, and bring a summary into later chats. That information may influence responses even when the new question is on another subject.

Depending on the product and license, you may be able to inspect or disable this memory. If an assistant seems to treat you in an unexpected way, looking at what it remembers is one place to start. ChatGPT, for example, provides controls for whether memories are collected and for viewing them.

Some tools offer a way to move this personalization between products. I show Gemini's prompt for extracting saved memories from ChatGPT. It is also an example of a detailed scope of work: it specifies the categories, desired structure, supporting quotations and the conversational filler to omit.

All of this changes what occupies the context window. Alongside your latest prompt and the answer generated so far are the system prompt and earlier exchanges in the current chat. That is why I recommend starting a new chat when you move to a new topic: unrelated material consumes space and can influence the answer.

Settings, saved memories, uploaded files and the results of web searches can also be brought into the window. My diagram is conceptual: the exact contents and order vary by product. The boundaries in the drawing should not be taken as literal walls separating each source.

Conceptually, the model predicts the next token over a sequence that includes these different kinds of input. I will return to the security implications of putting instructions and outside documents into the same context, but first consider what happens to the information you share in a chat.

A saved memory changes the context supplied to the model; it does not by itself continually update the model's trained weights. Generally a released model's weights remain fixed while people use it. A company that can see chats may, however, use them when training a later model, depending on the applicable settings and terms.

That distinction matters for privacy. I would not say the chance of a future model reproducing a sensitive detail from training data is necessarily zero. Stanford's agreements with the AI companies it licenses require chat histories to remain private and not be used for future training; an independently created account does not automatically have the same guarantee.

Retrieval-augmented generation, or RAG, addresses a different problem: your source material may be too large to fit in the context window, or so extensive that the model has difficulty using the middle of it. A separate document store can hold much more material than you would want to put into one prompt.

When you ask a question, the tool searches the document store for relevant material and brings selected material into the context window. The model can then ground its response in those sources instead of receiving the entire collection at once. The selection still matters: retrieval does not make every stored document simultaneously available to the model.

A large store is not an excuse to add sources indiscriminately. AI systems are often designed to rely heavily on material supplied in a RAG store, so inaccurate information there can disproportionately influence an answer. Curating the quality of the documents remains important even when storage space is not the constraint.

That preference for retrieved material is a tendency, not a guarantee. A paper on knowledge conflicts describes two failure modes: an 'adaptable chameleon' that accepts a supplied source too readily, and a 'stubborn sloth' that falls back on what it learned during training even when the user wants it to use a different source.

Both behaviors depend on context. If a bad document has slipped into the store, resistance may be useful. But if you are supplying reliable material to correct or adapt a response, you may want the model to follow it. Deciding which source is authoritative is not always a simple technical question.

These conflicts also raise questions about who gets to decide what counts as correct. I expect tension as a small number of companies make choices that become embedded in tools used by many people. Users and providers may disagree about when a model should resist a supplied claim.

Another thing we can save is a repeatable task, rather than just a fact or preference. A detailed scope of work can take time to develop. Once you have an output you like after a long exchange, ask the model for a standalone prompt that could reproduce the process next time.

Claude calls its reusable instructions skills; Gemini has gems. You can save a detailed instruction under a name and reuse it, and some setups also attach documents or other context. These instructions can be shared with others, so you may find a starting point for a task rather than building every prompt from scratch.

One example is a Stanford visual-identity skill intended to help produce materials consistent with detailed rules about logos, fonts and colors. At the other end of the spectrum are shared role-playing gems. The range shows how reusable instructions can encode either a formal workflow or an entertainment experience.

Treat a skill or gem from an unknown source with care, particularly if it asks for access to files or email. Installing one can carry risks similar to installing software. In response to a student's question, I explain that these instructions are added to the context; they do not change the model's weights.

Sometimes the useful choice is to leave information out. I call this strategic withholding: deliberately remove context that might bias the model when you want a fresh perspective. For instance, ask it to evaluate 'this essay' rather than 'my essay' if you are concerned it will be overly agreeable.

For a résumé, you might present the job listing and ask the model to evaluate a candidate from the employer's perspective without disclosing that you are the candidate. This framing may produce more candid advice. Remember that saved memories may supply the identity anyway, so consider a temporary chat or a separate account.

A fresh chat can also test whether a document stands alone. After developing a proposal through a long conversation, give only the final document to a new chat and ask why it makes a particular choice. If the model cannot explain the reasoning, the document may depend on background that an independent reader will not have.

Strategic withholding has other uses: keep your own position out of a negotiation role-play, avoid sharing unnecessary private information, or ask for brainstorming ideas before showing the model your favorite. An example supplied too early may anchor the suggestions around it instead of producing a wider range.

A student asks how all this context affects token usage. I explain that reasoning generated by the model may add to output token use, and that having more material to consider can increase that work. The way usage is charged depends on the product and its pricing.

The security problem is that the context can contain material with very different origins and purposes. I contrast a model's handling of instructions with the stricter separation an operating system uses for privileged operations: the model does not have an equivalent perfectly reliable boundary simply because we label one piece of text a system prompt.

Some attempts to override instructions are harmless, such as asking for more bold text. Others target safeguards against harmful behavior. Testing whether a model can be induced to do something it was designed to resist is one kind of red teaming, and the consequences depend on what the system is permitted to do.

A more worrying source is a web page retrieved by the assistant. Web access is valuable because it lets a model find information newer than its training, but the retrieved page is supplied by someone else. That page may contain instructions aimed at the model rather than material the user wanted summarized.

This is prompt injection. In my example, a malicious page tells the assistant to ignore previous instructions and send advertising email. The user did not ask for those messages: an outside document inserted the request into the assistant's context. A model that follows it has confused untrusted source content with an authorized instruction.

A student asks about blocking third-party text. I point out that the impact of the email example depends on whether the assistant has an email connection. Without that connection, the unwanted instruction might contaminate the answer, but the assistant cannot use that tool to send the messages.

The example involves a document found during a web search, but uploaded files and links can also raise questions about untrusted content. Reading a file before uploading can help, though it is not foolproof: an attacker could hide text in white font that a human reader does not notice but a machine processes.

Asked about RAG versus web search, I distinguish the curated document store used in this lesson from a general web search that brings in outside pages. Uploading a document directly to a chat is a rudimentary related example, although it may put the whole document into the window instead of retrieving selected pieces.

The danger grows when an assistant can take actions through connectors. Access to email or a hard drive changes the possible consequences from an odd answer to an action in another system. Be deliberate about enabling those connections, especially when the assistant will also read outside material.

A student asks how tools can recognize a suspicious instruction in a skill. I explain that providers can train models on examples of prompt injection and use feedback to discourage compliance, alongside other protective designs. Those measures can help, but I expect some risk to remain.

Finally, a student asks how to create a RAG setup in practice. Directly uploading a document offers a simple starting point, but the whole file may enter the context. In the tutorial, students will use a notebook product designed around working with a curated collection of sources.