James Briggs on Vector Databases, Retrieval and AI Memory

James Briggs recalls that ChatGPT brought a sudden wave of interest in language models and Pinecone sign-ups far beyond what the company had previously seen. Matthew Berman introduces Briggs as Pinecone’s developer advocacy manager and an AI educator. Briggs’s perspective reflects his position at the company; the interview does not identify a separate sponsor.

Berman asks what Pinecone and a vector database do. Briggs starts with an internal company search bar: a traditional keyword search may require users to guess the precise words in a document, even when they know what they mean. Semantic retrieval instead attempts to find content related to the meaning of the query.

Briggs says the approach can extend beyond text: with an appropriate multimodal embedding model, a text query such as “a cute dog” can retrieve relevant images. The representation and training matter; a vector database does not inherently understand every kind of content.

Berman asks what a developer actually stores. Briggs explains that an embedding model turns text or another input into a numerical vector. Similarity is then computed between vectors; the database indexes and retrieves them. The underlying source text or a reference to it must also be available when the application needs to show or pass the result on.

Briggs uses points on a two-dimensional graph as an illustration. Real embeddings often have hundreds or thousands of dimensions. Nearby points should correspond to inputs that are similar according to the model and its training, not according to an objective measure of all possible meanings.

Some embedding models are trained for general semantic similarity; others are tuned to match questions with useful answers. The choice depends on the task. A query and a relevant answer can be close in a retrieval space even if they do not resemble each other as paraphrases.

Berman checks his understanding with a poem: instead of storing only the literal text, an application can turn it into a sequence of numbers and index the vector. Briggs agrees, then distinguishes the embedding step from the database’s role in storing and retrieving those vectors.

At the time of the 2023 interview, Briggs describes developers generating embeddings themselves and sending them to Pinecone. He mentions hosted APIs such as OpenAI and Cohere, as well as Sentence Transformers for a more open route. Product capabilities evolve; this description is tied to the interview date.

For some data, Briggs says fine-tuning an embedding model may help, though an off-the-shelf model can be a useful first test. Berman restates the workflow: call an embeddings endpoint, receive a vector and send it to Pinecone for later retrieval.

Berman asks why vector search became associated with language-model applications. Briggs says one common use is retrieval: the model may not have a company’s private records, recent events or specialized facts in its training data, and it may generate an unsupported answer when asked.

Briggs describes taking a user’s query, searching an index and returning relevant passages, then placing those passages alongside the question in the prompt. This pattern is commonly called retrieval-augmented generation. Relevant evidence can help ground an answer, but retrieval mistakes and hallucinations remain possible.

The external material can be updated independently of the model’s training. Rather than retraining every time an internal document changes, an application can index revised source content and retrieve relevant pieces. This does not alter the model’s parameters or ensure that the retrieved answer is correct.

Berman calls this a form of long-term memory and a way to work with material too large for a context window. He uses a long PDF as an example: index portions of the document, retrieve passages relevant to a question, and pass those passages to the model. Briggs agrees with the description.

Asked about other uses, Briggs points to semantic search and recommendations. He gives a simplified example in which a user’s viewing history and other users’ behavior are represented so a system can find similar patterns and suggest shows. Actual recommendation systems can use many signals and architectures; these examples do not establish the implementation of any particular company.

Berman says search across a website or a software product is another appealing application. In his experience, a vector service made a relevant-search prototype easy to set up. Briggs agrees that preparing and cleaning the underlying data can take more work than the initial search function.

Berman recalls a difficult customer-search implementation at a previous company using Elasticsearch, with contact information and message history in the searchable records. He contrasts that experience with his quick Pinecone prototype, while acknowledging his older implementation was years earlier.

Briggs associates their past Elasticsearch setup with lexical methods such as BM25 and TF-IDF, which rank matches to terms. That experience should not be generalized to the entire product: [Elastic’s documentation](https://www.elastic.co/docs/solutions/search/hybrid-search) describes both vector and hybrid search. Lexical and semantic methods can complement each other.

Berman asks about Pinecone’s history. Briggs estimates that product development had been underway for several years, with the recognizable service released roughly one to two years before the conversation. He says he encountered Pinecone through freelance work and had been at the company for almost two years.

Before ChatGPT, Briggs says, the team was small and part of his job was explaining vector search from first principles. In his recollection, even many people he met in information retrieval did not discuss the topic. His account is a personal impression, not a measurement of the entire field.

Earlier conversations with potential users focused on semantic search and recommendation systems rather than the language-model memory applications now attracting attention. Briggs recalls a small, specialized audience and the work of showing why similarity retrieval could be useful.

Around May 2022, Briggs says, Pinecone worked more closely with OpenAI engineers. They demonstrated how to combine Pinecone retrieval with GPT-3. He clarifies that this was not secret advance access to ChatGPT: GPT-3 was already available, and the key lesson was how to structure the prompts and retrieved context.

His earlier question-answering work often extracted a span directly from retrieved text. With GPT-3 and a well-constructed prompt, Briggs says, the application could also produce a bulleted answer, summarize in two sentences or pull out relevant Python libraries. The flexible output impressed him.

Then ChatGPT changed the size of the audience. Briggs remembers a New Year gathering in the Philippines where people working in marketing and search optimization were talking about it. For someone accustomed to explaining the technology to a narrow audience, the change felt abrupt.

Berman contrasts the earlier retrieval conference with that social gathering. Briggs says that as people noticed language models could hallucinate or lack up-to-date domain knowledge, many began looking for retrieval-based approaches and learning about vector databases.

Inside Pinecone, the team already understood a potential connection between language models and vector search, but Briggs says ChatGPT exceeded their expectations. Interest and sign-ups grew by orders of magnitude compared with their previous baseline. He describes this as the company’s experience rather than publishing a verified growth dataset.

Berman asks whether AutoGPT produced another spike. Briggs says that after a period of rapid growth he returned from a weekend away and saw a single day’s sign-ups dwarf previous activity; colleagues explained that an AutoGPT integration was driving the increase.

Briggs says many of those early AutoGPT users had only about ten embeddings on average. A hosted vector service was not necessary for such a tiny collection, and Pinecone helped the project add other options. Usage later eased after defaults were changed. The account illustrates a trade-off between adoption and a tool’s suitability.

Berman describes AutoGPT as software that iterates on a high-level objective using model-generated steps. He frames its possible relation to future AGI as speculation. The popularity of an agent project does not establish an AGI architecture or make autonomous operation reliable.

For large external knowledge bases, Briggs points to collections of hundreds of thousands, millions or even billions of items as the kinds of problems where a managed vector database may be useful. These figures are examples, not universal cutoffs: latency, filters, update rate, cost and operational requirements also matter.

For a smaller collection, Briggs suggests keeping embeddings in a NumPy array, embedding the query and calculating similarity scores such as dot products. The highest-scoring results can then be selected. A few lines may suffice for a prototype, though the appropriate similarity metric depends on the model.

Briggs says he still sometimes uses Pinecone for small projects because it is managed and offers features such as filtering and hybrid search. A local array avoids a service dependency, while a database can offer operational features; neither size alone nor vendor affiliation determines the right choice.

Berman asks whether expanding context windows will make retrieval obsolete. Briggs welcomes larger windows but says models at the time could still overlook instructions even with a few thousand tokens. Fitting information into a window does not by itself guarantee that the model will reliably use it.

Even assuming perfect attention to a long prompt, Briggs argues that a 100,000-token or million-token window is small compared with large indexed collections. A token count and a number of stored embeddings measure different things, and the amount represented by each vector depends on chunking.

Briggs expects retrieval systems to remain useful for selecting, updating and managing large data collections, although they may overlap with long-context approaches for smaller tasks. His prediction about how efficiently future context windows will scale is a 2023 opinion rather than a demonstrated limit.

Berman mentions large search platforms as examples of collections too large to place in a single prompt. Briggs says Pinecone is oriented toward larger workloads and regards tens of millions of vectors as less extraordinary than billion-scale collections. The appropriate architecture still depends on the actual workload.

The conversation turns to what AI “memory” means. Briggs explains that a model’s parametric knowledge reflects its training and later updates, while a basic inference call does not automatically preserve a conversation as durable state. An application can manage state separately from the model.

In a chatbot, earlier turns can be supplied again with each new request so the response reflects the conversation. Briggs calls that conversational memory. It is a property of the surrounding application and prompt history, not proof that the model’s weights are learning from each exchange.

A separate external knowledge base can store vectors linked to documents or other content. When the application retrieves relevant records and puts them in the model’s context, the model gets a window onto selected outside information. Briggs calls this long-term memory, while noting that “external knowledge” may be the more precise description.

Berman summarizes three ways to provide or change information: include it directly in the context, retrieve relevant outside material into that context, or fine-tune model parameters. Fine-tuning consumes time and compute and is a different operation from retrieving documents; neither approach automatically guarantees reliable factual recall.

Briggs adds tools and agents as other ways to access information, such as searching the web or calling an API. Their results still have to be passed to the model in some form. He groups the approaches into parametric knowledge, source information supplied in the prompt and parameter updates through fine-tuning.

Berman asks whether memory is required for artificial general intelligence. Briggs says he would like to think so and recalls a science-fiction film whose AI drew on a search engine. He imagines retrieval paired with language models as a possible architecture; this is his speculation, not an established requirement for AGI.

Berman thanks Briggs and recommends his channel and Pinecone. The conversation closes with the same practical theme: retrieval can make selected information available to a model, while the right system depends on the scale, quality and management of the underlying data.