Why Retrieval Quality Matters for AI Agents Working with Large Document Collections

Aamir Shakir introduces Mixedbread, a company that builds retrieval models and a search product for agents. He argues that agents often struggle less with reasoning than with finding the relevant evidence in an organization’s data.

He uses an equity analyst as an example: a question about Intel’s memory choices may require searching reports, interviews, podcasts and research from multiple providers. A large collection can produce many keyword matches without revealing which passage is most useful.

Shakir says an agent that reads too many candidate documents can spend substantial tokens on discovery. PDFs add complications because text, tables, scanned pages and images cannot always be searched reliably as plain text.

He describes a conventional retrieval pipeline: parsing or transcription, OCR, chunking, embeddings, vector search and reranking. In his view, each transformation can lose information, especially when table structure or diagrams are not preserved.

The pipeline can also require teams to evaluate and integrate multiple vendors. Shakir calls this a “Frankenstein stack” and argues that coordinating these components can consume months of work. This is his assessment as a retrieval vendor, not a universal result for all RAG systems.

He illustrates a limitation of representing a document chunk as one vector. A complex financial report contains many facts; reports from different years may look similar in embedding space even when the query requires a precise figure from a particular period.

As a corpus grows, he says, near-duplicates and extraction errors can make evidence harder to find. An agent may compensate by making more queries or spawning more workers, increasing context size and cost.

Mixedbread’s position is that access to relevant data is a bottleneck separate from model intelligence. Shakir says the company built its product to handle retrieval across text and other modalities, while allowing customers to choose the model used for the downstream task.

He sketches the evolution from keyword search such as BM25 to embedding models and presents a preview of Mixedbread’s new retrieval model. He characterizes the improvement as a step change; the chart is a vendor-presented comparison.

The model uses late interaction: it represents documents and queries with multiple embeddings rather than a single vector. Shakir says this can retain finer-grained information while contextualizing those representations within the page.

In a demo, he uploads Intel earnings material. He argues that a visual representation of a page can preserve relationships in a figure that plain OCR might miss, while OCR and layout information remain available for downstream models.

A query about an Intel Foundry operating margin returns candidate pages. Shakir shows how visual layout and bounding boxes can support citations, allowing a user or agent to trace an answer back to its evidence. The live demo encounters slow network responses.

For more complex qualitative questions, he describes an agentic search model that decomposes a request into multiple searches. He claims it is faster and cheaper than larger general models on the company’s comparison; no independent methodology is provided in the talk.

Shakir presents a hedge-fund customer case. According to the slide, the benchmark contained 55 questions on the customer’s documents and changed the search backend. He reports that answer accuracy rose from 36% to 98% after adopting Mixedbread.

He also says the customer used fewer tokens and reduced language-model spending by roughly threefold because the agent received better context. These are vendor-reported results from one customer, not a guarantee for other collections.

Shakir concludes that retrieval quality can affect both answer quality and cost, and invites teams to benchmark the product on their own corpora. His central recommendation is to evaluate how well the system finds the evidence needed for a task.