How Bridgewater Keeps Its AI Analyst Inside Permission Boundaries

RÉSUMÉ

En 30 secondes

Aaron Linsky, head of product engineering at Bridgewater Associates, describes the firm's Vault: about 100 million external and internal documents, each with its own permission rules. A useful AI analyst must retrieve private research while ensuring that a user only sees material they are allowed to access. Bridgewater's first approach broke down as the Vault grew. Nikhil Benesch, CTO of turbopuffer, explains how their search engine balances permission filtering with vector search. Post-filtering can miss relevant authorized documents, while pre-filtering can force a slow scan; turbopuffer's native filtering uses document permissions and vector clusters together. They report 120-millisecond search at the 90th percentile across hundreds of millions of vectors, while noting operational complexities such as nested groups and metadata updates.

1Permissions are part of AI searchNikhil Benesch frames an AI analyst as a loop that searches both public information and private company documents. Public results are broadly visible, but emails, meeting notes, and research reports can each carry different access rules.
2Bridgewater's Vault holds 100 million documentsAaron Linsky describes a collection of licensed external research and internal writing, recordings, and reports. Different sources have different permission constraints, and both investors and AI agents read from the Vault.
3Brute-force search misses the latency targetAt one kilobyte per document, scanning 100 million documents from memory would take about five seconds under Benesch's estimate, far above the roughly 100-millisecond search target. Cold object storage makes a full scan slower still.
4Post-filtering sacrifices recallSearching for nearest neighbors first and removing unauthorized results afterward can leave too few useful documents. A user may know an accessible, relevant file exists, yet the search fails to reach it before its time or result limit.
5Pre-filtering sacrifices speedFinding every document a user may see and scanning them all restores recall, but can reduce the query to a brute-force search. If a user can access half of a 100-million-document vault, that scan remains too large.
6Native filtering uses both indexesTurbopuffer combines its vector index with permission indexes. It discards clusters where the user has no accessible documents, then searches the nearest eligible clusters and scans within them.
7The system reports 120-millisecond searchBenesch reports search latency of about 120 milliseconds at P90 across 450 million vectors in 400 search namespaces. Users express their permissions through filters, while turbopuffer handles the query planning needed to apply them at scale.

À retenir

An enterprise AI analyst must enforce each document's permissions throughout retrieval, not just after generating an answer.
Large unstructured research libraries can be valuable only when they remain searchable and securely permissioned.
Post-filtering nearest neighbors can have poor recall because the closest documents may be inaccessible to the user.
Pre-filtering every authorized document can preserve recall but become too slow at enterprise scale.
Combining permission indexes with vector clusters can reduce the search space while keeping relevant authorized results.
Simple permission expressions can hide substantial query-planning complexity and ongoing metadata maintenance.

ThèmesIA en entreprise Privacy & data governance Search & retrieval Vector search AI security

TRANSCRIPTION

Lire la transcription complète par chapitre

Why an AI analyst needs permissions

Nikhil Benesch introduces an AI analyst that searches the public web and private organizational documents. Unlike public sources, internal documents can have different user- and group-level permissions, which retrieval must respect.

Nikhil Benesch, CTO of turbopuffer, and Aaron Linsky, head of product engineering at Bridgewater Associates, introduce their question: how do you build an AI analyst that can keep a secret? In a simplified workflow, a user asks a question and an AI agent investigates by searching for evidence. Some evidence comes from the public web, where everyone can see the same documents; other evidence comes from private organizational data, where access depends on the person and the document.

A company's emails, meeting notes, and reports can each have different permissions, including group access and individual assignments. That creates a difficult information-retrieval problem: the analyst needs relevant documents, but cannot show every document to every user. Benesch says turbopuffer was designed to address this kind of permission-aware search, then hands over to Linsky to explain why it matters at Bridgewater.

Bridgewater's research process

Aaron Linsky describes Bridgewater's effort to understand markets systematically, evolving from handwritten rules through software and toward machine-centered research. Its process moves from observing the world to forming hypotheses, investigating them, and integrating what holds up.

Linsky says Bridgewater has spent five decades trying to understand how markets and economies work, then encode that understanding as logic that can be tested over time. An early bond-trading system fit on a yellow legal pad as a set of if-then statements. The process later moved through Excel, Scala, and Python, with humans at its center; the firm is now exploring how machines can take a more central role in that learning process.

Bridgewater calls its research process a five-step circle: perceive what is happening in the world, form a hypothesis about why, investigate whether it is true, systematize what holds up, then industrialize and run it every day. Linsky says he has worked at Bridgewater for about 20 years. For people and machines to participate in this process, they need high-quality data, including both time-series information and unstructured research.

The Vault and its permission model

Bridgewater's Vault contains roughly 100 million external and internal documents, including licensed research, company reports, and meeting records. Each source can have different access rules, and Bridgewater's initial search implementation failed as the collection grew.

Bridgewater's Vault contains about 100 million documents that the firm indexes, organizes, and secures. It includes material from 135 external sources, each negotiated with legal and data teams and potentially subject to different permissions, as well as internal writing, meeting recordings, and reports. Investors and AI agents read from the Vault, and the content they produce can feed back into it.

Linsky sketches a simplified permission model: a user belongs to groups, and each document can grant access to particular users or groups. The query needs to match both the content and the viewer's permissions. Bridgewater initially built this without turbopuffer and expected it to work, but the system fell over as they added more documents. They looked for alternatives, tested turbopuffer, and found a way to handle the problem at scale.

A search engine built on object storage

Benesch explains how turbopuffer stores data on cloud object storage and brings active data into faster storage when queried. It supports several search modes, but he focuses on vector search and the challenge of permission-aware retrieval.

Benesch describes turbopuffer as a fast, scalable, low-cost search engine built on cloud object storage such as S3, Google Cloud Storage, or Azure Blob Storage. Data can remain cold and inexpensive until a query needs it, at which point active data moves into faster storage such as SSDs or memory. When the queried data is cached, it can be fast; less frequently accessed data can remain on object storage.

This design is intended to scale to terabytes at substantially lower cost than systems that keep all data on disk, while offering comparable performance once the data is hot. Turbopuffer supports keyword and regular-expression search as well as vector search. Benesch narrows the rest of the talk to vector search so he can explain the permission problem and the tradeoffs in more detail.

Why brute-force search does not scale

A full scan of 100 million documents would take seconds even from memory, far beyond the target latency for search. A vector index narrows the search to nearby embeddings, but permissions complicate the simple nearest-neighbor approach.

Benesch starts with a simple baseline: compare a query with every stored document and return the closest matches. Assuming, very conservatively, one kilobyte per document, 100 million documents occupy 100 gigabytes. At roughly 20 gigabytes per second of memory bandwidth, scanning that data would take about five seconds, while typical search workloads want results in roughly 100 milliseconds. If the data is cold on object storage, pulling it all in would take longer still.

A vector index avoids comparing the query with the entire collection. Vector embeddings represent concepts as points in a high-dimensional space: a query about red shoes should land near documents about maroon sandals and far from a concrete block. The system can search near the query vector instead of traversing every document. Embeddings commonly have around a thousand dimensions, although the diagram reduces them to two dimensions for illustration.

The nearest-neighbor index speeds up similarity search by narrowing the candidates to nearby vectors. But permission checks change the problem: some of those nearest documents may be hidden from the current user, while accessible documents could be farther away. The index alone cannot simply return its closest matches if some of them are unauthorized. Benesch says turbopuffer has invested heavily in addressing this interaction between vector similarity and filters.

Post-filtering loses recall

If the system finds nearest neighbors first and removes inaccessible documents afterward, relevant authorized results may sit outside the retrieved set. The system can stop before finding enough documents the user is permitted to see.

The first tempting approach is post-filtering: retrieve the nearest vectors, then discard documents the user cannot access. This is fast and uses the vector index, but can have poor recall. A user who is permitted to see a relevant document may not receive it because unauthorized neighbors occupied the top results; searching farther outward can still hit a time limit before enough accessible documents are found.

Pre-filtering can be too slow

Filtering down to all documents a user can access before measuring vector distance can preserve recall, but it may require scanning tens of millions of documents. That gives up the speed benefit of the vector index.

The opposite approach is to apply permissions first, then compare the query against every document the user can access. This can return the closest permitted results, but if Aaron can access 50 million of the Vault's 100 million documents, the system is again scanning an enormous collection. That gives up the benefit of the vector index and can take seconds or minutes, so pre-filtering alone is also inadequate.

Native filtering combines permissions and clusters

Turbopuffer uses permission indexes to find vector clusters containing accessible documents, then searches nearby eligible clusters and scans within them. Benesch presents the combination as a way to preserve useful recall while avoiding both a broad post-filter and a massive pre-filter scan.

Turbopuffer's answer, which Benesch calls native filtering, combines the two strategies. Its vector index groups similar documents into clusters, and permission indexes identify which clusters contain at least one document a user can access. The system can discard clusters with no eligible documents, search the nearest remaining clusters, and scan within those clusters. Benesch says the clusters currently contain about 200 documents, a value the team continues to tune.

Benesch says combining the vector index and permission indexes makes large-scale filtered search practical. He reports 120-millisecond searches across 450 million vectors and 400 namespaces, describing the figure as P90 latency in his spoken explanation. The important design choice is to combine pre- and post-filtering around the vector clusters rather than rely on either strategy alone.

Latency, permission filters, and limitations

Benesch reports 120-millisecond P90 searches across 450 million vectors and 400 namespaces, and shows how users translate permission rules into filters. Nested groups and group-membership changes remain operational challenges, while the broader goal is isolation-aware search across organizational and infrastructure boundaries.

A customer expresses its permission model using turbopuffer's filter language. For example, a query can require a matching viewer, membership in one of several groups, and a document that has not been deleted. The customer-facing expression is relatively simple, while turbopuffer handles the complicated query planning required to use those filters efficiently at 100-million-document scale.

The approach has limitations. Nested groups make permissions more complicated, and a change in group membership can require metadata updates across documents. Benesch says turbopuffer is working to handle more of these cases automatically and invites teams with such problems to discuss them. He presents permission filtering as one part of a larger effort to make search aware of isolation requirements.

Beyond document permissions, turbopuffer is aiming for isolation-native search. Depending on an organization's security and regulatory needs, that can involve regional data residency, keeping data in a customer's own VPC, encryption at rest and in cache, dedicated compute, or keys controlled by the customer. Benesch describes these as different layers of isolation, with permission filters at the document level as one layer among them.

Benesch closes by describing the partnership between Bridgewater and turbopuffer as a way to perform fast, permission-aware search over a large research collection. He thanks the audience, and the session ends.