Vinoo Ganesh on Building AI Answers from Verifiable Facts

Vinoo Ganesh, CEO and co-founder of Kepler, returns to a theme from an earlier talk: financial institutions need ways to verify AI-generated work. He argues that adding a verification layer only after an answer has been generated misses important design choices earlier in the pipeline.

Ganesh says Wall Street's trust problem is not just that a model occasionally invents a number. A firm's own research and internal data may never have been retrieved or represented in the answer. At the same time, he values the creative analysis that can give an investor an edge, which he calls alpha.

Financial firms are buying retrieval systems, building their own and running agent pilots, but a convincing demonstration does not necessarily produce an answer a professional will rely on. Ganesh frames trust partly as an accountability problem: someone must be able to understand and take responsibility for the work.

He identifies three routes to a wrong numerical answer. First, retrieval may return the wrong document; even defining the right source can depend on whether the firm prefers its own analysis or a particular external analyst.

Second, a system can retrieve the right document but extract the wrong figure. A later annual report might contain figures for several years, for example, so a 2023 revenue question must not be answered with the 2025 value. Citation checks can help, but an after-the-fact filter alone does not change how the answer was produced.

Third, a model may make an arithmetic error. Ganesh advocates letting deterministic code perform arithmetic rather than relying on generated text to calculate the result. His broader aim is to reserve the model for planning and interpretation while using conventional tools for operations they handle well.

In Kepler's design, he says, preprocessing extracts facts from documents into structured storage and assigns identifiers or pointers to them. The model plans which sources and entities to consult; code then retrieves the referenced facts and performs calculations. This moves work from repeated model reading at query time into a preprocessing stage.

Ganesh says the output path is designed to resolve a pointer before a number can appear, followed by a separate verification pass. Readers can click a citation to see the source. Later in the talk he acknowledges that models can still generate unsupported numbers during inference, which is why additional post-processing is needed.

Drawing on his former work in business engineering at Citadel, Ganesh compares the model to a skilled analyst paired with deterministic tools. Investors need reliable data, but they also need judgment about incomplete information. The goal is to look up known facts and reason carefully about what remains unknown.

He describes storing information that can be represented deterministically and preprocessing less structured information into a hybrid system. Search and reranking identify candidate material; pointer dereferencing then retrieves a particular stored value. A lookup can return the stored value reliably, but choosing the wrong pointer or storing a wrong value remains a separate risk.

For broad retrieval, the system combines approximate nearest-neighbor search and BM25, using reciprocal rank fusion and a cross-encoder to narrow the candidates. His slide shows a funnel from up to 200 candidates to 100, then 20 documents and finally 10 results; the displayed RRF parameter k is 60.

Ganesh favors searching broadly before narrowing. In his experience, an agent can be tempted to accept an early result instead of exploring a wide set of sources, partly because examining more material is costly. The design tries to make that breadth a property of the retrieval system rather than relying only on the model's search plan.

He then turns to security in document retrieval. Examples of failures include omitting tenant filters, accepting a caller's claimed identity without proper checks, returning stale results, allowing direct reads to bypass search rules and querying an unpopulated index.

His recommendation is to build authorization into the scope of every query and document access path. A financial firm's research can be highly sensitive, so retrieving useful context is not enough: the system must also prevent a user or tenant from receiving material they are not entitled to access.

For verification, Ganesh describes identifiers and hashes that connect stored facts back to source material. If a requested pointer cannot be resolved, it should not yield a fact. This provides a way to check provenance, though it does not by itself prove the underlying source or the choice of source is correct.

He presents the intended contract as citations that are verified, corrected at search time or refused. The same pattern could apply outside finance, such as checking whether a cited legal precedent actually exists. The broader objective is to make the provenance of a fact visible rather than merely attaching a plausible citation afterward.

Ganesh worries that unreliable outputs could make institutions reject useful AI tools. He argues that wider adoption requires systems whose numbers and sources can be inspected, not just systems that produce fluent presentations. His remarks about industry adoption are his own assessment.

Showing production results, he says unsupported-number refusals dropped after the team introduced pointer-based handling. They did not fall to zero: he acknowledges that a probabilistic model can still generate a number during inference, so the implementation also uses a subsequent checking step. He does not provide enough context here to treat the result as a general benchmark.

His practical challenge to builders is to make a number inspectable, whether it was extracted, calculated or both. He asks vendors whether every citation and arithmetic step is checked, whether access controls prevent unauthorized documents reaching clients, and what happens when a fact has no source.

Ganesh closes by arguing that better models still need reliable data infrastructure and carefully designed software around them. A missing database value should not quietly become an invented fact; trustworthy applications need retrieval, controls and verification to work together.