Enterprise Google Failed for Twenty Years. A Search Veteran Finally Made It Work.

Sonya Huang introduces Arvind Jain as co-founder and CEO of Glean, an early Google search engineer, and later co-founder and CTO of Rubrik. Glean began as enterprise search and has grown into a general-purpose work assistant. Bringing AI onto company data is notoriously hard, she says, because of the integrations, permissions, ranking, and parsing required before a model can be useful. Pat Grady joins her as co-host of Sequoia’s Training Data.

Arvind’s pitch is simple: think of Glean as Google or ChatGPT inside the enterprise. Employees ask questions, and the product answers from company knowledge no matter which system that knowledge lives in. Glean is also an AI platform. Teams that want their own applications can use its RAG stack to build them without assembling the retrieval layer from scratch.

Sonya immediately tests the analogy. “Google for work,” she says, is something every CIO has described as their holy grail, and the industry has two decades of failed attempts to show for it — including Google itself. Search is hard, Arvind replies, because it is “magic” in the user’s mind: you can ask anything and you expect the right answer. Inside a company the information is scattered across many systems, so even getting hold of it is difficult, and judging what is still good versus what has gone stale is harder still.

Past attempts were weak, Arvind argues, because the problem demanded so much research and investment that it was not startup-friendly. Before SaaS, you could not even ship a product: connecting to one customer’s data meant sitting with that enterprise for a year just to ingest information, and only then could you start on the actual search problem. The real work never got a chance to begin.

Huang calls Glean one of the first and best examples of what an enterprise AI application company can look like. Under the AI surface, though, the stack is thick: infrastructure, connectors, a governance engine, and a knowledge graph. Pat Grady asks what had to be built *before* the models got good enough to matter. Arvind’s answer is three layers of unglamorous work that most products never finished.

For the first four years Glean was evangelizing a dead market — no competitors, and little customer interest. The team had to create the category one account at a time. Then ChatGPT arrived, search became hot overnight, and every company Arvind talks to now wants to build a product like Glean. He treats the rush as good news: buyers finally care, even if the vendor list got crowded.

Search starts with data. The first slice of Glean’s stack is deep integrations with the systems where work actually lives — Salesforce, Confluence, Jira, Google Drive, SharePoint, ServiceNow. Enterprise knowledge is scattered across those tools, and nothing useful happens until it is pulled into one place. That connector layer is unglamorous, and it is also the price of admission.

The second slice is governance, and it is the sharpest difference from web search. Most company information is private. A Google Drive doc may be visible only to its author or to a handful of colleagues, so you cannot dump the whole corpus into an index and serve it to everyone. The engine has to understand permissions on each piece of content, know who is asking, and retrieve only what that person is allowed to see — across hundreds of systems, each with its own access model.

The third slice is where most products failed: a knowledge graph of people, documents, and the relationships between them. Keyword or semantic match is not enough, because the right answer depends on who is asking. A new hire who wants an onboarding guide should not get the same document in marketing and in engineering. Glean models employees and their roles, models documents and which departments they belong to, and scores popularity and person-to-document ties so the results are relevant to *this* person.

AI was not bolted on later. Language models were part of Glean’s core search from day one in 2019, when the team was using BERT-style models — “we just called them language models at the time,” Arvind says. Semantic understanding meant the product was never limited to the exact keywords a user typed. When large language models exploded, five years of retrieval infrastructure was already in place.

Google’s ranking advantage is the public web: links, clicks, and a single gateway where knowledge-seeking happens. Enterprises do not have that. Huang asks how much of Glean’s magic is ranking, and Arvind says it is the core of the product. The signals have to be mined from how employees actually behave inside the company’s own tools.

A document that people across the company keep opening is important. A document written in the last week or two that is still drawing engagement is probably not obsolete. A document that looks unpopular company-wide but is heavily used inside one team is a treasure for that team. And if someone asks a question in Slack, a colleague pastes a doc, and the asker thumbs it up, that interaction is a labeled pair: this document answers that question. Keep the association, and the next similar query can surface it immediately.

The collection problem is different from Google’s. Google can watch what happens on Google, because that *is* the gateway to knowledge-seeking on the web. Inside a company, most knowledge work does not go through a search box. Glean has to watch communication systems and document systems and learn from that human behavior. The more of those traces you collect, the better the ranking gets — because what you are really learning is how this company works.

Huang asks Arvind to explain RAG as if she were five. After a brief negotiation over whether five or ten is the right age, they keep it at five. GPT, Gemini, and Claude, he says, are trained on the world’s public knowledge. Ask ChatGPT how many PTO days you get and it cannot answer, because the policy is private company knowledge the model never saw. RAG is how you bring that private data to the model.

The loop is straightforward. A question hits a retrieval engine; the engine finds documents that might contain the answer; those fragments go to the model with an instruction to answer using this company knowledge. That, Arvind says, is how most enterprise AI applications are built today. The search engine sitting in the middle is the only bridge between private data and the power of language models. Glean’s five-year search engine is what lets it ship both an end-user assistant and a platform other teams can build on.

The architecture is canonical and still full of holes. Models hallucinate. Retrieval misses. You are chaining two things that are both imperfect. A RAG app often fails at the retrieval stage: it never found the right knowledge, or it found stale knowledge, and then handed that to the LLM. “You can build a great AI demo in two hours, maybe in a day,” Arvind says. Making it robust, stable, and actually valuable inside an enterprise is a different problem.

Grady asks how enterprise search and RAG relate — subset, superset, or the same thing. Arvind treats them as one core technology: put company knowledge in a system you can query and get relevant information back. You can ship that as a standalone search product, or hide it as an API inside a larger AI app. The industry’s RAG stacks usually use a much simpler retriever, typically vector search without full enterprise context. Glean’s test is harsher: the search system has to be good enough to put in front of users as its own product. Pass that test, then drop it behind RAG, and the AI experience holds up. DIY-ing a pipeline and a little retriever in a few weeks is not the same job.

Grady notes that Glean is quadrupling year over year and that Sequoia uses it internally. The real test, he says, is how customers’ days change. Engineers use it as a troubleshooting tool: the error you just hit is rarely original, and Glean pulls how someone else already solved it. Support work flips. Instead of hunting across knowledge bases, agents see Glean’s draft first, validate it, and send it. The job becomes checking what AI said rather than finding the answer.

Salespeople prep with a 360-degree view of the customer — last meeting, open opportunities — and ask Glean live when a call throws a curveball. Inside Glean, the first few sales calls are not allowed to bring a sales engineer; the rep has to answer. The one use case Arvind sees across every role is finding people: given your question, connect you to the right expert. That, he says, is what everybody in the company uses a lot.

The north star is how many daily questions Glean answers successfully. They hold that number at 80% of sessions. Measurement is implicit, not a thumbs-up: if you click one of the top two or three results, stay at the destination, and do not immediately come back to rephrase, the session counts as a win. Huang recognizes the pattern from Google’s Search SAT. The harder gap is not a missing feature. Expectations are infinite — answer any question, then perform any task — so the work is simply getting harder questions right. Arvind puts the company at maybe 2% of the way to an assistant that uses company knowledge to answer anything and eventually does half your work. He later clarifies that 2% is made up, a way to say how early this still is.

Huang cites coding-agent numbers: pure RAG completion around 3–4%, agentic reasoning with reflection and retries around 14–15%. Glean’s move is more cautious. Ask it to write a weekly status report for your team, and it drafts a workflow — look up direct reports in HR, then pull each person’s work from the systems where work happens — but it asks the user to confirm, correct, or ignore the plan and build the workflow themselves. Those human-built workflows become training data for future agent loops. The surface area is too wide to hard-code every task. And the compounding math is unforgiving: ten steps at 90% each leaves you around 35% overall. If you cannot nail simple questions, complicated tasks fail worse. Human assist stays critical.

Act one, enterprise search, is what gives Glean the right to attempt act two, an application platform. Asking a company to hand a startup all of its data is a huge demand if there is no immediate value. The search product is that value: people understand it, they want to deploy it, and once it is running it is already connected to the enterprise corpus. Selling Glean as the core AI data platform is then a much easier conversation, because you are not asking for the data a second time. Grady’s analogy: Tesla had an advantage in self-driving because it already sold cars; Glean has an advantage in agents because it already sells the data platform.

Developers feel the same gravity. Customers who want their own AI apps discover that 90% of the work is boring infrastructure — pulling data from multiple SaaS systems, running ETL, building retrieval, getting permissions right — before they even get to play with models. Glean’s APIs collapse that. Integrations, governance, and search are done; you get a search API and spend your attention on the business problem. Huang hears a Snowflake story, except the central platform is for text and unstructured data.

Five years out, Arvind’s belief is that the majority of knowledge work will not be done by us. Assistants will have the company’s data, the context of past conversations and meetings, no memory gaps, and reasoning on top. Glean wants to be that workplace assistant — the place where most of the work happens. Today’s AI is still mostly you asking and it answering. The future looks like an executive assistant: some help on request, more of it proactive, managing the day. That luxury used to belong to a few executives. He thinks everyone will have it.

Huang ranks Glean among the most successful application-level AI companies, “probably number two behind Copilot and Scale,” and notes the awkward fact that OpenAI is both a provider and a top competitor. Arvind does not dwell on it. The slide he quotes from Sequoia puts software at $600 billion and AI expanding the market to $12 trillion or $15 trillion. His advice: if you found an important problem, go work on it, and do not worry that someone else is solving it — they probably will not solve it the way you will. AI is a tool in the arsenal, not a new kind of company; Glean does not think of itself as an AI company. They will not train GPT-4-scale models, but they do train smaller per-customer models on each enterprise corpus so the system learns that company’s lingo, acronyms, and code names. Frontier scale is for partners.