Featured snippets stalled at 7%. A Google veteran used language models to hit 70%

Sridhar Ramaswamy co-founded Neeva in 2019 after 16 years at Google, where he had most recently run ads, commerce, and privacy — the internet’s most profitable business. Talking with No Priors hosts Sarah Guo and Elad Gil, he describes the company as a return to basics. He left Google knowing he wanted to start something. After a long stretch with co-founder Vivek Raghunathan, the pair decided they were still excited about search: the geeky pleasure of helping people find information, plus the ambition that 20 years in, the product could be rethought.

The aha moment was abstract. If they did not have to deal with ads, if they did not have to worry about monetizing, they could start from first principles. Startups, he reminds the hosts, are as much about taking advantage of opportunity as about the original direction. The first three years of Neeva were spent building a better private, ads-free search engine. That stretch also delivered harsh lessons about whether consumers were ready to change.

What happened with AI and large language models the previous year became a second aha: they could keep the principles they started with and still create a much better experience. At the core, Ramaswamy says, Neeva was a bet that there must be a better search product — that it cannot be that there is one company, one religion, one product for the whole world. Plenty of daily Google users would say the incumbent is pretty good. He thinks a default that strong also makes people blind to what could be better.

Early Neeva leaned on being private and ads-free. Ramaswamy is blunt about the result: they underestimated how much people, especially in the United States, would care. Figuring out consumers is a very tricky thing. People often will not do what they say they will do, and will not even admit to the things they will or will not do. That, he says, is the nature of the game.

One surprise was Europe. Neeva did much better there than in the United States, even though the two markets do not look that different on paper. In practice, the number of people who care enough to change search engines is very different. Integrations of personal data and preferences had varying degrees of success. The fundamental challenge, especially in the U.S., was getting people to take the initial step of caring enough to want to change.

Once people actually switch, the job gets considerably easier. They begin to see all the things that were not really that great about the old experience. For a consumer founder, he calls these pretty harsh lessons in consumer psychology, and ones you have to learn. Product quality and the willingness to break a habit are not the same problem. The hard part is always the first step.

Neeva’s later breakthrough was AI summaries, which look nothing like a list of blue links. Ramaswamy says Google engineers and executives would be entitled to reply that they have been doing this for 15 years, and in a sense they have. Google launched featured snippets around 2010 or 2011. It was always known that an answer right in the main search experience trumps all. Elad, who overlapped with him at Google, would remember the proof: Google knocked out Live.com — then the world’s best image search — by putting images into the main search box. The same thing happened with Yelp and local. It did not matter how good Yelp was if the answer sat on the results page.

Featured snippets, which pick out the two or three lines from a website that match what the user wants, were always a big win. People love the product for a reason that goes back to Occam’s razor: anything that minimizes work wins. Give an answer instead of a click, and of course they like it. That is why Google’s currency-conversion widget is wildly popular. It is not that you cannot click through to another site. It is that you do not have to. Answers, in that sense, are old. What search actually returned for decades was a set of opaque links. Google’s trillion-dollar business is built on the fact that users cannot really tell a good link from a bad one. The New York Times domain makes the brain say the site is good. For most other sites, you click and find out. Opacity plus linear scanning is how search has worked.

Users, whether they state it or not, want the answer in the fastest possible way. Featured snippets were never deployable at scale. The technology simply was not there. Even if Google put the full might of its machine against the problem, coverage never really extended beyond five, six, seven percent. Website owners were unhappy: you are taking away my clicks. So the feature stayed edgy — something Google could show, but not too much. The aha with large language models was that, for the first time, a model could take any content and write a summary that got to the heart of the page, often in the context of the query. A blog with six sections and a query about one of them needs the right section summarized. Summaries had been a frustratingly vague concept: people can do a reasonable job, but making a model do it in general is hard.

Neeva spent a year making that work at scale and decided not to be beholden to OpenAI’s API for summarizing a four-billion-page index. A lot of the technology was built in-house. The cumulative product is cited summaries: one fluid, authoritative answer to a complicated query. Ramaswamy says they can write that kind of answer for 50, 60, 70 percent of queries. The same general-purpose learners that summarize text can also pull out structured information. He realized they were sitting on a gold mine beyond compare. The 50 to 70 percent figure excludes one-boxes that already fire, such as stock and weather cards. While working on a Poe integration — Poe being Quora’s chatbot aggregator — the other team’s feedback was to just give the weather back if someone asked for it. The core claim is simple: if you can provide a believable answer, people will always prefer it over any number of links. People do not like clicking on links.

Elad asks about the other 30 or 40 percent, and about cost. When Google built one-boxes, each type of result meant custom indices, crawls, and ranking. Is the idea eventually to do everything via LLMs, and is that prohibited by cost? Ramaswamy does not see a natural limit on how much language models can be used with search. He does see a strong limit on how many questions can be usefully answered. Search engines, he says with some shock, are actually pretty terrible at a lot of tail queries that people now type into a chatbot without thinking twice. Jason Calacanis typed “how are the Knicks doing this year” into Neeva and other engines and concluded the AI did not work. The real point is that nobody in their right mind used to type that into Google, because it never gave great answers. Tail queries have always been served poorly. That will not change instantly.

Neeva’s approach is retrieval-augmented generation: treat the search engine as a tool a large language model uses. Even that is only the beginning. There is no reason the system cannot recognize an arithmetic expression and fire a Python interpreter, or some other API. The difference between a chatbot and a search engine that combines a chatbot with retrieval is going to look more and more blurry. Hard questions will continue to be hard. A lot of questions people expect answers for will be eminently answerable with LLMs as one of the tools. Triggering used to be a giant ball of regular expressions. Extracting structured information from queries, and the old nightmare of Beautiful Soup over ever-changing websites, generalizes with even the smallest models now available.

Cost is the other constraint. Elad notes that when Satya Nadella talked about integrating LLMs into Bing, it almost had a “your margin is my opportunity” feel — as if Bing were willing to subsidize the feature to hurt Google’s margins. Ramaswamy says the comment might have meant two things: the cost of serving, or the margin Google makes on a deal like Apple’s. OpenAI had just cut API prices by a factor of ten, which he calls a little insane this early. The arithmetic is still ugly. An average very large model call costs about five cents, or $50 in CPM to serve a thousand queries. Average RPM for U.S. queries is about $40 to $50; his memory of the rest of the world is on the order of $20. Bing’s Sydney will issue up to three queries per question. Put that way, the cost is astronomical.

His own view is that you do not need the full power of the largest model to get most things done. For page summarization, Neeva is comfortable with models in the 5- to 10-billion-parameter range, fine-tuned in-house, with a human feedback loop about to kick off. The attitude is to run six kinds of models rather than one model that will conquer them all. For known, well-specified problems, model size is not really going to be an issue, and size and serving cost should keep falling. If Nadella meant Apple’s traffic, Ramaswamy would offer Apple 100% of revenue to get it. There is precedent: Google gave more than 100% to AOL and close to 100% to Yahoo in its early years. That is how you make markets. Bing also lives with a public perception — deserved or not — that its quality is worse than Google’s. Very few people on the planet can objectively judge search quality. It is perfectly rational, he says, for Bing to start with a better product and then go out of its way to establish a beachhead.

Guo asks what has been hardest about building search as a startup that cannot spend a billion dollars. Ramaswamy says search needs a fair amount of scale before you have any kind of meaningful product. An ads system can be built by a three-person team because the data is limited. A new mail client is a small problem; scale issues show up after a million users, not on day one. Search is more like standing up a new mobile network. You have to do a lot of work just to be seen as vaguely competitive. Everything from the crawl to the index has been a struggle. It is one of those problems where grown men and women work on it for three months and run away: they cannot deal with this, they just need to go.

Neeva survived it with an unusually strong team. He singles out Asim, who built a system that ran completely on flash, with super-rapid iteration: replace the entire index in the space of two days, or drop in arbitrary amounts of data for experimentation. Problems that took Google 15 years to solve, they had solved out of the gate, because Asim had run into many of them before. They also took shortcuts. Language models are universal input-output machines. A lot of problems Google solved with massive scale and user data, Neeva could solve with models instead — query rewriting, extracting structured information. People ask about the weather in many wondrous ways. A hard-coded trigger system is being replaced with an LLM that extracts structure.

The single biggest positive for the team was launching answers. Until then they had the feeling that even if they were as good as Google, or better, no one would care, because people cannot tell between lists of links anyway. Once that became an actual answer that, in Ramaswamy’s phrase, “my mom can take a look at and say way better than a bunch of links,” excitement showed up. Over the past few quarters people realized this could be a transformational experience. He describes it as a big jolt of electricity through everybody — how excited they are, how hard they work.

Distribution, Ramaswamy says, is hard. Habits are hard to change. You can dislodge some of that with a superior product and some of it with dollars. A decade and a half earlier, Google was already spending hundreds of millions of dollars a year on distribution; the Apple default deal has only grown that number. Part of the reason Neeva released Gist, an experimental app, was a deliberate question: if search looked like Instagram Stories, what should it look like? Sometimes you have to look for change at the locus of change. They are also offering conversational search on a publisher’s own corpus — Reddit, Boston.com, and others — at a moment when search traffic to websites may disappear and LLMs may scramble the aggregator-publisher relationship.

Partnerships with privacy products have worked better than a frontal assault on the default search box. Dashlane is live; ProtonMail is in conversation. The idea is to show up where users already have a clear intent. Distribution continues to be easily his top worry for how Neeva gets scaled. On the business model, they opted early for a privacy-centric subscription without ads. For Neeva it was a way to stand out and to give the company a clear runway. Ads, thoughtfully done, are an incredible juggernaut: they bring scale and they can disconnect monetization from the product, almost as a separate team. When they are very successful they get annoying. He does not like watching broadcast TV anymore. Sports broadcasts drive him crazy for how many ads he has to sit through.

Ads, he says, come with elements of self-destruction built in. When you are doing it, it is always attractive to show more. Hybrid approaches — start ads-free, maybe use ads later as an additional mechanism — might be more sustainable, even though reasonable people argue that teams who come to ads later are even more aggressive. Google search ads, in his telling, tried very hard for a long time to hang on to quality bars and user metrics. Compare that with Amazon today: he finds the Amazon search experience a joke, so full of ads, including misleading ads, that it is really hard to find what you want.

If you are in the business of providing answers, like ChatGPT, ads become a whole lot harder, because you are betting on the quality of the answer. For casual-consumption products — social media, or even where search might go — it is an open question. He points out that a Gist-style experience, a summary followed by a series of cards, could stick ads in; Neeva is not planning to. Elad recalls Google’s early argument that willingness to pay was itself a quality signal. Ramaswamy’s reply is the sharpest line of the hour: smart people will come up with great explanations for everything they do as long as it is convenient. “The best religion to have are the ones that are aligned with your business interests.” There is some truth in the old argument, he says, but it is not an explanation for two screenfuls of ads on a phone. Stories about ads enabling free products for people in Ecuador, told by billionaires sitting in Palo Alto, he finds entirely self-serving. His attitude: yes, we can make money with ads, it works pretty well, we are rich, it is okay.

If cited summaries and chatbot answers are compelling to consumers, what happens to the people who publish the underlying pages? Ramaswamy calls it one of the big unknowns. Larger, forward-thinking creators — he puts Reddit and Quora in that bucket — will say they want to be part of search but not part of answers. Taking their data and sticking it into LLMs is not really allowed by their crawl policy. They will also ship their own chatbots so you can browse Reddit or Quora content in a conversation. Smaller publishers will not be able to do this.

He likens the moment to dropping a bomb, or a giant impulse, at the center of how a lot of people get information. The shock will radiate out to a whole bunch of sites and to the content ecosystem. It is hard to predict, but it feels as if there might be more centralization or consolidation in content creation. The average small blog that could subsidize itself with advertising is going to find it hard to compete in an answer world, especially if the expected experience becomes: I do not really want to read giant pages, give me a summary, then I will ask follow-up questions. Those experiences are possible, but not for every blog that exists. A different platform for how content is created may evolve.

Asked what else AI will disrupt beyond search, he is not even talking about synthetic content — that will happen, and detecting it will be a cat-and-mouse game. An obvious place where content is generated is advertising. Personalized, multimodal ads can play a pretty big role. He jokes that Michael Jordan is going to look you in the eye, speak your name, and tell you to buy Air Jordans. Advertising, with its closed loop for optimization and relentless focus on efficiency, is a natural area. Incumbents will also react faster than the SaaS narrative suggests. Microsoft being this innovative this late in the game is not something you hear from IBM, not at consumer and worldwide scale. Pure SaaS innovation on products we take for granted gets a little harder.

The area he is personally most excited about is generalization: if you think of LLMs as a kind of machine language, the natural next step is combining them with the tools we already use — search engines, calculators, APIs, programs, other websites. Action transformers could be incredibly powerful, not just for RPAs but for an AI SRE, an AI code reviewer, fill in the blank. The technology is much more nascent than what people have come to expect from language models. OpenAI can crank out new generations of LLMs; this tool-using layer is not yet at a point where people can build lots of applications on top of it. Guo calls it the agentization of the world. Ramaswamy is waiting with bated breath.

Wrapping up, Ramaswamy says it is a really exciting moment for a technologist. The new technology is powerful and it is getting democratized very rapidly. WhatsApp, for him, was the seminal moment of mobile computing: a team of 30 people could create a product for the whole world. That represented the power of mobile platforms. The equivalent test now is whether, two years from now, three college kids — 20 years old — can build a brand-new application that uses the things we already know, web servers and databases, plus language models in a fundamental way, and make people say they never thought of that.

That, he says, feels very possible, and that is what is really exciting about where we are. In the meantime he is super excited about where Neeva can take search, and he is counting on the hosts’ wisdom and support. The practical reading of the hour sits underneath that optimism. Long-tail questions of the “how are the Knicks doing this year” variety were never well served by keyword boxes; they belong in cited answer engines. Cost should be compared as CPM against the RPM of the market you are in, which is why most Neeva tasks run on fine-tuned 5- to 10-billion-parameter models rather than a single giant one. And if your product sells answers, ads get harder, because quality is the bet; if it sells time-on-site, ads remain an open option.