His 2018 Prompting Paper Was Rejected. His Search API Now Beats Google

Nathan Labenz introduces Richard Socher as a pioneer of deep learning for natural language processing, former chief scientist at Salesforce, and founder and CEO of You.com — first sold as a new kind of search engine, now as an AI assistant. Socher's path starts earlier than the GPT-3 conversion story most guests tell. In 2003 in Germany he studied linguistic computer science, loved languages and math, and found NLP a small niche that did not have enough math for him. A master's in computer vision at the Max Planck Institute introduced statistical learning and pattern recognition, and he went to Stanford for a PhD.

At Stanford he noticed that even top NLP papers advertised beautiful models — conditional random fields, latent-variable setups — while the actual gains came from feature engineering. Named-entity recognition meant adding "this word is capitalized" or "this token is on a list of city names." He called it graduate-student ascent. Andrew Ng was feeding fixed-size pixels into neural nets and restricted Boltzmann machines on the vision side; Socher wanted the same bargain for language: data in, output out, no hand-built features. In 2010 he published his first neural-net NLP paper. Rejections piled up. Almost nobody in the field was paying attention.

After the PhD he taught at Stanford, first as a lecturer and then as an adjunct professor, because still no one was teaching neural nets as the official way to do NLP. The Hugging Face founders were among his students; he invested in their first round. His first startup, MetaMind, built a general-purpose neural-net platform for vision and language, got acquired by Salesforce, and he became chief scientist there. In 2018 came what he calls his last and biggest rejection: a unified model for all of NLP. Instead of one sentiment model and one translation model, you would ask "what is the sentiment?" or "what is the summary of this sentence?" or "who is the president in this paragraph?" OpenAI later cited that paper in the GPT-2 and GPT-3 writeups. The move now has a name: prompt engineering. Nathan notes that peer review still fails the same way — he had just watched a confident reject on the Mamba paper.

Asked what has surprised him most since that generalist-model idea, Socher says the technical trajectory is close to what he hoped for. The shock was on the application side. By early 2022 You.com already had apps that wrote code and essays inside the search results. Whenever the team moved the default experience too far from Google, the vast majority of users said they were used to Google and did not want another way of finding answers. The product kept getting pulled back toward ten blue links.

Then ChatGPT shipped, and people suddenly got it: it could just be pure text. You.com had been trying to inch there; users needed a bigger jolt. "An actual answer is the best way to get an answer, and that's in text." Nathan, a self-described Perplexity fan who has had Aravind Srinivas on the show, says You.com has joined the tools he uses multiple times a week. For a comprehensive multi-page report, Research mode is often the best tool he has.

A year earlier, Socher says, You.com was the only search engine with a web-connected language model and millions of users. The idea has since been copied many times, including by Perplexity. Differentiation now has to come from better technology, not just from having a chatbot on the web. In a comparison on 500 real user queries, people preferred the You.com answer 50% of the time, saw no difference 30% of the time, and preferred Perplexity the rest. That was for default Smart mode: a complete factual answer with careful citations, including articles from yesterday or today.

He maps the modes onto Daniel Kahneman's fast-and-slow thinking, via an assistant metaphor. Smart mode is asking the assistant to spend two or three minutes on a quick lookup. Genius mode is a question that needs programming, web search, and math — give the assistant a couple of hours, and expect five to ten seconds of wall-clock time. An orchestrating model calls others, writes Python, executes it, and plots. In the live demo it looks up current population figures for India and the United States, assumes a growth rate, and draws a curve out toward 2100. When a library fails a security check, the system rewrites the code the long, messy way without that library and tries again.

Research mode is asking the assistant to spend a day or two. It searches the open web, not just the existing index, follows new leads with further queries, and synthesizes a report. Socher's demo is the background and consequences of the Peloponnesian War: a long essay in which each sentence carries one or two citations you can click. Nathan had thrown a 200-word barrage of mixture-of-experts questions — how many experts, how many active per layer, whether any scaling laws exist — at Research mode in one shot and kept coming back. He used Genius mode for a Fermi estimate of the world's big datasets versus GPT-4's training data and available compute. Socher's line for this whole layer: the future is already here, it is just not equally distributed.

Citation logic is not a cosmetic extra. The team found competitors who "just randomly add numbers and citations to sentences"; you click through and the page never mentions the fact. Socher thinks that undermines the entire space of chatbot search. Accuracy of citations is one of many sub-systems you have to get right. Emergence still startles him: ask for nice cancelling headphones that are not expensive, and the system builds a table with images, pros and cons, and prices without anyone having programmed a headphones template.

Under the hood Socher describes two stacks: search and chat. Bing was expensive and not high quality; Google is hard to access without special agreements, and scraping SERP APIs sits in a legal gray area he does not want. So You.com built its own web index. The old Google and Bing indexes assumed a person looking at ten blue links, so each URL stored a very short snippet. An LLM can read much longer passages, extract the answer, and show the user only that. When they benchmarked the API, it was more accurate than Google or Bing. The trick, he admits, is almost like cheating: the snippets are much longer.

The eval method is borrowed from question answering. Take HotpotQA, SQuAD, MS MARCO, FreshQA, and similar sets, which pair a paragraph, a question, and a short gold span. Throw the paragraphs away. The engine has to recover the right answer from the live web. That is how they scored themselves against Google and the public APIs. Nathan's gloss — richer context per URL, decoupled from whichever model does the reading — is mostly right, Socher says, with a caveat: they do not claim a larger overall index. Rare Indonesian kayaking-rental pages in a local dialect may still live only at Google. On Western news and the queries their users actually run, they return more text per result.

The chat stack he has started calling LLMOS, a metaphor inspired by Andrej Karpathy. The language model is the CPU; RAG over a vector database is the hard drive; the internet is the ethernet card You.com provides; other models you orchestrate are a kind of GPU; a Python interpreter is an app sitting on top, which is Genius mode. Customers using the search API are repeating You.com's own lessons. Open-source "LLM wrappers" look simple until you ask "where was he born?" after "when was Barack Obama born?" and the search backend has no idea who "he" is.

Query transformation is only one of about ten such systems, running millions of times a day for millions of users. "What's a big CRM company?" yields Salesforce; "what's their stock price?" cannot be sent to search as the word "their." Another subsystem is whether to search at all. "Write me a poem about a Bay Area sunset love story" does not need a citation on every line; stuffing poem history into the prompt just clutters it. Some sites slap search results on everything, including a conversation about your feelings. Two more subsystems, mentioned later: which model to call, and dynamically rewriting the prompt for that model based on the query.

Google trained people to expect search for free and ad-supported. The new AI-first engines have leaned toward subscriptions; Nathan has not noticed sponsored units in You.com. Socher is not a fan of ads, but he is blunt: venture money only lasts so long, free products have to survive, and if you will not pay you will see ads. Chat engines will likely grow an ad offering, maybe this year, to users' dissatisfaction. Search APIs have been a duopoly — Google on consumer search, Microsoft on the API — and Microsoft used that position to raise prices five to twenty times. Little companies had to scramble. He is glad there is more competition now.

Subscriptions are growing. You.com still shows classic links in the top right, because sometimes you want a Google-like recovery in one click, and apps that return a live stock ticker instead of a paragraph about the ticker. Safari will not let you set a third-party default, so they built a full iOS and Android browser. Socher is "super stoked" that You.com will be in the EU's iOS 17.4 browser-choice popup in March — the only chat-style browser on that list, next to Chrome, Firefox, and the usual suspects. That, plus getting young people onto the assistant and then into Genius and Research, is a big part of the distribution bet. Near the end of the show he also notes that Julien Chaumond, Hugging Face's CTO, has just become an angel investor.

Nathan floats an "AI bundle": he would spend $100 a month for fractional access to a thousand tools, because single-product freemium is brutal. You.com's $10 subscription is among the lowest he has seen. Socher says they will raise it soon; people willing to pay often do not care whether it is $10 or $20, and anyone who wants GPT-4 — "literally the same underlying model as ChatGPT" — should come in while it is still half the Plus price. Nathan's own company, Waymark, sees users buy once, cancel, and maybe return; VCs hate those metrics. Socher's reply to the bundle is polite and short: sounds like a great idea. Nathan jokes that he will either build it or take DMs from someone who will.

Nathan wonders whether the big tech suites — Microsoft with Teams and Bing, Google with Workspace, maybe Salesforce with Slack and You.com — will play musical chairs and absorb the young AI search companies. Socher thinks Google's monopoly will be harder to sustain. He does not love the analogy, but search may look more like fast food: not just McDonald's, also Burger King, KFC, and Taco Bell. A new generation grew up on TikTok and Reddit. Each of those apps already peels queries off Google.

Amazon was the most successful raid: if you want to buy something under about $50 or $100, you search where you can complete the purchase. TikTok took restaurant search for young people who want to see whether a place makes good videos, not a map pin. Chat also expands the pie. Nobody asked Google for a complex Peloponnesian War essay or a mortgage calculation with two interest-rate variables, because Google would not code or summarize fifty sites. You do not talk about your feelings to a search box. The more complex the facts get, the less you go to Google.

Nathan suggests the limit looks like cloud: data centers, compute, bandwidth, inputs only the big companies have. Socher is "very excited" and "up for it." They entered the space because they saw the Transformer, attention, and prompt engineering and thought the technology was ready to disrupt search. Google makes on the order of $500 million a day; a business that large does not want disruption, which is one reason the Transformer authors left. Open source has lowered the bar from billions of dollars and a decade of work, but turning that into a product that runs millions of times a day, stays up, cites accurately, and stays current is still hard. Distill You.com to two words, he says, and you get "amazing answers." The generation growing up on ChatGPT is not going back.

On limitations, Nathan starts with reasoning. Genius mode's orchestration usually taps GPT-4; coding often does too, but not always. That is subsystem three: which model to use, sometimes several. Subsystem four is dynamic prompting, so the same product sends a vastly different prompt depending on the query. Nathan asks where future reasoning comes from — thinking tokens, extra test-time compute, OpenAI's process-supervision work that rewarded intermediate math steps. Socher points first at the three modes as already variable compute, then at code.

The biggest future impact on reasoning, he thinks, is models that program, then execute. That is subsystem five, and it is a security problem the moment you turn it on: people ask it to mine crypto until the machine is gone, or to climb five directories and dump password files. Isolation has to come first. Naysayers are partly right that language models fail at math. Ask one, in natural language, to multiply 5,600.3 by 365 and it will pretend to do billions of inner multiplications and still get the wrong answer. Teach that same model to notice a math question, write Python, run it, and read the output, and it works. Some people say that is cheating. Socher says it is "the new way of reasoning," a different kind of intelligence that does not have to look like a human doing long division in her head.

He makes the same move on embodiment. Wanting robots that clean the apartment is useful; treating a body as a necessary ingredient of intelligence is a failure of imagination. Blind and deaf people are intelligent; missing a sense is not a disqualifier. Some aspects of intelligence are modal — explaining a sunset, hearing a sonata — but most are not. An AI can retrieve facts, write programs, orchestrate them, and produce answers without baking the entire internet into weights. Nathan agrees that because the systems speak English, people assume they are more like us under the hood than they are, and cites Eliezer Yudkowsky's writing on the space of possible minds as useful prep even if a FOOM scenario looks unlikely.

Socher loves science fiction — he mostly reads nonfiction, but enjoyed *The Three-Body Problem* and wants the adaptation to do it justice — and thinks sci-fi authors pretending to be AI-safety researchers have captured too much of the narrative. The Terminator, a movie he liked as a kid, is "actively hurting" the European Union in particular. The US is more a litigation society, Europe more a legislation society; both logics are reasonable, but sci-fi scenarios have gotten so much weight in law that Europe is trying to outlaw or over-regulate models above a parameter cutoff. GPT-2 was hyped as too dangerous to release. More powerful models are out. He has not seen the apocalypse, or even a huge LLM-driven jump in web disinformation.

No one, he says, is working on conscious AI that sets its own goals and objective functions, because that does not make anyone money. Imagine spending billions of dollars on a system that understands itself, then asking it to help you make money, and it would rather watch the sunset. Nobody pays for that. With so little research progress, it is hard to predict when such a system would appear. Nathan is more libertarian than not, wants robotaxis sooner, and still holds "radical uncertainty": Geoffrey Hinton's point that humans run the Earth because we plan, reason, and use tools, and AIs now do all three with a steeper improvement curve. Asked for P(doom), Nathan says 5% to 95% over a hundred years.

Socher's first correction is mathematical: P(doom) sounds like a prior; it should be a posterior given data, and current data does not show existential doom with humans as pets. He wants to coin a split between superhuman abilities and super-humanity. AI is already superhuman at translating a hundred languages, at predicting the next amino acid in protein sequences — work his Salesforce Research group published in 2018 that companies are running with — and at weather. Those used to be the pinnacle of AI; once solved they stopped being called AI. Language may be the exception that keeps the name. Predicting the next token, with enough capacity, is enough to learn that driving north from New York is more likely to yield Boston than Paris. Nothing in that objective, he says, implies the model should wipe out humanity so every future sequence is the letter A and perplexity goes to zero. "It's so absurd."

He engaged doom-worried interlocutors, including a conversation with Nick Bostrom published in a German newspaper, and found the scenarios fantastical: grey goo, a perfect virus that waits a year and then kills everyone, science that is not there. He is writing a book on AI for science. There are no breakthroughs in machines with a sense of self. The idea that the most intelligent entity always rules is not how politics works. He is "very, very optimistic," with real present problems: models pick up racism, sexism, and other biases the web is not proud of.

On Yoshua Bengio's three threat vectors — intentional misuse, accidental misuse, and loss of control — Socher puts intentional misuse first and calls it real. Open source, he thinks, helps people understand those vectors and compete with them. People should already distrust the internet; they should have distrusted photos since Photoshop, and they will have to extend that skepticism to video and voice. Cultures will answer the amplification of old harms differently. Holocaust denial is illegal in Germany and not in the United States. He does not see a meaningful probability of a full existential scenario. It is mostly people using more powerful tools against other people.

Nathan applauds the scarce resource of a positive future vision — *The Jetsons* is still near state of the art for 2030 — and shares a GPT-4 red-team anecdote: a hard-SF story in the style of *The Three-Body Problem* about text-to-protein models that climaxed in a prompting war over molecules. Absurd, he says, but maybe not entirely. He would order the risks the same way Socher does, with chaos-GPT-style actors and even grant language about "omnicidal tendencies" at the top of the stack. His remaining objection to P(doom) of zero is the integral: each sci-fi path can have vanishing probability while the space of paths is huge, and a function going to zero does not force the integral to zero.

Socher grants a tiny number — ten to the minus something — because over five billion years anything can happen, including a *Three-Body* alien species arriving with AI. That is not, for him, a reason to spend billions of dollars and public mindshare on it. Pessimist Archive on Twitter is his exhibit: people thought novels, newspapers, the internet, and Wi-Fi were the end of civilization. Designing countermeasures now is like writing a defense of vacuum-tube computers against machines going crazy, or like ARPANET TCP/IP experts in the 1970s predicting foreign interference in elections via social media. None of them did. Spend a couple of researchers on cool scenarios; do not scare lay people away from a technology that, even without consciousness, will be a bigger productivity step-function than agriculture, steam, electricity, or the internet. Jobs will change a lot. New ones will appear. It will not happen overnight: companies are still not on the cloud, and stretches of the US and Germany still lack full internet.

Proteins are his example of regulation that does make sense, and of why he is excited anyway. Everything in disease runs through proteins. A paper he read months earlier put iron molecules on one end of carbon nanotubes and a protein that binds only to brain-cancer cells on the other, injected the tubes into a mouse with a brain tumor, then used a magnetic field to spin the iron and perform nano-surgery on each cancer cell. The US had recently outlawed gain-of-function work that makes viruses more deadly — not because the scientists were cartoon villains, but because the tradeoff was judged not worth it for now. He does not think an open-source protein model is the deciding factor in building a new virus. If you already have the wet lab, you can do what Frances Arnold did to win the Nobel Prize in Chemistry: directed evolution, random permutation plus an experimental pipeline, iterated.

Emergent capabilities still wow people inside You.com: users ask if the headphones table was specially programmed; it was not, just next-token prediction putting pieces together. Actions are clearly on the roadmap. The most amazing answer, taken all the way, is "I did what you asked" instead of telling you how. Cool demos of that are easy. Killer use cases are not. Booking a flight in chat is hard: the system books the time you named and looks dumb for missing the option that adds a half-hour layover and an extra leg and saves $50. Expedia spent decades on a visual interface that puts every tradeoff on one screen. A human assistant who has worked with you for months knows your price sensitivity, when you will sit a red-eye, and still sometimes comes back with three options. There is an uncanny valley between the demo and that person, and pure text does not cross it. That is why You.com still ships a stock-ticker app and why links remain: language plus a designed UI is often the right combination.

On AGI timelines, Socher says you have to watch the terminology. The old definition, including his own, was superintelligence plus consciousness plus self-set goals, more capable than all humans. Marketing pulled "AGI" down toward shipping product and parked "superintelligence" as the long-term label — the way a multiplanetary company still has to put satellites in orbit. A pragmatic investor-and-finance definition he does not find crazy is that 80% of jobs can be automated up to 80%. Lots of work is repetitive and does not require novel science. He would restrict it further: digitized jobs in a browser or on a computer, where you can collect training data at scale. "No one's collecting training data for plumbers, for roofers, for tilers, for maids." Those jobs are not getting automated anytime soon.

Even radiology, he suspects, could catch 80% of what is wrong in a head CT; the remaining 20% is a long tail a radiologist may never see in a lifetime, only in a textbook, and one-shot learning is not good enough yet for that. Self-driving makes the liability philosophy concrete. In a utilitarian thought experiment, if 20 million miles of AI driving killed 10,000 people and humans killed five times more on the same mileage, you would deploy. If the one death in the AI car was your daughter, you sue and try to end the company. The silver lining is that the model can learn from the crash; a texting human is already illegal and still does it. Repetitive-job automation in the next few years he finds plausible. Full conscious superintelligence that sets its own goals he will not forecast, because nobody is making progress on it, and he would not call a system superintelligent if its only objective is still cross-entropy or perplexity.

In a short lightning round on retrieval, memory, and online learning, he is a fan of all three. Retrieval is RAG over the web and, now, over uploaded files. Short-term memory still lives in the prompt. Personalization is their pragmatic online learning: the system remembers things you say about yourself, is transparent, and can be turned off. What he wants next is models that adapt immediately to thumbs-up, thumbs-down, and a spoken "I didn't like that answer," then update future answers in a principled way. He would love a second episode. For now the pitch stays two words: amazing answers.