Richard Socher on Language Models, Search and the Limits of AGI Predictions

In this August 2023 interview, You.com founder and CEO Richard Socher tells Harry Stebbings that he began studying computational linguistics in 2003, moved into computer vision and later explored applying neural networks to natural language. He describes his own research trajectory; claims of priority for specific inventions are his account.

As Salesforce chief scientist, he says he learned about marketing, management and the value of research in a large company. He credits Marc Benioff with listening carefully and making consequential decisions. A startup, he adds, has less room for research that cannot lead to a product relatively soon.

Socher sees major progress in AI alongside inflated expectations. A chatbot is not necessarily the best interface for every task: a map can be better for comparing restaurants, and a live ticker can be better for a stock price. He cites You.com examples, and has a direct commercial interest in the product.

Increased funding brings talent and experimentation, he says, even if some spending is inefficient. He recalls early deep-learning workshops with a small community and the resistance to neural networks when hand-designed features dominated many research workflows.

Socher’s long-running goal was a single model able to perform different language tasks instead of training separate models for sentiment, translation and question answering. He says his research explored expressing tasks as questions and that this influenced his decision to build a search product.

He recalls rejection of a paper proposing a unified approach. His analogy is that people use one brain for multiple tasks, though that analogy does not prove a particular model architecture works. He says large models, data, attention mechanisms and hardware helped make a more general approach feasible.

Model size and data size are complementary, he argues: a simple model cannot represent arbitrary complexity merely by receiving more data, while a large model needs training data. He uses linear regression as an illustration of limited model capacity.

He describes next-token prediction as a way to train on large quantities of text. To predict well, a model can learn statistical relationships that encode useful information about the world. His illustration is a way to understand training, not proof that a model reliably reasons or possesses complete factual knowledge.

On data advantages, Socher says both public and proprietary data matter. A firm with labeled customer-service records may more easily adapt a system to its own workflow, while foundation models let a startup build an initial general-language solution more quickly than before.

He rejects treating every AI application as a trivial wrapper. Some may offer little differentiation, but others must build distribution, partnerships, sales processes or retrieval systems. He says You.com’s search backend is complex; that is a claim from the company’s CEO.

For factual and current answers, he favors retrieving relevant information and using it alongside a language model. Retrieval can provide sources and context, but the conversation does not establish that it eliminates errors. He contrasts this with relying solely on a model’s imperfect remembered information.

Asked whether hallucination is a feature, Socher distinguishes creative fiction from factual search. Generating an imagined story can be useful when requested, while inventing facts in response to a factual question is a problem. The system must still infer the user’s intent, and its output needs evaluation.

He expects models to be updated frequently, but thinks some general architectures could persist. On whether switching models is a decisive advantage, he emphasizes how an application tunes, prompts and combines models and retrieves information. He suggests specialized forecasting may need a different model from a language model.

In 2023, Socher predicted that an open model comparable to the then-current GPT-4 would appear before year-end, while acknowledging the closed model could also improve. This was a time-specific forecast, not a finding that the prediction came true. He expected open models to serve many use cases.

He recognizes that open development faces coordination and funding problems. His aspiration is a collaboratively improved model analogous to Wikipedia, with structures for deciding what information can be added. That is a vision; open access alone would not solve governance or accuracy.

Socher believes governments can help fund open science, while acknowledging the political and bureaucratic difficulty of supporting uncertain research. He sees search as a high-impact language application because people use it to learn, navigate and form opinions every day.

He describes a search interface that can answer questions, generate code or walk through a problem rather than only present links. Those are examples of what he wants You.com to offer; generated answers and code still need checking, especially where factual accuracy matters.

Stebbings asks how publishers survive if search answers replace visits to their pages. Socher proposes an open platform where content or application providers can participate in revenue or subscriptions. He admits adoption was slow at the time because You.com did not yet have a very large audience.

Distribution remains difficult for a startup, Socher says. He reports that changes to Google’s ranking reduced traffic to You.com sites and says the company was seeking partnerships. The interview provides his account but does not establish why the ranking changed.

He expects a mix of incumbents and startups to create value. He praises Salesforce and says Bing and Google introduced features similar to You.com’s earlier search chat, while acknowledging they may have worked on related technology independently. These are competitive observations from a direct participant.

Socher argues that a search incumbent’s advertising revenue can make a rapid shift to a chat-first interface difficult. He cites a daily revenue estimate and criticizes the ad experience; neither the estimate nor any alleged ranking choices are independently established here. He presents this as a possible opening for You.com, not a guaranteed advantage.

He praises Salesforce’s AI work and says competition has accelerated changes in search. As a former Salesforce executive and current search-company CEO, he has personal and commercial connections relevant to his assessments. He declines to name a large company he considers a laggard.

On AGI, Socher cautions against extrapolating recent progress indefinitely. He uses historical examples of rapid improvements followed by constraints to argue that progress may not follow a constant exponential path. These analogies illustrate his uncertainty rather than establish an AGI timeline.

He suggests image generation was approaching a practical ceiling for photorealism at the time, while video remained an area for progress. That was his 2023 opinion and is not a demonstrated technological limit. He also argues that human language constrains how people perceive “superhuman” language ability.

Stebbings observes that technical progress and public awareness can move at different speeds. Socher expects AI to affect more digital and data-rich tasks sooner than highly variable physical work. He predicts some physical services could become relatively more expensive; the interview supplies no labor-market forecast evidence.

Both discuss job displacement. Socher expresses empathy for workers facing rapid change and favors social systems that help people acquire skills and use new tools. His belief that new work will emerge is an opinion, not a guarantee that displaced workers will find equivalent employment.

He says further research breakthroughs may be needed for AGI, but their timing is unknown. He personally questions whether a system that only predicts tokens should count as generally intelligent and argues that independent goals may matter. These are contested conceptual views, not agreed criteria or proof that no researchers study agency.

In the quick-fire round, he emphasizes retrieval, wants more discussion of near-term AI risks and says he did not sign a proposed pause on model training. His position reflects his assessment of risks and benefits rather than a scientific consensus.

Looking toward 2033, Socher hopes You.com becomes a default for hundreds of millions or billions of people seeking answers. That is a founder’s aspiration, not an achieved user count or guaranteed outcome. The conversation closes with mutual thanks.