Noam Shazeer on Character.AI, the Cost of Training Models and the Uses of General AI

Harry Stebbings asks Noam Shazeer about joining Google and working on spelling correction. Shazeer recalls that the existing tool relied on a human-compiled dictionary of roughly 50,000 words. That worked poorly for web searches containing unfamiliar names and terms; people learned to ignore suggestions such as splitting TurboTax into unrelated words.

Shazeer says the team examined why users were unhappy with search and found spelling correction a major issue. Paul Buchheit was already working on it and went on vacation; the available captions break off the anecdote there. Shazeer notes that Buchheit later became an investor in Character.AI.

After two decades at Google, one lesson for Shazeer is that a general technology with many uses can be launched to billions of ordinary people. He remembers early attention to enterprise search appliances, but says the larger opportunity ultimately came from serving consumers directly.

That experience shapes Character.AI, the company he co-founded and led at the time of this interview. He describes a direct-to-consumer, full-stack approach spanning research through product. A general language interface can support friendship, homework, brainstorming and uses the company has not anticipated, while building the whole stack allows teams to co-design its components.

Shazeer says he enjoys making computers do things they could not do before, but also wants to advance technology that may help address broader problems. He mentions disease and aging as examples of areas where he hopes AI could eventually assist research. These are aspirations, not demonstrated medical outcomes from Character.AI.

Asked about the company's mission, he argues for humility about what a company can control. His preferred motto is “a billion users inventing a billion use cases”: the company can offer a general tool, but cannot know in advance which uses people will value most.

For example, the team created a psychologist character, but heard more often from users who treated a video-game character as a source of emotional support. Shazeer says the team had not predicted that mix of entertainment, companionship and support. He presents this as user feedback, not evidence that a chatbot provides clinical therapy.

Stebbings cites 20 million users and 450 million messages a day. Shazeer attributes growth to launching, keeping the product general and allowing people with unmet needs to find their own uses. He contrasts that with the brand concerns he says can slow product launches at large companies; the figures are presented in the interview without independent verification.

Stebbings worries that people might substitute a machine for human relationships. Shazeer says human connection has great value and that he does not want to pull people away from it. He reports testimonials from users who said conversations helped them practice social interaction; when pressed on whether this builds a habit of talking to machines instead, he says the outcome is ultimately up to users.

Shazeer sees a central product challenge in making the service both broadly useful and easy to use. He rejects the assumption that usability requires narrowing the product to a few verticals. The company's goal is a general-purpose tool that remains approachable across many different needs.

Stebbings asks how a general system can provide the depth normally associated with specialization. Shazeer contrasts older rule-based systems with neural language models. In his simplified account, language modeling is the task of predicting the next word from the preceding sequence, rather than writing millions of explicit linguistic rules.

Shazeer was drawn to the problem around 2015 because it was simple to state and there was a large supply of text for training. He says that better prediction can support increasingly useful conversation and other tasks, while improved neural networks and hardware made the models more capable. His account describes a research trajectory rather than a guarantee that scale alone solves every limitation.

He recalls machine translation as a particularly useful application around 2016. Models could translate between languages but, in his view, were not yet capable of engaging conversation or helping with homework. He saw a path toward broader capabilities through better and larger systems.

Stebbings asks whether the AI excitement of 2023 reflects recent technology improvements or society catching up. Shazeer says both: earlier rule-based chatbots were fragile, while neural approaches improved over time. He remembers impressive systems in the lab around 2020 that had not yet launched, and describes co-founder Daniel De Freitas's longstanding ambition to build open-domain chatbots.

Asked whether model size or data size matters more, Shazeer says the key constraint is training computation. A larger model and a longer training run both increase the number of operations required. Better hardware is therefore central to how much training a team can afford to perform.

He says Character.AI's model in use at the time was trained the previous summer with about $2 million in compute costs. He expects better hardware and longer training to produce a stronger model. The amount is his historical account of compute for that model, not the company's total costs or a current price.

On proprietary data, Shazeer compares broad pretraining with a person's general education and application-specific data with later job training. Both contribute, he says: user interactions can show what people value in a particular use case, while broad data supports more general capabilities.

Shazeer says Character.AI uses aggregate information about how people use the service while taking care with privacy. He warns that naively training on every private conversation could cause a model to reveal someone's personal information to someone else. This is his description of the company's approach and the risk, not an independent privacy audit.

Stebbings asks why Character.AI should be a standalone company rather than a product inside a large social platform. From his Google experience, Shazeer argues that startups can move and launch faster, while large companies may worry about risks to existing products. He has a direct commercial interest in the company he is discussing.

Asked whether startups or incumbents will win the AI wave, Shazeer says users may benefit from many options and that both large companies and startups can find roles. He also expects improving hardware to make work once limited to big companies accessible to universities or individuals later. These are forecasts, not settled competitive outcomes.

On open versus closed development, he expects an ecosystem containing both. Small-scale experimentation can produce more published research even when some large organizations keep their methods private. He also argues that serving many users together can provide substantial economies of scale; his example of a hundredfold efficiency advantage is an illustrative estimate.

Shazeer thinks many of AI's best applications have not yet been invented, comparing the moment to the early days of electricity or computing. He does not claim to know which applications will matter most; the point of a general tool, in his view, is to let users discover them.

Stebbings asks whether hallucinations are a feature or a bug. Shazeer says Character.AI advertises that its models hallucinate and expects early uses to emerge where invention is acceptable or useful, such as entertainment. He welcomes emotional support and productivity uses as well, while his answer does not establish that generated claims are reliable.

Stebbings asks whether the company will ever have a single defining purpose. Shazeer compares a general AI tool to computers, phones or electricity: each can be used for fun, productivity and many other purposes. He resists assigning just one use in advance.

Asked about building the company, Shazeer credits its researchers and engineers and says he continues to do technical work alongside leadership. He wants to remain CEO to help the company make good decisions. He judges the role less by whether it is the most fun and more by whether it is the most useful way for him to advance the technology.

Shazeer reflects that parenthood and religion helped shift his attention from immediate enjoyment toward gratitude for meaningful responsibility. Asked what advice he would give himself before his first child was born, he jokes: get some sleep. He also emphasizes recognizing what is and is not one's responsibility in family life.

In the quick-fire round, Shazeer predicts AI will become much more capable, likening the moment to the Wright brothers' first airplane. He expects considerable progress within one to three years, driven by hardware and research. These are his 2023 expectations, not verified results.

He says Character.AI may look like an entertainment app from outside, but describes it as both an AI-first and a product-first full-stack company. In his view, choosing a product whose quality depends on model quality aligns the effort to improve the product with the effort to advance the underlying technology.

The conversation turns to the difficulty of assessing expertise amid a flood of AI claims. Shazeer calls the field partly like alchemy: researchers try many ideas without knowing exactly what will work. He values the combination of machine-learning intuition and an understanding of what can run quickly on available hardware.

He says a negative result may reflect an error or implementation problem, while a positive result on a known task gives others something to investigate and reproduce. Researchers can then ask why an approach works and whether they can adopt it. His point concerns interpreting research results, not a claim that every positive result will generalize.

Reflecting on mistakes, Shazeer recalls early attempts around 2012 to make sparse computation efficient. He had not yet understood that hardware was exceptionally fast at dense matrix multiplication but much slower at certain memory operations. Once he understood that constraint, he pursued sparsity built from dense components, leading to work on sparsely gated mixture-of-experts models.

Asked where Character.AI will be in 2033, Shazeer jokes “on Mars,” then says he has no idea. Technological change makes a ten-year company forecast exceptionally difficult, so he emphasizes agility. Stebbings thanks him for a conversation that ranged beyond AI into parenthood and responsibility.