Stop Chasing AI News: Chip Huyen on What Actually Makes AI Products Better

Lenny Rachitsky opens with a table Huyen posted on LinkedIn that went viral. The left column lists what people believe will improve AI apps: following the latest news, adopting the newest agent framework, agonizing over vector databases, constantly comparing which model is smarter, and fine-tuning. The right column lists what actually helps: talking to users, building more reliable platforms, preparing better data, optimizing end-to-end workflows and writing better prompts.

Huyen says one of the questions she hears most is how to keep up with AI news, and her answer is to ask why you need to. When people ask her to choose between technologies such as MCP and an agent-to-agent protocol, she asks how much improvement the optimal choice would really bring. Often the honest answer is "not much", in which case the debate is a waste of time.

She adds a second test: how hard would it be to switch later? A new, little-tested technology may lock a team in, so she advises thinking twice before over-committing to something that has not been battle-tested.

Huyen describes language modeling as encoding statistical information about language. After "my favorite color is", the word "blue" is far more likely than an arbitrary word, and the model learns such distributions across many domains. She notes the idea is old, going back to Claude Shannon's 1951 paper on the entropy of English, and compares it to Sherlock Holmes decoding stick-figure messages by assuming the most common figure is the letter E.

Tokens sit between characters and words, which keeps vocabulary manageable while letting the model handle new words like "podcasting" as "podcast" plus "ing". Because the model outputs a distribution rather than a single answer, the sampling strategy, whether to take the most likely token or something more creative, can change performance a lot. She calls it hugely underrated. Weights, she says, are simply the function parameters adjusted until the model fits the training data.

Most people never touch a pre-trained model directly, and she says raw pre-trained models can behave horrendously. Pre-training raises general capability but needs huge amounts of data, and text from the internet is close to maxed out. Since labs have similar pre-training data, she argues post-training is where they now make the difference.

One form of post-training is supervised fine-tuning on demonstration data, where experts write ideal answers. Open-source models often do this by distillation, having a strong model generate the answers. Huyen respects the open-source community but stresses that emulating an existing good model is a very different achievement from training one from scratch.

Huyen explains reinforcement learning as encouraging a model to produce better outputs based on some signal. One source of signal is human feedback, usually comparisons, because people find it much easier to say which of two songs they prefer than to give one a score out of ten. Those comparisons train a reward model that scores outputs and pushes the main model toward better responses.

Other signals come from AI judges or from verifiable rewards, such as a math problem with a known answer of 42. The common thread is different ways of collecting signals. Lenny links this to RLHF and to the AI-driven variant Anthropic uses.

Model developers want models to do well on expert tasks in accounting, physics, law or engineering, and on specific tools such as QuickBooks or spreadsheets. That requires many human experts to write questions and expected answers, which feeds the data-labeling industry. Huyen finds the economics curious: a few frontier labs are the buyers, many startups supply data, and labs benefit from playing providers against each other. She says she is more curious than bearish about how it plays out.

Huyen separates two kinds of eval work. App builders need to know whether their chatbot is good or bad, while model developers design task-specific evals, such as deciding how to measure good code writing. She finds eval design creative and fun, with nothing dry about it.

On the debate over whether AI products need evals at all, she says you do not have to be perfect to win, only good enough and consistent. An executive looking at a feature that already works may ask what two engineers on evals would add, and hear "maybe 80% to 85%". That engineering time may be better spent on a new feature. She acknowledges the risk of having no clear metric, so evals become essential at scale, where failures can be catastrophic, or when the feature is a competitive advantage.

Asked how many evals a product needs, she rejects a fixed number. The goal is to guide development by revealing where the product performs poorly, for example a user segment where messaging fell flat. She has seen products with hundreds of metrics, and says what matters is enough coverage to be confident in performance and to see where it is failing.

Her deep-research example shows why. Such a system issues search queries, gathers and aggregates results, then writes a summary, so every step needs its own evaluation. Are the queries diverse or near-duplicates? Do the results overlap, and do they offer breadth, depth and relevance? Evaluating only the end result would miss these problems.

Huyen defines retrieval-augmented generation as giving the model relevant context so it can answer a question. Early work showed that retrieving passages from Wikipedia improved question-answering benchmarks, which sounds like a no-brainer in hindsight. Real documents are messier than Wikipedia pages, though, and she gives an example: a document that says "from now on, the podcast" without ever naming Lenny's Podcast may never be retrieved for a query about Lenny.

Fixes include adding context, summaries or metadata to each chunk, or generating hypothetical questions each chunk could answer, so a query can match those questions. Huyen says that in most companies she has seen, the biggest RAG improvement came from better data preparation rather than agonizing over which vector database to use. Databases matter for latency and access patterns, she says, but not much for answer quality.

She also covers chunk size. Longer chunks carry more relevant context but eat your retrieval budget, while short chunks cover more documents but each holds little information. One team saw a large gain from rewriting content into question-and-answer format, and another added an annotation layer for AI to documentation that was written for humans, for instance explaining what a returned value of 1 means on a scale.

Huyen sees two broad categories of company AI tools. Internal ones include coding tools, Slack chatbots and knowledge assistants that answer questions on policies and benefits. Customer or partner-facing ones include support and booking chatbots. She suspects the latter are popular because their outcomes, such as conversion rate against human operators, are easy to measure.

Internal tools are trickier. She says AI strategy rests on use cases and talent, and many companies spend heavily on subscriptions and upskilling workshops to build AI literacy, then see little usage and no clear gain. The core problem, she says, is that productivity is hard to measure: counting pull requests or lines of code is a poor proxy.

She tells a telling anecdote. Asked whether they would prefer expensive coding-agent subscriptions for the team or an extra headcount, managers almost always choose the headcount, while VP-level leaders choose the tools. Managers are still growing and a headcount is a big deal for them, whereas executives care about business metrics. She says the question is less whether AI makes people productive than whether we can measure the improvement.

Reports on who benefits also conflict. A friend's company of about 30 to 40 engineers ran a randomized trial, giving half of each performance bucket access to Cursor, and found the top performers gained most, then the average ones. Another company told her its senior engineers were the most resistant, because they are opinionated and think AI code is poor. She says she has not reconciled these accounts.

Lenny argues senior engineers who understand systems at scale are now very valuable, since AI tools act like endless junior engineers. Huyen says companies she works with are restructuring accordingly: senior engineers write guidelines and review code, while juniors or AI produce pull requests. The open question, she notes, is how anyone becomes a strong senior engineer in that world, and Lenny suggests AI-native juniors could learn fast if they use AI to learn rather than just delegate.

Huyen cites Stanford's Mehran Sahami, who said computer science is not about coding but about system thinking, and that coding is a means to an end. As AI automates more, problems simply get bigger, and understanding what caused an issue and designing a step-by-step solution will not go away. Lenny connects this to a similar point Bret Taylor made on the podcast.

She illustrates the limits of AI with a deployment bug on an unfamiliar hosting service. The assistant kept changing environment variables, code and language, but the real cause, which she found by reading the docs, was that the feature was not available on her pricing tier. AI kept trying to fix the wrong component, which makes her wonder how to teach AI, and humans, to think about how components interact.

On her book, she describes the difference between ML engineers, who build models, and AI engineers, who build products on existing models. No definition is perfect, she says, but model-as-a-service has lowered the entry barrier while widening what AI can be used for.

Huyen expects organizational change. Evals are a system problem that needs engineering and user understanding, so product, engineering and even marketing teams are moving closer together. She also sees companies already automating functions that were previously outsourced, and rethinking how to value junior and senior engineers.

She leans toward the view that base models will not keep making the huge jumps seen from GPT to GPT-3, and that GPT-5 was a debatable step. Therefore she expects more improvement in post-training and in the application layer. Lenny counters with the Anthropic co-founder's point that exponentials are hard to feel from inside and releases now come more often.

She is also excited about multimodality, especially voice, which she calls an entirely different beast. A voice chatbot adds latency from speech-to-text, text generation and text-to-speech hops, and it must decide when to stop talking if the user interrupts, versus just saying "mm-hmm". Interruption detection is a classical machine-learning problem and latency is largely an engineering challenge, while voice-to-voice models are promising but hard. Regulation may also require bots to disclose they are not human.

Huyen then explains test-time compute. Beyond compute spent on pre-training and fine-tuning, a model can spend more on inference, such as generating several answers and taking the majority, or producing more thinking tokens before answering. That improves perceived performance even though the base model does not change.

Huyen has helped companies run hackathons and internal challenges, and says many people simply do not know what to build. She calls it an idea crisis: tools can now design, code and ship a website, yet people are stuck. She suspects society's push toward specialization makes big-picture ideas hard.

Her tip is to spend a week noticing what frustrates you at work, ask whether it could be done differently, and build something around it. Lenny shares a vibe-coded app he made to download all images from a Google Doc, and Huyen says she is bullish on AI-built micro tools that make life a little easier.

In the lightning round, Huyen recommends The Selfish Gene and Lee Kuan Yew's From Third World to First, which she values as a study of system thinking at the scale of a country. She is writing a first novel, a drama that she recently sold, and watches TV such as the Chinese series Yanxi Palace to understand what stories people like.

She says content creation is about predicting audience reactions, and fiction exposed her to a very different audience from engineers. An editor taught her to think about the emotional journey, with a hook, moments of intensity and calmer stretches so readers are not exhausted. Technical writing is impersonal, but readers of fiction care about whether characters are likeable.

Her first draft had a very rational protagonist, and a friend admitted he hated her. In the second version she added vulnerability and setbacks so readers could relate. She says she is starting a Substack on system thinking and perhaps a YouTube channel reviewing books that help you think better.