Chips, Agents and a Reckoning: Sarah Guo and Elad Gil on the 2023 AI Boom

Answering a listener question about the GPU crunch, the hosts start with the structure of the industry. A very small number of producers, mostly Nvidia and AMD, make the chips, and Nvidia is far ahead on the high-end processors best suited to large-scale training and inference. Supply is still disrupted from the pandemic and depends on a few major foundries such as TSMC.

Expanding capacity is hard. New fabs cost billions, and yield is a precision-manufacturing problem where temperature, pressure, chemical concentration and tool imperfections can all lower output. The host compares the situation to half the world's companies deciding within a year that they need supercomputers made of gigantic networked GPUs, a pace the physical process cannot match.

Asked how big the gap is, the host says there is no way to test price elasticity yet. Deliveries arrive in small quantities in September and larger ones in December and January, and most large cloud providers are sold out through April of the following year. Even Microsoft is buying near-term supply from other providers. Elad Gil adds that a specialized bonder used in GPU assembly is itself a constraint, and that labs keep scaling models and training time by orders of magnitude, so he sees little reason for that demand to fade.

Gil sees second-order effects. First, the bottleneck is creating new business models around GPU access, from companies like Weave and Foundry ML that aggregate or federate GPU sources to operators running large GPU clouds. Because crypto is down, GPUs once used for mining may now earn more by being rented out for AI training and inference.

Second, startups that build their own AI-specific semiconductors are seeing strong pull. Gil points to Cerebras, which he says just signed a $100 million deal with the UAE to build nine supercomputers, and says Groq and others will benefit while buyers are desperate and willing to do extra work to use other silicon.

Guo adds that blocked scaling pushes people toward efficiency, an area that has gotten little focus because most have followed Chinchilla scaling as the simplest path. She names undervalued lines of research: dynamically routing to efficient models, as in FrugalGPT, distillation, and smarter data choices for pre-training and fine-tuning mixes. She does not expect the crunch to end soon, since inference already dominates OpenAI's compute usage and that demand keeps rising.

Gil agrees demand will rocket in the short run and says much depends on how fast Nvidia, and perhaps AMD, can scale manufacturing. Assuming AI growth continues, he expects bottlenecks to persist.

Gil is writing a blog post on the idea that we are very early in this wave of adoption. In his telling, transformers brought a new capability set rather than a gradual improvement, and ChatGPT arrived only about eight months earlier, with GPT-4 around five months before. The first adopters were AI-native companies like OpenAI and Midjourney, followed by startups such as Perplexity, Harvey and Character and by founder-driven incumbents like Notion and Zapier.

He says real enterprise adoption in products at scale is close to zero. Big companies take six to nine months to plan and about a year to prototype before launching, so he expects large-scale AI applications from incumbents in one to two years. That leaves room for more hype, sometimes irrational, and for more adoption of semiconductors and other infrastructure.

Guo agrees that we are early in the collective exploration of applications and constraints. She views ChatGPT as the starting gun, but shipping real products, learning how to make models useful, turning them into workflows and then improving them takes time. Her summary is that we are in inning one, and that so far it has all been demos.

Guo describes an agent as more than a chatbot: a model-driven planning mechanism that takes actions autonomously, often with other tools, and reports back. She expects both broad and specialized agents, from consumer products such as Inflection and Minion's web agents to enterprise automation. She also sees growing attention to agents that write executable code so they can call APIs and use many more tools. Her enterprise example is Harvey, where one legal task is a composition of planning, research and writing.

Gil looks to past technology waves, where most things that work start as vertical applications that do one job well and sometimes broaden into platforms. Facebook began with five colleges, then added all colleges, work emails and eventually everyone, while Google was broad from day one. He is wary of founders who pitch an assistant that does "everything" and will "make you happy." His advice is to pick one or two things such as meeting research or scheduling, because a focused use case tells you who you are building for. He also recalls the old Y Combinator line that it is better to delight a few people than leave many indifferent.

Guo sees a classic tension between a research-and-infrastructure mindset, which says the technology is general and should not be narrowed, and a product mindset focused on completing tasks. She thinks exploration has skewed too far toward the first. Her example is an agent that fixes bugs, which can use bug-specific techniques such as test-time search to check whether generated code executes, and she says such narrow research goals are far more tractable.

Gil adds a third approach, infrastructure tooling, where you build the infrastructure that lets others build agents quickly. He says it works in two ways: as an outgrowth of a vertical product that then opens up, like Facebook's auth platform after hundreds of millions of users, or as something everyone keeps rebuilding, like payments for Stripe. Builders need to know which case they are in.

Gil expects four markets next year. AI will keep running and look expensive, though a handful of its companies will look cheap in hindsight, as in every technology wave. For mid-to-late-stage private tech companies outside AI that raised in 2021, he predicts a three-way split among unicorns: a third will go under or be fire-sold, a third will hit the highest valuation they will ever have, and a third will grow past it. He points to companies from 2014 that went public later and never exceeded their market cap again.

He expects a lot of carnage, with three consequences. Hiring gets easier, a second shoe drops for commercial real estate, and venture funds feel the hit as big unicorn successes become worth little. He thinks the venture effects will take two to three years to play out, and says the other shoe has not dropped in private tech markets because companies still have money raised in 2021, so 2024 and 2025 could be bumpy.

On advice, one host urges companies that raised at healthy valuations in that period to disassociate from them, since people contort themselves to do flat or up rounds that make no sense. The host cites an analysis of the strongest dot-com survivors, which took about a decade to regain their pre-bubble valuations, time startups do not have.

The other host is less worried about valuation, since public tech stocks lost 30 to 90% of their value and were repriced daily. The bigger danger is burning $50 or $100 million to generate $5 or $10 million in revenue, which is a business-model issue. Gil closes by saying a founder's time is the most precious asset, that early years are the best moment to take risk, and that you should recognize when a business is not working.