The Hyperscalers Are Spending $200 Billion — and the Next Bottleneck Is a Power Plant

After a round of hat jokes, Sarah Guo and Elad Gil turn to Suno and Udio, two music models that have taken off. You specify a style, write lyrics, and the system adds vocals. Gil treats it as early but as a glimpse of what content creation looks like next. He lays the formats in sequence: GPT-3-era text tools such as Jasper, then the image wave of Midjourney and Stable Diffusion, then ChatGPT as a new interaction modality, then video with tools such as Pika, and now music. Drake has already put out a song that voice-cloned two or three other rappers. Assuming permissions, Gil says, using anybody's voice could become ordinary.

Guo is less sure the market explodes. On media platforms the ratio varies, but readers and feed-scrollers vastly outnumber creators. Make it much easier to produce something that is any good, and it is still unknown how many people actually want to create music, or whether the music we get simply changes. A founder told her everybody should have a personalized soundtrack for their life, in the voice and style of Taylor Swift. Guo says she already has one of those. She will not share it on the podcast.

Apple has entered the chat, Guo says, with relatively small models now on Hugging Face. Mixtral and Llama gave developers impressive open reasoning, but the unmet demand is useful capability at about one and three billion parameters, small enough for phones and laptops. On-device inference changes the latency paradigm for consumer experiences. Not paying cloud compute for every call also makes ongoing, passive, and proactive features affordable. Her prediction: Apple will create interfaces for running models locally as part of its ecosystem.

Gil is already seeing Mac and iPhone apps that index everything on a computer, search it, build embeddings, or drop ChatGPT onto the desktop. In a couple of years, he suspects, that will ship standard from Apple or a partner. Guo is skeptical that independents persist: Android launcher and search-bar experiments rarely stuck when they tried to rewrite a core OS experience. Gil's test is the footprint of content. Stay inside local files and the OS vendor can copy you. Reach through the browser into third-party apps and you might have a reason to exist. Platforms, he adds, tend to integrate the most valuable things into themselves. That is the strategy, and usually the ending.

The odd exception he names is Veeva, a roughly $40 billion vertical SaaS company for pharma. It began as a thin Salesforce app, sold that way for years, then swapped out the backend. Salesforce's market cap, they note, is about $266 billion, so Veeva is on the order of 20 percent of the platform it sat on — and it was not subsumed. Guo's read is compliance: workflows for selling to doctors in life sciences that Salesforce did not have the expertise, or the urgency, to attack. Gil's historical rhyme is Microsoft Office. Lotus became Excel; there was a PowerPoint company; Word replaced a pile of separate document apps. Microsoft folded them into its distribution. The default is that the OS company eventually owns the feature.

How smart can a small model get? Gil splits what people want from an LLM into three or four pieces: reasoning, synthesis, multimodality, and the knowledge actually resident in the weights. You can only stuff so much into a 3B or 7B model running on a device. The live product question is which capabilities can be device-resident and which always go to the cloud. Devices get more capable over time, but the line can be drawn analytically and almost nobody has published that cut. Guo's analogy is the old client-server religious war: there is no principled answer except that it varies by application, and by how fast the needed capability fits in hardware people already own.

AI-specific hardware is already a company category. If a product is meant to feel AI-native, Guo says, you have to decide how much compute sits in the device, how much you preprocess, and how much you send to a large model in the cloud. From a consumer angle, one of the more interesting theories is a passive device that continually collects visual data about the world — which is why Ray-Ban Meta glasses exist, rather than a phone or watch in a pocket. You could also bolt small cameras onto other objects. The form-factor question collapses to whether you need that extra data, and when you would use it. Gil notes that early mobile winners often rode a single new primitive: the accelerometer for fitness, GPS for Uber. Vision will throw off unexpected applications. You still have to say what you will do with it.

Guo has tried Meta AI and was very impressed. Meta entered with multiple products and modalities at once, yet has not pushed the assistant aggressively into WhatsApp and Messenger — surfaces with, as she puts it, unlimited distribution. The standalone meta.ai product looks like restraint. Gil guesses it phases in as a channel or bot on those chats, another account you talk to. Her kids have been making images across Playground, OpenAI, and Meta; they loved the one-click animation. As a launch, she calls it impressive.

On the open-source side, Meta also made a splash. Research has argued over whether to move the efficient frontier of training or just ride the curve and scale. If you are Meta, with something like 22,000-GPU clusters and 350,000 GPUs available, continuing to train Llama 3 past supposedly optimal points still improves performance. It does not asymptote as soon as a lot of the research community predicted. That is a point in favor of firms willing to spend. Efficiency and architectural creativity still matter: really large models are impossible to serve for the vast majority of use cases on cost and speed. If all you want is a technical demonstration, stacking scale works. Meta's GPU-related spend this year, she notes, may be $30 to $35 billion. That table does not have many seats.

Snowflake and Databricks have started launching models, while also hosting other people's, including Mistral, and those models show up again on Microsoft Azure. As an investor, Guo says the map has blended into one landscape of competition among models, inference, and compute. In the immediate term it is less obvious that a data platform needs to own the model, especially with a thick open-source catalog. What it does need is to prove to customers that it can train, fine-tune, deploy, and customize — expertise that is itself a product, plus the marketing that goes with it.

Gil pulls the camera back to capital. Frontier models take more and more compute, which means more money, which is why so many of them have hyperscaler sponsors. Other players have to decide how long they keep chasing bigger, or whether they specialize in medium-to-small models for narrower jobs. In the long run, he argues, inference platforms probably live in that mid-to-small range. A handful of hyperscalers and their partners sit at the frontier, because only their business models can subsidize the training on the bet that ROI arrives later, as intelligence spreads into more applications. The question each company actually has to answer is how far ahead it can invest.

Investors keep asking Guo, stressed, how much money the world should spend on this, and whether it is rational. *Should*, she says, is the wrong verb: it is a business decision made by a very small number of people. What feels unique in AI is that those people believe in strong intelligence, can reason about exponential growth, have a high risk appetite, and can marshal $10 billion-plus — Meta perhaps $30 to $35 billion a year on GPUs. Put it in context. Hyperscalers are spending almost $200 billion this year on AI compute. Oil majors spend about $80 billion a year. Broadband providers have been in the neighborhood of $100 billion a year of infrastructure. Railroad freight has had years of huge capex; so have chip fabs. The scale is not unheard of. Whether it is worth it depends on where you think this goes. Gil adds one more unbounded checkbook: sovereigns, if countries want models fitted to their language, customs, and culture.

One hot research slogan is "unlimited context," which Guo treats mostly as a term for actively managing context. Gil is an investor in Magic, among the first to ship a really long window: 5 million tokens, six or twelve months before this taping. Google's Gemini 1.5 came out at a million. It seems likely, he says, that a lot of players will be in the 10 million-plus range in the next year or two. Longer windows change what you put in a prompt. Entire code repositories, stacks of legal documents, a giant customer-support queue — things that used to be too big — start to fit.

The example that struck him most is not an assistant at all. Biology models have shown that just increasing the context window for protein folding makes a big difference in the fidelity of the result. That is the kind of payoff he did not necessarily expect. The analogy is microprocessors and bandwidth in the 1990s: each generation of fatter pipes, from dial-up to broadband to fiber, unlocked a new set of applications. Context will have a long expansion, then some day a max-out. He jokes that a "long period" in AI usually means two weeks, then corrects himself: this one will be much longer than that.

Guo lists the bottleneck chain: first chips, then packaging, then data centers — and she expects everyone to miscalculate how many of those they need — with energy tied to that last step. Nobody has built a 500-megawatt to gigawatt data center yet. Think of it as a nuclear plant's worth of power going to a single hall. Training still needs GPUs collocated because of how much data moves between chips; physics says you have to deliver that much electricity to one campus. Sam Altman has argued that once the constraint is permitting and physical-world construction, progress slows. It is no longer only a software problem. Guo thinks a lot of this will get worked through. The fear is that you cannot just throw more compute at the problem because throwing more compute is physically hard.

Then there is the data wall: cheap available tokens on the internet have been used, and the next supply is either collected in the real world or generated as synthetic data. Synthetic data, she notes, is still a bits problem, and bits get solved fast. The signal is already in the guest list. The newly announced DHS AI safety board includes the obvious names — Sam Altman, Satya Nadella — and also people who work on energy and infrastructure security. Microsoft is posting nuclear-engineering jobs. It also put $1.5 billion into Abu Dhabi's G42: an AI data center in an energy center, broadening global access by sitting the GPUs next to the power.

Gil zooms out to a century of abundance versus scarcity. One of the scarcity camp's biggest wins, in his telling, was shutting down new nuclear in the United States in the 1970s. Nuclear is still about 17 or 18 percent of US generation, about 70 percent in France, around 30 percent in Japan. It could have been 70 percent in the US. Cheap, abundant power would be a game changer for compute. The problems are solvable if we choose to solve them — which is why that Microsoft job posting interested him. Guo wants that reversed even beyond general energy needs. If AI is a strategic and national-security issue, not using every energy technology already on the shelf is another dependency on other countries we are creating without needing to. Widespread nuclear, Gil adds, would rewrite geopolitics toward Russia, Venezuela, the Gulf, Iran. The hats come back out. The infrastructure joke, this time, is millinery.