The Analyst Tracking 200 GPU Clouds Breaks Nvidia Into a Three-Headed Dragon

Sarah Guo introduces Dylan Patel as the chief analyst at SemiAnalysis, the shop hedge funds, chip companies, and frontier labs actually read on chips and AI infrastructure. The conversation that follows ranges from OpenAI’s open-weight model to whether you can build a data center the size of Manhattan, then geopolitics, then poker as a tell for founders. Patel is listed as founder and CEO in the episode materials; on tape he is the person who will not let a round number go unchallenged.

Guo notices the Samsung watch. Patel’s origin story is stubbornly hardware-nerd. As a kid he moderated forums after his dad’s first Android, the Motorola Droid; on road trips he rooted it, underclocked it, and chased battery life until he was a moderator of r/android and related Nvidia and Intel subreddits. He still carries an Ultra watch and a Fold, not as a statement against iMessage but because he can keep Slack and email open at once and, more to the point, actually use a spreadsheet. When someone quotes a GPU purchase number that is slightly off his forecast, he checks it on the phone.

Asked for a reaction to OpenAI’s rumored open-source model, Patel’s first word is amazing. He expects the world to be shocked: it is the first time America has had the best open-weight model in six, nine, maybe twelve months. Llama 3.1 405B was the last such peak; Mistral held the crown briefly; then Chinese labs—DeepSeek, Moonshot, Alibaba—dominated. The new model probably will not win generic chat. It is more reasoning-focused, and he is excited about code and tool use, even if tool use will be confusing without OpenAI’s closed tool stack.

The rollout is the other shock. The weights leaked, but the open-source community still could not run inference because of oddities in the architecture. Most labs dump weights and tell you to write your own serving code. OpenAI is dropping weights and custom kernels, working with partners so everyone has an optimized stack on day one. That changes the inference-provider game. When DeepSeek dropped, Together (with Tri Dao and other kernel talent) and Fireworks (with former Meta PyTorch people) could pull ahead on low-level performance. If OpenAI ships the optimizations, those firms have to differentiate somewhere else.

Guo’s premise is that the model-optimization layer is becoming open-source commodity, so the real fight moves to infrastructure. Patel pushes back: those optimizations have not all been open, and the pace of change is fast. Nvidia is open-sourcing pieces around Dynamo; OpenAI is pushing more open. But the hard work is not a single server. Caching across turns, caching tool-use calls, DeepSeek-style inference on something like 160 GPUs—over $10 million of hardware for one replica, then shared cache servers across replicas—is an ugly distributed-systems problem. He lands where Guo wanted: single-node optimization is not necessarily commoditized; owning the orchestration abstractions is.

What does an American open model mean for apps? Patel says some enterprises are still iffy about the *best* open models because they are Chinese. He does not think DeepSeek, Moonshot, or Alibaba (with teams in Singapore) are planting Trojan horses; they are racing each other to release. Anthropic papers on embedding hidden behavior found it fairly detectable. He is not an interpretability researcher; this is a vibes call. The tell is behavioral: companies that refuse to run a Chinese model will use a service backed by one and feel fine. They just do not want the weights on their own machines.

The real damage is raising the commodity bar. Kimi may be strong, but it is huge and hard to run. OpenAI’s model is relatively small, so you do not need gigawatt-brain infrastructure to serve it, which kneecaps closed-source APIs that are not at the absolute frontier. SemiAnalysis has been building alternative data on token flows, and the result is counterintuitive: people are not using reasoning models that much in APIs. Anthropic has overtaken OpenAI on API revenue, and the bulk is not thinking mode—it is Claude 4 for code. The same pattern shows up for OpenAI and Google. Cost and latency are the brakes.

Guo invokes Jevons paradox: if you strip a layer of margin and make the model smaller, restrained customers will use more reasoning. Patel’s cost example is stark. For a long time OpenAI charged more per token for o1 and o3 than for GPT-4o even though the architecture was basically the same and only the weights differed. Longer context is a partial excuse; roughly 4x per-token pricing was not. They were taking margin because they could. DeepSeek, then Anthropic and Google, commoditized that. This open release, he says, cuts everyone off at the hip again for anything short of leading-edge APIs.

Every day SemiAnalysis still finds a new neocloud. The database is around 200. Should they all exist? Obviously not. Differentiation is already extreme. Some have horrible utilization. CoreWeave is sold out on four-, five-, even six-year contracts and often will not quote startups, or quotes a stupid number because it does not want the business. Time to deploy, reliability, and software vary wildly. Many cannot even install Slurm. From the investor side, commercial-real-estate capital—debt and equity—is pouring in after a bad few years in offices. Those LPs can live with 10–15% return on equity. Venture cannot. Patel puts about a hundred of these firms between a rock and a hard place: no longer VC-worthy, or unable to get utilization up against CoreWeave and Crusoe.

Some are already in a death spiral. They bought GPUs on a loan, utilization is too low, cash burn cannot cover debt service, so they dump remaining capacity at insane discounts. Patel commends the startups that have learned to harvest those desperate deals—and notes the vendors will go bankrupt anyway. ClusterMax, SemiAnalysis’s review of neoclouds and major clouds, already finds some newcomers better than Amazon, Google, and Microsoft on uptime, availability, reliability, network, and security, because they have no old baggage. Most are worse. Gold and platinum today will be table stakes in a year.

The paths, as he lists them, are few. Move up the stack: Together and Nebius already sell inference APIs, and *The Information* has reported CoreWeave trying to buy Fireworks for the same reason. Go absurdly big: Crusoe building gigawatt campuses, where almost nobody competes. Accept commercial-real-estate returns. Or go bankrupt. Guo’s reason-to-exist is eating hyperscaler GPU margins that still look like CPU-era ROIC. Patel’s punchline is a software inventory: PyTorch is open source, Nvidia libraries are largely open, models are open, vLLM and SGLang are open. If you are just renting GPUs, you are not shipping software the end user actually uses, so you do not deserve Amazon-and-Google margins. Guo agrees the software *could* exist; the majors have not delivered it. Startups still keep multiple people on infra so a random AI SaaS does not go down for eight hours.

The multi-trillion-dollar question: what would it take to challenge Nvidia? Patel’s short answer is a three-headed dragon. Head one is hardware engineering; they are genuinely great at GPUs. Head two is networking, with thousands of people on it. Head three is software: he will call CUDA only “okay,” except that everyone else is terrible, and twenty-plus years of libraries still matter even when the driver install is painful. Hyperscalers play a different game. Google TPUs, Amazon Trainium, Meta MTIA are not doing anything crazy. Squint and Blackwell and TPU memory hierarchies and systolic-array sizes are converging. That is fine for a volume-and-margin player. A chip startup must be unique, and uniqueness usually means you are worse on every other vector.

Stack the penalties. Miss the latest process node: 20–30% on cost, performance, and power. Miss the latest memory by a year: another round. Miss on networking: another. Your unique trick cannot be 2–4x faster; it has to be far more, and a flop is a flop—there is not 10x hiding in a standard von Neumann design. Nvidia has on the order of 50 people on memory controllers and HBM and thousands on networking. Dilute a 5x advantage down to 2x, misstep by six months, and the next Nvidia chip is out. Even Amazon, with a relatively unaggressive Trainium rack, blamed AWS’s miss on rack integration yield. Hardware timescales are long; many things can go wrong.

The deeper trap is that hardware and models must co-evolve, and nobody knows the model of two years from now—not even Greg Brockman. Microsoft’s chip efforts suffer because the company does not understand models; Meta’s chips work for recommendation because they co-design; Google is strong at co-design; AMD is fine at hardware and loses on networking and software. The first-wave startups—Cerebras, Groq, SambaNova, Graphcore—all bet more on-chip memory and less off-chip bandwidth. Nvidia’s on-chip memory rose maybe 30% from A100 through Blackwell; those firms had roughly 10x, and then models outgrew the chip. The opposite bet is in play now: giant compute units sized for dense Llama-70B matmuls, just as models went sparse MoE with tiny hidden dimensions, DeepSeek-style. Patel says that is happening at at least two companies he will not name. Edge had 40 or 50 AI chip startups; Qualcomm and Intel parts won. His second-choice bet is AMD GPUs, Amazon Trainium, or Google TPU (mostly internal), not a hardware startup. He still wants someone to succeed, because every point of Nvidia margin is a point of AI that does not get deployed.

Zoom out to data-center scale and people want one villain. Patel says if you solved one constraint you would get another 20% of output and hit the next wall. Company A’s bottleneck is not company B’s. He flips Noah Smith’s recession question: without the AI buildout, he is not sure the US economy would be growing much this year. Electrician wages have soared. Power gear with 15-to-30-year lives is being installed. That capex *is* the growth.

Imagine a facility the size of Manhattan, changing topology, novel hardware, real failure rates, dense networking. In 2023 the story was simple: Nvidia could not make enough chips because of CoWoS (chip-on-wafer-on-substrate) and HBM. Then data centers were the issue; then substation equipment and transformers; then generation. The old constraints did not vanish. CoWoS, HBM, and optical transceivers are still tight, and so are power and buildings. Meta is putting GPUs in tents because the building takes too long and too much labor. Ohio delayed a bring-up on grid issues, so they looked at on-site generators and found a multi-year GE turbine backlog. Elon buys an existing plant overseas, moves it to Memphis, and inherits permits and protesters.

Labor is the constraint pitches now brag about. Founders tell Patel they have booked every contractor in a region, so rivals will have to fly people in. There are not that many electricians in America; wages for data-center trades have jumped. He describes a group of Russians who used to wire Yandex facilities, now in the US on speed bonuses. Microsoft is not building Stargate for OpenAI the old way because it would be too slow, so it rents heavily from CoreWeave. OpenAI scatters demand across Oracle, CoreWeave, Finland’s Nscale, the Middle East’s G42—many baskets, whoever executes wins. That is also how he scores xAI: no leading-edge model released, a valuation above Anthropic’s, because Colossus was built faster than anyone else, and speed is part of being best at models. Trade-school supply is too slow; the Middle East imports labor; robotics is the other path.

Guo asks what “export the American stack” should actually mean. Patel detours to a week in Lebanon with Indian, Chinese, and Lebanese friends. Twelve-year-old girls complimented a Chinese friend’s skin and asked if she liked sushi; when they heard San Francisco, they asked whether people get shot in the streets. Their worldview was TikTok. Hollywood’s old American filter is broken; the same kids were weirdly fervent about Target because influencers posted store tours. The next generation will run on someone’s models, and models carry a worldview—talk to Claude and you can feel it. He wants that stack to stay American.

The last administration’s diffusion rule said the world could use US chips if American companies operated them: Microsoft and Oracle could build in Malaysia, random small firms could not. It was too complicated and made enemies. The current administration tore it up, then signed Middle East deals where GPUs are still mostly run by American firms or rented to OpenAI, Amazon, and Oracle. Patel’s value-chain ranking is consistent across both: sell services if you can, then tokens, then infrastructure, then GPUs; do not sell the tools that make chips. Enable competition at the high-margin layers. Do not let someone else win the bottom.

The China question is a gray line, not a slogan. San Francisco often wants zero GPUs sold; then China squeezes rare earths. The US briefly banned EDA software, China signaled it would ship minerals, and the ban lifted in weeks. Cut magnets and rare earths and American car plants stop, with hundreds of thousands idle. Huawei still gets TSMC wafers through shell companies; Korea memory and Taiwan chip sanctions have holes. Patel’s middle: some GPUs in exchange for rare earths, never the best ones, and actually enforce equipment and wafer bans. He does not assume China matches Nvidia quality. A chip that uses 3x the power on N-2 process can still be rational if China has 4x the electricity and will subsidize, as it did with solar and EVs, to collect prompts, databases, and service margin. That, more than peak flops, is what should keep Washington up.

If Guo lands Mark Zuckerberg on the show, Patel would not ask about infrastructure—he can reconstruct that from supply chain and satellites. He would ask the companion question Meta put on the page: when people talk to AIs more than to other people, and the interface is always on a face via Reality Labs hardware, what happens to the human psyche? Social feeds already break brains. Always-on AI companions are a different regime. What are the negative and positive ramifications, and how do you keep youth from slop and brain rot?

The closer is a joke at Patel’s own data-driven self-image. He had Cognition as NGMI: OpenAI, Anthropic, and xAI would just make better general code models; Claude Code already felt stronger than Devin internally. Then he went to an East Meets West gathering—Asian CFOs and CEOs, East Coast finance, West Coast tech, governments—and looked over at the high-stakes poker table. Scott Wu was taking chips off a table of heavyweights, including the CEO of a major Chinese company Patel will not name. Suddenly the vibe flipped: maybe this person can take from the lion. Guo’s punchline for founders: win the Cognition poker game and capital might follow. Patel admits it is a stupid reason. He still has not done real product diligence. The tell was the table.