Nine Moats, a Cloud-Shaped Market, and Why OpenAI Is Not Optional Yet

After a leaked Google memo argued that AI had no moats, Nathan Labenz and Erik Torenberg spend this Cognitive Revolution episode pressure-testing that claim. Labenz’s working picture is that the “utility intelligence” market of the next few years will look a lot like cloud computing. If algorithms keep commoditizing and open source keeps catching up, the scarce resource starts to look like computers: how much inference you can serve is how much compute you have. He does not think it is quite that simple, but the market structure is lining up on similar lines.

In cloud, the big-tech oligopoly — AWS, Microsoft, Google, Oracle — already holds about half the market. Labenz expects a similar split for where inference is served. Microsoft and OpenAI are in the lead with the best product and the cloud to back it. Google DeepMind, whose cloud also powers Anthropic, is next. AWS is a little behind but has Hugging Face, a huge existing customer base, and a natural home for homegrown open-source models that still have to live somewhere. The rest is a long tail: other service layers on the same public clouds, on-prem enterprise setups, and even consumer devices.

In keeping with what he calls “everything everywhere all at once,” Labenz sees no reason any significant pool of compute would go unused for AI. For that picture to fail, someone would have to run away with the market, and OpenAI, though ahead, does not look light-years ahead. He puts the commercial gap versus open source at about 18 months to two years — meaning what OpenAI sold then is roughly what open source can do now, not that open source will catch today’s frontier in two years. GPT-3-quality models can now be trained from scratch for under $500,000 at Mosaic if you bring your own trillion tokens. GPT-4 is still rumored to cost around $100 million, a much harder target to copy.

The “no moats” thesis, he says, has not held up under a few weeks of scrutiny. The negation is not true either: the memo was right that the open-source community is flourishing, but it is far from the case that nobody needs OpenAI anymore. Teams putting this into medicine are not that cost-sensitive. They want the best thing, and OpenAI has it, with Google trying to get on that level via systems like Med-PaLM 2. You can taste the difference in a quick demo, without a deep evaluation process. That position looks secure for the foreseeable future.

Erik zooms out: some platform shifts expand open source’s share, some do the opposite, and the characteristics that decide which are worth watching. Labenz is not a scholar of earlier software waves, but the practical question he keeps getting is how a company like Waymark should choose. In most contexts, he thinks you do perfectly well with an OpenAI product. At Athena, where he is the AI advisor, the explicit strategy is always start with GPT-4: it is number one on the leaderboards, follows instructions best, and is not that expensive. It is a little slow for testing, but it is the fastest path to seeing whether a task can be automated at all. Fine-tuning a narrower model is a different scale of project, and a valid reason to leave — but most of the reasons people cite are not.

Cost, in particular, is “pretty cheap actually,” especially on a total-cost-of-ownership basis. He can take a common task and get the model performing reasonably well in a few rounds of prompt engineering, often in 30 minutes or less. How many runs do you need before hunting for a cheaper option pays back a handful of extra hours of someone’s time? Most tasks will never scale to millions, let alone billions. At GPT-4 prices — about three cents per thousand tokens in and six cents out, call it four cents per thousand — you are paying four cents per two pages of text. Turning a long client-call transcript into a condensed profile might cost a quarter and replace four hours of work.

The exception is high-volume generation with a low hit rate. Waymark makes 30-second ads for small businesses; generative AI now writes the script, picks images, and layers voiceover, but the company still has a tight process. Maybe one in three to one in five generated videos actually gets rendered and downloaded. Jasper, now listed on Mosaic’s homepage as a client, is the contrast: open-ended copy tools where users rifle through generations the way Playground users make a thousand images a day. That economics can justify a cheaper or fine-tuned model. Task automation is the opposite hope — use essentially all of the output. GPT-4’s real frictions there are the 8,000-token context window and latency when you have to chunk a long transcript, not the invoice.

After a sponsor break, Labenz finishes the Athena example. The human review layer is “did this 90-minute call miss anything important,” not a rewrite from scratch. You can go from hanging up to a draft in a couple of minutes; the model’s cost is tiny next to the person’s time, so they would not even consider an open-source model. Downstream, matching 100 new clients a month against 200 already-vetted candidates implies on the order of 20,000 comparisons — something humans historically stop after finding one good-enough match. The model can run overnight, still cheaper than people, and then write a personalized intro blurb. In all of that, he says, cost is not an issue.

Asked whether the roughly two-year open-source gap will widen or shrink, Labenz guesses a yo-yo. The open-source line rises continuously; the labs drop punctuated next releases; the perceived public gap closes, then widens again. Whether the private gap — what they have internally that the rest of the world does not know — is shrinking is even harder to tell. Meanwhile the leading companies, OpenAI, Google DeepMind, and Anthropic, are coordinating at a “very meta level,” saying agreeable things about safety and broad benefit. Google CEO Sundar Pichai has tried to say they will not get into an AI race, language Labenz reads as validation for the AI-safety movement, and specifically the effective-altruism version of it. It is easy to imagine a counterfactual where cutting-edge labs dismissed tail x-risk; that they take it seriously, he thinks, traces in part to Eliezer Yudkowsky and that literature.

Erik pushes on actions versus words: if they were in a race, would they be acting any differently? There is a cynical reading that “this is not a race” is reverse psychology, useful with competitors and with regulators. Other technologies hyped themselves as the next great thing and then faced skeptical media; with AI, some people accused labs of marketing by emphasizing danger, which also emphasizes power. Erik expected more public alarm and instead sees more concern inside tech than outside it — media and government still seem more scared of social media, misinformation, and teen depression. Labenz answers with Sam Altman’s actual regulatory ask: rules should apply only to the leading companies doing the biggest training runs, and should not interfere with open-source research and startups. That, he says, is a sane position, not a hype campaign.

The labs do not need to market. Mosaic’s Jonathan and Abby told him they cannot run long enough, and that if you want to be a customer you had better call soon: they are making tough decisions between research and customers and will not cut research to nothing. The spend required to get OpenAI’s attention has jumped. A little over a year earlier, Waymark bought a $2,500-a-month service package for a regular call; that tier no longer exists, and an account manager now takes a six-figure or even quarter-million-dollar commitment. Altman also downplays launches: one of the first lines on GPT-4 day was that it seems more impressive at first than it really is. Labenz quotes a Sam Mays tweet in the same register — still makes stuff up, still has major weaknesses. That does not sound like a hype cycle to him.

OpenAI’s recent governance statement argues it is “counter-intuitively dangerous” not to develop these systems, because the fundamentals keep making powerful models easier, and a sudden breakthrough dropping into an unprepared society would be worse. Labenz is not sure he follows that all the way to “better us than them,” but he thinks their behavior is consistent with trying to rise to the occasion. The charter has long said they would combine forces with another effort credibly close to AGI rather than compete — a public commitment from around 2015, not a new talking point. He bets future frontier releases will keep clustering, as GPT-4 and PaLM 2 did, because leadership does not want a death match between Microsoft and Google. Google, he thinks, will be vindicated for not taking the bait and shipping before it was ready; it has not lost much search share. That judgment is, by extension, another vindication of the EA safety literature Sundar would not have grown up with on his own.

They then run OpenAI’s nine moats the way you would handicap NBA teams. Moat one is GPT-3.5 Turbo as the best value in the LLM game. If GPT-4 is already a huge saving over a human process, the fallback inside the same company is a model Labenz prices at about 0.2 cents per thousand tokens — two dollars per million words — which powers ChatGPT’s free tier. He has started watching an LMSYS.org arena that runs chess-style head-to-head battles and Elo ratings. GPT-4 sits at the top, Claude is second, and Claude Instant has recently pushed Turbo into fourth. Claude Instant might now have a slight edge on value, but both are cheap, fast, and good enough for marketing copy and structured formats. The leap from 3.5 to 4 is like going from the bottom 10 percent of the bar exam to the top 10; the cheaper model still does a lot. Open-source imitators, many trained on GPT output, are not that close even to this second tier on a blind user vote, so OpenAI has not left the door open for a pure-price steal.

Moat two is branding and trust. ChatGPT is accused of being too white-supremacist and too woke at once, and Labenz thinks almost any perspective can find unwanted biases. The open-source alternative, for a corporate customer, is “just way worse.” Radical free-speech users who are not major companies can go that route; a business putting a chatbot in product does not want its own “Sydney” experience. He jokes that open-source radicals overestimate the corporate appetite for large-language-model adventure. The old line was nobody got fired for going with IBM; now a CIO can say they used the industry standard that spent six months on GPT-4 safety. Google, he thinks, will get there soon on gravitas. When the safe choice is also cheap, why put yourself on the hook for an under-tested open-source model to save a little money you may not even save once you count the extra engineering hours?

Moat three is the feedback loop, and here OpenAI is “quite out in front.” Nobody has ChatGPT’s volume; Bing is doing decent volume but not as much, and OpenAI powers that too. API terms now retain data briefly then delete it and keep it out of training; the free ChatGPT tier is used in training by default, with a newly added opt-out. They also have a humming product-development process that others find hard to copy. Labenz has gone down the Bing rabbit hole: Microsoft tested for months in other countries, users reported on forums the same behavior that later made the New York Times cover, and one part of the organization was not talking to another. At launch, a user correcting the date was enough to send the whole thing off the rails. He wants people to distinguish a jailbreak that tricks ChatGPT into a bad word from a bot that “turns on” a user — a different failure of control. Well-behaved product AI, he says, does not cohere by accident.

Anthropic’s constitutional AI, using self-critique and synthetic data, tries to get similar control without that user base, and it seems to work quite well. They may even be ahead on safety profile if not raw capability; Labenz has also heard the reverse, and with a huge surface area you can be ahead and behind at once. Claude used to hallucinate less in his tests — ChatGPT inventing a Massachusetts property-tax rate while Claude hedged with a round number. Character.AI is a dark horse with a real at-scale product loop. Google does not have that unified flywheel yet: Bard sits down the LMSYS leaderboard not for lack of horsepower but because the product is not shaped the same way. Med-PaLM 2 can match GPT-4 on expert medical benchmarks, but it is a specialist; it cannot be your general chat advisor. Google is playing both sides, taking the specialist into hospital groups rather than a raw base model. Specialist AIs could even be a better overall architecture.

Labenz restates pricing power with a vivid unit. At two dollars per million tokens, you would have to serve 100 billion tokens to pay for one employee at $200,000 of revenue — a rounded-down San Francisco AI salary, before electricity and Microsoft’s cut. A hundred billion tokens is a ridiculous amount of stuff: a Llama-scale open-source run might train on a trillion tokens, a ballpark people also use for “the whole internet,” so you are generating on the order of a tenth of the internet for $200,000. Who needs that many tokens, and how do you assemble another world-class team only to then go hunting for many customers who each need 100 billion? He does not think imagination is exhausted — mass summarization, scanning everything, actually implementing “this call is recorded for quality control” on every call, now that it is cheap — but as a VC he would be wary of competing at that commodity layer.

Moat five is privileged access to cloud compute. You can only serve as much of this as you have chips, and shortages are starting. A friend at Google told him demand is beginning to bind even after a decade-long infrastructure bet; they are rationing. OpenAI said the same in the middle of last year: even with Microsoft, they could not scale compute as fast as they wanted, and DALL-E 2’s blow-up forced them to delay other launches to put resources behind it. That is the Azure cloud. Hitting capacity after hundreds of billions of capital investment is about as self-explanatory a moat as it gets. You can still build a business, but you will have a hard time accumulating anything on similar scale. Anthropic partnered with Google, Hugging Face with Amazon; leading labs without their own clouds are playing musical chairs for preferred-provider slots, and AWS will not take on that many.

Moat six is GPT-4 itself, used in ways that compound advantage. If you want a story for why leaders pull away from open source, this is one of the best: they already have models that scale work that used to be hard. OpenAI’s interpretability paper used GPT-4 to inspect what neurons are doing — Labenz thinks they were looking at GPT-2 — by running text, collecting high-activating examples, and labeling concepts. In their Tiny Stories interview, that process on small models makes categories jump off the page: a neuron that fires on dog, cat, bird is clearly “animal.” Bigger models get messier. Automating that with GPT-4 is how you scale over huge parameter counts. The same engine can clean and enrich training sets. Anthropic’s accidentally leaked pitch deck said companies that fall behind in the 2025–26 cycle may never catch up: the models become the engine of the next research, and without access you cannot run at the same pace.

He generally believes the calls for regulation are sincere, but this is where interests diverge. If exhaustive checks are required and you can run them with GPT-4 at one scale and not without it, that becomes a lock-in. One could imagine a rule that they must share certain capability with other developers; as it stands, their terms do not allow the imitator models trained on ChatGPT output, and OpenAI would probably sue and win. The follow-up is uncomfortable: how can they train on all of humanity’s data and then prohibit downstream distillation? Labenz is not sure that position stays tenable. China has issued guidelines people call impossible — training data must be “reliable,” must not violate IP, must meet quality adjectives — because they are more worried about Tiananmen Square than a chatbot space race. The way you meet those standards, he argues, is if you own a cloud and have GPT-4: filter and transform real data into synthetic data, then train the next model on a squeaky-clean set. They have also raised the drawbridge by dropping logits. GPT-3’s API used to return the top five tokens with probabilities, a gift to anyone studying or imitating the model; GPT-4 returns none of that.

Moat seven is team and talent density. A recent story had Satya Nadella asking Microsoft’s head of research how OpenAI did this with a couple of hundred people while Microsoft has all these people and is getting its butt kicked. Labenz has seen that quality in every part of the organization, including the old $2,500-a-month consulting engagement. The business contacts know the technology in a way he does not think you can sustain at huge scale; they do not have to check with the team. Moat eight is “insane distribution and partnerships.” A single thread from one of the business people listed Intercom, Wix, Morgan Stanley, Shopify, Khan Academy, Atlassian, Zoom, and Brex. There is a huge partnership with Bain and another with a global consulting firm. They are already in the door at basically all of corporate America. You can try to sneak in before the door closes, but those sales processes are in motion, if not already closed and moving toward model customization.

Moat nine, network effects, he puts last because they are weaker than in social media — a point they had also discussed with venture capitalists Elad and Sarah. When people ask about lock-in, he is honest: there is not much. An OpenAI API call is a few lines; you could flip to Claude in two seconds. Fine-tunes cannot be downloaded, so you are building to rent, but you still own the dataset and can run it on an open-source model. He has considered doing exactly that at Waymark out of curiosity, not cost. Their task is tightly defined: fill a video-script structure given user information. If the model malfunctions, the app errors; the user never sees wild output. That developer control means they could live without OpenAI’s safety bells, and switching still would not be hard.

What gravity there is, is social and ecosystem. Everyone is introduced to AI through these products; prompt-engineering techniques and tools get built for OpenAI first. Libraries are usually provider-neutral quickly, but nobody ships a new framework that does not support OpenAI on day one. If something works with OpenAI, the first thing a user trying a competitor will do is paste the same prompt. If it fails, they will think the other model sucks, even if a rewritten prompt would work. Competitor CEOs who say “read our prompt guide” are going to lose; they have to reduce friction, and in doing so they end up in a following position. The plugin architecture looks portable in theory — Microsoft will adopt the same basic design — but there are many ways to be worse at supporting it, and then you are playing their game from behind instead of developing your own.

That is why he flags Character.AI and Inflection’s new Pi as possible exceptions: they are trying to do something different. Pi is not a butler that fetches an answer; it is a more open-ended, exploratory dialogue. He does not think many people go to ChatGPT for companionship, and ChatGPT somewhat discourages that by reminding you it is an AI in ways that break immersion. The companion products he has seen do not deceive you about being AI, but they engage differently and could become a separate category.

Turning to what is investable, Labenz says the sophisticated analysis now agrees Salesforce should be fine: it has enough product, distribution, and contracts to layer on a copilot before anyone builds a worthy rival from scratch, no matter how smart their use of GPT-4. Adobe is in a similar position on the visual side. A lot of amazing generative work has shipped, but nobody has really canceled their Adobe accounts, and Adobe is announcing its own amazing stuff. He cites a Twitter thread from someone named Riley: Midjourney is groundbreaking, but if you were telling someone what to learn today you would tell them Photoshop, because Midjourney is a black box and Adobe now has both the generative layer and the “dig in and do it” layer. Building a credible Adobe rival will take a long time. He would be very wary of a thesis that you pick an incumbent and beat them because you use AI. They all know about AI, it is not that hard to implement, and a big platform’s documentation is already in the models.

You can ask GPT to write a tweet versus a LinkedIn post and it already knows the difference; a social network that does not exist yet does not get that for free. Salesforce will have its own copilot, better than default GPT-4, with perhaps a trillion tokens of docs, forum posts, and tickets to train on. Bottom line: incumbents should mostly be fine, and a “we win because we use AI” attack on them is a weak path unless the incumbent is flagrantly dropping the ball. The more exciting, hard-to-predict bets are new things that could not have existed before, where there is no incumbent. So far that list is short: AI chatbot as a category, text-to-image as a category. GPT-4 has only been out a couple of months, and a lot of the crazy ideas will not even come from AI developers — the way Uber was not built by the people who built the iPhone, and Apple could never have done what Uber did.

One idea he is watching is improved discourse tied, unusually for him, to crypto: a language model as a dispute resolver inside a smart contract, cryptographically guaranteeing you get the model that was agreed when the contract was signed, handling refunds and little escrow accounts. People will accept some unfairness, he thinks, because the alternative is small-claims court. New relationship paradigms are another. He is not excited for himself — he does not want an AI friend, and he sees “amusing ourselves to death” written all over it — but talking to Eugenia of Replika, the revelation was how badly a lot of people need companionship. How far she had gotten before GPT-3 was a society story, not an AI story; inject AI and it can easily go 100x. He does not see why the share of people with an AI friend would not explode, and for many it might even be good. The failure mode he fears is crowding out real relationships, the way social media is addictive, mixed, and crowds out other activities.

On healthcare, Google and Microsoft already have the hospitals: those systems get Docs from one of the two, or run on Epic. He expects Neal Khosla and Curai to be successful — they have been in the game, this was not a company started to ride GPT-4 — but that may mean past investors do well while next-round investors, buying after prices have dropped 97 percent, do not. The biggest corporations are ready to run at or near cost, and they already have distribution. What will be transformative for a while is task automation more than a totally different form factor, though you can imagine a true AI doctor with no humans and a new sensor stack. Hippocratic.ai is differentiating by training only on trusted medical information and refusing patient-facing diagnosis in favor of back-office work, a big enough market. Neal’s counter is that if a patient says they had cereal they probably had milk, and if they went to Burning Man they inhaled a lot of dust — general knowledge medical systems may not encode. Labenz’s guess is still that the Microsoft–Epic complex takes most of that market. “It’s tough to get your little green shoot up through the hornet’s nest.”