Worth Thousands, Priced at $20: Zvi Mowshowitz on OpenAI's Flurry, Gemini Bets, Deepfakes and a Compute Cap

Nathan opens by listing OpenAI's recent burst of launches: DALL-E 3 rolling out to ChatGPT users, image understanding added to ChatGPT and Code Interpreter, voice as a first-class feature, and browsing re-enabled. His own newest use case was adding a feature to a React app even though he had never coded in React before.

He describes asking it to explain the inherited project, having it write a command to print his file structure, and then dropping in screenshots when the feature wasn't working. He says it felt like a significant jump in the value of ChatGPT. Zvi agrees and says it tempts him to start building things himself, because it makes work an order of magnitude easier.

Zvi notes the pattern that the worse you are at a task, the more these tools help. Nathan counters that even expert programmers such as Andrej Karpathy get leverage from them, and says the tool rarely makes typo-style mistakes. He puts his speedup at three to five times in general and plausibly ten times on the React project.

Nathan shares a tip he calls coding by analogy: bring a snippet of working code and ask the model to map your goal onto it. That helps with outdated library knowledge, which was a known problem before browsing returned. He also admits that his DALL-E experiment to modify a graph didn't work, because the model reinterprets the image and generates something new rather than editing it.

Asked why the releases are arriving now, Zvi says OpenAI has repeatedly held back large capability advances; GPT-4 was held for months, and ChatGPT came after the technology had been ready for a while. The multimodal features are built straight off existing GPT-4 and could have come earlier. He suspects worries about adversarial attacks on vision are one reason, and he is curious what defenses they have.

He then lists the pressures now on OpenAI. Claude 2 offers a giant context window, reads PDFs natively and is free; Llama 2 is open source, and OpenAI doesn't want developers to build on it; and Google's Gemini is expected by year-end, possibly natively multimodal with AlphaZero-style ideas. If OpenAI is about to lose the best underlying model, he says, it must lock in users through features.

Nathan says he loves Claude 2 for long context and for imitating writing style. For example, it is the only model that can convert a podcast transcript into timestamped show notes without chunking it up. But it struggles near the full window: it works on chunks of an hour to 90 minutes and goes off the rails at the full length.

Zvi mentions a study on how recall degrades as more tokens come between the start of the context and the information you need, and suggests staying around half of the window. He used to think the free Claude 2 was enough, but says the new ChatGPT features have changed that for anyone doing knowledge work.

Nathan asks what ChatGPT is really worth against its $20 individual price and the roughly $60 per seat known for enterprise. Zvi says what it is worth and what people will pay are completely different things. He thinks GPT-4's marginal value over alternatives is at least four figures a month for him, and five figures or more versus nothing, yet he would recoil from paying that.

He compares this to people who spend two hours a day on Twitter and balk at eight dollars. He thinks enterprises should happily pay thousands of dollars per employee per month, and that $60 per seat is very reasonable if usage is effectively unlimited and comes with privacy and security. Adding optional features can raise price further, as with any slightly better version of a product.

Nathan says he would have paid up to $1,000 a month for original Copilot, and that the real question is your alternative. Zvi agrees that the best alternative matters: with free options such as Claude 2, Bing and Perplexity, it's hard to charge an arm and a leg, but once a product does things others can't, prices can climb. Nathan adds that he'd happily pay $60 for integration with his email and docs so that GPT-4 knows his context.

That leads to a shift in how they treat their inboxes: Nathan used to delete mail for tidiness but now wonders what to archive for a future model. Both note that too much distracting data hurts a model, much like the long-context issue. Nathan plans to purge about 90% of his mail, and Zvi recalls working at a financial firm where nothing could be deleted and search was hopeless.

Nathan asks whether Gemini will become the best model and whether scale is all you need. Zvi says he firmly believes it is not, pointing to large open-source models trained on lots of data by teams that, in his view, lack expertise, which end up useless. OpenAI's expertise is incredibly valuable, and larger models are also costlier to run.

Google, by contrast, has data, compute and DeepMind's expertise, but Zvi questions whether it can still execute. He notes its internal competition among overlapping projects and the need for many product teams to adopt Gemini effectively. He would set the over-under at about GPT-4.25: somewhat better, not earth-shattering, and he warns it may not be ready by year-end.

Asked whether Code Interpreter is a 4.5 on that scale, Zvi says it makes sense to count the GPT-4 core model plus everything people learn to build on it. Nathan observes that it feels more autonomous, running code, hitting errors, rewriting and examining data, and may reflect extra training on iterative problem-solving. Zvi adds that this kind of scaffolding probably cost little but was conceptually hard, which is scary for what comes next.

On what Gemini might bring, Zvi expects more raw intelligence and perhaps better understanding of what a question is about. He also flags a Google claim of continuous training, which would let a model's cutoff be the current week, a change that would matter because the lag is why he uses GPT-4 less.

Nathan says Perplexity has become his first choice for questions he does not already know the answer to, while Google remains better for quick lookups when he's sure he can surface the page. Bing has not impressed him. He values the sources and the freshness. Zvi says that for him, using a model with Bing is outsourcing his Google skills to an alternative that is no better at it.

Nathan reports that fine-tuning GPT-3.5 has been fast, easy and effective, and that Perplexity claims fine-tuned 3.5 matched or beat GPT-4 on its task. His tip is to train on GPT-4's reasoning, not just its output: have GPT-4 explain its reasoning before the task and fine-tune 3.5 on both. Zvi reads Perplexity's result as a statement about its users' questions, which tend to be one simple sentence, unlike long multi-paragraph prompts.

Nathan speculates that OpenAI might offer a mixture of models behind one API, routing each query to the right model at a price between 3.5 and 4. Zvi says third parties will do this, for instance by asking 3.5 to answer only with a short flag if GPT-4 would do much better. Because the cost ratio is large, calling the cheap model first adds little.

Nathan lists the growing pile of AI subscriptions, including ChatGPT, Claude Pro, Perplexity Pro, Windows Copilot, ChatGPT Enterprise, Google's Duet AI, GitHub Copilot and Replit Ghostwriter, and suggests a cable-style bundle. At Waymark, he says, users often take the free trial, download what they came for and cancel. A bundle login could grant free access across apps while power users still pay extra, and it might also cool the commercial race.

Zvi would be all for it and says we are in an unbundling era where everyone wants a subscription and nobody shares revenue, leaving people with impoverished choices. But he doubts it will happen: splitting the revenue is arbitrary and social, and companies do not cooperate that way. OpenAI would also not want to advertise rivals such as Claude.

On the games side, Zvi describes PlayStation Plus, Game Pass and Nintendo's subscription as bundles whose marginal delivery cost is close to zero and which give good value to non-picky players. His ideal would be a Steam service that divides revenue by play time, but it does not exist because it would alienate developers and cannibalize sales. He adds that these systems rely on whales and a curator who negotiates what content is included.

Both conclude that a bundle would probably need a real desire to coordinate and lower the temperature of the race, and even then it looks tricky. Nathan notes that cable bundles also had a physical delivery bottleneck that does not exist for AI.

Nathan recalls floating the idea of intentional information pollution as an emergency brake: flood the internet with synthetic content so that training data becomes unreliable. Zvi says it would not work because current models are good enough at filtering good from bad data, and he does not advocate it. Nathan points to tongue-in-cheek hidden-text injections on Perplexity, such as claiming Nat Friedman is known for his good looks.

Zvi distinguishes adversarial pollution from accidental pollution. His example is Quora putting a GPT answer at the top of its pages, which Google then picked up as the top answer, and he faults both. He also mentions people publishing endless AI-written books on Amazon and sites trying to trick advertisers with nonsense words.

His view is that Google is letting its product decay more than needed, and that obvious nonsense is easy for language models to detect. Defense, he thinks, probably wins most of the time, using models as filters plus PageRank-style webs of trust. Nathan compares it to low-background steel and notes that data from before 2022 may be especially precious.

Nathan says audio deepfakes are already very good, citing the AI-voiced versions of Zvi's blog posts, though he notices occasional misplaced emphasis. Images are good too, as in the Rand Paul bathrobe picture in Zvi's post, and video may follow. His worry is that fake content could someday outnumber real content of famous people and make everything muddy.

Zvi calls himself a deepfake optimist. He expects AI to learn to identify fakes because real images are internally consistent in shadows and details, and notes that no one has yet made a systematic effort at big identifiers. He also says a trained eye picks up a too-smooth, generic look in AI images.

On politics, he says nobody is really trying to convince people that a fake video is real, because it is a third rail. Instead campaigns use vibes and obvious spoofs, such as the Ron DeSantis Office-style video, where the point is a caricature rather than deception. Nathan cites NLW's view that people will be on high alert before an election.

The two also discuss Kevin Fisher's AI Souls demo, whose voices carry notable emotion. Nathan guesses perhaps one in four naive listeners would flag it as AI, and notes OpenAI's voices are deliberately restrained. They note that OpenAI took down its AI text classifier, and that camera filters blur real and generated content.

Zvi describes an idea from Emmett Shear: before recordings existed, we relied on testimony, and perhaps we are leaving a brief period when photos and video were trustworthy. He expects a strong cultural taboo against faking evidence, and says producing a video that survives scrutiny in court or Congress is harder than people think, because you can't know every detail it must get right.

Nathan introduces a recent paper on the reversal curse: if a model is trained that A is B, it doesn't necessarily learn that B is A. His example is that a model may know who a famous person's mother is, yet fail to name the famous person given the mother's name. Zvi says it makes sense given how forward passes and training work, and that Gary Marcus has raised a version of this complaint since the 1990s.

The obvious fix, Zvi says, is to put reversed facts into the training data, possibly by generating synthetic reversed versions, which is expensive but would work. He compares it to language learning, where knowing one direction of a flashcard does not guarantee the other. Nathan suspects GPT-4 already does some of this, noting its near-perfect recall of details in Wikipedia-style sports pages, and suspects OpenAI rewrites trusted data sets many times.

Nathan offers a speculative architecture hack: take the middle layers where facts seem to be stored, flip a copy and stitch it in as a reverse lookup, then train the model to use it. Zvi says such tinkering sometimes works and puts the odds at perhaps 10 to 20 percent, with Nathan higher but under 50. Zvi adds that he finds this kind of problem fascinating but does not want to help capabilities advance.

Nathan takes from this that there is still plenty of low-hanging fruit, so algorithmic improvements will keep arriving. He says that to the extent they do not raise central intelligence, they are mostly good, and to the extent they do, he is scared.

Nathan raises Connor Leahy's suggestion of banning models above 10^24 FLOPs, roughly where GPT-4 is thought to sit, with possible grandfathering of existing systems. Zvi says that six months ago this would have been beyond the pale. Now he says it is outside the Overton window of things likely to pass into law, but inside the one of things you can discuss without being dismissed.

Zvi says Leahy and his team care about what they think will make us die, not about the Overton window. He thinks the right regime would set a limit around 10^26 FLOPs now and lower it over time as algorithmic efficiency improves. That would mean tracking compute more carefully, and he lists dangers such as misuse, recursive self-improvement, replication and agentization.

On whether the world could have been better, Zvi's best counterfactual is one where DeepMind is the sole serious player with a large lead and a responsible CEO, releasing carefully. Nathan doubts such a secret could be kept, since the know-how needed for something interesting is modest, and notes that California does not enforce non-competes. Zvi replies that ideas often sit ignored, and that the alternative might be worse if less responsible competitors emerged.

Nathan briefly mentions Sam Altman's Reddit post that AGI had been achieved internally, which Altman then walked back as a joke. Both treat it as a comment on how uncertain everything is.

Nathan says that after Anthropic's earlier raise from Google, he wondered why Amazon had no horse in the race and why Anthropic's valuation wasn't higher. He reads the new deal as a mild signal that Gemini is probably pretty good, since otherwise he would expect a bigger bidding war. Zvi cautions against giving Google too much credit as a unified entity that updates on information.

Zvi calls the deal the steal of the century for Amazon, saying it gets a huge portion of the company for a few billion dollars. He guesses it includes a large compute commitment on AWS, perhaps at favorable prices, which could lower Anthropic's costs permanently. Nonstandard terms are only alluded to in public.

On why investors would put in money, Zvi says the main reason to stay out would be thinking it is bad for the world, or an affinity preference. He notes that companies such as Meta and Uber show shareholders may treat a company as worth its theoretical profits even if cash never reaches them, a fiction he calls one of several in play. They close by planning a future survey of the AI safety landscape, including the open question of whether Anthropic's work advances safety or capabilities.