GPT-4 as Teacher, GPT-3.5 as Student: A Nine-Tenths Cost Cut and an AI Bundle Nobody Will Start

A couple of weeks after OpenAI opened GPT-3.5 Turbo fine-tuning, Nathan Labenz is already using it in production at Waymark, the marketing-video company he founded. Llama 2 had been on his to-do list. He crossed it off. Even a successful open-weight fine-tune, he says, still leaves the ugly questions: where to host, what load it can take, whether to auto-scale. Solutions exist; they are not mature. OpenAI’s version is a drop-in. Once training finishes, the new model is live with the same high rate limits as the stock APIs. His developers swap a model ID and the rest of the stack does not move.

The new flow is chat-native. You set a system message for role and job, then a few user–assistant turns, and you are off. The old prompt-and-completion format still works; chat is now the primary one. Waymark already ships a working product: a small business types what it needs and, in 20 to 30 seconds, gets a script, images from its own library, and a voice-over. Scripts are often not quite good enough, image picks are sometimes “wonky,” and timing between voice and scenes drifts. Fine-tuning is the lever he wants for those gaps, without handing the team a new inference stack.

The first round was the old method: inputs and outputs only. Waymark’s fine-tune sets have always lived between about 100 and 1,000 scripts—small by internet standards, historically enough. It was “not that great.” Labenz took a walk. GPT-4 already does the task well with a couple of examples, so more raw pairs did not look like the missing piece. The idea he came back with was to train 3.5 not just on GPT-4’s answers but on the reasoning that leads to them: analyze the task, explain step by step, form a strategy, then produce the video. Classic chain of thought.

The twist is that chain of thought did not obviously make GPT-4 itself any better. In that sense it felt like a waste. Port those traces onto 3.5 and the smaller model jumps. Labenz’s analogy writes itself: the honor student already had the answer; handing the weaker student the scratch work is what opens the lock. The production math is ugly for GPT-4. Boilerplate is about a thousand words, two examples run about 2,000 tokens each, and he leaves 3,000 for the output, bumping the 8,000-token window. That is 30 to 35 cents a call and a 60-to-90-second wait, too dear and too slow for production.

Fine-tuned 3.5 has a 4,000-token cap (16,000 is coming), so the two examples get cut. Token count falls by more than half. Cost lands in the ballpark of a 90% reduction. Latency is often under 10 seconds, sometimes into the 20s; the slowest 3.5 calls sit about where the fastest GPT-4 calls used to. Training takes minutes. The model is usually callable in seconds, with the same scalability as stock 3.5. Human-written Waymark scripts never included three paragraphs of planning—that lived in people’s heads—so he just asked GPT-4 to generate chain of thought and the video together. First-pass quality was roughly as good as the human set. Anyone with a GPT-4 prompt that already works can mint reasoning traces and distill them downward. He expects that loop to become common.

The last episode with Carnegie Mellon’s Zico Kolter and researcher Andy Zou argued that jailbreaks are downstream of pretraining on internet sludge, then papered over with alignment. Their suggestion was to feed models less of the sludge. Labenz’s narrow loop is a small-scale version of that: for a vertical task, he now controls what the student eats. OpenAI itself may not get away with calling GPT-4 outputs “synthetic” and stopping there—people will ask how GPT-4 was trained—but for an app builder standing on those giants, the provenance story is cleaner.

At Athena, the executive-assistant company he often cites, the worry is practical. Assistants put sensitive and personal information in prompts. The team is not de-identification experts and does not want to get caught making a huge mistake. This experiment nudges him toward a policy: never train on client data; train on synthetic data that looks like, or is inspired by, client data. Explicit identifiers—phone numbers, names, prices, a SQL schema—are easy to fudge. Conceptual IP is not. Recent work, he notes, shows models can memorize a text from very few examples, sometimes one.

Process chemistry is his cautionary case. A pharma notebook’s heating rates and reagent additions look innocuous page by page; stacked together they are the secret. Scientists rarely read one another’s notebooks. A model that read all of them might see patterns no person would. He would train his own model on that corpus; he would not hand it to OpenAI. That, he thinks, is why the company now repeats “we don’t train on your data.” He is running the loop in HumanLoop, which lets him track every run, overlay a correction, and fold it into the next batch. Waymark will probably do about ten more fine-tune iterations in the coming weeks. For script writing, he is not even sure they need GPT-5.

Labenz’s policy is two-handed. Fine-tuning 3.5 so everyday apps get cheaper, faster, and more reliable is something to accelerate. New frontier systems on the GPT-5 scale are something to slow down. Leading labs, he says, have already staked a reasonable position: systems more powerful than GPT-4 deserve special scrutiny, including independent auditors and red teams. Talk of licensing is often caricatured as Big Brother on your laptop. The actual proposals, as he hears them, apply only above GPT-4 and are a sliding scale: if GPT-5 is safe, the line moves.

The concrete fear is biosecurity. Anthropic’s Senate testimony, with leading biosecurity people including MIT faculty, said a language model cannot yet do every step of making a new pathogen, but it can do some of them and can help you figure out the rest. That knowledge is already in the weights because the models were trained on everything. RLHF tries to refuse; a universal jailbreak, especially on an open-source release, peels the refusal off. One free-speech absolutist strips the guardrail, someone else walks the remaining steps, and control cascades. Up through GPT-4 he is reasonably confident it will not design a pathogen. GPT-5 very well might.

He would rather know what is in the dataset, and know that certain sensitive material is not, before anyone trains that next jump. OpenAI seems to want that path; Anthropic is “definitely all about that”; DeepMind’s posture is less clear, though AlphaFold shows they take scientific risk seriously. Meta is the wildcard. Scuttlebutt says it is training a GPT-4-class model to open-source and does not care who objects. Whether it filters the data will tell you how the company thinks. A design he likes is not a seal after training but two classes of model—bio-aware and not—so ordinary users lose almost nothing and a few biologists lose a little literature access. Synthetic data and cleaning need time to mature. Meanwhile, he says, we can have all the cool 3.5 apps we want.

Erik Torenberg asks where AI is a copilot and where it just *does the job*. Labenz currently splits use into two modes, with a third coming. Copilot: the human is the primary agent and the model assists. In a recent coding sprint he almost never writes by hand. He takes existing code or docs to GPT-4, says what he has and what he wants, and pastes error messages back in. It is a multiple-times speedup, fewer stalls, fewer typos, because the model rarely makes those. He is still driving, one class at a time.

Delegation, a word he borrows from Athena, has one core test: you trust the system enough that you do not review every item. A thousand applications come in; nobody will read them. He spends the time of reading 50 to 100 validating a prompt, then lets it split the bottom 80 from the top 20 and never opens the 800. Time savings around 90%. Hiring decisions stay human. Rachel Woods’s three-tier hierarchy is the mnemonic: work that just needs to get done goes to automation first; “good enough” work is borderline; insight and strategy stay with you, copilot at most. Agents, when they arrive, are the bridge—real-time delegation without today’s scaffolding.

Then the awkward case. A friend of a friend asked a legal question. Labenz is not qualified to answer it. He is qualified to put it into ChatGPT. The call to a lawyer never happened. He got enough to tell the friend they probably need a new lawyer. That is copilot for *him*, and it still substituted for billable time. He now sends contractor agreements to both Claude and GPT-4; if both say nothing of concern, he signs. “To deny displacement at this point is definitely head in the sand.” Expert calls that would have billed hours are already not being made. That is not the end of those jobs. It is already displacement. Torenberg cites Sam Lessin and Seth Rosenberg arguing cheaper costs explode demand. Labenz’s reply is the farmers tweet: aggregate demand can rise while a particular person loses the work.

He is wary of crisp market-size bets because prices keep collapsing—98% cuts year to year scramble the metric. Still, OpenAI’s growth has beaten even his GPT-4-red-team expectations: high-20s millions of revenue in 2022, now about a billion-dollar run rate, low-80s millions a month, 15 to 20 times in nine months. Naively, at GPT-3.5 Turbo’s $2 per million tokens, $80 million a month is about 40 trillion tokens, or 5,000 tokens a month for every human on Earth, ignoring free usage and mix of models. Another developer-conference price cut is already being rumored. The pie is growing faster than he thought, and he had already thought it would grow a lot.

Labenz’s own bill is the prompt. ChatGPT is $20 retail, $60 for the new Enterprise edition—the highest AI platform price he has seen. Windows Copilot $30, Google Duet $30, Perplexity Pro $20, Claude Pro $20, GitHub Copilot $10 personal or $19 company, Replit Ghostwriter $10 a seat. Those are only the tools he uses every week. Drop Enterprise ChatGPT and pick one of Windows or Google and he is still at $110. The long tail is worse: DALL-E, Midjourney, Playground, Lexica, and Stable Diffusion for images; WellSaid, Play.ht, and ElevenLabs for voice; Waymark, Gamma, Tome, and Jasper at $20-plus; Khan Academy tutoring he subscribed to and cancelled. Occasional-use apps add up to about $500 a month, 80–90% of them not worth a standalone sub.

Supply is twisted the same way. Serving one random Waymark visitor costs about 15 cents in AI APIs. Lowest subscription tiers sit at $10 to $30. One $30 payer has to cover roughly 200 free trials. Enterprise customers—Spectrum, Fox, Gray TV, shops that sell video ads all day—retain well. Retail is generate, download, cancel: they never meant to stay. His sketch is a cable bundle: $100 a month, editorially curated access to a hundred apps going on a thousand, baseline quota included, power users upsold. Waymark’s honest slot is ESPN Classic, not ESPN: people need a video every so often, not every day.

The arithmetic is the pitch. If the pack is $100 and Waymark takes one-thousandth, that is 10 cents a subscriber. A million bundles is $100,000 a month, a million dollars a year, plus a light-user audience it could never convert. The United States still has about 50 million cable subscriptions, so a million AI packs is small. At 10 million, share might not scale linearly; the catalog might grow from 1,000 apps to 2,000. Who runs it? OpenAI could publish take-it-or-leave-it tiers by scale and skip a thousand negotiations. Or a Wirecutter-style editor could just say what is in. Developers he talks to all see the same high-churn pattern: plenty of signups, rotten retention. Zero learning curve, he notes, is also zero switching cost. People never have time to get attached.

Torenberg is sympathetic and unconvinced. Ben Thompson has spent years arguing people underappreciate bundle economics, yet SaaS never produced a general pack: each vendor wants the customer relationship as a wedge into more products. Direct-to-consumer works for a power-law few. Disney can command a Plus; CNN+ died; he never bought Discovery+. Labenz answers that he wants the billing relationship with his *best* customers and would happily give up the worst. Cable already has a “log in with your provider” pattern; the app would still know the user. AI’s magic, Torenberg adds, cuts against retention: value arrives instantly, and so does the next app.

Labenz’s counter is the moment. Frontier labs are competing, but softly. There is a Frontier Model Forum, papers still go out, and OpenAI’s charter says that if another credible org nears AGI they will work with it rather than race. Claude 2 and GPT-4 have not triggered that clause. If they are sincere about not cutting safety corners, a rules-based commercial split could take the heat off every subscriber. Nobody would need to panic that the other guy shipped a day earlier. Google’s rushed demo the day before Microsoft’s Bing event is the pattern he does not want in a GPT-5 era. New apps would make the pack more valuable, not steal the checkout.

Torenberg’s Uber–Lyft story is the cold water. The products were the same; venture money subsidized a dual burn. Emil Michael, then Uber’s chief business officer, tried to merge them. Travis Kalanick said they had to kill Lyft; Lyft said they hated Uber. “Businesses just are irrational at the end of the day,” especially when VCs fund take-over-the-world visions. Labenz concedes a likelier shape, visible on Sling TV’s orange pack and blue pack: an OpenAI–Microsoft bundle versus an Anthropic–Google bundle, smaller apps picking a side, probably required to use that side’s models. Less beautiful. The market, he quotes, can stay irrational longer than you can stay solvent. If irrationality is the only blocker, he still sees a chance. They leave it there: let’s see if anyone can be talked into it.