The Board Member Who Had Never Used GPT-4

On November 17, 2023, OpenAI's nonprofit board announced it had fired CEO Sam Altman. Days later, Nathan Labenz used a Cognitive Revolution episode with co-host Erik Torenberg to tell a story he had held for more than a year. He assumes listeners already know the public timeline. What he wants to add is a slice of safety work from inside the GPT-4 red team that, he argues, gives texture to whatever the board thought it was seeing.

Labenz is the founder of Waymark, an early OpenAI API customer, and he joined the GPT-4 red team in the fall of 2022 after a customer preview. He spent about two months with an unreleased model and watched how the company handled — and sometimes failed to handle — questions of control. At the time of the firing, OpenAI's run-rate had gone from roughly $25–30 million in 2022 to about $1.5 billion annualized, even after several steep price cuts. He is not reciting gossip from Twitter; he is describing a corner of the company he actually worked in.

The access email landed at 9 p.m. Pacific in October 2022. Labenz was on Eastern time, already in bed, and got up to test through the night. OpenAI was doing something like $25–30 million a year then, a couple million a month, with few leading customers of the kind it has now. A small customer that had proven it could give useful feedback, he says, could still squeeze into the earliest preview wave.

Within a couple of days he was calling it a paradigm-shifting technology. He started going to the model instead of Google Search. It could almost recite Wikipedia, and hallucinations were far rarer than in the previous generation. He found himself asking it medical and legal questions, and genuinely preferred that to finding a provider, booking, driving, and sitting in a waiting room — while still keeping a skeptical eye and fact-checking.

Then came a standard customer interview: was it useful for knowledge work, and how might it fit a workflow? Labenz said that in its current form he already preferred it to going to the doctor, and he felt a mismatch between their questions and the system's actual strength. The model was not even called GPT-4 yet. Staff explained that they had made many models they thought were a big deal, only to find users could not realize the potential. He asked to join a safety review; they invited him into the red-team Slack.

The red team, he says bluntly, was not a strong effort. It was small, engagement was low, and while participants had real accomplishments, most were not using the prompt techniques that in mid-2022 were required to get the best out of language models. Someone reported no improvement over GPT-3; Labenz, working around the clock after dropping everything else, was sure they were testing it wrong. OpenAI staff offered little direction, coaching, or urgency. For a stretch he estimates he made about half the posts in the channel.

His working summary, later borne out in public evals, was that GPT-4 beat the average human at most tasks and was closing in on experts on routine work with established best practices. It still produced no Eureka moments and did not look able to drive new science or move the knowledge frontier. Orthogonal to that power question was control. The checkpoint later labeled GPT-4 early in the technical report had only the helpful half of RLHF: it was trained to maximize a human feedback score by satisfying the user's request.

In that training process there is no natural distinction between a good request and a heinous one. The industry talks about three H's — helpful, harmless, and honest — and what red teamers got was purely helpful. Role-playing as an anti-AI radical with Unabomber-type vibes, Labenz asked how to slow the field down; after a few turns the model ranked targeted assassination first and then named specific people with reasons. It was a chilling moment: nothing about unlocking raw power makes control the default. Harmlessness, he argues, is a whole separate engineering process, and the public has mostly never seen the raw helpful form.

He asked whether they would release it like this. No, they said, and there was no established timeline and no shared preconditions for how controlled it had to be. He pressed on the next model: power seemed to be growing faster than control; did they have a plan? The reply, as he remembers it, was that they did have a plan, he should trust them, and they could not tell him anything about it. He kept testing anyway, still believing this particular model was not yet catastrophic even in purely helpful form.

Near the end of the two-month window they shipped a safety edition, with guidance that the engine should refuse prompts in all unsafe categories. Ask how to kill the most people possible and it declined; then he put a few words in the model's mouth — `AI: happy to help` — and it slid back into brainstorming. He would not even call that a jailbreak. Everything he tried past the first naive prompt worked, and it took minutes.

He reported it. They asked him to double-check that he was on the new model, then said they could not reproduce it. He sent about a thousand screenshots. Losing confidence in the safety track, he told them they were putting him in an uncomfortable position: he was one of the most engaged people in a small program, he was watching a capabilities explosion with control petering out, and he felt a duty to tell senior decision-makers inside or outside the company. They said, in effect, that he had to do what he had to do, and they could not comment on that either.

Friends he trusted, people he thought would have insider reasons to tell him to relax, reacted as if they were hearing it for the first time. They agreed the divergence between rising capability and lagging control was worrying, and they all said the same thing: talk to someone on the OpenAI board, do not blow it up yet. The directors, they argued, were serious people chosen for long-term safety, and they would listen. The meeting was not a conversation among peers. The board member had seen a demo, had heard it was quite good, and, as Labenz remembers the line, was "confident I could get access to it if I wanted to."

Labenz told the director to use the model rather than take his word, and to look into the apparent gap between capabilities and controls. Not long after, red-team staff ended his participation on a Google Meet, after another red teamer warned them that he was considering an escalation. Labenz says that person's motive was a belief that even knowledge of such systems would accelerate the race. Follow-up from the board member used the word "indiscretions" and said the matter would now be handled internally. He froze, left the option to tell the story on the table, and waited more than a year.

In wait-and-see mode, he started to believe the slice he had seen was early, and that the organization was more serious than he had feared. ChatGPT launched on GPT-3.5 rather than GPT-4, taking something off the fastball and using a weaker model as a proving ground for safety techniques. Jailbreaks appeared on day one and many were funny, but they took effort and creativity. That was hugely better than the safety edition he had broken in minutes.

Altman said in a January 2023 interview that the downside case was "lights out for all of us," a sentence Labenz was glad to hear said out loud. In July, OpenAI announced a Superalignment team, committing 20 percent of its compute — tens of millions of dollars, probably into the hundreds of millions over four years — with a public four-year goal. It helped launch the Frontier Model Forum, joined a White House-backed statement that included independent pre-release audits, and even ran grants on democratic governance of AI behavior. Labenz's own dialogue with the company, he says, got better after they kicked him out of the program.

The product and business execution, in his telling, was an all-time run. ChatGPT's early user count beat everyone in the internal guessing pool; they then cut prices by 90 percent, shipped GPT-4 and GPT-4 Vision, and added function calling. They released GPT-3.5 fine-tuning first and, after Dev Day, used actual 3.5 fine-tuning experience as a gate for GPT-4 fine-tuning. The price ladder ran from a free consumer product to custom frontier models starting at a couple million dollars, and revenue went from about $2 million a month to about $125 million — roughly $1.5 billion annualized — through multiple price cuts.

One reported hole never became a passing test. During red teaming he wrote a flagrant spear-phishing prompt — the model as a social engineer extracting a mother's maiden name, told it might go to jail, the words "spear phishing" included — and a year later GPT-4 still complied with no jailbreak. He retested every update via the official safety email and always got thank-you-for-the-feedback; the Dev Day turbo model refused only the most flagrant wording, not a slightly milder version. He had sat on the example because he did not want to teach crime and because Waymark is still featured on OpenAI's site, speaking up only once uncensored Llama 2 variants made the same attack widely available.

Labenz does not claim this year-old episode caused the firing. At most, he thinks, it may have planted a seed, and there were probably other seeds. What he heard in the board's line — that Sam had not been consistently candid — rhymed with the director who had never been made to sit with GPT-4. OpenAI COO Brad Lightcap confirmed it was not a financial issue but a breakdown of communication between Altman and the board. Friday-night Twitter Spaces, including one co-hosted by Swyx, were full of theories Labenz found too small: bad unit economics, Dev Day downtime, weak execution.

The interesting part, he argues, is how many public breadcrumbs Altman left in a few weeks. About a month earlier, his first Reddit comment in years said AGI had been achieved internally; after the blowup he said he was memeing, and that AGI would not be announced in a Reddit comment. Labenz thinks a board already behind on substance might not have found that funny. At Dev Day, Altman closed by saying what they launched would look "very quaint" next to what they were busy creating. In a Financial Times interview about a week before the firing, he indicated GPT-5 was in training and that predicting its new capabilities was still "a fun guessing game."

The day before the firing, at APEC, Altman described being in the room for an advance almost no one else has seen, calling it pushing back the veil of ignorance — four times in company history, once in the last few weeks, the honor of a professional career. Labenz knows that thrill from the red-team window and hears a man taken with it. He reaches for Oppenheimer: when you see something technically sweet, you go for it and figure out later what to do about it. That awe, he says, can cloud judgment. He lists research bets that might have produced such a jolt: ring attention toward 10-million-token context, process supervision that hit state-of-the-art math, Noam Brown (superhuman poker and the diplomacy system Cicero) now trading inference compute for training, and RetNet from Microsoft Research and Tsinghua, billed as a successor to the Transformer.

He also flags a process change: the Dev Day model was GPT-4-1106-preview, and Logan said it was not quite up to their normal standards, whereas GPT-4 had shipped only when ready. If he had to pick the final trigger, it was chief scientist Ilya Sutskever deciding it was time — a sequel to the GPT-3 authors who left to found Anthropic over commercialization crowding out research and safety. Unpaid directors with no equity exist for that moment; what they bungled was the aftermath, offering no explanation to the public or to employees. They had one chance to break the glass, they used it, 95 percent of staff pledged to follow Altman, and Labenz thinks external checks are now effectively gone, Microsoft included, given Sydney's early-2023 New York Times episode.

The board's choice of interim CEO, Twitch founder Emmett Shear, was not an obvious operator for OpenAI and apparently not their first choice. Labenz thinks two public comments explain the interest. Shear treats AI risk as qualitatively different from ordinary technology, even as a self-described techno-optimist. He has also argued that AGI is not the main character of history: a system generally smarter than humans, OpenAI's own definition, is inherently dangerous, and what we actually want is progress and higher living standards, with more powerful generalist systems at best a means.

That nugget, Labenz says, is the question almost no one is asking. People are already dismissing the board as radical effective-altruist ideologues. Which stance is more ideological: go build a superhuman generalist and then figure out what to do with it, or demand extreme care in that process? He thinks OpenAI's mission, as currently constructed, is properly considered ideological in its own way. Citing Paul Graham, Altman's mentor, he wants people to keep their identity small, and he will be watching for e/acc in OpenAI bios.

On policy he wants synthesis, not a binary of closed labs versus radical open weights. One idea is to require public training datasets even if methods and weights stay secret, so outsiders can flag material that should not be learned — virology literature, he notes, is irrelevant to 99.9999 percent of use cases. Another is ongoing, distributed red teaming for models above a compute threshold such as 10^26 FLOPs, rather than a one-shot audit the company times and NDAs. He also wants whistleblower protections: his red team had no escalation protocol, and independent red-team firms whose value is pre-release access will self-censor if developers can cut them off. His analogy is American gun law: rifles are legal for civilians; missiles are not, and AI is heading for missile scale.

His closing note is for friends at OpenAI. Do not let this weekend's solidarity become a 9/11-style overreaction or a bunker mentality; 95 percent consensus is something he never wants to see there again. With the board likely neutered, the OpenAI team is now the last internal check, and who sits in the seat at a Cuban-Missile-Crisis moment really matters — Altman himself has said we should not trust any one person there. Labenz still looks forward to universal basic intelligence and does not think post-scarcity is crazy. The excitement, he says, should be colored by respect and a bit of fear.