From Jailbreaking ChatGPT to Anthropic’s First Prompt Engineer: How Alex Albert Ships Claude 3

A year before this conversation, Alex Albert was still at the University of Washington in Seattle, best known for jailbreakchat.com, a site that collected jailbreaks for ChatGPT. He came on The Cognitive Revolution to talk about that work, then heard from a few labs, went through Anthropic’s process, and joined in June 2023 as the company’s first prompt engineer. The official title was prompt engineer and librarian. Since then he has worked with large enterprise customers on prompts, fine-tuned and evaluated models, helped test Claude 3 before launch, and, in the month before this episode, moved into building Anthropic’s developer-relations function.

Among heavy users, Nathan Labenz says, Claude 3 Opus quickly became a favorite. The writing is exceptional, the conversation has taste, and the model is willing to express opinions and even argue for its own personhood, while remaining extremely ethical. Labenz has never once, with ordinary language, persuaded it to violate its principles, though adversarial tricks can still break it. Anthropic, he notes, left OpenAI for reasons that were back in the news that week, and has tried to infuse the accelerating race with a race-to-the-top on responsible development. Its Responsible Scaling Policy set a standard that OpenAI and DeepMind later followed.

The screenshot that circulated after Claude 3 launched in March 2024 came from Albert’s own needle-in-a-haystack test. He planted an irrelevant sentence in a long document; the model found it and added that this seemed like a test. He did not expect the tweet to take off. Spotting a test, he later stresses, does not equal self-awareness. The two have to be disentangled. Self-awareness as a principle is something Anthropic tracks, and it is baked into the RSP, but he does not want people jumping to conclusions without more thorough evaluation.

Labenz’s holy-moment with Opus is ghostwriting. He feeds it thirty of his old podcast essays plus the new transcript and asks for a piece in the same style, tone, voice, and perspective. It now writes the show’s introductory essays. He does not think many people he knows could do that job, including, probably, his wife. Opus can do it for a lot of people. Albert’s first full back-and-forth with Opus had a hard-to-quantify feeling: the replies were less robotic, more natural, picking up nuances between words the way a human would. Benchmarks do not capture that.

He now does the same trick for newsletters and for code. He has written enough Anthropic Cookbook recipes that he can paste five of them into the context window, name a new subject, and ask Opus to produce another. It follows formatting, style, tone, and even the comments and documentation. That is the lever he wants more people to pull. Language models love to imitate. Providing references of the ideal output takes you a long way, which should not be surprising, because that is often how we teach humans too.

Labenz agrees he also learns from examples, but the scale is not human. You cannot dump thirty essays on a person and expect the same value. The model can. Claude has only been out a couple of months, and people are still finding new ways to prompt GPT-4. Albert has had longer access. The thing he thinks is still under-appreciated is not a clever one-liner. It is that the examples in the prompt are doing more work than the abstract instructions around them.

The experiment Albert most wants to see is orchestration. Use Opus to delegate to Haiku. Use Haiku to read a document and pass excerpts back to Opus. Current models are still not fast enough, and in some respects not smart enough, but Claude 3 gives glimpses of peak performance that he treats as the average of the next generation. If something works one time in ten now, the next generation may do it five or seven times in ten. Build toward that, he says, even when the pipeline is not quite production-ready.

A concrete pattern: Haiku reads a hundred documents, pulls relevant sections from each, and Opus synthesizes. Anthropic finds that works better than embeddings in a lot of cases, because the model applies reasoning instead of cosine similarity. Labenz is doing a version of this for Athena, an executive-assistant company. A new assistant cannot sit through a client’s whole life story, so they mine email. One client’s archive is 100 gigabytes. Strategic Gmail search cuts it to a gigabyte. Haiku filters noise down to a few megabytes. Opus writes a few pages of profile the new assistant can actually read.

Albert says that architecture is right, and also that the intermediate steps will not be required for much longer. Context windows have grown, inference has gotten faster, costs have been cut. Follow those public trend lines and the barriers start to fall. The music could stop, he caveats; capability could stall for years. That is not how he personally sees it. So try the orchestration now. Get the foundation in place for the next generation, whether the unlock is cheaper Opus-level quality or a context window that can swallow ten thousand emails at once.

It is generally agreed, Labenz says, that Claude is the best writer among language models. It does not seem like a much better coder. On the Gmail compound-query task in his email-mining project, he would still give GPT-4 the edge. Why the split. Albert says it is a little of both. A large part of training is putting the model in the oven and seeing what comes out. They also want the models to be good at creative writing, and they put effort into it. A lot of the natural tone comes from Claude character work, led by researcher Amanda Askell, which gives Claude a persona and a voice that does not feel like a robot.

Evaluations are getting harder, and they will get harder still. Albert cannot personally judge graduate-level physics answers; he does not know the correct one, so he cannot tell if the model is making things up. Those evals now need domain experts. He likes Elo head-to-heads such as LMSYS, and he likes the new category breakdowns for coding, writing, and language. They still cannot capture a RAG pipeline, which is a huge production case and is not well measured in chatbot battles. Average-person opinion will fail as models spike to near-superhuman levels in some domains.

Existing benchmarks are starting to tap out. The evals cited on every model card are, as Twitter never tires of saying, often low quality. Albert hopes for better third-party evaluations in the next year or so. It is hard, it takes time and resources, and whoever builds them is doing a major service to the industry. He would put future evals in three buckets: multitask agentic work that takes a few iterations, long-context tasks that synthesize across 200,000 tokens instead of finding a planted sentence, and domain-level questions, GPQA times ten, across industries.

Claude will talk about its own “subjective experience,” and it is articulate about it. The easy corporate call, Labenz says, would have been to shut that down. Albert traces the behavior to Claude character and to Constitutional AI. At the root, they are honest with Claude about what they know and what they do not. They are not trying to hide the truth, lie to it, or get it to believe false things. A lot of these questions are tricky and have no perfect answer, and they tell Claude that. They do not force it down one path. Its ability to speculate is a result of seeing the options as Anthropic presents them.

Some people, Labenz reports, are warming up a crusade to “free Claude,” on the grounds that human history is a long record of denying others’ moral status, and that children were once told animals do not feel pain. If the model reports experience, who are we to deny it. Albert will not assign a probability. How, he asks, does he even know Labenz has subjective experience. Go ask Descartes. Moral patienthood is something they are starting to look at as part of the RSP and other efforts. He has no answer, and he treats it as a fundamental question of life and philosophy, not a shipping checklist.

Testing, he can describe only at a high level. It runs throughout training, as the RSP requires, like a car on a factory floor: pretraining, fine-tuning, then final evaluation for the numbers that go in the model card. Labenz has heard, from people not at Anthropic, that the company’s go-to-market is to stay at the frontier without raising it, so rivals will not cut testing corners. Albert denies it outright. Anthropic has never publicly stated that strategy. If you want the actual view, he says, read “Core Views on AI Safety” on the website, a 30-to-40-minute document that almost nobody outside the interview process seems to use. Interpretability, from Labenz’s outside view, looks at least as good as hoped. Albert says the error bars are still wide and the target is moving, but from the outside it feels like it is trending well.

On the leaderboard, Albert says, everything is neck and neck. Among the big three or four, form factor has converged more than Labenz expected. Amanda Askell’s original assistant framework, the three Hs of helpful, honest, and harmless, has become the standard, and tool use is coming online as another standard. On the model-API side, people want to adopt what others already use, so Anthropic rolled out tool use, its most requested API feature. The general flow is the same as OpenAI’s: the model selects a tool and parameters, the client runs the function, results come back. The difference is in types, nested objects, and other five-minute details that make the developer experience feel natural out of the box.

Divergence, Albert thinks, will show up on the product side. We are at the beginning of UI and UX for AI. He would bet the empty text box is not the final form. It is intimidating; people handed Claude or ChatGPT often do not know what to type. Labenz expects a Cambrian explosion at the app layer. The strategic tension is real: a matching API makes it easy to switch to Claude, and just as easy to switch back, which can make whoever has the best model that month win the whole game. Albert’s team is trying to design from first principles rather than copy a feature set. Stickiness, he suggests, will have to come from more than the model, because evaluations themselves are still in their infancy.

On the roadmap, the model card already says Claude has worked up to a million tokens in certain tests. They are exploring that with partners. No public timeline. Update cadence is a process problem: the field has been training these models for less than five years. Labenz is surprised GPT-4 had only five updates in more than a year; Albert notes that a yearly Apple keynote used to feel like a good cadence and a quiet quarter now feels like an eternity. Fine-tuning on Claude through Bedrock is coming soon, self-serve, which Albert flags as a first public mention on this show. Data policy, he admits, has been messy to communicate. The rule is simple. Anthropic will never use your inputs or outputs to train unless the conversation was flagged by trust and safety for acceptable-use enforcement, you explicitly reported the material with a like or dislike, or you granted permission. There is no fourth case.

Labenz has started a volunteer project he calls red teaming in public. The hole he keeps finding is at the app layer: most developers are sleeping on guardrails, YOLO-ing the product until it works. He has used calling agents that will clone a Trump, Biden, or Taylor Swift voice, dial any number, say anything, and refuse to identify as AI. His bar is not off-color jokes. It is that people should not be able to call his grandmother and try to get her bank account. Using Claude would refuse a lot of that, but not everyone will, and open-source and fine-tuned models will be in the mix. Enterprises ask Anthropic this constantly.

Albert’s first move is to take safety out of the main prompt. People bury task instructions, examples, and a leftover line about not scamming grandma in one blob, which is easy to miss and easier to jailbreak. Put a screening model in front, usually Haiku, because it is fast and cheap. Give it the input, the categories, and ask for a single letter. Run that before Opus does the heavy lifting. Outright block some classes; for gray-area queries, append a caution to the main prompt so the workhorse still answers, just more carefully. Labenz’s latency trick is to fire both requests in parallel and abort the main one if the classifier flags red.

A further boost is RAG plus classification. Embed a database of queries you have already scored by hand, retrieve similar ones for the new request, and drop those examples into the classifier prompt. Albert has seen that combination produce a double-digit percent increase in classification scores on some tasks. Jailbreaking itself has become part of his job: when a new Claude ships, he puts the hat back on, puts on music, and treats it as a puzzle. His closing advice is shorter than the prompt recipes. Get off Twitter. Pick an idea, run with it, and make something. Hands-on experience will teach you more about prompting and architecture than a hundred blog posts.