Theo on Fable’s Safeguards, Data Retention, and Transparency

Fable is an impressive model, but I am concerned about Anthropic’s restrictions: who can access it, when requests fall back to another model and how those interventions affect evaluation. Strong benchmark results can include an Opus 4.8 fallback, and a reported ProgramBench run saw the model refuse all 200 tasks.

A later insert provides context: while I was recording, news arrived that the US government had asked Anthropic to restrict the models. The main discussion was recorded before that suspension and examines the safeguards Anthropic had already introduced.

Anthropic describes Fable 5 and Mythos 5 as the same underlying model, with some safeguards lifted for trusted Mythos users. I compare them to two doors into the same place, one with additional checks. My idea that additional reinforcement learning separated the earlier Mythos preview from the later model is an assumption.

Anthropic said some requests would be routed to Opus 4.8 because of risks in areas such as cybersecurity. It said the safeguards were conservatively tuned, could catch harmless requests and triggered in fewer than 5% of sessions on average. In my experience, rerouting did sometimes happen on benign work.

For example, I asked for help with a cryptographic puzzle and saw a fallback followed by a response that stopped. I find that frustrating: a benign request can be rerouted and then fail under the fallback model’s restrictions too. Anthropic acknowledged that false positives were possible and said it intended to improve the classifiers.

The company described separate classifiers for potential misuse, including cybersecurity, biology, chemistry and model distillation. For the visible fallback categories it said users would be informed and billed for the model handling the request. I would prefer an explicit choice before an unexpected fallback incurs a different charge.

Restrictions can also change the meaning of a benchmark: a low result on a security evaluation may reflect safeguards or refusals rather than the underlying model’s raw capability. Some areas involve serious dual-use risks, so both the safety purpose and the measurement limitation deserve attention.

A separate issue is data retention. Anthropic said business traffic on Mythos-class models would require 30-day retention on first- and third-party surfaces, even where customers might otherwise expect zero data retention. It said the data would not be used to train new Claude models or for non-safety purposes, and would normally be deleted after 30 days.

The company also described exceptions for safety investigations or legal obligations. I worry about the longer retention periods mentioned for flagged material and what they mean for customers with sensitive data. My interpretation of how different policies interact, including whether any training exception applies, is uncertain and should not be treated as a legal conclusion.

I found that the system card I had downloaded differed from the version online. Initially I accused the company of changing the document without noting it; later in the video I corrected myself after finding a changelog. The substantive change still matters to my criticism, but the initial claim about an absent changelog was wrong.

The earlier system card described invisible safeguards for requests targeting frontier language-model development. It listed methods such as prompt modification, steering vectors or parameter-efficient fine-tuning to limit effectiveness, and estimated that about 0.03% of traffic would be affected. Those were the document’s described interventions and estimates.

I interpreted that as a serious trust problem: a user could pay for an answer without knowing an intervention had affected it. My example of a prompt being rewritten to deliberately make code worse was hypothetical; the text I show does not establish that this exact rewrite occurred.

Researchers raised a related evaluation concern: if an intervention is invisible, an evaluator may not know whether a failure reflects the model’s capability or a safeguard. I share that concern, particularly where legitimate research resembles a restricted category. That uncertainty is different from proving a particular answer was deliberately sabotaged.

I discuss criticism from Redis creator Salvatore Sanfilippo and other researchers, who objected to restricting useful research and to the implications for openness. These are their arguments and mine; reasonable people can disagree about how to balance competitive, safety and transparency concerns.

I then found Anthropic’s update and corrected my earlier statement about the missing changelog. The company said it was changing frontier-model-development safeguards to be visible: flagged requests would fall back to Opus 4.8, and API users would receive a refusal reason, with server-side fallback to follow.

Anthropic said invisible safeguards were initially chosen because they could be narrowly targeted while the company worked on robustness, but acknowledged that this was the wrong transparency tradeoff. It warned that making safeguards visible could lead to more false positives and said it was tuning cyber and bio classifiers.

I welcome the reversal but remain unhappy with the earlier hidden intervention and think affected customers deserve compensation. Anthropic also described feedback and appeal paths for requests people believe were mistakenly flagged. This discussion records the rollout as described at the time, not a statement of current product behavior.

Why did Anthropic choose this approach? I offer a theory based on its research into models assisting AI researchers. It is speculation, not an established explanation from Anthropic. The company’s stated rationale was the safety and transparency tradeoff described in its update.

The research chart examined 129 internal sessions at points where a researcher’s next step had room for improvement. Mythos Preview’s alternative was judged better in 64% of those selected cases. This does not mean the model was better than researchers on average; the cases were chosen because the human direction was already improvable.

I speculate that proprietary research information may have entered the model and influenced the company’s decision. I do not provide evidence proving that happened, that the information can be extracted, or that it caused the safeguards. Those are conjectures rather than findings of the chart.

John Ready’s article frames invisible degradation as a supply-chain concern: as more ordinary software work involves AI models, developers may not know why an answer was poor. I agree that undisclosed interventions can make evaluation and trust harder. I am glad the specific approach was changed, but remain concerned about the precedent.