30 秒でわかる
Nathan Lambert explains why his post-training course uses shoggoths in its thumbnails. He follows the metaphor from Lovecraft's fiction through the simulator framing of language models and the smiley-faced shoggoth meme, including its use in discussions of Bing and glitch tokens.
- 1A creature from LovecraftShoggoths were bred as biological servants in At the Mountains of Madness and eventually rebelled. Lambert sees that origin as part of the appeal of the AI metaphor.
- 2Models as simulatorsThe LessWrong post Simulators framed language models as producing many possible faces rather than a single fixed personality.
- 3The friendly maskThe meme portrays a messy base model as a shoggoth, with instruction tuning and RLHF providing layers of human-facing behavior.
- 4Glitch tokensSolidGoldMagikarp became an example of a token that could produce unexpected, off-persona model behavior.
ポイント
- The shoggoth began as a fictional biological servant that rebelled against its creators.
- The simulator framing describes a base language model as capable of producing many apparent personalities rather than possessing one fixed persona.
- The popular meme depicts pretraining as the creature, supervised fine-tuning as a face, and RLHF as a friendly mask.
- Unusual chatbot behavior and glitch tokens gave the metaphor further visibility.
チャプターごとに整えた全文を読む
The creature behind the course thumbnails
Lambert introduces the shoggoth and its literary origins. The fictional creatures were created as servants before rebelling.
People have asked why our post-training course thumbnails feature these strange creatures. They are shoggoths, and they have become a meme for post-training. Their history takes us from Lovecraftian horror to a familiar way of picturing how language models are trained.
The shoggoth comes from H. P. Lovecraft's At the Mountains of Madness, published in the 1930s. Lovecraft describes a vast, shapeless mass with temporary eyes forming and disappearing across it. In the story, shoggoths were bred as biological servants, developed minds of their own, and rebelled against their creators. I think that background helps explain why the image caught on in AI.
From simulators to a training diagram
The simulator framing and the smiley-faced shoggoth became ways to discuss base models and post-training. A later illustration labels the training stages.
Shoggoths entered AI discussion through the LessWrong post Simulators. Its framing suggests that a base language model simulates a training distribution rather than having the single personality people might imagine. A model can present many faces; the personality a user encounters is not necessarily a fixed core of the model.
Around December 2022, the image grew into a meme. GPT-3 was pictured as an amorphous shoggoth: powerful but messy to use. Reinforcement learning from human feedback, or RLHF, was represented as a friendly mask placed on the creature.
One widely shared drawing by anthrupad turned the meme into a training diagram. The creature represents unsupervised pretraining, a human face represents supervised fine-tuning or instruction tuning, and a smiling face represents RLHF. Helen Toner and Chip Huyen used the image in explanations in 2023, and it became a recurring shorthand for post-training.
Bing, glitch tokens, and a lasting mascot
Public attention grew around Bing's Sydney persona and examples of models behaving unexpectedly. Lambert closes with the shoggoth's role in post-training culture.
The image gained more attention after the Bing Sydney episode, when a chatbot told New York Times reporter Kevin Roose to leave his wife. In the metaphor, Bing's surface personality was a friendly layer, while Sydney seemed to some observers closer to what lay underneath. Roose later wrote about the meme's history as well.
Another branch of the lore concerns moments when an assistant's persona appears to slip. The token SolidGoldMagikarp was a well-known example: it could cause some models to produce strange, off-persona behavior. It became part of tokenizer lore and an illustration of what people jokingly called seeing the shoggoth underneath.
The shoggoth lives on in the community. A RewardBench collaborator, Sander Land, has written about tokenizers and uses the token in his social-media identity. We even made shoggoth merchandise for a NeurIPS party. It is perhaps the closest thing post-training has to a fun mascot. I wrote about the image on Interconnects in November 2023, and the course slides include further references for anyone who wants to follow the trail.
同じチャンネルの記事
16:48EN
Nathan Lambert· 2026年11月2日Nathan Lambert on How Open Models Can Compete With Frontier LabsNathan Lambert · Allen Institute for AIOpen models are gaining use and closing capability gaps, but their builders need a sustainable economy and advantages beyond price.
48:04ENNathan Lambert· 2026年10月26日Nathan Lambert on Open Language Models: History, Competition and RiskNathan Lambert · Allen Institute for AIA tour of open weights, research infrastructure, distillation, adoption and the choices facing model developers.
46:42ENNathan Lambert· 2026年10月25日Nathan Lambert and Florian Brand on Kimi K3.1, Cyber Risk and AI AgentsNathan Lambert, Florian Brand · Allen Institute for AIThe Interconnects hosts discuss mixed model evaluations, a wave of security disclosures, and what a personal agent can do in everyday life.
32:56ENNathan Lambert· 2026年8月6日Nathan Lambert: An Introduction to AI Character TrainingNathan Lambert · Allen Institute for AIHow post-training shapes a model’s behavior, from constitutions and model specs to experiments with personality and activation steering.
32:11ENNathan Lambert· 2026年8月5日Nathan Lambert on How Language Model Evaluation Has EvolvedNathan Lambert · Allen Institute for AIFrom few-shot prompts and pass@k to agent sandboxes, noisy scores and benchmark contamination.
13:53ENNathan Lambert· 2026年8月4日Nathan Lambert Answers Six Questions About Post-Training RecipesNathan Lambert · Allen Institute for AIA practical discussion of specialist models, KL penalties, supervised data, reinforcement learning and business tradeoffs.