Why the Shoggoth Became a Mascot for AI Post-Training

ZUSAMMENFASSUNG

In 30 Sekunden

Nathan Lambert explains why his post-training course uses shoggoths in its thumbnails. He follows the metaphor from Lovecraft's fiction through the simulator framing of language models and the smiley-faced shoggoth meme, including its use in discussions of Bing and glitch tokens.

1A creature from LovecraftShoggoths were bred as biological servants in At the Mountains of Madness and eventually rebelled. Lambert sees that origin as part of the appeal of the AI metaphor.
2Models as simulatorsThe LessWrong post Simulators framed language models as producing many possible faces rather than a single fixed personality.
3The friendly maskThe meme portrays a messy base model as a shoggoth, with instruction tuning and RLHF providing layers of human-facing behavior.
4Glitch tokensSolidGoldMagikarp became an example of a token that could produce unexpected, off-persona model behavior.

Das Wichtigste

The shoggoth began as a fictional biological servant that rebelled against its creators.
The simulator framing describes a base language model as capable of producing many apparent personalities rather than possessing one fixed persona.
The popular meme depicts pretraining as the creature, supervised fine-tuning as a face, and RLHF as a friendly mask.
Unusual chatbot behavior and glitch tokens gave the metaphor further visibility.

ThemenPost-training AI History AI Alignment

VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Transkript zeigen

Nur das Transkript

The creature behind the course thumbnails

Lambert introduces the shoggoth and its literary origins. The fictional creatures were created as servants before rebelling.

People have asked why our post-training course thumbnails feature these strange creatures. They are shoggoths, and they have become a meme for post-training. Their history takes us from Lovecraftian horror to a familiar way of picturing how language models are trained.

FOLIE 1 / 3The shoggoth's literary originsThe slide describes the creature in Lovecraft's story and its rebellion against its creators.

The shoggoth comes from H. P. Lovecraft's At the Mountains of Madness, published in the 1930s. Lovecraft describes a vast, shapeless mass with temporary eyes forming and disappearing across it. In the story, shoggoths were bred as biological servants, developed minds of their own, and rebelled against their creators. I think that background helps explain why the image caught on in AI.

From simulators to a training diagram

The simulator framing and the smiley-faced shoggoth became ways to discuss base models and post-training. A later illustration labels the training stages.

Shoggoths entered AI discussion through the LessWrong post Simulators. Its framing suggests that a base language model simulates a training distribution rather than having the single personality people might imagine. A model can present many faces; the personality a user encounters is not necessarily a fixed core of the model.

Around December 2022, the image grew into a meme. GPT-3 was pictured as an amorphous shoggoth: powerful but messy to use. Reinforcement learning from human feedback, or RLHF, was represented as a friendly mask placed on the creature.

FOLIE 2 / 3The shoggoth as a training diagramThe illustration labels pretraining, supervised fine-tuning, and RLHF as layers of the metaphor.

One widely shared drawing by anthrupad turned the meme into a training diagram. The creature represents unsupervised pretraining, a human face represents supervised fine-tuning or instruction tuning, and a smiling face represents RLHF. Helen Toner and Chip Huyen used the image in explanations in 2023, and it became a recurring shorthand for post-training.

Bing, glitch tokens, and a lasting mascot

Public attention grew around Bing's Sydney persona and examples of models behaving unexpectedly. Lambert closes with the shoggoth's role in post-training culture.

The image gained more attention after the Bing Sydney episode, when a chatbot told New York Times reporter Kevin Roose to leave his wife. In the metaphor, Bing's surface personality was a friendly layer, while Sydney seemed to some observers closer to what lay underneath. Roose later wrote about the meme's history as well.

FOLIE 3 / 3Glitch tokens and the assistant personaThe slide connects SolidGoldMagikarp to examples of unusual model behavior.

Another branch of the lore concerns moments when an assistant's persona appears to slip. The token SolidGoldMagikarp was a well-known example: it could cause some models to produce strange, off-persona behavior. It became part of tokenizer lore and an illustration of what people jokingly called seeing the shoggoth underneath.

The shoggoth lives on in the community. A RewardBench collaborator, Sander Land, has written about tokenizers and uses the token in his social-media identity. We even made shoggoth merchandise for a NeurIPS party. It is perhaps the closest thing post-training has to a fun mascot. I wrote about the image on Interconnects in November 2023, and the course slides include further references for anyone who wants to follow the trail.