1.5 Million GPT-4 Fairy Tales Let a 33-Million-Parameter Model Out-Reason GPT-2

Here is a fill-in that should not be hard. Lily likes cats and dogs. She asked her mom for a dog. Mom said no, so she asked for a __. The nearby noun, and the one that already appeared twice, is dog. GPT-2 XL, with 1.5 billion parameters, still prefers dog. A TinyStories model with 33 million parameters—about forty-five times smaller, and only 2% the scale of GPT-2—completes cat. The gap is not charm. It is a claim about how language models spend a finite brain.

In June 2023 Nathan Labenz sat down on The Cognitive Revolution with the two Microsoft Research authors behind that result, Ronen Eldan and Yuanzhi Li. They had built TinyStories, a dataset of about 1.5 million children's stories generated so that an advanced three-year-old could understand every word, then trained models from 1 million to 33 million parameters on it. The small networks not only tell fluent stories. On basic exclusion and negation tests, they beat a much larger predecessor trained on the open web.

Ronen came from pure math and hit the same wall a lot of new LLM researchers hit: ideas are cheap, and checking one almost always means a GPU-heavy experiment. Train a small model and it will not generate coherent English, so the experience is nothing like using a real LLM. Train something BERT-sized and you get downstream-task scores that still do not feel like ChatGPT. The only way to get the real experience, he says, was an experiment with tons of GPUs.

Yuanzhi Li had spent seven or eight years on machine-learning theory, much of it in vision, where CIFAR-10 and MNIST held about 50,000 images. You could train in a day, and the results still reflected what large models did. Language flipped that. He had been reading blog posts that said it was impossible to do a machine-learning PhD without something like eight A100s, and that perhaps only 1% of PhD students have that compute. They wanted the old iteration speed back, without giving up authentic natural language for toy arithmetic. TinyStories is their bid for a language CIFAR-10.

Nathan reads a sample straight from the paper so the dataset stops being an abstraction. Tom has a big pot of soup and wants to share it with Jane. She takes a spoonful and makes a face. The soup is __. Their 28-million-parameter model continues without a hitch: the soup is very bitter, Tom apologizes, he finds bread and cheese, and he tells Jane they are not bitter, they are sweet and yummy. A model that small produces a complete children's scene with emotion, a turn, and dialogue.

The design idea, Ronen says, was already sitting in human development. Young children speak English that is mostly coherent, which means the knowledge required to speak coherent English is not that large. Constrain every example to what a small child can understand, spread the knowledge that child has as widely as possible, and put it in story form so grammar, facts, and a little reasoning sit in the same short text. GPT-4 wrote most of the stories; GPT-3.5 wrote some. GPT-4 is clearly better at the job.

Diversity is the whole game. If you just ask GPT-4 for a short story, even at temperature 1, about one fifth of the stories will be about children scared of the slides at the park. They ran that experiment. Unconditioned, the model dumps the most likely plot on the internet. So they built a vocabulary of about 2,000 words a three-year-old would know, sampled a random verb, noun, and adjective, and asked GPT-4 to weave them into a creative story. They made about a million such calls and ended up with roughly 1.5 million stories, plus extra requests for a plot twist, a bad ending, or dialogue.

The GPT-4 versus GPT-3.5 gap shows up on awkward triples such as ancient, thunder, and sad. GPT-4 usually folds them in fluently; GPT-3.5 gets the words on the page without a story that holds. A base model with no RLHF may not even treat "write me a story" as an instruction—the plausible web completion might be "I don't feel like it." Nathan then does the arithmetic the authors confirm: from 2,000 words, the space of verb-noun-adjective bags is about a billion; a million stories is roughly a thousand-to-one undersample. At about 300 tokens a story, rebuilding the set at GPT-4 retail would cost on the order of $10,000. If a trained model later writes a fluent story from a triple it never saw, it is not retrieving a memorized plot.

The authors do not mean theorem-proving when they say reasoning. They mean global consistency: the next word should agree with something far back in the prompt, not just with the previous few tokens. They split that skill into what they call logical primitives. Grammar is first: after "Amy wanted," the next word is probably "to," and you need no world knowledge. Facts and local semantics come next: "Jack was hungry, so he went looking for some…" wants food. Only then does exclusion show up. In the Lily puzzle, a grammar-only model knows it needs a noun; a context-tracking model sees cat and dog and picks dog, which is more frequent and more recent; a model that can eliminate knows mom already said no to a dog, so the leftover is cat. GPT-2 XL stops at step two.

That is also their picture of emergence, a word they treat as poorly defined. There is no sharp phase change, but qualitative abilities do appear as you scale. A 1-million-parameter TinyStories model can hardly generate a coherent story; at 10 million, almost all of them are coherent. Reasoning prompts all fail between 1 and 5 million parameters and almost all succeed around 30 million. On the open web the incentives are inverted. Yuanzhi argues that GPT-2 is not big enough to drive loss all the way down, so it has to choose, and web text is stuffed with knowledge. Ronen's conjecture, which he did not test but is "pretty sure" about, is that the model learns that Joe Biden is president of the United States long before it learns to complete the dog-and-cat sentence.

Open a random Wikipedia page, Ronen says, and ask of every word what ability you need to guess the next one. Grammar is needed every three or four words; reasoning, Yuanzhi estimates, only once every 20 to 30. That rarity is why emergence feels magical. If Bob does not like sweet food and the model still completes "candy" instead of "pizza," it looks as if the model understands nothing—but that is one word in ten or twenty, maybe less than a 10% loss difference. In image classification, going from 90% to 95% accuracy is not called an emergent ability. In language, that last 5% can be the handful of globally consistent tokens that decide whether the model knows what it is talking about.

Human children look inverted because the objective is inverted. Nathan's kids lock onto ice cream and treat grammar as optional. Children, Yuanzhi says, are running something like RLHF with parents as the reward model, and parents are extremely robust to grammar mistakes. A language model is penalized for bad grammar on almost every token and for global inconsistency only rarely, so it studies grammar first. GPT-2 XL's snowman continuation makes the point: after a dog knocks the snowman down, the model riffs sentence by sentence—red real hair, a kiss on the eyes, "he has saved the world"—with what Ronen calls a good sense of humor and no consistency beyond the previous line. The dataset implication is a mixing problem. Knowledge is fuel, but too much of it and a small model memorizes. Ronen doubts you can teach facts and logic in separate modules and have the network combine them the way a student cramming for the SAT does; Nathan's MosaicML interview points the same way, because even huge client datasets still need a mix of general pretraining or the model catastrophically forgets.

The paper also splits a Transformer along an axis practitioners argue about constantly: number of layers versus width of a layer. None of this, Ronen warns, is well established. Still, the working picture is that depth tells you how many times information can percolate from token to token. Each global-attention layer is one chance for an instruction, a negation, or a constraint to soak into the rest of the sequence. The cat-and-dog completion is two hops. First the token "not" has to enter "dog," so the model represents "dog is not allowed." Then that combined feature has to enter the set {cat, dog} so only cat remains. Two rounds of attention, two logical leaps. A shallower net can look grammatically fine and still fail the exclusion.

Facts look more like a table. For "the capital of France is," you do not need many rounds of global attention. You bind "capital" to "France" and look up Paris. Here dimension matters, because a larger vector space can hold more entities and a wider MLP can store more of those pairs. Depth is for multi-step pointers; width is for inventory. Nathan's gloss is that attention is not immediately transitive: if the current token looks at a token that itself looks further back, you need a second round to move the two hops.

It is not a hard cap. Yuanzhi notes that you can simulate two-layer leaps inside one layer if you enumerate combinations, at the cost of growing the intermediate size from n toward n squared. Their earlier Microsoft Research paper on a synthetic task they call Lego makes the layer-versus-steps link very direct, and a follow-up, not in TinyStories, found that models invent sophisticated ways to pack multiple leaps into a single layer. The same deflationary story, Ronen thinks, may explain why code pretraining seems to help reasoning. In Lego, a pretrained BERT did not look as if it had "learned to reason" in a deep sense. It had grown simple attention heads that look at a relative position or compare two tokens precisely. The components coding needs are already sitting there. They are not a hidden reasoning circuit.

The interpretability results are the part of the paper that most surprised Nathan. In a single attention block, the heads split into a clean dichotomy: some care only about distance between tokens, others only about semantics, with hardly any mix. The distance heads look almost exactly like ALiBi, the decay-with-distance scheme now behind long-context models such as Claude's 100K window and MosaicML's 65K release. Short-range distance heads pick up grammar; longer ones keep global consistency, or let a later "Alice" find an earlier "Alice." ALiBi is an engineered positional prior. In TinyStories the same pattern grew on its own, because the model had to discover multi-scale distance attention from absolute positions.

The semantic heads do a different job. The most striking one attends symmetrically to main-character names—Tom and Lucy, for example—as if the network were tagging who the story is about so later sentences do not wander. Other heads latch onto objects, so "the" or "a" can point at banana rather than invent a new noun. Nobody should have expected that cleanliness. The default guess is that every head attends a little to everything. They have only really checked one attention block so far, but even that is enough to make the point: when the model is small, you can actually give meaning to heads.

For neurons they reused an idea from a 2015 paper by Li et al., *Visualizing and Understanding Neural Models in NLP*: run text through, collect the tokens that most activate each neuron, and look for a theme. On GPT-2 XL they could find no pattern—the same unit fired on nouns sometimes, verbs sometimes, nothing shared. In the small model, one neuron reliably activates when the main character is introduced. That is what a programmer would write first if the task were to autocomplete a story: find the name, because you will need it everywhere. They have not fully confirmed it, but there is probably an attention head that then reads that neuron's output, and together the two copy the name around the generation. Scale the Transformer up and the luxury returns: five heads can simulate one, three layers can do one layer's work, and the coordinate-aligned solution disappears. Interpretability, Ronen says, is a side effect. Most solutions to the same problem are one big mess, and that is where the entropy is. A small net does not have enough nonlinearities to hide there.

The open question hanging over the whole project is transfer. If an architecture wins on TinyStories, why should it win on a trillion-token LLM? Ronen names the Chinchilla paper and OpenAI's scaling-laws work as hints that some phenomena are universal, then immediately says they have only a few indications and that the question is completely open. They hope it is universal anyway, for energy's sake and so PhD students can do language-model research on a laptop GPU. Yuanzhi's next step is to grow TinyStories toward elementary-school knowledge, aiming at a model of perhaps 300 million parameters that can talk about third-grade content. Train that in a day or two on a laptop GPU, and the old CIFAR loop might actually come back.

Even if the findings never fully scale, Ronen still wants the playground. What we are watching in LLMs, he says, is a mathematical miracle: gradient descent plus a clever but still simple architecture, fitted to a huge corpus, does not merely memorize text. It synthesizes new content and shows signs of reasoning. TinyStories is a compact version of the same miracle. It is less exciting than GPT-4, but it is a place to start building the mathematical foundations for why neural nets generalize at all.

Asked for interpretability work that inspired them, they point to their own Lego paper, which found attention heads that copy matching tokens and others that do simple deduction, and to *Transformer Feed-Forward Layers Are Key-Value Memories*, which gave some neurons in BERT-scale models a readable role. The theory, Ronen thinks, is still in its infancy. We have a limited understanding of the human brain; there is no a priori reason a gradient-descent solution will be anything but messy; he is not optimistic that we will interpret large models well enough to control them from the inside. Yuanzhi's alternative is closer to psychology: talk to the model the way the Sparks of AGI paper did, and map what it likes, what it fails at, and when it breaks. Ronen catches that with an older technology. Humanity got a lot of mileage out of horseback riding without knowing what every neuron in the horse was doing. The Mongols did not understand equine biology, and they still used the horse reliably. Tame the behavior. Do not wait to read the cells.