In 30 seconds
3Blue1Brown explains how text tokens are associated with vectors and illustrates relationships through vector arithmetic, while such analogies depend on the particular embedding.
- 1EmbeddingA token is associated with a vector in a high-dimensional space.
- 2Vector relationshipsSome word relationships appear as differences between vectors.
- 3An illustration, not a ruleThe arithmetic depends on the embedding and does not always work.
Takeaways
- Tokens can be represented by high-dimensional vectors.
- Learned vector geometry can reflect relationships between words.
- Analogy arithmetic is illustrative and does not universally hold.
Read the edited text by chapter
Tokens and vectors
A short 3Blue1Brown explanation asks how word vectors can reflect relationships between concepts. It is an excerpt from a longer video about language models.
A short 3Blue1Brown explanation asks how word vectors can reflect relationships between concepts. It is an excerpt from a longer video about language models.
A language model divides text into pieces called tokens and associates them with vectors: long lists of numbers. An embedding can be imagined as a direction or position in a high-dimensional space.
Relationships in space
Some learned embeddings encode relationships in their geometry. The example in the video says that a difference between gender-related word vectors can move the vector for “uncle” near that for “aunt.”
Some learned embeddings encode relationships in their geometry. The example in the video says that a difference between gender-related word vectors can move the vector for “uncle” near that for “aunt.”
Limits of analogy
The narrator gives another historical analogy involving Italy, Germany, Hitler and Mussolini. Such arithmetic illustrates a pattern in a particular embedding space; it is not a reliable rule for every model or pair of words.
The narrator gives another historical analogy involving Italy, Germany, Hitler and Mussolini. Such arithmetic illustrates a pattern in a particular embedding space; it is not a reliable rule for every model or pair of words.
More from this channel
40:44EN
3Blue1Brown· Oct 16, 2026Grant Sanderson: There's never been a better time to go into mathGrant Sanderson, Constantin Kogler · 3Blue1BrownStudents fear AI has solved mathematics. Grant Sanderson argues the opposite: even if machines prove theorems better than we do, the institution of math stays valuable—and this is a rare moment when newcomers can reshape it.
51:55EN3Blue1Brown· Sep 18, 2026The last IMO problem AI could not solveGrant Sanderson · 3Blue1BrownGrant Sanderson walks through IMO 2025 Problem 6 — a tiling puzzle that frontier models missed — and the visual patience it takes to prove a construction is optimal.
0:55EN3Blue1Brown· Jul 24, 2026The 64 Sugar Cubes Puzzle: Six Equal DistancesA 4×4×4 block has three colors; prove that six same-colored pairs share a distance.
33:51EN3Blue1Brown· Jul 16, 2026What Cross-Entropy Connects: Compression, Language Models and KL DivergenceGrant Sanderson · 3Blue1BrownGrant Sanderson builds cross-entropy from code lengths and uses it to explain next-token training and distillation.
2:25EN3Blue1Brown· Jun 12, 2026How Shannon Measured the Entropy of EnglishA guessing game helped Claude Shannon estimate how much information English text contains, and why prediction is central to compression.
1:02EN3Blue1Brown· Jun 10, 2026A Robot Walk and the Puzzle of Optimal Binary EncodingHow few bits does it take to send directions that occur with unequal frequencies?