In 30 seconds
Nathan Lambert introduces his RLHF book and accompanying course, explaining the gaps in public post-training documentation that motivated them. He outlines the book, slides and code resources, and invites feedback as the lectures are released.
- 1Why the book existsLambert wanted to document methods he encountered in post-training.
- 2Book structureBackground leads to core methods, data and cross-cutting practical questions.
- 3Course materialsThe site brings together slides, recordings and code.
Takeaways
- The book records practical post-training methods and intuitions that were not always publicly documented.
- Core chapters cover training methods and mathematics, followed by data and broader practical topics.
- The course adds recordings, slides, PDFs and code examples.
Read the edited text by chapter
Documenting post-training
Lambert explains the motivation for the book and gives examples of underdocumented techniques.
I am putting together a course to accompany Reinforcement Learning from Human Feedback. The book grew out of notes I kept while working closer to the frontier of post-training, especially on intuitions and methods that were used in models but poorly documented publicly.
Rejection sampling and character training are examples of practices I wanted to explain. The book is available on the web and in other formats, with a print edition planned through Manning at the time of this recording.
Book and course resources
He outlines the chapters, accompanying slides, code and plans for releasing lectures.
The book starts with background and related work. Chapters 4–9 cover the central training methods and mathematics; chapters 10–12 address data; later sections discuss broader practical questions that cut across individual techniques.
The accompanying course will provide recordings, interactive slides, PDFs and related educational talks on the book’s site. I am building a lightweight slide tool that uses Markdown and renders HTML with mathematics, code, citations and charts.
Code repositories are being built for policy gradients, reward modeling and DPO-like methods. Lambert invites feedback and contributions as he records and releases the lectures iteratively; he also mentions the possibility of future online cohorts.
More from this channel
16:48EN
Nathan Lambert· Nov 2, 2026Nathan Lambert on How Open Models Can Compete With Frontier LabsNathan Lambert · Allen Institute for AIOpen models are gaining use and closing capability gaps, but their builders need a sustainable economy and advantages beyond price.
48:04ENNathan Lambert· Oct 26, 2026Nathan Lambert on Open Language Models: History, Competition and RiskNathan Lambert · Allen Institute for AIA tour of open weights, research infrastructure, distillation, adoption and the choices facing model developers.
46:42ENNathan Lambert· Oct 25, 2026Nathan Lambert and Florian Brand on Kimi K3.1, Cyber Risk and AI AgentsNathan Lambert, Florian Brand · Allen Institute for AIThe Interconnects hosts discuss mixed model evaluations, a wave of security disclosures, and what a personal agent can do in everyday life.
5:42ENNathan Lambert· Aug 12, 2026Why the Shoggoth Became a Mascot for AI Post-TrainingNathan Lambert · Allen Institute for AINathan Lambert traces a creature from Lovecraft to a visual metaphor for language models, fine-tuning, and RLHF.
32:56ENNathan Lambert· Aug 6, 2026Nathan Lambert: An Introduction to AI Character TrainingNathan Lambert · Allen Institute for AIHow post-training shapes a model’s behavior, from constitutions and model specs to experiments with personality and activation steering.
32:11ENNathan Lambert· Aug 5, 2026Nathan Lambert on How Language Model Evaluation Has EvolvedNathan Lambert · Allen Institute for AIFrom few-shot prompts and pass@k to agent sandboxes, noisy scores and benchmark contamination.