Video · Matthew Berman ·
Anthropic Researcher explains how to train LLMs
In short
The video explains how large language models are built, from neural-network fundamentals and pre-training through instruction tuning and reinforcement learning. It matters because these stages turn a numerical pattern-matching system into an assistant that can reason, improve through trial and error, and potentially discover new knowledge.
Key points
- Neural networks learn functions by adjusting numerical parameters, or “knobs,” to minimize prediction loss.
- PyTorch automates parameter updates using gradients and a learning rate through backpropagation.
- Language models are neural networks trained to predict the next token from large text datasets.
- Pre-training creates an autocomplete model that reflects its data but is not necessarily helpful, truthful, or coherent.
- Post-training uses formatted examples to teach the model assistant behavior, response quality, and when to stop.
- Reinforcement learning trains models such as DeepSeek-R1 to improve reasoning through repeated attempts and scored outcomes.
- The speaker argues that models trained to reason over longer horizons could develop genuinely new knowledge.
Written by an AI model from the video’s captions and description.
In this story
A tool or model opens in the Heidelberg AI Curriculum: what it does, what it costs, how to start. A repo opens on GitHub.



