Heidelberg AI News

Video · Matthew Berman ·

Anthropic Researcher explains how to train LLMs

In short

The video explains how large language models are built, from neural-network fundamentals and pre-training through instruction tuning and reinforcement learning. It matters because these stages turn a numerical pattern-matching system into an assistant that can reason, improve through trial and error, and potentially discover new knowledge.

Key points

  • Neural networks learn functions by adjusting numerical parameters, or “knobs,” to minimize prediction loss.
  • PyTorch automates parameter updates using gradients and a learning rate through backpropagation.
  • Language models are neural networks trained to predict the next token from large text datasets.
  • Pre-training creates an autocomplete model that reflects its data but is not necessarily helpful, truthful, or coherent.
  • Post-training uses formatted examples to teach the model assistant behavior, response quality, and when to stop.
  • Reinforcement learning trains models such as DeepSeek-R1 to improve reasoning through repeated attempts and scored outcomes.
  • The speaker argues that models trained to reason over longer horizons could develop genuinely new knowledge.

Written by an AI model from the video’s captions and description.

Watch on YouTube The opening of the article at youtube.com.

In this story

A tool or model opens in the Heidelberg AI Curriculum: what it does, what it costs, how to start. A repo opens on GitHub.

Related stories

All stories