How Neural Networks Learn, from the ground up

A long-form, interactive explainer

Underneath every modern AI system is one deceptively simple idea: a machine that gets better at a task just by looking at examples. No one hand-codes the rules. The machine adjusts a pile of numbers until its guesses stop being wrong. This explainer builds that idea from absolutely nothing.

We start with something concrete — predicting house prices from a straight line — and follow a single thread all the way up: what a prediction is, how we score it, how the machine figures out which way to adjust its numbers, what happens when you stack these adjustable functions into a linked network of them, and how it works out whether each of a million, billion or even trillion numbers should go up or down.

No machine-learning background is assumed, and there is no heavy math. Every term gets defined the first time it appears; hover any underlined word for a tooltip, or visit the glossary any time.

Where this sits: it's the on-ramp to the LLM Pre-training explainer, which assumes exactly the machinery that this one builds from scratch. The last section connects the two directly: a large language model turns out to be this same idea, learning to predict the next piece of text instead of the price of a house.

Start reading → ~6–8 hours, 23 sections

Contents

The learning game

  1. 01 The prediction game — Input → model → guess → score
  2. 02 Kinds of learning — Supervised, unsupervised, self-supervised, reinforcement
  3. 03 Turning the world into numbers — Features, encoding, and why inputs get scaled
  4. 04 The simplest model — Weights, bias, and why parameters are just knobs
  5. 05 Measuring wrongness — How we measure a wrong answer
  6. 06 Drawing a boundary — Turning scores into yes/no decisions

How learning happens

  1. 07 Which way is downhill? — Slopes and just enough calculus
  2. 08 Gradient descent — The loss surface, step size, and getting stuck
  3. 09 Tuning the descent — Making the descent faster and smarter
  4. 10 Batches, epochs & noise — Why we estimate the gradient from samples

From a line to a network

  1. 11 Neural networks — Layers, activations, and why nonlinearity matters
  2. 12 Why depth wins — Features built on features
  3. 13 Backpropagation — How the model assigns blame for a mistake
  4. 14 The training loop — Forward → loss → backward → update → repeat
  5. 15 Where training starts — Where the numbers start
  6. 16 When gradients vanish — The problem that stalled deep learning, and the fixes
  7. 17 Shapes of networks — Matching the architecture to the data

Learning vs. memorizing

  1. 18 Learning vs. memorizing — Why memorizing isn't learning
  2. 19 Data quality & the three splits — Garbage in, and holding data back
  3. 20 Judging a model — Why the loss is not the metric you care about

The bigger picture

  1. 21 A short history — From the perceptron to deep learning, and why now
  2. 22 Embeddings — Meaning as geometry
  3. 23 From this loop to an LLM — The same loop, where the prediction is the next token