Section 01

The prediction game

Input → model → guess → score

Many systems in modern AI — the model that finishes your sentence, labels your photo, or writes code — are, underneath, trained by the same simple game. The model looks at some input, makes a guess about an answer, and finds out how wrong the guess was. Then it changes itself a little so that next time the guess is closer. Do that a few billion times and you get something that feels intelligent. This explainer is about that loop, built up one piece at a time, starting here with the game itself.

Input, model, prediction

Strip away the jargon and a machine-learning model neural network A function built by stacking many simple operations — mostly matrix multiplies with nonlinearities between them — whose behavior is shaped by tuning billions of internal numbers (its parameters) from data. See in glossary → is just a function function A rule that takes an input and gives back exactly one output — like a machine: put something in, get something out. Often written f(x); for example f(x) = 2x turns 3 into 6. A model is just a (very elaborate) function from input to prediction. See in glossary → : something takes an input and produces a prediction.

  • Input: the size of a house. Prediction: its price.
  • Input: an email. Prediction: spam or not.
  • Input: the text so far. Prediction: what comes next.

The model is whatever sits in the middle turning one into the other. For now, picture it as a box with a few numbered dials on the side. The same input, with the dials in different positions, produces different predictions. Those dials are the learned part of the model: training changes their settings so the predictions improve and get more and more accurate.

The model as a box of dials
An input goes in and a prediction comes out. The same input, with the dials in different positions, gives a different answer — and "training" is just turning the dials until the answers are good.
input size = 2,000 ft² the model a function with knobs the dials · one number each w₁ w₂ w₃ b prediction price ≈ $410k

A wrong guess needs a score

A guess is only useful if you can say how good it was. So we need a single number that measures wrongness: bigger when the prediction is far from the truth, smaller when it’s close, zero when it’s perfect. That number is the loss loss function A single number measuring how wrong the model's predictions are on a batch of data. Training works by adjusting the model to make this number smaller. See in glossary → , and it’s the compass for everything that follows. We’ll spend a whole section on how to define it well; for now, all that matters is that low loss = good.

Play the game yourself

Here is the smallest possible version. The machine predicts using one dial, w, with the rule prediction = w × input. The gold dots are the real answers we want it to match. Drag w and watch the score. Then hand the job to the machine: “Take one step” lets it feel which way the score is falling and nudge w that way; “Let it learn” repeats that automatically.

One knob, one rule
The machine guesses ŷ = w · x. The dots are the real answers. Turn the one knob w to make the line pass through them — or let the machine adjust it for you.
x (input) →ŷ (guess)
Each red stick is one prediction's error. The score adds up their squares — one number for "how wrong are we overall." "Take one step" measures which way the score falls and nudges w that way. Repeat, and the machine finds the rule on its own. That measure-and-nudge move is the entire idea; the rest of this explainer just makes it bigger.

Notice what just happened. You didn’t tell the machine the right value of w. You gave it examples and a way to score itself, and the “learning” button found the answer by repeatedly measuring and nudging. No rules about houses or prices were ever written down — only predict, score, adjust.

Where we’re going

We’ll build up in a deliberate order. First we’ll place this game on the map of how machines learn and see how raw data becomes numbers a model can read; then a real model with many dials, a proper definition of the score, the calculus calculus The math of how things change. The one piece we need is the slope: at any point on a curve, how steeply it is rising or falling, and in which direction. That single idea is what lets us tell which way to turn each dial to lower the loss — no heavy math required. See in glossary → trick for knowing which way is downhill, the algorithm that turns that into learning, what happens when you stack these functions into a network, and finally how the whole thing becomes a Large Language Model (LLM). Each step is small. Let’s start by asking what “learning from examples” even means.