← All explainers
An explainer
Neural
Networks
Cover & TOC
The learning game
  • 01 The prediction game
  • 02 Kinds of learning
  • 03 Turning the world into numbers
  • 04 The simplest model
  • 05 Measuring wrongness
  • 06 Drawing a boundary
How learning happens
  • 07 Which way is downhill?
  • 08 Gradient descent
  • 09 Tuning the descent
  • 10 Batches, epochs & noise
From a line to a network
  • 11 Neural networks
  • 12 Why depth wins
  • 13 Backpropagation
  • 14 The training loop
  • 15 Where training starts
  • 16 When gradients vanish
  • 17 Shapes of networks
Learning vs. memorizing
  • 18 Learning vs. memorizing
  • 19 Data quality & the three splits
  • 20 Judging a model
The bigger picture
  • 21 A short history
  • 22 Embeddings
  • 23 From this loop to an LLM
§ Glossary
Section 08

Gradient descent

The loss surface, step size, and getting stuck

Premium section

Unlock “How Neural Networks Learn, from the ground up”

This section is part of the paid explainer. Buy it once for lifetime access to all 23 sections. Sections 01, 02, 03, 04 are free to read.

Sign in to buy · $8

New here? Create an account

← Previous · §07
Which way is downhill?
Next · §09 →
Tuning the descent
© 2026 Knowlific Inc. All rights reserved.
Terms of Service Privacy Policy