Glossary

Every term, in one place

Every defined term across the explainers, alphabetized. Each entry links to the section where the term is first introduced.

"aha moment"Introduced in §27 · GRPO & DeepSeek-R1

The observation in DeepSeek-R1-Zero that, under pure RL with verifiable rewards, the model spontaneously learns to pause, reconsider, and backtrack — reasoning behaviors no one demonstrated.

3D parallelismIntroduced in §07 · Parallelism

Combining data, tensor, and pipeline parallelism (three axes) at once to train a model too big for any single axis to handle. Frontier runs add expert parallelism as a fourth.

6ND ruleIntroduced in §06 · Compute & memory

A rule of thumb: training a dense model with N parameters on D tokens costs about 6ND floating-point operations (≈2ND forward + ≈4ND backward).

acceptance lengthIntroduced in §21 · GLM-5.2

In speculative decoding, the average number of drafted tokens the target model accepts per verification pass. Longer accepted runs mean fewer expensive target-model steps per generated token.

accuracyIntroduced in §20 · Judging a model

The fraction of predictions that are correct. Simple, but misleading on imbalanced data where one class dominates.

activationsIntroduced in §11 · Neural networks

The values produced by a layer after applying its activation function. During training, intermediate activations are often kept for the backward pass.

active parametersIntroduced in §18 · DeepSeek-V3

In a Mixture-of-Experts model, the subset of parameters actually used to process a given token. DeepSeek-V3 has 671B total but only 37B active per token, so compute tracks the smaller number.

actorIntroduced in §15 · Value & advantage

The model itself, in its acting role: during a rollout it proposes next-token predictions and we sample from them to build the response. It's called the actor to contrast with the critic, which judges the actions instead of taking them.

AdaFactorIntroduced in §13 · T5

A memory-efficient optimizer (used to train T5) that factorizes Adam's second-moment matrix into row and column statistics, drastically cutting optimizer-state memory for very large models.

AdamIntroduced in §09 · Tuning the descent

Adaptive Moment Estimation — an optimizer that tracks running averages of the gradient (first moment) and its square (second moment) to give each parameter its own adaptive step size.

AdamWIntroduced in §09 · Tuning the descent

Adam with decoupled weight decay. It applies weight decay directly to the parameters instead of folding it into the gradient, which often regularizes more cleanly.

adapterIntroduced in §01 · What is post-training?

A small set of extra parameters trained on top of a frozen base model, so fine-tuning updates only the adapter rather than the full network. Adapters cut memory and storage cost and can be swapped in and out per task.

advantageIntroduced in §15 · Value & advantage

How much better an action was than the baseline expectation: A = reward − value. Positive advantage pushes an action’s probability up, negative pushes it down.

agentic RLIntroduced in §30 · Agentic & tool-use RL

Reinforcement learning over multi-step, tool-using trajectories — the model acts, observes results, and acts again — rather than producing a single response. The 2025–26 frontier.

AI winterIntroduced in §21 · A short history

A period of collapsed funding and interest in AI after expectations went unmet — first triggered in part by the perceptron's exposed limits.

alignmentIntroduced in §04 · The alignment problem

The problem of making a model behave in accordance with human intent and values — helpful, honest, and harmless — rather than merely continuing text plausibly.

all-reduceIntroduced in §19 · Scaling out

A collective op where every GPU contributes a tensor and every GPU ends up with the sum (or other reduction). The TP workhorse.

annealingIntroduced in §08 · The data pipeline

A final pre-training phase that upsamples small amounts of the highest-quality data (math, code, curated text) while the learning rate decays to its floor. Reliably boosts quality and can be used to gauge a dataset's value.

arithmetic intensityIntroduced in §06 · Compute & memory

The ratio of compute (FLOPs) to memory traffic (bytes) for an operation. High-intensity ops keep the GPU's math units busy; low-intensity ops stall on memory.

attention scoreIntroduced in §04 · Attention

A single number sᵢⱼ measuring how much token i wants to attend to token j. Computed as the dot product of i's query vector and j's key vector (scaled by √d_k), then softmaxed across j so the weights for each i sum to 1. High score = i finds j relevant.

autoregressiveIntroduced in §01 · What is an LLM?

Generating one token at a time, where each new token is conditioned on every token that came before it.

auxiliary-loss-free load balancingIntroduced in §18 · DeepSeek-V3

DeepSeek-V3's load-balancing method that adjusts a per-expert routing bias instead of adding a balancing loss term, avoiding the quality hit that auxiliary losses impose on the main objective.

backpropagationIntroduced in §13 · Backpropagation

The algorithm that computes the loss gradient for every parameter efficiently by applying the chain rule backward through the network, reusing intermediate results from the forward pass.

backward passIntroduced in §16 · When gradients vanish

The second half of a training step: backpropagation walks from the loss back through the network, computing each parameter's gradient.

banditIntroduced in §13 · Policy gradients & REINFORCE

The simplest reinforcement-learning setting: a single decision with a fixed set of actions, each giving a reward, and no states or follow-on consequences. Named after the "one-armed bandit," old slang for a slot machine.

The name comes from gambling. A slot machine was nicknamed a "one-armed bandit" — its lever is the single "arm," and it "robs" the player over time. The classic "multi-armed bandit" problem then imagines a gambler facing a whole row of slot machines, each with an unknown, different payout, who must decide which arms to pull to win the most. That captures the purest form of the explore-vs-exploit tradeoff — try unknown options, or cash in on the best one found so far — which is why the name stuck for the simplest RL setting.

base modelIntroduced in §01 · What is post-training?

A model straight out of pre-training — a powerful text continuator that has not yet been taught to follow instructions, hold a conversation, or refuse harmful requests.

baselineIntroduced in §14 · The baseline trick

A reference value subtracted from the reward to reduce gradient variance without adding bias. Can be a learned critic, a group mean (GRPO), or a leave-one-out average (RLOO).

batchIntroduced in §10 · Batches, epochs & noise

The group of training examples used for one gradient estimate. Bigger batches reduce gradient noise but use more memory and compute per step.

batch normalizationIntroduced in §16 · When gradients vanish

A layer that re-centers and re-scales activations using batch statistics to keep them well-conditioned mid-network, easing and speeding training of deep models.

best-of-NIntroduced in §22 · Rejection-sampling alignment

Sampling N responses and selecting the highest-reward one. Used both at inference time and as the data-generation step in rejection-sampling fine-tuning.

BF16Introduced in §05 · Precision & numerics

Brain Floating-point 16-bit: 1 sign + 8 exponent + 7 mantissa bits. Keeps FP32's wide exponent range (so it rarely overflows) at the cost of precision — the workhorse format for modern pre-training.

biasIntroduced in §04 · The simplest model

The constant term added to a weighted sum — the model's baseline output when the inputs contribute nothing. It lets a line shift up and down rather than being pinned to the origin. (Unrelated to "bias" in the fairness sense.)

bidirectionalIntroduced in §11 · BERT

Able to use context from both the left and the right of a token. BERT is bidirectional; a causal language model is left-to-right only.

bits-per-tokenIntroduced in §02 · The objective

Cross-entropy loss measured in bits (log base 2) instead of nats. A compression-flavored view: a better language model encodes the next token in fewer bits.

BooksCorpusIntroduced in §10 · GPT-1

A dataset of around 7,000 unpublished books (~800M words) used to pre-train GPT-1. Long contiguous passages made it good for learning long-range structure.

BPEIntroduced in §02 · Tokens

Byte-Pair Encoding — the most common tokenization algorithm. It merges frequent byte pairs into tokens.

Bradley–Terry modelIntroduced in §11 · Reward models

A simple rule for turning "A beats B" comparisons into a single score per item: the bigger an item's score over another, the more likely it wins. A reward model produces exactly such a score.

byte-level BPEIntroduced in §08 · The data pipeline

Byte-level Byte Pair Encoding — running BPE over raw bytes rather than Unicode characters, so any possible input (emoji, code, any language) is representable with a small base vocabulary. Introduced by GPT-2.

C4Introduced in §13 · T5

Colossal Clean Crawled Corpus — the ~750 GB cleaned web-text dataset built from Common Crawl for training T5, and widely reused since.

calculusIntroduced in §01 · The prediction game

The math of how things change. The one piece we need is the slope: at any point on a curve, how steeply it is rising or falling, and in which direction. That single idea is what lets us tell which way to turn each dial to lower the loss — no heavy math required.

categorical featureIntroduced in §03 · Turning the world into numbers

An input whose values are discrete categories (color, country) rather than meaningful numbers. Usually one-hot encoded before a model can use it.

causal language modelIntroduced in §02 · The objective

A model that predicts each token using only earlier tokens (never future ones). "Causal" because information flows strictly left to right. The GPT family are causal LMs (Language Models).

causal maskIntroduced in §09 · Attention Is All You Need

A mask applied before the attention softmax that sets future positions to −∞, preventing each token from attending to tokens that come after it. What makes a decoder autoregressive.

chain ruleIntroduced in §13 · Backpropagation

The calculus rule for differentiating composed functions. Backpropagation is just the chain rule applied layer by layer, from the loss back to the inputs.

chain-of-thought (CoT)Introduced in §23 · Bootstrapping reasoning

Having a model write out intermediate reasoning steps before its final answer. Improves accuracy on multi-step problems and is the substrate reasoning RL optimizes.

chat templateIntroduced in §06 · The SFT stage in practice

The fixed formatting (with special tokens marking roles like system/user/assistant) that turns a multi-turn conversation into the single token stream a model is trained and served on.

chunked prefillIntroduced in §17 · Chunked prefill

Splitting a long prompt into multiple smaller prefills so decoding requests aren’t blocked behind one giant compute step.

class imbalanceIntroduced in §20 · Judging a model

When one class vastly outnumbers another, making raw accuracy a poor metric — a do-nothing model can score high by always guessing the majority class.

classificationIntroduced in §02 · Kinds of learning

A prediction task whose answer is one of a fixed set of categories (spam / not spam, or which of 100,000 tokens comes next). The model outputs a probability per option and is scored with cross-entropy.

clipped surrogate objectiveIntroduced in §17 · TRPO to PPO

PPO’s loss: maximize the probability-ratio-weighted advantage, but clip the ratio to [1−ε, 1+ε] so a single update can’t move the policy too far.

clusteringIntroduced in §02 · Kinds of learning

An unsupervised task that groups examples resembling one another, without being told what the groups are.

cold-start dataIntroduced in §27 · GRPO & DeepSeek-R1

A small amount of high-quality SFT data used to "warm up" a base model before RL, so reasoning RL is more stable and readable. DeepSeek-R1 adds it; R1-Zero skips it.

Common CrawlIntroduced in §08 · The data pipeline

A free, monthly public crawl of the web — petabytes of raw HTML. It is the raw feedstock for most large pre-training corpora after heavy filtering.

compactionIntroduced in §34 · GLM-5.2

Compressing an agent's long interaction history (summarizing earlier turns, dropping stale tool output) so a long-horizon task keeps fitting in the model's context window; the trajectory then continues from the compacted state.

completionIntroduced in §01 · What is an LLM?

The text the model generates in response to a prompt.

Compressed Sparse AttentionIntroduced in §26 · DeepSeek-V4

Compressed Sparse Attention (CSA) — a DeepSeek-V4 attention variant that attends to a compressed, sparsely-selected subset of past tokens to make million-token context affordable.

compute-optimalIntroduced in §16 · Chinchilla

The allocation of a fixed compute budget between model size and training tokens that minimizes loss. Chinchilla showed it means scaling both roughly equally — about 20 tokens per parameter.

confusion matrixIntroduced in §20 · Judging a model

The 2×2 table of true/false positives and negatives that summarizes a classifier's outcomes and from which accuracy, precision, and recall are computed.

Constitutional AIIntroduced in §12 · RLAIF & Constitutional AI

Anthropic’s method where a model critiques and revises its own outputs against a written set of principles (a "constitution"), then trains on AI-generated preferences — a form of RLAIF.

context lengthIntroduced in §02 · The objective

The maximum number of tokens the model can attend to at once (also called the context window or sequence length). Pre-training picks a context length; later stages often extend it.

continuous batchingIntroduced in §14 · Continuous batching

A scheduler that swaps finished requests out and queued requests in at every decode step instead of waiting for the whole batch to finish.

convolutional network (CNN)Introduced in §17 · Shapes of networks

A network that slides small shared filters over grid-like data such as images. Reusing each filter at every position saves parameters and makes shifted patterns easier to recognize.

coreferenceIntroduced in §04 · Attention

When two words in a text refer to the same thing. In "Marie went home because she was tired," the pronoun "she" co-refers to "Marie." Resolving coreference — figuring out which earlier mention a pronoun, "this", "the company", etc. points back to — is one of the relationships transformer heads learn to track during training.

corpusIntroduced in §19 · Data quality & the three splits

The body of text a model is trained on. Modern pre-training corpora are measured in trillions of tokens drawn from web crawls, books, code, and more.

cosine decayIntroduced in §04 · Optimizers & schedules

A learning-rate schedule that follows a half-cosine curve from the peak down to a small floor, decaying slowly at first and fast at the end. The most common LLM schedule.

cosine similarityIntroduced in §22 · Embeddings

A score based on the angle between two vectors: 1 means the same direction, 0 means a right angle, and −1 means opposite directions. It is common for comparing embeddings, but the score's meaning depends on the learned space.

criticIntroduced in §15 · Value & advantage

A model trained to predict the value function. PPO uses an actor (the policy) and a critic; GRPO drops the critic and uses a group average instead.

cross-entropy lossIntroduced in §05 · Measuring wrongness

A classification loss that penalizes the model according to the negative log-probability it assigned to the correct answer.

d_modelIntroduced in §03 · Embeddings

The hidden dimension that flows through the whole transformer. Llama-3-8B uses 4096, GPT-3 12288.

DAPOIntroduced in §28 · GRPO refinements

A fully open GRPO refinement (ByteDance/Tsinghua, 2025) combining Clip-Higher, dynamic sampling, token-level loss, and overlong-reward shaping to stabilize large-scale reasoning RL.

data contaminationIntroduced in §19 · Data quality & the three splits

When test or benchmark data leaks into the training corpus, inflating scores. Careful pipelines try to detect and remove contamination before training.

data leakageIntroduced in §19 · Data quality & the three splits

When information about the answers (e.g. test examples) sneaks into training, inflating scores without real learning. Why LLM corpora are deduplicated and benchmarks scrubbed.

data mixtureIntroduced in §08 · The data pipeline

The recipe specifying what fraction of training tokens comes from each source (web, code, books, math, multilingual). Tuning the mixture is one of the highest-leverage data decisions.

data parallelismIntroduced in §07 · Parallelism

Replicating the whole model on each GPU, giving each a different slice of the batch, then averaging gradients across GPUs with an all-reduce. The simplest way to scale out.

data wallIntroduced in §21 · Synthetic data

The looming limit where the supply of high-quality human-written text is exhausted relative to models' appetite for tokens, motivating interest in synthetic data and better filtering.

decision boundaryIntroduced in §06 · Drawing a boundary

The surface in feature space where a classifier flips from one predicted class to another — a straight line or plane for a linear model, a curve for a network.

decision thresholdIntroduced in §06 · Drawing a boundary

The probability cutoff at which a classifier commits to the positive class (often 0.5). Moving it trades false alarms against missed detections.

decodeIntroduced in §11 · Prefill and decode

The autoregressive phase: one forward pass per generated token. Memory-bandwidth-bound — the GPU mostly waits on weights.

decoderIntroduced in §09 · Attention Is All You Need

The half of a transformer that generates a sequence one token at a time using masked (causal) self-attention. GPT-style language models are decoder-only.

deduplicationIntroduced in §19 · Data quality & the three splits

Removing duplicate or near-duplicate documents from the corpus. Dedup improves quality, reduces memorization, and stops the model wasting capacity on repeated text.

DeepSeek Sparse AttentionIntroduced in §28 · GLM-5.2

DeepSeek Sparse Attention (DSA): an attention variant where a lightweight indexer scores every past token with cheap dot products, a top-k selection keeps the most relevant ones, and full attention runs over only that subset.

denoising objectiveIntroduced in §13 · T5

Any pre-training objective that corrupts the input (masking, deleting, or shuffling tokens) and trains the model to restore the original. Masked LM and span corruption are both denoising objectives.

derivativeIntroduced in §07 · Which way is downhill?

The slope of a function at a point: how fast the output changes as you nudge the input, and in which direction. In training, the derivative of the loss with respect to a parameter tells you which way to turn that knob.

dimensionality reductionIntroduced in §02 · Kinds of learning

Compressing many correlated features into a few underlying dimensions that capture most of the variation.

document attention maskingIntroduced in §08 · The data pipeline

Restricting attention so each token can only attend within its own document (never across a separator) when several documents share one packed sequence. Prevents one document's tokens from leaking into another's predictions. Also called intra-document masking.

document separatorIntroduced in §08 · The data pipeline

A special token (often an End-Of-Sequence / EOS marker such as <|endoftext|>) inserted between documents packed into one training sequence, marking where one document ends and the next begins.

dot productIntroduced in §04 · Attention

A single number summarizing how aligned two vectors are. To compute a · b: multiply matching components and add the results. Positive means they point partly together, zero means they are at right angles, and negative means they point partly apart; what that means for the data depends on how the vectors were learned.

downstream taskIntroduced in §01 · What is pre-training?

Any specific job (translation, question answering, coding) a pre-trained model is later applied to. Pre-training is deliberately task-agnostic so it transfers to many downstream tasks.

DPOIntroduced in §20 · Direct Preference Optimization

Direct Preference Optimization (Rafailov, 2023) — a closed-form supervised loss that optimizes the RLHF objective directly from preference pairs, with no separate reward model and no RL loop.

Dr.GRPOIntroduced in §28 · GRPO refinements

A corrected GRPO that removes length and standard-deviation normalization biases, so the gradient is unbiased and long wrong answers aren’t implicitly favored.

dropoutIntroduced in §09 · Attention Is All You Need

A regularizer that randomly zeroes a fraction of activations during training, forcing the network not to rely on any single unit. Common in early models; large modern pre-training often uses little or none.

dynamic rangeIntroduced in §05 · Precision & numerics

The span between the smallest and largest magnitudes a number format can represent, set by its exponent bits. BF16 has FP32-like range; FP16 does not.

EAGLEIntroduced in §18 · Speculative decoding

A draft-model architecture that predicts feature vectors of the target model, achieving high acceptance rates.

embeddingIntroduced in §22 · Embeddings

A dense vector representation of a token (typically d=2k–8k floats). Similar tokens get nearby vectors.

embedding matrixIntroduced in §22 · Embeddings

A table with one row per vocabulary entry. Looking up a token = indexing into this matrix.

emergent abilitiesIntroduced in §15 · GPT-3

Capabilities that are absent in smaller models but appear, sometimes abruptly, once a model is large enough — e.g. multi-step arithmetic or in-context learning of novel tasks.

encoderIntroduced in §09 · Attention Is All You Need

The half of a transformer that reads an input sequence with full (bidirectional) attention, producing a contextual representation of it. BERT is encoder-only.

encoder-decoderIntroduced in §09 · Attention Is All You Need

An architecture with an encoder that reads the input and a decoder that writes the output, connected by cross-attention. The original transformer and T5 are encoder-decoder models.

entropyIntroduced in §03 · From next-token to behavior

A measure of how spread-out (uncertain) a probability distribution is. In RL post-training, keeping entropy up preserves exploration and prevents premature collapse onto one answer.

epochIntroduced in §10 · Batches, epochs & noise

One full pass over the training dataset. Frontier LLMs are often trained for roughly a single epoch over a deduplicated corpus, so each token is seen about once.

expectationIntroduced in §02 · Probability, policies & gradients

The probability-weighted average of a quantity over a distribution: E[f] = Σ p(x)·f(x). RL maximizes the policy's expected reward — the average reward over the responses it would generate.

expert parallelismIntroduced in §07 · Parallelism

Placing different experts of a Mixture-of-Experts layer on different GPUs, so each device holds only some experts and tokens are routed across the network to reach them.

exploding gradientIntroduced in §16 · When gradients vanish

When the combined effect of many layers makes the backward learning signal enormous, causing unstable updates or NaNs. The mirror image of a vanishing gradient.

featureIntroduced in §03 · Turning the world into numbers

An individual input the model reads — a house's size, an email's word counts, a pixel. Each feature gets its own weight saying how much it matters.

few-shotIntroduced in §15 · GPT-3

Giving the model a handful of worked examples in the prompt before the real query, so it infers the task from them. Contrast with zero-shot (instructions only) and one-shot (a single example).

fill-in-the-middleIntroduced in §25 · Qwen3-Coder-Next

Fill-in-the-Middle (FIM) — a code pre-training objective that gives the model a prefix and a suffix and asks it to generate the missing middle, teaching it to edit and complete code in place, not just continue it.

fine-tuningIntroduced in §01 · What is pre-training?

Continuing to train a pre-trained model on a smaller, task- or behavior-specific dataset. This explainer is about pre-training; fine-tuning and other post-training steps are out of scope.

floating pointIntroduced in §05 · Precision & numerics

How computers store real numbers: a sign, an exponent (range), and a mantissa (precision). The trade-off between range and precision is central to training numerics.

FLOPIntroduced in §06 · Compute & memory

Floating-Point Operation — one multiply or add. Training compute is measured in total FLOPs; a frontier run is on the order of 10^24–10^25 FLOPs.

forward passIntroduced in §03 · How a model learns

Running inputs through the network to produce outputs (logits) and the loss, caching intermediate activations that backpropagation will need.

foundation modelIntroduced in §01 · What is pre-training?

A large model pre-trained on broad data that can be adapted to many downstream tasks. The pre-trained LLM is the foundation; fine-tuning specializes it.

FP16Introduced in §05 · Precision & numerics

16-bit half-precision Floating Point: 1 sign + 5 exponent + 10 mantissa bits. Half the memory of FP32 but a narrow exponent range, so it can overflow/underflow without loss scaling.

FP32Introduced in §05 · Precision & numerics

32-bit single-precision Floating Point: 1 sign + 8 exponent + 23 mantissa bits. The traditional "full precision" format; accurate but memory- and bandwidth-hungry.

FP8Introduced in §05 · Precision & numerics

8-bit Floating Point (typically E4M3 or E5M2 layouts). The newest training precision, used on H100/Blackwell GPUs to roughly double throughput; needs careful scaling to stay numerically stable.

fragmentationIntroduced in §15 · Paged attention

Wasted memory from allocations that don’t fit cleanly. Paging trades internal fragmentation (≤1 page per request) for none of the external kind.

FSDPIntroduced in §07 · Parallelism

Fully Sharded Data Parallel — PyTorch's implementation of ZeRO-style sharding: each GPU stores a shard of the parameters and gathers the rest just in time for each layer's compute.

functionIntroduced in §01 · The prediction game

A rule that takes an input and gives back exactly one output — like a machine: put something in, get something out. Often written f(x); for example f(x) = 2x turns 3 into 6. A model is just a (very elaborate) function from input to prediction.

GAEIntroduced in §16 · Generalized advantage estimation

Generalized Advantage Estimation — a way to trade bias against variance in advantage estimates using a decay parameter λ. The standard advantage signal inside PPO.

GELUIntroduced in §07 · The MLP block

Gaussian Error Linear Unit — a smooth nonlinearity used inside the MLP. SiLU/SwiGLU are common modern variants.

generalizationIntroduced in §18 · Learning vs. memorizing

How well a model performs on data it never saw during training. The whole point of pre-training is to generalize, not to memorize the corpus.

generative reward modelIntroduced in §33 · Generative reward models

A reward model that is itself a language model: it writes out a critique or reasons step by step before scoring a response, rather than emitting a single opaque scalar. More accurate and interpretable than a bare scalar head.

Goodhart’s lawIntroduced in §19 · Reward hacking & over-optimization

"When a measure becomes a target, it ceases to be a good measure." Optimizing a proxy reward (the measure) eventually diverges from the true objective it stood in for.

GPUDirectIntroduced in §13 · GPU memory hierarchy

Nvidia tech that lets the NIC or NVMe DMA straight into/out of GPU HBM, bypassing host RAM.

GQAIntroduced in §12 · The KV cache

Grouped-Query Attention — multiple query heads share one K/V head, shrinking the KV cache by 4–8× with minimal quality loss.

gradientIntroduced in §07 · Which way is downhill?

The vector of partial derivatives of the loss with respect to every parameter — it points in the direction of steepest loss increase, so we step the opposite way to reduce the loss.

gradient accumulationIntroduced in §07 · Parallelism

Summing gradients over several mini-batches before doing one optimizer update, to simulate a large batch size that wouldn't fit in memory all at once.

gradient ascentIntroduced in §02 · Probability, policies & gradients

Taking small steps along the gradient to maximize an objective — the same operation as gradient descent with the sign flipped. "Maximize the log-likelihood" and "minimize the negative log-likelihood" describe one update.

gradient checkpointingIntroduced in §06 · Compute & memory

Activation recomputation — saving memory by discarding most activations in the forward pass and recomputing them during the backward pass, trading extra compute for far less memory.

gradient clippingIntroduced in §16 · When gradients vanish

Capping the overall size (norm) of the gradient before the update, to stop occasional huge gradients from destabilizing training.

gradient descentIntroduced in §08 · Gradient descent

The core training algorithm: repeatedly nudge each parameter a small step in the direction that lowers the loss, as told by the gradient.

greedy decoding (argmax)Introduced in §02 · Probability, policies & gradients

Always picking the single most probable next token (the argmax of the distribution). Deterministic — it explores nothing, which is why RL relies on sampling instead.

group-relative advantageIntroduced in §27 · GRPO & DeepSeek-R1

GRPO’s advantage estimate: a response’s reward minus the mean reward of its group of siblings (often divided by their standard deviation), replacing a learned value function.

GRPOIntroduced in §27 · GRPO & DeepSeek-R1

Group Relative Policy Optimization (Shao, 2024) — drop PPO’s critic; sample a group of responses per prompt and use their mean reward as the baseline, giving a group-relative advantage. Memory-cheap RL that powered DeepSeek-R1.

HBMIntroduced in §13 · GPU memory hierarchy

High-Bandwidth Memory — the DRAM stack soldered next to the GPU die. H100 SXM has 80 GB at ~3.35 TB/s.

He initializationIntroduced in §15 · Where training starts

A starting rule for ReLU-like networks that uses a weight scale around √(2/width), aiming to keep values and gradients at workable sizes across layers.

headIntroduced in §05 · Multi-head attention

One independent attention computation. Multi-head splits d_model into N parallel heads, each learning its own pattern.

head_dimIntroduced in §05 · Multi-head attention

d_model / num_heads — the dimension each attention head operates in.

helpful, honest, harmlessIntroduced in §04 · The alignment problem

The "HHH" framing (from Anthropic) of what an aligned assistant should be: useful to the user, truthful, and unlikely to cause harm.

hidden layerIntroduced in §11 · Neural networks

A layer of a neural network between the input and the output. "Hidden" because its values are internal scratch space you don't directly observe. Stacking hidden layers (with nonlinearities) is what gives networks their power.

HopperIntroduced in §08 · Learning from human preferences

A standard reinforcement-learning benchmark: a simulated one-legged hopping robot (from the MuJoCo physics engine / OpenAI Gym) that an agent learns to control. Not a real robot — a toy continuous-control environment used to test RL algorithms.

hyper-connectionsIntroduced in §26 · DeepSeek-V4

A learnable replacement for the residual connection: instead of always adding a layer's output straight back to its input, the model keeps several parallel copies of the signal and learns how much of each to mix between layers. DeepSeek-V4's Manifold-Constrained variant (mHC) keeps those learned weights in a safe range so training stays stable.

hyperparameterIntroduced in §19 · Data quality & the three splits

A training setting you choose rather than learn — learning rate, batch size, number of layers, etc. Tuning these well is much of the craft of pre-training.

implicit rewardIntroduced in §20 · Direct Preference Optimization

In DPO, the reward is never trained explicitly; it is implied by the log-ratio between the policy and the reference. Optimizing the DPO loss is equivalent to RLHF under that implied reward.

importance samplingIntroduced in §17 · TRPO to PPO

Reweighting samples from one distribution to estimate expectations under another, via the probability ratio π_new/π_old. The ratio PPO clips comes from here.

in-context learningIntroduced in §15 · GPT-3

A model performing a new task purely from examples or instructions placed in its prompt, with no gradient updates. GPT-3 showed this emerges from pure next-token pre-training at scale.

IndexShareIntroduced in §28 · GLM-5.2

Sharing one sparse-attention indexer across a block of consecutive layers (or across speculative-decoding draft steps): the first computes which past tokens matter and the rest reuse its top-k selection, eliminating most indexer compute.

inferenceIntroduced in §01 · What is an LLM?

Running a trained model to produce outputs. Training learns the weights once; inference uses them many times.

inference scalingIntroduced in §25 · Inference scaling & o1

The empirical finding that accuracy improves predictably as you spend more test-time compute (longer reasoning, more samples) — a second scaling axis beyond model and data size.

initializationIntroduced in §15 · Where training starts

The scheme for setting parameters before training starts. Good initialization keeps activations and gradients at sane scales through a deep network so training can get going.

instruction tuningIntroduced in §05 · Instruction tuning is born

Fine-tuning on many tasks phrased as natural-language instructions so the model learns to follow instructions in general — including ones it never saw in training.

interconnectIntroduced in §07 · Parallelism

The high-speed network linking GPUs — NVLink within a node, InfiniBand/Ethernet across nodes. Its bandwidth and latency cap how aggressively you can shard a model.

IPOIntroduced in §21 · The DPO zoo

Identity Preference Optimization — a DPO variant that replaces the logistic loss with a squared loss to avoid overfitting to deterministic preferences.

IsoFLOPIntroduced in §16 · Chinchilla

A curve of loss versus model size at a fixed compute budget ("iso" = equal FLOPs). Its minimum reveals the compute-optimal model size; Chinchilla used IsoFLOP profiles to find the 20:1 rule.

ITLIntroduced in §11 · Prefill and decode

Inter-Token Latency — gap between consecutive generated tokens during decode.

JacobianIntroduced in §16 · When gradients vanish

A table containing all the local slopes from a function's many inputs to its many outputs. Backprop uses these slope tables to carry gradients through a layer.

keyIntroduced in §04 · Attention

A vector saying “what I represent”. Compared against queries to compute attention scores.

KL divergenceIntroduced in §03 · From next-token to behavior

Kullback–Leibler divergence — a measure of how far one probability distribution is from another. Used in post-training as a "leash" that keeps a model close to a reference policy.

KL penaltyIntroduced in §18 · PPO for RLHF in practice

A term added to the RLHF reward that subtracts β times the KL divergence from the reference policy, keeping the optimized model from drifting too far while chasing reward.

knowledge distillationIntroduced in §20 · Gemma 2

Training a smaller "student" model to match the full output probability distribution of a larger "teacher" model, rather than just the one-hot next token. Richer targets let the student learn more per token.

KTOIntroduced in §21 · The DPO zoo

Kahneman–Tversky Optimization — a preference method using a prospect-theory loss on unpaired, binary good/bad labels, so you don’t need matched preference pairs.

KV cacheIntroduced in §12 · The KV cache

The stored keys and values from all past tokens, so attention at step t only needs to compute Q for the new token.

label noiseIntroduced in §19 · Data quality & the three splits

Incorrect answers in labeled data. They can mislead training, though their exact effect depends on whether the mistakes are random, predictable, or repeated.

label smoothingIntroduced in §09 · Attention Is All You Need

Softening the one-hot target so a little probability mass is spread over all other tokens. It slightly worsens perplexity but discourages overconfidence and often improves downstream quality.

language modelIntroduced in §01 · What is pre-training?

A model that assigns probabilities to sequences of tokens — in practice, one that predicts the probability distribution of the next token given the preceding ones.

layerIntroduced in §12 · Why depth wins

One stage of a neural network: a group of units that transform the values from the previous stage before passing them on.

LayerNormIntroduced in §16 · When gradients vanish

Layer Normalization — rescales each token's activation vector to zero mean and unit variance (then applies learned scale/shift), stabilizing training. RMSNorm is the cheaper modern variant.

learning rateIntroduced in §07 · Which way is downhill?

The size of each parameter step. Too high and training can diverge; too low and it crawls.

learning-rate scheduleIntroduced in §04 · Optimizers & schedules

A plan for changing the learning rate over training — typically a short warmup ramp up followed by a long cosine or linear decay down to a small final value.

length / format rewardIntroduced in §28 · GRPO refinements

Auxiliary reward terms that shape output length or enforce a required format (e.g. putting reasoning in tags, the answer in a box) — used to keep reasoning-RL outputs usable.

likelihoodIntroduced in §03 · From next-token to behavior

The probability a model assigns to observed data. Supervised fine-tuning maximizes the likelihood of human-written target responses given their prompts.

LLMIntroduced in §01 · What is an LLM?

Large Language Model — a neural network trained on huge text corpora to predict the next token given previous tokens.

LM headIntroduced in §09 · Stacking into a full model

Language-Model head — the final linear projection from hidden states (d_model) back to vocab size, producing logits over every token. "Head" because it sits atop the transformer stack like the head of a body; "LM" because it's the layer specialized for the language-modeling (next-token-prediction) objective.

load balancingIntroduced in §18 · DeepSeek-V3

Keeping tokens spread evenly across a Mixture-of-Experts layer's experts, so no single expert (or the GPU holding it) becomes a bottleneck while others sit idle.

logistic regressionIntroduced in §06 · Drawing a boundary

A linear classifier that passes a weighted sum of inputs through the sigmoid to output a class probability, trained with cross-entropy.

logit / scoreIntroduced in §06 · Drawing a boundary

The raw, unbounded linear output (w·x + b) before it is squashed by a sigmoid or softmax into a probability.

logit soft-cappingIntroduced in §20 · Gemma 2

Bounding the model's logits (and/or attention scores) with a scaled tanh so they can't grow without limit, improving training stability. Used in Gemma 2.

logitsIntroduced in §09 · Stacking into a full model

The raw, pre-softmax scores the model produces — one per vocabulary token, per position. Bigger logit = the model finds that token more likely; the actual value can be any real number, positive or negative. Applying softmax across the vocabulary turns logits into a probability distribution that sums to 1. Sampling then picks one token from that distribution.

long chain-of-thoughtIntroduced in §25 · Inference scaling & o1

Extended internal reasoning — thousands of tokens of self-correction, backtracking, and exploration — that reasoning-RL elicits and that test-time scaling rewards.

loss functionIntroduced in §01 · The prediction game

A single number measuring how wrong the model's predictions are on a batch of data. Training works by adjusting the model to make this number smaller.

loss landscapeIntroduced in §08 · Gradient descent

The (extremely high-dimensional) surface of loss as a function of the parameters. Training is a walk downhill on this surface toward a low-loss region.

loss scalingIntroduced in §05 · Precision & numerics

Multiplying the loss by a large constant before backprop (and dividing it back out before the update) to push small FP16 gradients up into the format's representable range.

manifoldIntroduced in §26 · DeepSeek-V4

A smooth space that, up close, looks like ordinary flat space — e.g. the surface of a sphere is a 2D manifold. "Constraining weights to a manifold" means restricting them to a well-behaved subset (such as matrices of bounded size or fixed norm) instead of letting them take any value.

mantissaIntroduced in §05 · Precision & numerics

The significant-digits part of a floating-point number; more mantissa bits means finer precision. FP16 has 10, BF16 only 7.

masked language modelIntroduced in §11 · BERT

Masked Language Model (MLM) — a pre-training objective (used by BERT) that hides a fraction of tokens and trains the model to fill them in using context from both sides. Contrast with next-token prediction.

mean squared errorIntroduced in §04 · The simplest model

A common loss for predicting numbers: average the squared gap between each prediction and its true value. Squaring makes all errors positive and gives especially large misses extra weight. L = (1/n) Σ (ŷ − y)².

MedusaIntroduced in §18 · Speculative decoding

Adds multiple parallel “medusa heads” onto the base model to propose several future tokens at once — no separate draft model.

memory bandwidthIntroduced in §06 · Compute & memory

How fast data moves between GPU compute units and high-bandwidth memory. Many training kernels are bandwidth-bound, not compute-bound, so bandwidth often sets real speed.

metricIntroduced in §20 · Judging a model

A measurement used to judge whether a model succeeds at the real task — such as accuracy, recall, response time, or cost. It need not be the same number used as the training loss.

MFUIntroduced in §06 · Compute & memory

Model FLOPs Utilization — the fraction of a GPU's peak floating-point throughput actually used for useful model math. Real large-scale runs often land around 30–50%.

mid-trainingIntroduced in §08 · The data pipeline

A phase between the main pre-training run and post-training, used to inject specialized data or capabilities (e.g. long context, code-from-execution) while still training the base model on a next-token-style objective.

mini-batchIntroduced in §08 · Gradient descent

The chunk of training examples processed together in one step. Gradients are averaged over the mini-batch, trading off gradient noise against memory and compute.

mixed-precision trainingIntroduced in §05 · Precision & numerics

Doing the heavy matrix multiplies in a low-precision format (BF16/FP8) for speed while keeping a high-precision (FP32) copy of the weights and accumulating sensitive sums in FP32 for stability.

Mixture of ExpertsIntroduced in §18 · DeepSeek-V3

Mixture of Experts (MoE) — a layer with many parallel sub-networks ("experts") where a router sends each token to only a few. The model has a huge total parameter count but activates only a fraction per token, so compute stays modest.

MLPIntroduced in §17 · Shapes of networks

Multi-Layer Perceptron — a stack of dense (matrix-multiply + nonlinearity) layers applied per-token. The transformer’s feed-forward block.

modeIntroduced in §03 · From next-token to behavior

A peak of a probability distribution — an outcome (or region) of locally maximal probability. A distribution can have several modes; "mode collapse" is when a model piles nearly all its probability onto just one of them.

model collapseIntroduced in §21 · Synthetic data

Degradation that can occur when models are trained on too much model-generated data over generations, as rare patterns in the distribution get washed out. Observed for some pure-synthetic mixtures, not for moderate rephrased-data ratios.

momentumIntroduced in §09 · Tuning the descent

An optimizer trick that accumulates a running average of past gradients, letting updates build up speed in consistent directions and damp out oscillations.

Monte-Carlo estimateIntroduced in §02 · Probability, policies & gradients

Estimating an expectation you can't compute exactly by drawing samples and averaging. Accuracy improves with the number of samples; it underlies every policy-gradient estimator.

MQAIntroduced in §12 · The KV cache

Multi-Query Attention — extreme GQA where all query heads share a single K/V head.

multi-head attentionIntroduced in §17 · Shapes of networks

Running several attention operations ("heads") in parallel, each with its own learned projections, so the layer can track many kinds of relationships at once, then concatenating their outputs.

Multi-head Latent AttentionIntroduced in §18 · DeepSeek-V3

Multi-head Latent Attention (MLA) — DeepSeek's attention variant that compresses the keys and values into a small shared low-rank latent vector, drastically shrinking the KV cache while keeping multi-head expressivity.

Multi-Token PredictionIntroduced in §18 · DeepSeek-V3

Multi-Token Prediction (MTP) — a training objective where the model predicts several future tokens at each position (not just the next one), densifying the learning signal and enabling faster speculative decoding later.

multi-turn RLIntroduced in §30 · Agentic & tool-use RL

RL where an episode spans many interaction turns (with a user or an environment), requiring credit assignment across turns rather than within one response.

multimodalIntroduced in §22 · Gemma 3

A model that handles more than one input type — e.g. text plus images (or audio). Pre-training can fold in non-text data via encoders that turn it into token-like embeddings.

MuonIntroduced in §24 · Kimi K2.5

A newer optimizer (Momentum Orthogonalized by Newton-Schulz) that orthogonalizes each weight-matrix update instead of scaling it per-element like Adam. Used at scale by Kimi K2.5 via the MuonClip variant.

MuonClipIntroduced in §24 · Kimi K2.5

A stabilized variant of the Muon optimizer (used by the Kimi models) that clips/rescales attention query-key logits to prevent the loss spikes that can derail very large training runs.

native multimodal pre-trainingIntroduced in §24 · Kimi K2.5

Training on a mix of text and other modalities (e.g. vision) from the very start, with a constant ratio, rather than bolting a modality onto a finished text model late in training. Kimi K2.5's approach.

negative log-likelihoodIntroduced in §02 · The objective

Another name for the cross-entropy LM (Language Model) loss: −log of the probability the model gave to the correct token. Big when the model was confidently wrong, small when it was confidently right.

neural networkIntroduced in §01 · The prediction game

A function built by stacking many simple operations — mostly matrix multiplies with nonlinearities between them — whose behavior is shaped by tuning billions of internal numbers (its parameters) from data.

neuron (unit)Introduced in §21 · A short history

A single computing unit in a network: a weighted sum of its inputs plus a bias, passed through an activation. Loosely inspired by biological neurons.

next sentence predictionIntroduced in §11 · BERT

Next Sentence Prediction (NSP) — a secondary BERT objective: given two sentences, predict whether the second actually follows the first. Later work found it largely unnecessary.

next-token predictionIntroduced in §23 · From this loop to an LLM

The pre-training objective for GPT-style models: given the tokens so far, predict a probability distribution over the next token. Also called causal or autoregressive language modeling.

nonlinear functionIntroduced in §01 · What is an LLM?

A function whose output isn't just a scaled, shifted copy of its input — e.g. ReLU, GELU, sigmoid. Stacking nonlinearities between matrix multiplies is what lets a neural net represent anything more interesting than scaling and rotation.

NTK-aware scalingIntroduced in §06 · Positional encoding

A RoPE-extension trick: instead of linearly shrinking all positions (which over-compresses the fast-spinning low-i pairs), adjust the rotation base — the 10000 in 10000^(2i/d) — so high-frequency pairs are preserved while only the slow pairs get stretched. Named after the Neural Tangent Kernel theory it was originally motivated by. Better quality than plain position interpolation at modest extension factors.

NVLinkIntroduced in §13 · GPU memory hierarchy

Nvidia’s high-speed GPU-to-GPU interconnect. H100 NVLink ≈ 900 GB/s per GPU — much faster than PCIe.

off-policyIntroduced in §13 · Policy gradients & REINFORCE

RL that learns from data generated by a different (older or separate) policy. DPO and rejection-sampling methods are off-policy / offline.

offline RLIntroduced in §20 · Direct Preference Optimization

Optimizing from a fixed dataset of responses and preferences without generating new rollouts during training. DPO and rejection-sampling methods are offline.

omni-modalIntroduced in §27 · Qwen3.5-Omni

A model natively pre-trained to handle all major modalities at once — text, images, audio, and video — jointly, rather than text plus a single added modality.

on-policyIntroduced in §13 · Policy gradients & REINFORCE

RL where the data used to update the policy was generated by the current policy. PPO and GRPO are (approximately) on-policy; they resample as the policy changes.

on-policy distillationIntroduced in §31 · On-policy distillation

Distillation where the student generates its own rollouts and a teacher grades every token of them. Combines the dense per-token signal of distillation with the on-policy benefit of RL — much cheaper than RL for transferring a teacher's reasoning.

one-hot vectorIntroduced in §03 · Turning the world into numbers

A vector that is 1 at a single index and 0 everywhere else. Useful for representing categories without inventing a fake numeric order.

online RLIntroduced in §18 · PPO for RLHF in practice

RL that generates fresh rollouts from the current policy during training (e.g. PPO, GRPO). Expensive but adaptive, since the data tracks the improving policy.

optimizerIntroduced in §09 · Tuning the descent

The rule that turns gradients into parameter updates. Plain gradient descent is the simplest; Adam-family optimizers add per-parameter adaptive step sizes.

optimizer statesIntroduced in §06 · Compute & memory

Extra per-parameter values an optimizer maintains — for Adam, the first and second moment estimates. In FP32 these add 8 bytes per parameter, often dwarfing the weights themselves.

ORPOIntroduced in §21 · The DPO zoo

Odds-Ratio Preference Optimization — folds SFT and preference optimization into a single reference-free stage using an odds-ratio penalty term.

outcome reward model (ORM)Introduced in §24 · Process vs outcome rewards

A reward model that scores only the final answer of a solution, ignoring how it was reached. Simpler than a PRM but gives sparser credit.

over-trainingIntroduced in §17 · Llama 3

Deliberately training a model on far more tokens than the compute-optimal ~20 per parameter. It costs more training compute for a slightly better, much smaller model that is cheaper to run at inference.

overfittingIntroduced in §18 · Learning vs. memorizing

When a model memorizes training-set quirks instead of learning general patterns, so it does well on training data but poorly on new data. Rarely the main worry in single-epoch LLM pre-training, but it shapes data choices.

paddingIntroduced in §08 · The data pipeline

Filler tokens added to a sequence to reach a fixed length. Padding wastes compute — the model still processes the meaningless tokens — which is exactly what sequence packing exists to avoid.

pageIntroduced in §15 · Paged attention

A fixed-size slab of KV cache memory (e.g. 16 tokens). The unit vLLM allocates and frees.

page tableIntroduced in §15 · Paged attention

Per-sequence mapping from logical position → physical page in the KV cache. Same idea as OS virtual memory, applied to attention.

pairwise comparisonIntroduced in §08 · Learning from human preferences

Asking a labeler which of two responses is better, rather than scoring each on an absolute scale. Easier and more reliable for humans, and the basis of preference learning.

parametersIntroduced in §04 · The simplest model

The numbers (weights) inside a model that get adjusted during training. A “7B model” has 7 billion of them.

PCIeIntroduced in §13 · GPU memory hierarchy

The bus between the GPU and the host (CPU/RAM/NVMe). PCIe Gen5 x16 ≈ 64 GB/s — far slower than HBM.

perceptronIntroduced in §21 · A short history

Rosenblatt's 1958 single trainable linear unit — the earliest artificial neuron. Its inability to solve XOR helped trigger the first AI winter.

perplexityIntroduced in §02 · The objective

The exponential of the cross-entropy loss — roughly "how many equally-likely tokens is the model choosing between?" Lower is better; a perplexity of 1 means perfect prediction.

pipeline bubbleIntroduced in §07 · Parallelism

Idle GPU time at the start and end of a pipeline-parallel batch, while stages wait for the first micro-batches to flow through. Smaller micro-batches shrink the bubble.

pipeline parallelismIntroduced in §19 · Scaling out

Splitting the model layer-wise across GPUs. Each GPU owns a contiguous slab of layers; activations flow from one to the next.

policyIntroduced in §13 · Policy gradients & REINFORCE

In RL, the thing that chooses actions — here, the language model itself, viewed as a distribution over next tokens given the context. RL post-training optimizes the policy.

policy gradientIntroduced in §13 · Policy gradients & REINFORCE

A family of RL methods that directly adjust the policy’s parameters in the direction that increases expected reward, using the score-function (REINFORCE) estimator.

position interpolation (PI)Introduced in §06 · Positional encoding

A RoPE-extension trick: linearly scale incoming positions down so a model trained at length L "sees" a longer context as if it were still length L. To go from 4k to 16k, divide all positions by 4 before rotating. Cheap, effective for short extensions, but degrades quality on the tasks the model was already good at.

positional encodingIntroduced in §06 · Positional encoding

Information added to embeddings so the model knows where each token sits in the sequence.

post-trainingIntroduced in §01 · What is post-training?

Everything done to a model after pre-training to turn a raw next-token predictor into a useful assistant: supervised fine-tuning, RLHF, and RL from verifiable rewards.

power lawIntroduced in §14 · Scaling laws

A relationship of the form y = a·x^(−b): on log-log axes it's a straight line. Pre-training loss follows a power law in scale, so each 10× of compute buys a roughly constant drop in loss.

PPOIntroduced in §09 · Optimizing against the reward

Proximal Policy Optimization (Schulman, 2017) — nudges the policy toward higher reward in small, clipped steps with a KL leash to a reference model. The RLHF workhorse: stable, simple, widely used.

pre-normIntroduced in §12 · GPT-2

Placing the normalization layer before each sub-layer (inside the residual branch) rather than after it. Pre-norm transformers are far more stable to train at depth, and became standard after GPT-2.

pre-trainingIntroduced in §01 · What is pre-training?

The first phase of building a language model: training on an enormous corpus of raw text to predict the next token, learning general-purpose language ability before any task-specific tuning.

precisionIntroduced in §20 · Judging a model

Of the cases the model flagged positive, the fraction that truly are positive: TP / (TP + FP). High precision = few false alarms.

preference dataIntroduced in §08 · Learning from human preferences

Data where humans (or an AI) compare two or more model responses to the same prompt and mark which is better. The training signal for reward models and DPO.

prefillIntroduced in §11 · Prefill and decode

The first forward pass that processes the entire prompt at once. Compute-bound, parallel over prompt tokens.

prefix cachingIntroduced in §16 · Prefix caching

Sharing KV pages across requests that start with the same tokens (system prompts, few-shot prefixes), so the prefill is computed once.

probability distributionIntroduced in §02 · Probability, policies & gradients

An assignment of a non-negative probability to every possible outcome that sums to one. A language model produces one over the vocabulary at each step — its bet on the next token.

process reward model (PRM)Introduced in §24 · Process vs outcome rewards

A reward model that scores each step of a reasoning chain, not just the final answer — giving denser, better-targeted credit. Trained on per-step correctness labels.

process supervisionIntroduced in §24 · Process vs outcome rewards

Training or rewarding a model on the correctness of intermediate reasoning steps rather than just outcomes — the idea behind PRMs and "Let’s Verify Step by Step."

promptIntroduced in §01 · What is an LLM?

The input text fed to the model — what you want it to continue or respond to.

quality filteringIntroduced in §08 · The data pipeline

Discarding low-value text (spam, boilerplate, gibberish) using heuristics and trained classifiers, keeping the corpus closer to the kind of text you want the model to learn.

queryIntroduced in §04 · Attention

A vector asking “what am I looking for in other tokens?”. Computed per token, used to score against keys.

RAFTIntroduced in §22 · Rejection-sampling alignment

Reward-rAnked Fine-Tuning — iteratively sample, rank by reward, and fine-tune on the top responses. Offline, RL-free preference alignment.

random variableIntroduced in §02 · Probability, policies & gradients

A quantity whose value is an outcome of a random process — e.g. the next token, which isn't fixed until it's drawn from the model's distribution.

RDMAIntroduced in §13 · GPU memory hierarchy

Remote DMA — letting one node’s NIC write directly into another node’s memory without involving the CPU. The basis of InfiniBand and RoCE.

reasoning modelIntroduced in §25 · Inference scaling & o1

A model trained (usually with RL) to produce long internal chains of thought before answering, trading inference compute for accuracy on hard problems. o1 and DeepSeek-R1 are examples.

recallIntroduced in §20 · Judging a model

Of all truly positive cases, the fraction the model caught: TP / (TP + FN). High recall = few misses. Usually traded against precision.

receptive fieldIntroduced in §17 · Shapes of networks

The region of the input that a given unit actually looks at. Small near the input in a CNN, growing with depth.

recurrent network (RNN)Introduced in §17 · Shapes of networks

An architecture that processes a sequence one step at a time, carrying a memory (hidden state) forward. A pre-transformer approach to sequences.

red teamingIntroduced in §04 · The alignment problem

Deliberately probing a model for failures — adversarially crafting prompts that try to elicit harmful, unsafe, or otherwise unwanted behavior — to surface weaknesses that ordinary testing misses and generate data for fixing them.

reference modelIntroduced in §10 · RLHF scales to language

A frozen copy of the policy (usually the SFT model) that RLHF and DPO stay close to via a KL penalty, preventing the optimized policy from drifting into degenerate text.

regressionIntroduced in §02 · Kinds of learning

A prediction task whose answer is a number (a price, a temperature), as opposed to a category. Usually scored with squared error.

regularizationIntroduced in §18 · Learning vs. memorizing

A change to training that tries to improve performance on new data, often by limiting how freely the model can fit the training examples. Weight penalties, dropout, and early stopping are common forms.

REINFORCEIntroduced in §13 · Policy gradients & REINFORCE

The basic Monte-Carlo policy-gradient estimator (Williams, 1992): scale the gradient of each action’s log-probability by the reward (or advantage) it earned. Everything else builds on it.

REINFORCE++Introduced in §28 · GRPO refinements

A critic-free baseline that adds PPO-style stabilizers (token-level KL, clipping) to plain REINFORCE, aiming for robustness without a value network.

reinforcement learningIntroduced in §02 · Kinds of learning

Learning from trial and error: an agent takes actions and receives a single-number reward signal, with no labeled "right answer" for each step.

rejection samplingIntroduced in §22 · Rejection-sampling alignment

Generate several candidate responses, keep only the best-scoring one(s) by some reward or verifier, and fine-tune on those. A simple, stable, RL-free way to improve a model.

representation learningIntroduced in §12 · Why depth wins

When a network discovers useful features from raw data on its own — building higher-level features out of lower-level ones — instead of relying on hand-designed inputs.

residual connectionIntroduced in §16 · When gradients vanish

output = x + f(x). Lets gradients flow through deep stacks and means each block adds a refinement rather than rewriting.

returnIntroduced in §13 · Policy gradients & REINFORCE

The total (often discounted) reward accumulated over a trajectory. Policy-gradient methods push up the probability of actions that led to high return.

reverse KLIntroduced in §31 · On-policy distillation

The KL divergence measured with the student's distribution in front, D(student ‖ teacher). It is mode-seeking — it pushes the student to concentrate on what the teacher actually does rather than spread mass thinly. Used as the per-token signal in on-policy distillation.

rewardIntroduced in §02 · Kinds of learning

A single-number feedback signal in reinforcement learning that says how good an outcome was. The agent tries to collect as much reward as possible over time.

reward ensembleIntroduced in §19 · Reward hacking & over-optimization

Using several reward models and aggregating (e.g. taking the minimum) to make hacking harder — a policy must fool all of them at once.

reward hackingIntroduced in §19 · Reward hacking & over-optimization

When a policy finds ways to score high on the reward model without actually being better — exploiting quirks of an imperfect proxy. A central danger of RL post-training.

reward model (RM)Introduced in §11 · Reward models

A model trained from human preference data to output a scalar score for how good a response is. Stands in for a human judge so RL can query reward millions of times.

reward over-optimizationIntroduced in §19 · Reward hacking & over-optimization

Pushing the policy so hard against a proxy reward that true quality starts to fall even as the proxy keeps rising — the quantitative face of reward hacking (Gao et al., 2022).

RewardBenchIntroduced in §35 · Recap

A standard benchmark for evaluating reward models across chat, safety, and reasoning, making reward-model quality measurable and comparable.

RL scaling lawsIntroduced in §32 · RL scaling laws

Empirical curves predicting how RL post-training performance grows with compute — the RL analogue of pre-training scaling laws. Recent work fits a sigmoidal curve that can be extrapolated from small runs.

RLAIFIntroduced in §12 · RLAIF & Constitutional AI

Reinforcement Learning from AI Feedback — replace human preference labels with labels from another model (or the model itself), making the feedback loop cheap and scalable.

RLHFIntroduced in §08 · Learning from human preferences

Reinforcement Learning from Human Feedback — train a reward model on human preference comparisons, then optimize the policy against that reward with RL (typically PPO), with a KL leash to a reference.

RLOOIntroduced in §28 · GRPO refinements

REINFORCE Leave-One-Out — use the average reward of the other samples in a group as each sample’s baseline. A simple, critic-free policy-gradient method for LLMs.

RLVRIntroduced in §26 · RL from verifiable rewards

Reinforcement Learning from Verifiable Rewards — use an automatic checker (unit tests, an answer key, a math grader) as the reward instead of a learned reward model. No reward hacking of a neural proxy.

RMSNormIntroduced in §08 · A full transformer block

Root Mean Square Normalization — a normalization layer that divides each activation by the root-mean-square (√(mean(x²))) of the whole vector, then multiplies by a learned per-dimension scale. Cheaper than LayerNorm (no mean subtraction, no learned bias) and empirically just as good. Standard in Llama-class models.

RMSPropIntroduced in §09 · Tuning the descent

An optimizer that divides each parameter's step by the root-mean-square of its recent gradients, giving every parameter its own adaptive step size. A precursor to Adam.

rolloutIntroduced in §02 · Probability, policies & gradients

A complete generated sample from the policy — for an LLM, one full response to a prompt. RL collects rollouts, scores them, and updates the policy.

RoPEIntroduced in §06 · Positional encoding

Rotary Position Embeddings — rotates Q/K vectors by an angle proportional to position. Standard in modern LLMs.

rubric-based rewardIntroduced in §33 · Generative reward models

Scoring a response against an explicit written checklist of criteria instead of a black-box score. More transparent and harder to game — the policy must satisfy named requirements — and the criteria can be generated automatically.

saddle pointIntroduced in §07 · Which way is downhill?

A flat-looking point that is downhill in some directions but uphill in others — like the center of a horse saddle. A zero gradient can mark a saddle rather than a minimum.

samplingIntroduced in §10 · Sampling

Choosing the next token from logits — greedy (argmax), temperature scaling, top-k, top-p, etc.

scalable oversightIntroduced in §12 · RLAIF & Constitutional AI

The challenge of supervising models on tasks too hard or numerous for humans to label directly — addressed by AI feedback, critiques, and verifiers.

scaled dot-product attentionIntroduced in §04 · Attention

softmax(QKᵀ / √d_k) · V — the canonical attention formula from “Attention is All You Need”.

scaling hypothesisIntroduced in §12 · GPT-2

The idea — crystallized around GPT-2 — that simply scaling up model size, data, and compute keeps improving capabilities, without needing fundamentally new architectures.

scaling lawsIntroduced in §14 · Scaling laws

Empirical formulas showing that test loss falls as a smooth power law in model size, dataset size, and compute. They let you predict a large model's performance from small experiments.

schedulerIntroduced in §14 · Continuous batching

The component that picks which requests run in the next forward pass given GPU memory and policy constraints.

score-function estimatorIntroduced in §13 · Policy gradients & REINFORCE

The identity ∇E[R] = E[R · ∇log π] that lets us estimate a reward gradient by sampling, even though the reward itself isn’t differentiable in the model’s parameters.

self-attentionIntroduced in §17 · Shapes of networks

Attention where the queries, keys, and values all come from the same sequence, so each position can gather information from other allowed positions in that sequence.

self-consistencyIntroduced in §23 · Bootstrapping reasoning

Sample many chain-of-thought solutions and take the majority-vote answer. A test-time technique that trades extra compute for accuracy.

Self-InstructIntroduced in §07 · Synthetic & self-generated data

A method that bootstraps instruction-tuning data from a model itself: seed it with a few tasks, have it generate many more, filter, and fine-tune. Made instruction data cheap and synthetic.

self-supervised learningIntroduced in §02 · Kinds of learning

Training where the labels come from the data itself — e.g. hide part of an example and ask the model to predict it. No human annotation needed.

SentencePieceIntroduced in §08 · The data pipeline

A tokenizer toolkit that operates directly on raw text (treating spaces as symbols), so it works language-agnostically without pre-splitting on whitespace.

sequence packingIntroduced in §08 · The data pipeline

Concatenating many short documents into full-length training sequences (with separators) so no compute is wasted padding to the context length.

sequence parallelismIntroduced in §07 · Parallelism

Splitting the work along the token/sequence dimension across GPUs, often paired with tensor parallelism to shard the normalization and dropout activations it leaves behind.

SGDIntroduced in §08 · Gradient descent

Stochastic Gradient Descent — gradient descent using a noisy gradient estimated from one mini-batch at a time rather than the whole dataset.

shared expertIntroduced in §18 · DeepSeek-V3

In DeepSeekMoE, an expert that every token always passes through (alongside a few routed experts), used to capture common knowledge so the routed experts can specialize.

sigmoidIntroduced in §06 · Drawing a boundary

The logistic function σ(z)=1/(1+e⁻ᶻ), which squashes any real score into a probability between 0 and 1. Turns a linear model into a binary classifier.

SimPOIntroduced in §21 · The DPO zoo

Simple Preference Optimization — a reference-free DPO variant using a length-normalized implicit reward plus a target margin, removing the need for a reference model.

sliding-window attentionIntroduced in §20 · Gemma 2

Restricting attention to a fixed-size window of nearby tokens instead of the whole sequence. Cheaper and smaller-KV than global attention; modern models interleave local (windowed) and global layers.

SLOIntroduced in §20 · Throughput vs latency

Service Level Objective — a target like “p99 TTFT < 1 s”. Serving systems are tuned to maximize throughput subject to SLOs.

softmaxIntroduced in §06 · Drawing a boundary

Function that turns any vector into a probability distribution (positive, sums to 1) by exponentiating and normalizing.

span corruptionIntroduced in §13 · T5

T5's pre-training objective: replace random contiguous spans of tokens with sentinel placeholders and train the model to reconstruct the missing spans. A denoising objective.

special tokensIntroduced in §06 · The SFT stage in practice

Reserved tokens (e.g. role markers and end-of-turn markers) added to the vocabulary to delimit structure that ordinary text tokens cannot express.

speculative decodingIntroduced in §18 · Speculative decoding

A small draft model proposes K tokens; the big target model verifies them all in one pass. Net effect: more tokens per target-model step.

SRAMIntroduced in §13 · GPU memory hierarchy

Static Random-Access Memory — the on-chip scratchpad / L1+shared memory inside each SM. Tiny (~100s of KB per SM) but ~10× faster than HBM.

SSM / hybrid architecturesIntroduced in §12 · The KV cache

State-Space Models (SSMs) replace attention with a recurrent operator (Mamba, RWKV) that compresses the entire past into a fixed-size hidden state — no KV cache to grow with sequence length. Hybrids (Jamba, Zamba, RecurrentGemma) interleave a few attention layers with many SSM layers, keeping most of the recall power of attention while shrinking the KV cache by 5–20×. They're a different bet on the same memory problem.

standardizationIntroduced in §03 · Turning the world into numbers

Putting numeric features on comparable scales by subtracting each feature's training-set average and dividing by its typical spread. This often makes slope-following training easier.

STaRIntroduced in §23 · Bootstrapping reasoning

Self-Taught Reasoner (Zelikman, 2022) — generate chain-of-thought rationales, keep those that reach the correct answer, fine-tune on them, and repeat. Bootstraps reasoning from a model’s own correct attempts.

stochasticIntroduced in §02 · Probability, policies & gradients

Involving randomness, so the outcome varies from run to run rather than being fixed. Sampling is stochastic — the same prompt can yield different responses; greedy decoding is deterministic.

supervised fine-tuning (SFT)Introduced in §05 · Instruction tuning is born

Training a pre-trained model on curated (prompt, response) pairs with the ordinary next-token objective, so it imitates demonstrated assistant behavior. The first stage of post-training.

supervised learningIntroduced in §02 · Kinds of learning

Training on examples where each input is paired with a human-provided correct answer (a label). Regression and classification both live here; this explainer focuses on it.

SwiGLUIntroduced in §07 · The MLP block

A gated MLP variant (Llama, PaLM): output = SiLU(xW₁) ⊙ (xW₂), then projected. Outperforms plain MLPs at the same param count.

SXM5Introduced in §13 · GPU memory hierarchy

Server PCI eXpress Module, 5th generation — Nvidia's proprietary mezzanine board form factor for datacenter GPUs. (Despite the name, SXM bypasses PCIe entirely.) An H100 SXM5 module plugs directly into the motherboard via the SXM socket, which gives it more power (700 W vs ~350 W for PCIe), more NVLink bandwidth (900 GB/s per GPU), and higher HBM bandwidth than the PCIe variant of the same chip. Standard in HGX/DGX servers; what you get in most cloud H100 instances.

sycophancyIntroduced in §04 · The alignment problem

A failure mode where a model tells the user what it thinks they want to hear rather than what is true or correct — often a side effect of preference optimization.

symmetry breakingIntroduced in §15 · Where training starts

Initializing weights to different random values so units in a layer can specialize, instead of all computing the same thing and receiving identical gradients forever.

synthetic dataIntroduced in §21 · Synthetic data

Training text generated by another model or an automated pipeline, rather than scraped from humans. Used to augment scarce high-quality data; its benefits in pre-training are conditional.

system promptIntroduced in §06 · The SFT stage in practice

A special leading instruction that sets the assistant’s persona, rules, and constraints for a conversation, separate from the user’s turns.

teacher forcingIntroduced in §02 · The objective

During training, feeding the model the true previous tokens (not its own guesses) at every position, so all next-token predictions in a sequence can be learned in parallel.

temperatureIntroduced in §10 · Sampling

Divides logits before softmax. <1 sharpens (more deterministic), >1 flattens (more random). 0 = greedy.

tensor parallelismIntroduced in §19 · Scaling out

Splitting each weight matrix across N GPUs. Every GPU does a slice of every layer; activations get all-reduced across them.

test-time computeIntroduced in §25 · Inference scaling & o1

Compute spent at inference — longer chains of thought, more samples — to improve answer quality, as opposed to compute spent during training.

text-to-textIntroduced in §13 · T5

T5's framing in which every task — translation, classification, summarization — is cast as "input text → output text", so one model and one objective handle all of them.

The StackIntroduced in §23 · The big picture

A large dataset of permissively-licensed source code (~3 TB, 30+ programming languages) scraped from public GitHub repositories, from the BigCode project; used to train code models like StarCoder. Not related to the Stack Overflow Q&A site.

throughputIntroduced in §11 · Prefill and decode

Total tokens generated per second across all concurrent requests. Often traded against per-request latency.

tokenIntroduced in §22 · Embeddings

The atomic unit of text the model sees. Roughly a word-fragment — “tokenization” is a piece of text → list of token IDs.

token IDIntroduced in §02 · Tokens

An integer index into the vocabulary that uniquely identifies a token.

tokenizerIntroduced in §08 · The data pipeline

The program that converts raw text into a sequence of integer token IDs (and back). Its vocabulary and merge rules are fixed before pre-training begins.

tokens per parameterIntroduced in §16 · Chinchilla

The ratio of training tokens to model parameters (D/N). Chinchilla's compute-optimal point is around 20; modern models often deliberately exceed it to get smaller, cheaper-to-serve models.

tool-use RLIntroduced in §30 · Agentic & tool-use RL

Training a model with RL to call external tools (search, code execution, calculators) effectively, rewarding trajectories that use tools to reach correct outcomes.

top-kIntroduced in §10 · Sampling

Only sample from the k highest-probability tokens; zero out the rest.

top-pIntroduced in §10 · Sampling

Nucleus sampling — keep the smallest set of tokens whose cumulative probability ≥ p, sample from that set.

train/test splitIntroduced in §18 · Learning vs. memorizing

Holding back part of your data from training so you can measure the model on examples it never saw. Good results there are evidence that its patterns carry over to similarly collected new data.

training stepIntroduced in §10 · Batches, epochs & noise

One iteration of the loop: forward pass on a batch, backward pass to get gradients, optimizer update. A large model is trained for hundreds of thousands of steps.

trajectoryIntroduced in §02 · Probability, policies & gradients

The sequence of states and actions in a rollout. For text generation, the tokens generated one after another, each conditioned on those before it.

transfer learningIntroduced in §10 · GPT-1

Learning general skills on one task (here, next-token prediction on huge text) and reusing them on other tasks. Pre-training plus adaptation is the transfer-learning recipe behind modern LLMs.

transformerIntroduced in §17 · Shapes of networks

A neural-network architecture introduced in "Attention Is All You Need" (2017), built from stacked self-attention and feed-forward layers.

translation equivarianceIntroduced in §17 · Shapes of networks

A shift-in, shift-out property: when the input image moves, the grid of detected features moves with it. This differs from invariance, where the final answer would not change at all.

translation invarianceIntroduced in §17 · Shapes of networks

A property where shifting an input does not change the final answer. CNNs can become less sensitive to shifts, but ordinary convolution does not guarantee exact invariance.

triangle inequalityIntroduced in §03 · From next-token to behavior

The rule that a direct distance is never longer than a detour: d(a, c) ≤ d(a, b) + d(b, c). It is one of the properties a true distance metric must satisfy — and one that KL divergence does not, which is why KL is a divergence rather than a distance.

TRPOIntroduced in §17 · TRPO to PPO

Trust Region Policy Optimization (Schulman, 2015) — take the largest policy-gradient step that stays within a trust region (a KL bound), guaranteeing stable improvement. PPO’s parent.

truncationIntroduced in §08 · The data pipeline

Cutting a document off at the model's maximum context length and discarding the rest. It avoids overflow but throws away data and can split documents mid-thought.

trust regionIntroduced in §17 · TRPO to PPO

A bound on how far the policy may move in one update (measured in KL divergence), so the update stays in the region where the local approximation is trustworthy.

TTFTIntroduced in §11 · Prefill and decode

Time-to-First-Token — wall-clock from request submitted to first generated token returned. Dominated by prefill.

Tülu 3Introduced in §29 · Scaling open post-training

Allen AI’s fully open post-training recipe (2024) — SFT, then DPO, then RLVR — released with data, code, and evals. A reference manual for open post-training.

turn-level rewardIntroduced in §30 · Agentic & tool-use RL

A reward assigned to individual turns or tool calls within a multi-turn trajectory, giving denser feedback than a single end-of-episode reward.

underfittingIntroduced in §18 · Learning vs. memorizing

When a model is too simple (or too constrained) to capture the real pattern, so it does poorly even on the training data. The opposite failure from overfitting.

unsupervised learningIntroduced in §02 · Kinds of learning

Finding structure in unlabeled data — grouping, compressing, or otherwise organizing it — with no answer key to imitate.

validation setIntroduced in §19 · Data quality & the three splits

A held-out slice of data looked at repeatedly to compare models and tune hyperparameters — kept separate from the final, touch-once test set.

valueIntroduced in §04 · Attention

A vector representing the content actually mixed into the output when a token gets attended to.

value functionIntroduced in §15 · Value & advantage

The expected return from a given state under the current policy. A learned value function (the critic) provides a baseline that reduces the variance of policy-gradient updates.

vanishing gradientIntroduced in §16 · When gradients vanish

When the combined effect of many layers shrinks the backward learning signal toward zero, leaving early layers with almost nothing to learn from.

VAPOIntroduced in §28 · GRPO refinements

Value-Augmented PPO (2025) — brings a well-trained critic back for long chain-of-thought RL, building on DAPO’s tricks to beat critic-free methods on reasoning.

verifierIntroduced in §26 · RL from verifiable rewards

An automatic, often rule-based checker that returns whether a response is correct (e.g. runs unit tests, compares to a known answer). Provides the reward in RLVR.

virtual memory pagingIntroduced in §15 · Paged attention

The operating-system technique that gives each program the illusion of one large contiguous memory while physically storing it as small fixed-size pages scattered across RAM. vLLM's PagedAttention applies the same idea to the KV cache.

vision encoderIntroduced in §22 · Gemma 3

A module (such as SigLIP) that converts an image into a sequence of embedding vectors the language model can attend to, as if they were tokens. The bridge that makes a text model multimodal.

vLLMIntroduced in §01 · What is an LLM?

An open-source LLM inference engine, originally from UC Berkeley, that introduced paged attention and is now one of the most widely used serving systems for open-weight models.

vocabularyIntroduced in §23 · From this loop to an LLM

The fixed set of tokens a model knows about. Modern LLMs have ~32k–200k entries.

warmupIntroduced in §04 · Optimizers & schedules

Starting training with a tiny learning rate and ramping it up over the first few thousand steps, to avoid blowing up the still-random early model.

WebTextIntroduced in §12 · GPT-2

The dataset behind GPT-2: ~8 million web pages reached via outbound Reddit links with at least 3 karma, used as a quality filter. About 40 GB of text.

weightIntroduced in §04 · The simplest model

One adjustable number that scales an input inside a model — how strongly that input pushes the prediction up or down. The weights (plus biases) are the parameters training adjusts.

weight decayIntroduced in §09 · Tuning the descent

A regularizer that shrinks parameters toward zero a little each step, discouraging large weights and improving generalization.

weight sharingIntroduced in §17 · Shapes of networks

Reusing the same weights at many positions (as a CNN filter does), so a pattern learned once is recognized everywhere and far fewer parameters are needed.

WordPieceIntroduced in §08 · The data pipeline

A subword tokenization algorithm (used by BERT) closely related to Byte Pair Encoding, building a vocabulary of word pieces from frequent character sequences.

Xavier initializationIntroduced in §15 · Where training starts

A starting rule that chooses weight sizes from a layer's number of inputs and outputs, aiming to keep values and gradients from rapidly shrinking or growing. Also called Glorot initialization.

YaRNIntroduced in §06 · Positional encoding

Yet another RoPE eNtension method. Combines NTK-aware scaling with a length-dependent attention-score scaling and a "ramp" that smoothly transitions between high- and low-frequency treatment. Currently the highest-quality way to extend a RoPE model's context length without retraining; used to ship Llama-3, Qwen-2, and others at 128k+ contexts.

ZeROIntroduced in §07 · Parallelism

Zero Redundancy Optimizer — a family of techniques that shard optimizer states, gradients, and optionally parameters across data-parallel GPUs so no device holds a full redundant copy.

zero-shotIntroduced in §12 · GPT-2

Performing a task from instructions alone, with no examples given. GPT-2 showed a pre-trained LM can do many tasks zero-shot, just by being prompted.