GLM-5.2

IndexShare and the solid 1M-token context

Sources: GLM-5.2: Built for Long-Horizon Tasks — Z.ai, 2026; IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse — Bai et al., 2026

GLM-5.2 (Z.ai, 2026) closes our frontier tour with the same target DeepSeek-V4 chased in chapter 26: a million-token context windowcontext lengthThe maximum number of tokens the model can attend to at once (also called the context window or sequence length). Pre-training picks a context length; later stages often extend it.See in glossary → that is actually affordable. But where V4 attacked the problem by compressing attention, GLM-5.2 attacks the piece that compression itself introduced: the little model-inside-the-model that decides which history to look at. And, in keeping with the open-model theme of this whole era, it ships under an MIT license, with weights on HuggingFace.

Sparse attention needs an index

GLM’s architecture uses DeepSeek Sparse Attention (DSADeepSeek Sparse AttentionDeepSeek Sparse Attention (DSA): an attention variant where a lightweight indexer scores every past token with cheap dot products, a top-k selection keeps the most relevant ones, and full attention runs over only that subset.See in glossary →), from the same lineage as V4’s compressed sparse attentionCompressed Sparse AttentionCompressed Sparse Attention (CSA) — a DeepSeek-V4 attention variant that attends to a compressed, sparsely-selected subset of past tokens to make million-token context affordable.See in glossary →. The idea: instead of attending to all past tokens, a lightweight indexer scores every past token with cheap dot products, a top-k selection keeps only the most relevant ones, and full attention runs over just that subset.

That makes the expensive part of attention scale with $k$ instead of with the sequence length. But look at what’s left: the indexer still touches every past token, in every layer. At a million tokens of context, the “lightweight” scoring pass and its top-k selection stop being a rounding error and become the dominant per-token cost. The sparse attention’s bodyguard is now the one eating the budget.

IndexShare

GLM-5.2’s answer, IndexShareIndexShareSharing one sparse-attention indexer across a block of consecutive layers (or across speculative-decoding draft steps): the first computes which past tokens matter and the rest reuse its top-k selection, eliminating most indexer compute.See in glossary →, is disarmingly simple: every 4 transformer layers share a single indexer. The indexer runs once, at the first layer of each block of four, and the next three layers reuse its top-k indices as-is. Three quarters of all indexer dot-product and top-k computation vanishes. At a 1M-token context, this cuts per-token FLOPs by 2.9×.

The underlying paper (published as IndexCache, and validated on the production GLM-5 model) shows why this is safe: neighboring layers largely agree about which past tokens matter, so a nearby layer’s selection is a very good stand-in for your own, with negligible quality loss even when 75% of indexer computation is removed.

Just as interesting is when the trick was trained in. GLM-5.2 adopts IndexShare from mid-trainingmid-trainingA phase between the main pre-training run and post-training, used to inject specialized data or capabilities (e.g. long context, code-from-execution) while still training the base model on a next-token-style objective.See in glossary → onward, with 128K-token sequences: the base model’s early training is unchanged, and the shared-index behavior is learned during the long-context phase where it matters. The result outperforms GLM-5.1 on long-context benchmarks while spending less compute to get there.

Z.ai’s framing of “solid 1M context” is worth keeping too: a long context window is easy to claim, and much harder to keep useful across long, messy agent trajectories. GLM-5.2’s 1M-context training was deliberately expanded over coding-agent scenarios (large-scale implementation, debugging, performance work) rather than just long documents. On coding benchmarks it is the strongest open-weights model of its moment: 81.0 on Terminal-Bench 2.1 against GLM-5.1’s 63.5, within a few points of the closed frontier.

That completes our tour of the papers, from a 65-million-parameter translation model in 2017 to omni-modal, million-token, trillion-parameter systems in 2026. The final chapter steps back to trace the through-line: what changed, what didn’t, and why.