DAILY BRIEF / ATTENTION

Sparse attention moves the bottleneck to the indexer.

Published Source

What changed

At 128K input, the indexer and Top-k take 47% of a DeepSeek V3.2 prefill layer; one key head per group avoids a 7.6x decode slowdown.

Why it matters

When profiling sparse-attention models, measure indexer and Top-k selection as their own costs, not as part of attention.

Scope and limitations

NVIDIA reports normalized runtimes with no GPU model or absolute times. The 2.1-2.4x Top-k gain over radix select is for TensorRT LLM on DeepSeek V3.2/V4 decode.

Go to the source

A sourced technical brief published 18 Nov 2026. The implication is turingfold’s interpretation. We have not independently benchmarked this change.

← Back to Daily Brief