DAILY BRIEF / SPECULATIVE DECODING
Speculative decoding gains depend on load and hardware.
What changed
On RTX 5090, DFlash gave 3.50x autoregressive throughput at low concurrency but 0.76x at high concurrency with short outputs.
Why it matters
Tune speculation per GPU and per load level. SpecSearch reports 2.01x over autoregressive and 1.36x over fixed speculation in vLLM.
Scope and limitations
A new preprint with no code release stated. Results cover RTX 5090, H100, L40 and MI300X.
Go to the source
A sourced technical brief published 18 Nov 2026. The implication is turingfold’s interpretation. We have not independently benchmarked this change.