DAILY BRIEF / MEMORY

SGLang pools KV and recurrent state in one arena.

Published Source

What changed

With --enable-unified-memory, GPT-OSS-20B reached about 1,000 concurrent requests vs 618, with 14% more throughput, on one H200.

Why it matters

For hybrid sliding-window or recurrent models, one shared pool can raise concurrency without adding GPUs.

Scope and limitations

Experimental and for hybrid models only. Not for full-attention models or DeepSeek-V4, and not compatible with LMCache, FP4 KV or two-batch overlap.

Go to the source

A sourced technical brief published 18 Nov 2026. The implication is turingfold’s interpretation. We have not independently benchmarked this change.

← Back to Daily Brief