DAILY BRIEF / SERVING ENGINES

vLLM v0.34.0 changes memory and block-size defaults.

Published Source

What changed

Extensible KV cache is on by default, gpu_memory_utilization defaults to 1.0, block size is now 64, and Model Runner V1 is gone.

Why it matters

Re-check memory headroom and capacity before upgrading. Use --gpu-memory-utilization 0.9 or --no-enable-extensible-kv-cache to keep the old sizing.

Scope and limitations

VLLM_USE_V2_MODEL_RUNNER=0 now fails at startup. ROCm 7.2 artifacts, QuTLASS/fp_quant and Marlin W4A8 are removed. Quoted speedups (up to 1.95x decode on GB200) are model- and hardware-specific.

Go to the source

A sourced technical brief published 18 Nov 2026. The implication is turingfold’s interpretation. We have not independently benchmarked this change.

← Back to Daily Brief