Hybrid Attention Models#

Step 1 — find the model’s unified block size N#

N is the single number that drives every other setting: the LMCache server’s --chunk-size and vLLM’s --max-num-batched-tokens are both derived from it (step 2). Get it wrong and LMCache raises at engine startup.

For a Mamba / GDN hybrid, vLLM forces one block size across all KV cache groups, chosen large enough that an attention page is at least as big as a Mamba state page. It depends on the model’s head dimensions and GDN state size, so it is model-specific — never assume a value, read it from the model. vLLM prints it once at startup:

INFO ... interface.py:670] Setting attention block size to 784 tokens to
ensure that attention page size is >= mamba page size.

You do not need LMCache, a full serving run, or the weights to be quantized to read it — just launch vLLM until the line appears, then stop. The snippet below does exactly that and prints N:

MODEL=Qwen/Qwen3.6-27B
LOG=$(mktemp)

# Launch vLLM just far enough to size the KV cache; cheap settings only.
vllm serve "$MODEL" \
    --mamba-cache-mode align --enable-prefix-caching \
    --max-model-len 8192 --gpu-memory-utilization 0.5 \
    --port 8011 > "$LOG" 2>&1 &
VLLM_PID=$!

# Wait for the block-size line (or a fatal error), then stop vLLM.
until grep -qiE "Setting attention block size|Error|Traceback" "$LOG"; do
    sleep 3
done
grep -i "Setting attention block size" "$LOG"
kill "$VLLM_PID"

The number in to N tokens is your N. Values grow with model size; for example:

Step 2 — derive the required flags from N#

1. LMCache server --chunk-size = N (or any multiple of N) and --separate-object-groups. The chunk-size rule is enforced by the connector (it must be a multiple of vLLM’s unified block size, or registration fails); --separate-object-groups gives the linear-attention layers their own cache objects and is required for these hybrids:

   lmcache server --chunk-size 784 --separate-object-groups \
       --l1-size-gb 100 --eviction-policy LRU
   

2. vLLM --max-num-batched-tokens ≥ N — a scheduler step must advance at least one whole block (align snapshots the Mamba state only at the end of each step, on a block boundary). Within [N, 2·N) every step advances exactly one block, so LMCache snapshots every block boundary — the finest partial-prefix reuse, and N is the simplest always-valid choice. --separate-object-groups additionally allows values ≥ 2·N, which raise prefill throughput with larger steps but snapshot only the last block of each step (cached prefixes then align to step boundaries, not every block).

3. vLLM --mamba-cache-mode align --enable-prefix-caching — align is mandatory (GDN backends do not support the all mode):

   vllm serve <model> \
       --enable-prefix-caching --mamba-cache-mode align \
       --max-num-batched-tokens 784 \
       --kv-transfer-config \
       '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both"}'
   

So for a freshly-probed model the whole derivation is: read N (step 1), then pass --chunk-size N --separate-object-groups to the server and --max-num-batched-tokens N to vLLM.

No --no-disable-hybrid-kv-cache-manager or attention-backend flag is needed; LMCacheMPConnector advertises hybrid support and vLLM auto-selects the GDN backend.

Caveats#

• Generation is not bit-exact between a cached and a fresh run: GDN backends do not support vLLM’s batch-invariant mode. Validate with a score-level comparison (see Verifying Correctness), not a token-level diff.

• The cached pages are byte-opaque, so content-aware features (CacheGen compression, CacheBlend) do not apply, and cache entries must not be shared across engines with different attention backends or kernel block sizes.

• Several of these models are vision-language (they load a vision tower). The validated, supported path is text KV caching; image/video KV caching is not validated.

• vLLM’s Mamba prefix caching in align mode is marked experimental upstream.

See the Qwen3.5 / Qwen3.6 recipe for the validated end-to-end commands and the per-model block sizes.