Memory explosion / leak when serving Qwen3.6-27B-ConfigI-MLX with TurboQuant kv_scheme=turbo4

[heykb]

Environment

• Hardware: Apple MacBook Pro (M4 Pro, 48GB unified memory)
• vllm-swift version: vllm-swift 0.6.2
  • dylib: /opt/homebrew/Cellar/vllm-swift/0.6.2/lib/libVLLMBridge.dylib
  • vLLM: 0.19.1
• Model: Qwen3.6-27B-ConfigI-MLX (local path: ~/models/Qwen3.6-27B-ConfigI-MLX)
• Command:

vllm-swift serve ~/models/Qwen3.6-27B-ConfigI-MLX \
  --served-model-name qwen3.6-27b \
  --max-model-len 40960 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --additional-config '{"kv_scheme": "turbo4", "kv_bits": 4}'

Problem Description

After the server starts successfully and I send a chat completion request, the response begins streaming — exactly 2 characters are emitted immediately — then the stream hangs. Within seconds, memory usage explodes and the system becomes unresponsive (48GB unified memory saturated).

This behavior is 100% reproducible on my machine.

Observed Behavior

1. Server starts without error.
2. First request arrives.
3. 2 tokens/characters are returned instantly.
4. Stream freezes.
5. Memory usage spikes rapidly until the OS kills the process or the machine swaps heavily.

Expected Behavior

The request should stream to completion without unbounded memory growth.

Questions / Hypotheses

• Is this a memory leak in the TurboQuant KV cache path (kv_scheme: turbo4)?
• Could the hang after 2 tokens be related to a missing kernel or a dequantization failure that causes an infinite retry / allocation loop?
• I noticed issue TurboQuant unavailable on Qwen3.5 (head_dim=256): kernel turbo_dequant_rotated_4_256_bf16 missing in v0.6.0 metallib bottle #17 mentions a missing turbo_dequant_rotated_4_256_bf16 kernel for Qwen3.5 (head_dim=256). Qwen3.6-27B also uses head_dim=256. Could this be the same root cause manifesting as a memory leak instead of a clean error?

Logs

No explicit crash log is produced — the process is killed by the OS due to memory pressure. Happy to provide any diagnostics if instructed.

Related Issues

• TurboQuant unavailable on Qwen3.5 (head_dim=256): kernel turbo_dequant_rotated_4_256_bf16 missing in v0.6.0 metallib bottle #17 — TurboQuant unavailable on Qwen3.5 (head_dim=256): kernel turbo_dequant_rotated_4_256_bf16 missing
• Long-context prefill (40k+ on Qwen3.5-9B at 131k ctx) blows up memory; chunked prefill is force-disabled in platform plugin #18 — Long-context prefill blows up memory; chunked prefill force-disabled

---

Please let me know what additional information (traces, memory profiling, model config) would help diagnose this.