[MLX] Low speed on Qwen3.5-27B-4bit: 15.5 tok/s with 80.9% acceptance (M4 Pro 48GB)

[heykb]

dflash --model mlx-community/Qwen3.5-27B-4bit --draft z-lab/Qwen3.5-27B-DFlash --prompt "Write a Python quick sort" --max-tokens 1024

Output: 815 tokens | 15.5 tok/s | 80.9% acceptance

Hardware: M4 Pro 48GB

High acceptance (80.9%) but low speed. Per-step verification latency seems too high.

[bstnxbt]

I retested this with the current benchmark path and the same prompt/max token shape. I cannot reproduce the 15.5 tok/s result on the current stack.

Command used:

dflash benchmark \
  --model mlx-community/Qwen3.5-27B-4bit \
  --draft z-lab/Qwen3.5-27B-DFlash \
  --prompt "Write a Python quick sort" \
  --max-tokens 1024 \
  --repeat 1 \
  --cooldown 180 \
  --no-eos

Result on my local Apple Silicon run:

metric	value
prompt tokens	15
baseline mlx_lm generation	33.40 tok/s
DFlash generation	65.24 tok/s
speedup	1.95x
acceptance	86.3%
DFlash TTFT	200.7 ms
DFlash peak memory	16.60 GB

So this looks different from the reported 15.5 tok/s despite similar acceptance. A few likely differences:

• dflash --model ... --prompt ... is not the current canonical command; use dflash benchmark for baseline-vs-DFlash numbers, or dflash generate for one-shot generation.
• current large drafts default to the validated w4 draft path; bf16 draft A/Bs are much heavier and can be slower.
• thermal pressure matters a lot on M-series laptops, so the benchmark command should use a cooldown and report the artifact JSON.

Could you retest with the command above on the latest main or the next PyPI release and paste the generated results.json / summary? I am leaving this open until we have your exact artifact, but the low-speed symptom is not reproduced on the current benchmark path.