perf(deepseek_v4): fuse switch_mlp gate_proj + up_proj into single gather_qmm#1999

Open

adurham wants to merge 2 commits intoexo-explore:mainfrom

adurham:dsv4-fused-moe-gate-up

Commits on Apr 27, 2026

fix(deepseek_v4): drop full-attention sharding for MoE-only strategy
Adam Durham
committed
perf(deepseek_v4): fuse switch_mlp gate_proj + up_proj into single gather_qmm
Adam Durham
committed