Add DeepSeek-v4 (Flash/Pro)#1192

Open

Blaizzy wants to merge 53 commits intoml-explore:mainfrom

Blaizzy:pc/add-deepseekv4flash-model

Commits on Nov 12, 2025

Commits on Dec 2, 2025

Merge branch 'ml-explore:main' into main
Blaizzy
authored

Commits on Mar 4, 2026

Merge branch 'ml-explore:main' into main
Blaizzy
authored

Commits on Apr 2, 2026

Merge branch 'ml-explore:main' into main
Blaizzy
authored

Commits on Apr 24, 2026

Commits on Apr 25, 2026

Commits on Apr 26, 2026

Fix RoPE init
Blaizzy
committed
Refactor RoPE initialization in V4Attention to streamline handling of compressed layers and update associated tests for consistency.
Blaizzy
committed
Improve DeepseekV4Cache and V4Attention with improved trimming logic and mask handling for compressed layers
Blaizzy
committed
Add scores parameter and updatie projection logic
Blaizzy
committed
Optimize DeepSeek V4 performance

0xClandestine
and
Blaizzy
committed
Refactor dtype handling in _ensure_cached method and improve pooled tensor processing logic (fix cache blowing up)
Blaizzy
committed
format
Blaizzy
committed
Fix matrix multiplication in _hc_expand_op to use original comb tensor instead of swapped axes
Blaizzy
committed
Refactor output projection
Blaizzy
committed
Refactor kv tensor reshaping in V4Attention for improved dimensionality handling
Blaizzy
committed
Remove redundant cache type check in V4Attention to streamline processing logic
Blaizzy
committed
Fix DeepSeek V4 sparse pooled prefill memory

Blaizzy
and
adurham
committed
Remove sparse pooled attention test block
Blaizzy
committed

Commits on Apr 27, 2026

Commits on Apr 28, 2026

Commits on Apr 29, 2026