Hybrid recurrent cache#13979

Merged

ggerganov merged 39 commits intoggml-org:masterfrom

gabe-l-hart:HybridRecurrentCache

Jun 19, 2025

Commits on Jun 17, 2025

feat: Add llama_model_is_hybrid API call
gabe-l-hart
committed
feat: Add c++ side constants for attention layer indices hparam
gabe-l-hart
committed
feat: Add support for distinguishing recurrent vs non-recurrent layers in hparams
gabe-l-hart
committed
feat: Auto-fill hparams.recurrent_layer_arr based on whether the model is recurrent
gabe-l-hart
committed
refactor: rename *_is_hybrid -> *_is_hybrid_recurrent
gabe-l-hart
committed
feat: Add layer filter to recurrent cache
gabe-l-hart
committed
fix: Use per-layer sizing everywhere in kv caches
gabe-l-hart
committed
feat: First pass at llama_kv_cache_hybrid_recurrent
gabe-l-hart
committed
feat: Construct hybrid recurrent cache for hybrid recurrent models
gabe-l-hart
committed
fix: Fix wrong bool condition for split equal in hybrid cache
gabe-l-hart
committed
fix: Fix shift logic to defer to unified cache
gabe-l-hart
committed
feat: Support hybrid recurrent in llama-graph
gabe-l-hart
committed
fix: Fix logic for initializing inputs and attn layers for hybrid caches
gabe-l-hart
committed
fix: Update recurrent cache for changes to remove intermediate kv_cache interface
gabe-l-hart
committed
fix: Fix status for init_update sig for recurrent cache state
gabe-l-hart
committed
fix: Add missing padding to n_ctx for hybrid cache construction
gabe-l-hart
committed
fix: Update clear signature for data argument after rebase
gabe-l-hart
committed
fix: Remove errant virtual destructor leftover from previous impl attempt
gabe-l-hart
committed
fix: Use per-layer n_embd_k/v_s calls for mamba (1) layers
gabe-l-hart
committed
refactor: Remove n_embd_k/v_s from unified cache
gabe-l-hart
committed
refactor: Remove layer index from n_embd_k/v_s
gabe-l-hart
committed
refactor: Remove n_embd_k/v_gqa from recurrent cache
gabe-l-hart
committed
feat: Allow custom layer filters for hybrid recurrent
gabe-l-hart
committed
fix: Remove logits_all after rebase
gabe-l-hart
committed
fix: Remove llama_model_is_hybrid_Recurrent public API
gabe-l-hart
committed
refactor: Use llama_memory_state_ptr for child states in hybrid memory state
gabe-l-hart
committed
feat: Overhaul build_recurrent_state / build_inp_s_copy to match attention pattern
gabe-l-hart
committed
fix: Fix resize vs reserve and skip null tensors in size computation
gabe-l-hart
committed
fix: Fix initialization of child states
gabe-l-hart
committed
refactor: Use a common build_recurrent_state method that is cache-agnostic
gabe-l-hart
committed

Commits on Jun 18, 2025