mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-08-12 22:29:39 +04:00
31018dc511
Adds ggml_latent_attn_prefix_ext / ggml_latent_attn_indexed_ext: MLA latent-cache attention with an always-visible learned K/V prefix (openPangu's 128 param_sink rows), joint softmax over [prefix | cache], reading the raw F32/F16/Q8_0 latent cache directly. CUDA implementation plus a scalar CPU reference that pins the op's semantics; the CPU backend reports support truthfully, and openPangu adopts the op only on a non-CPU backend as builder policy. openPangu routes its dense/SWA/MTP full-span attention and the gathered DSA path through the op, capability-gated per layer on the attention output projection's scheduled backend, with the latent cache required resident on that same backend (--no-kv-offload keeps the unfused chain); any layer whose backend cannot run the candidate keeps the exact unfused chain. Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com>