LLM-Inference/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-08-12 22:29:39 +04:00
Code Issues Packages Projects Releases Wiki Activity

state: fix V cache compatibility check for models with no V cache (#2212)

Browse Source 
read_kv_cache_data gated the restore on kv_self.v_trans != (v_state == 1).
v_state == 2 records that the writer had no V cache, while v_trans tracks
flash attention rather than V allocation, so with -fa 0 on a K-only or MLA
cache the two disagree and the restore is refused.

Compare V cache presence in both directions, and transposition only when a
V cache exists on both sides. The write path and serialized layout are
unchanged.

Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com>
This commit is contained in:
Joel Farthing
2026-08-01 09:17:04 +03:00
committed by GitHub
co-authored by Joel Farthing
parent 7064b7b6b7
commit 8ba790e8ad
1 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama.cpp
+8 -3
View File
@@ -9907,9 +9907,14 @@ struct llama_data_read {
return false;
}
// Currently the only way there is no V cache (and thus v_state is 2) requires flash_attn, and flash_attn sets kv_self.v_trans to false
if (kv_self.v_trans != (v_state == 1)) {
LLAMA_LOG_ERROR("%s: incompatible V transposition\n", __func__);
// v_state == 2 means the writer had no V cache at all. Transposition is meaningless in that
// case, and kv_self.v_trans is independent of whether v_l was ever allocated, so it says
// nothing about compatibility here. V cache presence has to match in both directions;
// transposition only has to match when there actually is a V cache on both sides.
if (((v_state == 2) != kv_self.v_l.empty()) ||
(v_state != 2 && kv_self.v_trans != (v_state == 1))) {
LLAMA_LOG_ERROR("%s: incompatible V cache state (v_state = %u, v_l %s, v_trans = %d)\n",
__func__, v_state, kv_self.v_l.empty() ? "empty" : "present", (int) kv_self.v_trans);
return false;
}