LLM-Inference/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-08-12 22:29:39 +04:00
Code Issues Packages Projects Releases Wiki Activity

GLM-DSA: fix -fa 0 garbage perplexity on the batch>8 indexer path (#2069)

Browse Source 
The batch >8 indexer path in build_deepseek2_dsa_indexer accumulates the
per-head scores with ggml_add_inplace into an accumulator that is seeded
from a view of KQ_mask. On -fa 0, KQ_mask is the raw F32 input tensor, so
the in-place writes land in the shared KQ_mask buffer and corrupt the causal
mask that build_deepseek2_dsa_sparse_mask and the later softmax layers read
back, which gives garbage perplexity.

-fa 1 is unaffected (its F16 mask is cast to a private F32 buffer), and the
small-batch path added in #2067 is unaffected (it uses a non-inplace add).
Take a private copy of the seed in the batch >8 -fa 0 path (raw F32 mask)
before the accumulation, matching what those two paths already do.

4K -fa 0 --dsa PPL goes from thousands to 2.7134 (dense 2.6972, -fa 1 --dsa
2.7111). -fa 1 and non-DSA builds are byte-identical.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
mb8565
2026-07-02 19:11:30 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent dbe2ecbe47
commit 87fc8701ff
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/graphs/build_deepseek2.cpp
+7
View File
@@ -486,6 +486,13 @@ ggml_tensor * llm_build_context::build_deepseek2_dsa_indexer(
indexer_score = ggml_add(ctx0, indexer_score, score);
cb(indexer_score, "dsa_indexer_score", il);
} else {
// -fa 0: KQ_mask is the raw F32 input tensor, so the seed above still aliases it and the in-place
// per-head accumulation below would corrupt the shared causal mask that the sparse-mask and later
// softmax layers read back. Copy the seed first. On -fa 1 the F16 mask was already cast to a private
// F32 buffer above (so this does not fire), and the small-batch path uses a non-inplace add.
if (KQ_mask->type == GGML_TYPE_F32) {
indexer_score = ggml_cont(ctx0, indexer_score);
}
for (int head = 0; head < n_ihead; ++head) {
int il_cb = 1000*(il + 1) + head;
// [1, n_tokens]