From 87fc8701ff4da81a7d2a91ec0695f95eb3066a47 Mon Sep 17 00:00:00 2001 From: mb8565 <244351746+mb8565@users.noreply.github.com> Date: Thu, 2 Jul 2026 12:11:30 -0500 Subject: [PATCH] GLM-DSA: fix -fa 0 garbage perplexity on the batch>8 indexer path (#2069) The batch >8 indexer path in build_deepseek2_dsa_indexer accumulates the per-head scores with ggml_add_inplace into an accumulator that is seeded from a view of KQ_mask. On -fa 0, KQ_mask is the raw F32 input tensor, so the in-place writes land in the shared KQ_mask buffer and corrupt the causal mask that build_deepseek2_dsa_sparse_mask and the later softmax layers read back, which gives garbage perplexity. -fa 1 is unaffected (its F16 mask is cast to a private F32 buffer), and the small-batch path added in #2067 is unaffected (it uses a non-inplace add). Take a private copy of the seed in the batch >8 -fa 0 path (raw F32 mask) before the accumulation, matching what those two paths already do. 4K -fa 0 --dsa PPL goes from thousands to 2.7134 (dense 2.6972, -fa 1 --dsa 2.7111). -fa 1 and non-DSA builds are byte-identical. Co-authored-by: Claude Opus 4.8 (1M context) --- src/graphs/build_deepseek2.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/graphs/build_deepseek2.cpp b/src/graphs/build_deepseek2.cpp index e39a2bc77..1502e1c7a 100644 --- a/src/graphs/build_deepseek2.cpp +++ b/src/graphs/build_deepseek2.cpp @@ -486,6 +486,13 @@ ggml_tensor * llm_build_context::build_deepseek2_dsa_indexer( indexer_score = ggml_add(ctx0, indexer_score, score); cb(indexer_score, "dsa_indexer_score", il); } else { + // -fa 0: KQ_mask is the raw F32 input tensor, so the seed above still aliases it and the in-place + // per-head accumulation below would corrupt the shared causal mask that the sparse-mask and later + // softmax layers read back. Copy the seed first. On -fa 1 the F16 mask was already cast to a private + // F32 buffer above (so this does not fire), and the small-batch path uses a non-inplace add. + if (KQ_mask->type == GGML_TYPE_F32) { + indexer_score = ggml_cont(ctx0, indexer_score); + } for (int head = 0; head < n_ihead; ++head) { int il_cb = 1000*(il + 1) + head; // [1, n_tokens]