mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-08-12 22:29:39 +04:00
Petr VilímandGitHub
af4e0cbdb6
* ggml : assert the delta-net value head dim equals the key head dim The CPU forward sizes its result from the value head dim (src[2]->ne[0]) but indexes it with the key head dim (src[0]->ne[0]). A model where the two differ was mis-indexed silently. The CUDA op has asserted this all along. * ggml : fuse the delta-net recurrent state copy into the op The problem: the delta-net op produces the new recurrent state into the tail of its result. Then llama copies that tail into the KV slot the state was read from. The copy buys nothing - the kernel could write the slot itself. The change: the slot's two halves are written by two narrow CPY nodes instead of one CONCAT. That isolates the state write in an ordinary node, which the scheduler places by the rules it already has. A backend that recognises the pattern lets the kernel write the slot directly and skips that node. One that does not implement the fusion runs the copy as before. No public header changes. Notes: this leaves ggml_concat_inplace(), added in #1777 for exactly this site, without a caller.