LLM-Inference/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-12 22:31:11 +04:00
Code Issues Packages Projects Releases Wiki Activity

metal : avoid threadgroup matrix array instantiation in kernel_lightning_indexer (#26646)

Browse Source 
- In MSL, declaring an array of matrix types like `threadgroup half4x4` causes
a 'no matching constructor' compilation error because MSL matrix types do not
have zero-argument default constructors and threadgroup variables cannot have
initializers.

- Fix this by declaring a POD `threadgroup half` array instead and casting
to `threadgroup half4x4 *` for matrix indexing.

Signed-off-by: JamePeng <jame_peng@sina.com>
This commit is contained in:
JamePeng
2026-08-07 07:49:14 +03:00
committed by GitHub
parent 15586e2d71
commit e40bf88642
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-metal/ggml-metal.metal
+2 -2
View File
@@ -11328,8 +11328,8 @@ kernel void kernel_lightning_indexer(
const int i_kv_0 = tgpig.x*NK; // first key of this threadgroup
const int i_kv = i_kv_0 + sgitg*NKPSG; // first key of this simdgroup
threadgroup half4x4 sk4x4[NK*DK16];
threadgroup half * sk = (threadgroup half *) sk4x4;
threadgroup half sk[NK * DK16 * 16];
threadgroup half4x4 * sk4x4 = (threadgroup half4x4 *) sk;
for (short i = tiitg; i < NK*DK16; i += NTG) {
const short ik = i/DK16;