LLM-Inference/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-08-12 22:31:22 +04:00
Code Issues Packages Projects Releases Wiki Activity
4,839 Commits 65 Branches 88 Tags
2ca53bb45e38748d07b310eeb36245a7157ac882
Commit Graph
54 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
lhezandGeorgi Gerganov 75f1f0c063 opencl: fix potential crash in aos reconstruct (llama/25383) 2026-07-10 13:06:42 +03:00
lhezandGeorgi Gerganov 759197a754 opencl: allow loading precompiled binary kernels from library (llama/23042) 
* opencl: allow loading binary kernel

* opencl: add libdl.h

* ggml-backend-dl is in ggml, which depends backend libs, thus
  ggml-opencl cannot depend on ggml-backend-dl
* add libdl.h to break cyclic dep

* opencl: allow loading bin kernel lib

* opencl: load `gemm_moe_mxfp4_f32_ns` from kernel lib if available

* opencl: load q8_0 gemm from kernel lib

* opencl: load q4_0 moe gemm from kernel lib

* opencl: load q4_1 moe gemm from kernel lib

* opencl: load q4_k moe gemm from kernel lib

* opencl: always declare `get_adreno_bin_kernel_func_t`

* opencl: rephrase message

* opencl: fix for rebase

* opencl: update doc
 2026-07-10 13:06:42 +03:00
lhezandGeorgi Gerganov 9804f9736c opencl: initial q1_0 support (llama/25160) 
* opencl: general q1_0 support

* opencl: add Adreno GEMM/GEMV for q1_0
 2026-07-10 13:06:42 +03:00
lhezandGeorgi Gerganov fe37d2e489 opencl: support non-contig rows in norm (llama/24965) 2026-06-26 16:03:57 +03:00
lhezandGeorgi Gerganov 9f785839a3 opencl: optimize mul_mat_f16_f32_l4 for decode (llama/24504) 2026-06-19 12:53:43 +03:00
lhezandGeorgi Gerganov a87e950a06 opencl: improve get_rows, cpy, concat and q6_k flat gemv (llama/24160) 
* opencl: allow multiple workgroups for large rows

* opencl: improve small cpy

* opencl: packed concat for small input

* opencl: tweak flat q6_K gemv, increase N_DST and remap threads
 2026-06-08 14:36:36 +03:00
lhezandGeorgi Gerganov f110ff540c opencl: use flat variants of q4_K and q6_K gemv for very large M (llama/24006) 2026-06-08 14:36:36 +03:00
lhezandGeorgi Gerganov 9a0265d13b opencl: fix compiler warnings for non-adreno path (llama/23922) 
* opencl: fix compiler warnings for non-adreno path

* opencl: fix const cast warning
 2026-06-08 14:36:36 +03:00
lhezandGeorgi Gerganov 1c0d1f0f7c opencl: support bf16 by converting to f16 (llama/23839) 2026-06-08 14:36:36 +03:00
lhezandGeorgi Gerganov 94922ce12c opencl: move backend info printing into its own function (llama/23702) 
* opencl: move backend info print into its own function

* opencl: move new log line

* opencl: fix for non adreno path
 2026-05-29 09:47:30 +03:00
lhezandGeorgi Gerganov 896718eacf opencl: refactor backend initilization (llama/23318) 
* opencl: refactor initialization

* opencl: refactor GPU identification

* opencl: rename for consistency

* opencl: cache global mem size in dev_ctx

* opencl: adjust log level

* opencl: load argsort and flash_attn kernels in supports_op

* argsort kernel must be built for supports_op for querying the max
  workgroups
* flash_attn kernel has many variants, only load them when needed
 2026-05-25 12:26:07 +03:00
lhezandGeorgi Gerganov b19beb6027 opencl: fix crash when warming up MoE on Adreno (llama/22876) 2026-05-14 21:26:48 +03:00
lhezandGeorgi Gerganov f83b6bdc44 opencl: refactor Adreno q4_0 (llama/22335) 2026-05-14 21:26:48 +03:00
lhezandGeorgi Gerganov 93a3f37642 opencl: add iq4_nl support (llama/22272) 
* opencl: add general support for iq4_nl

* opencl: add iq4_nl gemm/gemv for adreno

* opencl: pack 2 lut entries into a uint
 2026-04-30 11:29:19 +03:00
lhezandGeorgi Gerganov 77c0630ce6 opencl: refactor q8_0 set_tensor and mul_mat host side dispatch for Adreno (llama/21938) 
* opencl: refactor q8_0 gemm/gemv Adreno dispatch

* opencl: refactor q8_0 set_tensor

* opencl: fix whitespace
 2026-04-30 11:29:12 +03:00
lhezandGeorgi Gerganov 08108512c7 opencl: fix leak in Adreno q8_0 path (llama/21212) 2026-04-30 11:28:59 +03:00
lhezandGeorgi Gerganov 1848f994e3 opencl: allow large buffer for adreno (llama/20997) 2026-03-29 15:04:36 +03:00
lhezandGeorgi Gerganov 624be93425 opencl: add q6_K gemm and gemv kernels for Adreno (llama/20089) 
* opencl: add q6_K noshuffle kernels, initial q6_K gemv, some host code

* opencl: add q6_K transpose

* opencl: fix cvt kernel name

* opencl: add call to q6_K gemv

* opencl: fix q6_K scale transpose

* opencl: fix loading for gemv q6_K, refactor

* opencl: fix transpose_8_buf kernel assignment, refactor

* opencl: refactor q6_K transpose

* opencl: add gemm_noshuffle_q6_k_f32

* opencl: fix qh loading

* opencl: refactor q6_K gemv host side, release bufs and imgs

* opencl: refactor

* opencl: fix q6_K dequant and scale selection

* opencl: workaround compiler bug, fix dump_tensor

* opencl: refactor q6_K convert kernels

* opencl: unpack transformed q6_K in get_tensor

* opencl: refactor, handle non-uniform workgroups

* opencl: support non-vector subgroup bcast
 2026-03-29 15:04:36 +03:00
lhezandGeorgi Gerganov a31600d8e3 opencl: fix l2_norm (llama/20480) 2026-03-16 13:10:15 +02:00
lhezandGeorgi Gerganov d5772cf7b2 opencl: use larger workgroup size for get_rows (llama/20316) 2026-03-16 13:10:15 +02:00
lhezandGeorgi Gerganov 910034df28 opencl: add l2_norm (llama/20160) 2026-03-16 13:10:15 +02:00
lhezandGeorgi Gerganov 1d94b0be4f opencl: add neg, exp and diag (llama/20127) 
* opencl: add `neg`

* opencl: add `exp`

* opencl: add `diag`
 2026-03-16 13:10:15 +02:00
lhezandGeorgi Gerganov 2c50962528 opencl: add SET, support i32 for CPY, minor refactor for cpy (llama/20101) 2026-03-16 13:10:15 +02:00
lhezandGeorgi Gerganov 9f87eeccdf opencl: add basic support for q4_1 (llama/19534) 
* opencl: add q4_1 mv

* opencl: clean up

* opencl: add flattened q4_1 mv

* opencl: clean up

* opencl: add basic q4_1 mm

* opencl: fix whitespace

* opencl: add general q4_0 mm
 2026-02-15 21:44:37 +02:00
lhezandGeorgi Gerganov 0326fd37dd opencl: add general Q6_K mm and Q4_K mv (llama/19347) 
* opencl: add general q6_k mm

* opencl: refine condition for q6_K mm

* opencl: add general q4_K mv

* opencl: fix whitespace
 2026-02-15 21:44:37 +02:00
lhezandGeorgi Gerganov 591072fcc8 opencl: refactor some ops, concat, repeat, tanh and scale (llama/19226) 
* opencl: refactor concat

* opencl: refactor repeat

* opencl: refactor tanh

* opencl: enable fp16 for tanh

* opencl: refactor scale

* opencl: fix unused variables
 2026-02-08 09:29:10 +02:00
lhezandGeorgi Gerganov 56f82a9f33 opencl: add flattened q6_K mv (llama/19054) 2026-01-30 15:56:40 +02:00
lhezandGeorgi Gerganov 0e030b852a opencl: enable the general fp mm for non-cont input and as a fallback for specialized kqv kernel for adreno (llama/18970) 
* opencl: add `copy_to_contiguous` and utilize mm kernels

* opencl: only copy to cont for f32 and f16 tensors

* opencl: use cont mm for fallback when dst is large

* opencl: use nb local to copy-to-cont

* opencl: use local offset as well
 2026-01-30 15:56:40 +02:00
lhezandGeorgi Gerganov 62a09b106d opencl: fix q6_K mv for m=1 (llama/18893) 2026-01-30 15:56:40 +02:00
lhezandGeorgi Gerganov a800a3acd1 opencl: allow resizing transpose buffers (llama/18384) 
* opencl: allow resizing transpose buffers instead of using fixed sizes

* opencl: remove commented code
 2025-12-31 17:52:09 +02:00
lhezandGeorgi Gerganov f2fe1e5baf opencl: unpack q4_0 for adreno in get_tensor (llama/18278) 2025-12-31 17:52:09 +02:00
lhezandGeorgi Gerganov 74ef5dd1a9 opencl: add sqr, sqrt, mean and ssm_conv (llama/17476) 
* opencl: add sqr

* opencl: add sqrt

* opencl: add mean

* opencl: add ssm_conv

* opencl: add missing cl_khr_fp16

* opencl: do sqrt in f32 then convert to f16 for better precision
 2025-12-12 17:53:09 +02:00
lhezandGeorgi Gerganov 5c0e4a9cc5 opencl: refine condition for kqv mm (llama/17392) 2025-12-12 17:53:05 +02:00
lhezandGeorgi Gerganov a75525cad0 opencl: fix rms_norm_mul (llama/17250) 
* opencl: use subgrroup reduce for reduction in rms_norm_mul

* opencl: add comment about workgroup size
 2025-11-17 21:05:46 +02:00
lhezandGeorgi Gerganov 46615d74d3 opencl: add fastdiv and use it in set_rows, ported from cuda (llama/17090) 
* opencl: add fastdiv for mm q8_0

* opencl: use uint4 for fastdiv vals

* opencl: use fastdiv for set_rows

* opencl: do not use fastdiv for q8_0 mm
 2025-11-17 21:05:46 +02:00
lhezandGeorgi Gerganov f856023f46 opencl: support imrope (llama/16914) 
* opencl: support imrope

* opencl: fix whitespace
 2025-11-09 23:38:03 +02:00
lhezandGeorgi Gerganov 7fdd53ac0d opencl: fix boundary handling for mul_mm (llama/16875) 2025-11-09 23:38:03 +02:00
lhezandGeorgi Gerganov 5c4c477d00 opencl: fix warnings and clean up profiling (llama/16688) 
* opencl: remove unused headers, fix warnings

* opencl: clean up profiling, only keep kernel time
 2025-10-22 12:58:11 +03:00
lhezandGeorgi Gerganov bef9f74553 opencl: add q8_0 mm support (llama/16469) 
* opencl: add mm_q8_0_f32

* opencl: fix data loading for incomplete tile

* opencl: use q8_0 mm for larger matrix

* opencl: add some tests to cover the path
 2025-10-22 12:58:11 +03:00
lhezandGeorgi Gerganov 16dab3d122 opencl: fix FA for f32 (llama/16584) 2025-10-22 12:58:11 +03:00
lhezandGeorgi Gerganov 66b0fc2fb7 opencl: fix build targeting CL 2 (llama/16554) 2025-10-15 09:29:17 +03:00
lhezandGeorgi Gerganov 31bb869929 opencl: support pad_ext (llama/15888) 2025-10-12 11:16:23 +03:00
lhezandGeorgi Gerganov 199626d79e opencl: support ne3 in get_rows (llama/15866) 2025-10-12 11:16:23 +03:00
lhezandGeorgi Gerganov 4b7f09ac0b opencl: fix concat crash on win arm64 with Adreno (llama/15944) 2025-09-29 15:18:08 +03:00
lhezandGeorgi Gerganov 0a7096f4f3 opencl: initial q8_0 mv support (llama/15732) 2025-09-29 15:18:08 +03:00
lhezandGeorgi Gerganov 582ef379ab opencl: fix support ops condition for rms_norm (llama/15560) 2025-09-20 13:42:41 +03:00
lhezandGeorgi Gerganov db1d2380a0 opencl: mark argsort unsupported if cols exceed workgroup limit (llama/15375) 2025-09-20 13:42:37 +03:00
lhezGeorgi Gerganovshawngu-quic
 		77771b2711 opencl: add initial mxfp4 support via mv (llama/15270) 
* opencl: add reference `mul_mv_mxfp4_f32`

* opencl: add reference `mul_mv_id` for mxfp4

* Q4_0 tranpose fix for Adreno

---------

Co-authored-by: shawngu-quic <shawngu@qti.qualcomm.com>
 2025-08-18 20:30:45 +03:00
lhezandGeorgi Gerganov 8a36cd924a opencl: support sink in soft_max (attn sinks) (llama/15152) 2025-08-18 20:30:45 +03:00
lhezandGeorgi Gerganov d153cfb507 opencl: add swiglu_oai and add_id (llama/15121) 
* opencl: add `swiglu-oai`

* opencl: add `add_id`

* opencl: add missing `add_id.cl`
 2025-08-18 20:30:45 +03:00
lhezandGeorgi Gerganov 53d8c5179f opencl: fix adreno compiler detection logic (llama/15029) 2025-08-18 20:30:45 +03:00
lhezandGeorgi Gerganov 4904bc3bda opencl: add mul_mat_f32_f32_l4_lm and mul_mat_f16_f32_l4_lm (llama/14809) 2025-08-18 20:30:45 +03:00
lhezandGeorgi Gerganov d414c3f6ac opencl: add fused rms_norm_mul (llama/14841) 
* opencl: add fused `rms_norm` + `mul`

* opencl: improve workgroup size for `rms_norm_mul`
 2025-07-28 13:02:32 +03:00
lhezandGeorgi Gerganov de49384ab3 opencl: remove unreachable return (llama/14806) 2025-07-28 13:02:32 +03:00