HyperWinX/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-08-12 22:29:04 +04:00
Code Issues Packages Projects Releases Wiki Activity
3,729 Commits 65 Branches 88 Tags
sync-ggml-25-12-17
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi Gerganov fcaa8f8a7b talk-llama : sync llama.cpp 2025-12-17 15:20:22 +02:00
Georgi Gerganov 82ee376b93 sync : ggml 2025-12-17 15:19:57 +02:00
Georgi Gerganov e500fa6ce4 ggml : arm repack fix build (llama/0) 2025-12-17 15:19:48 +02:00
Georgi Gerganov f0c9017a2f ggml : arm repack fix build (#0) 2025-12-13 08:04:09 +02:00
Georgi Gerganov 179d8b1c9c talk-llama : sync llama.cpp 2025-12-12 18:15:27 +02:00
Georgi Gerganov 48cdc06e91 sync : ggml 2025-12-12 17:55:11 +02:00
Georgi Gerganov 72714d169c whisper : adjust to ggml changes (#0) 2025-12-12 17:54:58 +02:00
Georgi Gerganov 1da1a6865c ggml-alloc : fix reuse-parent logic for misaligned sizes (llama/17884) 2025-12-12 17:53:24 +02:00
Georgi Gerganov cd9b8c6d18 ggml : remove GGML_KQ_MASK_PAD constant (llama/17910) 
* ggml : remove GGML_KQ_MASK_PAD constant

* cont : remove comment
 2025-12-12 17:53:24 +02:00
Georgi Gerganov b6ae0b29d1 metal : print node names for debugging (llama/17882) 2025-12-12 17:53:22 +02:00
Georgi Gerganov 41cf229d72 metal : fix build(#17799) 
* metal : fix build

* tests : fix context destruction
 2025-12-12 17:53:20 +02:00
Georgi Gerganov aefcd75f4f rpc : fix alloc size logic (llama/17116) 
* rpc : fix alloc size logic

* rpc : bump version
 2025-12-12 17:53:18 +02:00
Georgi Gerganov 322903fa67 metal : add residency sets keep-alive heartbeat (llama/17766) 
* examples : add idle

* metal : attach residency sets to queue

* idle : add link

* idle : adjust intervals

* metal : add residency sets keep-alive heartbeat

* cont : adjust default keep-alive time
 2025-12-12 17:53:18 +02:00
Georgi Gerganov 194d016456 metal : use params per pipeline instance (llama/17739) 2025-12-12 17:53:16 +02:00
Georgi Gerganov 4a00f2e3a4 metal : fix data race in pipeline library (llama/17731) 2025-12-12 17:53:16 +02:00
Georgi Gerganov 32090930f7 metal : add FA head size 48 (llama/17619) 2025-12-12 17:53:14 +02:00
Georgi Gerganov 7cd3de89bf ggml : extend the GGML_SCHED_NO_REALLOC debug logic of the scheduler (llama/17617) 2025-12-12 17:53:14 +02:00
Georgi Gerganov 968db8bcfa ggml : add ggml_top_k (llama/17365) 
* ggml : add ggml_top_k

* cont : add ggml_argsort_top_k

* metal : add top_k support

* ggml : cleanup

* tests : add virtual err() function for test_case

* ggml : add comments
 2025-12-12 17:53:08 +02:00
Georgi Gerganov b12abefa9b sync : llama.cpp 2025-11-17 21:05:46 +02:00
Georgi Gerganov 0e5deca8e2 sync : ggml 2025-11-17 21:05:46 +02:00
Georgi Gerganov 661567357c metal : support I32 -> I32 copy (llama/17317) 2025-11-17 21:05:46 +02:00
Georgi Gerganov 74bb8a8b23 metal : faster argsort (llama/17315) 
* metal : faster argsort

* cont : keep data in registers
 2025-11-17 21:05:46 +02:00
Georgi Gerganov 57c0e6f8b6 metal : add cumsum (llama/17305) 2025-11-17 21:05:46 +02:00
Georgi Gerganov 9549cc1051 metal : remove obosolete asserts (llama/17295) 2025-11-17 21:05:46 +02:00
Georgi Gerganov 523a6c27ea metal : support argsort for ne00 > 1024 (llama/17247) 
* metal : refactor argsort

* cont : sort chunks

* cont : merge sorted buckets

* cont : cleanup
 2025-11-17 21:05:46 +02:00
Georgi Gerganov b4d7df3ba2 metal : make the FA extra sizes consistent (llama/17143) 2025-11-17 21:05:46 +02:00
Georgi Gerganov 6a1d830dfd Revert "ggml-cpu: handle 3d tensors in repack mat_mul (llama/17030)" (llama/17233) 
This reverts commit 1c398dc9eca9c366ce98deb0e6f3538e444ebc8a.
 2025-11-17 21:05:46 +02:00
Georgi Gerganov 3810a6180b ggml : use std::sort in ggml_argsort CPU implementation (llama/17211) 
* ggml : use std::sort in ggml_argsort CPU implementation

* cont : add missing header
 2025-11-17 21:05:46 +02:00
Georgi Gerganov 40aebfe8bf metal : cap threadgroups size of set_rows (llama/17146) 2025-11-17 21:05:46 +02:00
Georgi Gerganov bb92c79f56 metal : enable tensor API for A19 (llama/17087) 2025-11-17 21:05:46 +02:00
Georgi Gerganov a1867e0dad sync : llama.cpp 2025-11-09 23:38:03 +02:00
Georgi Gerganov e67dfbc51b sync : ggml 2025-11-09 23:38:03 +02:00
Georgi Gerganov a4339e2ea7 metal : retain src and dst buffers during async ops (llama/17101) 2025-11-09 23:38:03 +02:00
Georgi Gerganov 32ed574370 metal : initial Metal4 tensor API support (llama/16634) 
* metal : rework mat-mat multiplication

* metal : initial Metal4 support

* cont

* metal : detect tensor support

* cont : better ifdefs

* metal : support tensors in mul_mm_id

* metal : add env for disabling tensor API

* tests : restore

* metal : remove unused constants

* metal : fix check for bfloat tensor support

* cont : handle API incompatibilities

* cont : handle even more incompatibilities

* metal : use tensor API only on M5 and later
 2025-11-09 23:38:03 +02:00
Georgi GerganovandXuan Son Nguyen 39834fde1b clip : use FA (llama/16837) 
* clip : use FA

* cont : add warning about unsupported ops

* implement "auto" mode for clip flash attn

* clip : print more detailed op support info during warmup

* cont : remove obsolete comment [no ci]

* improve debugging message

* trailing space

* metal : remove stray return

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2025-11-09 23:38:03 +02:00
Georgi Gerganovcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ggerganov
 		addda802dd ggml : fix conv2d_dw SVE path (ggml/1380) 
* Fix test-conv2d-dw failure on ARM SVE by using runtime vector length

The ggml_compute_forward_conv_2d_dw_cwhn function was using a hardcoded GGML_F32_EPR (8) for SIMD vectorization, but on ARM SVE the actual vector length varies by hardware. This caused incorrect computation when processing CWHN layout tensors on ARM machines.

Fix by using svcntw() to get the runtime SVE vector length instead of the compile-time constant.

Co-authored-by: ggerganov <1991296+ggerganov@users.noreply.github.com>

* ci : reduce sam score threshold

* ci : update bbox checks for sam test

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ggerganov <1991296+ggerganov@users.noreply.github.com>
 2025-11-09 23:38:03 +02:00
Georgi Gerganov 322c2adb75 talk-llama : sync llama.cpp 2025-10-22 12:58:11 +03:00
Georgi Gerganov 35ea5ced60 sync : ggml 2025-10-22 12:58:11 +03:00
Georgi Gerganov 8ed913da0e metal : avoid using Metal's gpuAddress property (llama/16576) 
* metal : avoid using Metal's gpuAddress property

* metal : fix rope kernels buffer check
 2025-10-22 12:58:11 +03:00
Georgi GerganovandGitHub 23c19308d8 server : set no_context == true (#3482) 2025-10-20 15:39:48 +03:00
Georgi Gerganov 4979e04f5d release : v1.8.2 2025-10-15 10:29:42 +03:00
Georgi Gerganov 8ba3c13b0c talk-llama : sync llama.cpp 2025-10-15 09:29:17 +03:00
Georgi Gerganov ff2253b08a sync : ggml 2025-10-15 09:29:17 +03:00
Georgi Gerganov 25ac94a6cb metal : FA support F32 K and V and head size = 32 (llama/16531) 
* metal : FA support F32 K and V and head size = 32

* graph : remove obsolete comment [no ci]
 2025-10-15 09:29:17 +03:00
Georgi Gerganov ccac1b4772 ggml : fix scalar path for computing norm (llama/16558) 2025-10-15 09:29:17 +03:00
Georgi Gerganov a91dd3be72 release : v1.8.1 2025-10-12 11:17:59 +03:00
Georgi Gerganov ea174c62bc bench : update [no ci] 2025-10-12 11:16:23 +03:00
Georgi Gerganov ff4c1a5a53 talk-llama : sync llama.cpp 2025-10-12 11:16:23 +03:00
Georgi Gerganov ed6a3063ec sync : ggml 2025-10-12 11:16:23 +03:00
Georgi Gerganov d201705e71 metal : fix mul-mm condition + fix mul-mv permuted kernels (llama/16494) 2025-10-12 11:16:23 +03:00
Georgi Gerganov 7ef78a72e1 metal : mark FA blocks (llama/16372) 
* metal : better unroll in the FA kernels

* metal : index FA blocks

* tests : restore [no ci]

* metal : prevent division by zero in FA kernels

* metal : fix -INF detection logic
 2025-10-12 11:16:23 +03:00
Georgi Gerganov 4bce4fa5e9 metal : add support for non-padded FA KV (llama/16148) 
* metal : pad K, V and Mask when needed

* cont : simplify

* cuda : add TODO about KV padding requirement

* metal : add comments

* metal : remove mask padding requirement
 2025-10-12 11:16:23 +03:00
Georgi GerganovandJeff Bolz 6cf0c21b09 tests : add -INF blocks to the KQ mask in the FA tests (llama/16380) 
* tests : add -INF blocks to the KQ mask in the FA tests

* cont : bump -INF block size to 64

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>

* ggml : prevent division by zero in FA CPU op

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-10-12 11:16:23 +03:00
Georgi Gerganov 1a4116f942 metal : various optimizations + refactoring (llama/16446) 
* metal : ssm_scan minor opts

* metal : get_rows optimize

* metal : cpy optimize

* metal : ssm_conv opt

* metal : ssm_scan simplify

* metal : ssm_Scan opt
 2025-10-12 11:16:23 +03:00
Georgi Gerganov 0e431b3cea ggml : fix unaligned access in AMX code (llama/16315) 2025-10-12 11:16:23 +03:00
Georgi Gerganov 93c1305565 metal : fix loop bound in ggml_mem_ranges (llama/16412) 2025-10-12 11:16:23 +03:00
Georgi Gerganov d3a29d7b88 minor : fix code style (#3463) 2025-10-10 11:33:01 +03:00
Georgi Gerganov 98930fded1 whisper : clean-up headers 2025-10-09 10:48:52 +03:00
Georgi Gerganov 8c0855fd6b bench : update [no ci] 2025-09-30 21:40:32 +03:00
Georgi Gerganov 47fcd7da8b scripts : add -nfa option [no ci] 2025-09-30 21:37:00 +03:00
Georgi Gerganov 8a67c55c8a wchess : fix link [no ci] 2025-09-30 21:28:03 +03:00
Georgi Gerganov 41fc9dea6a release : v1.8.0 2025-09-30 21:25:36 +03:00
Georgi GerganovandGitHub 0b3587acdd whisper : enable flash attention by default (#3441) 2025-09-30 15:47:20 +03:00
Georgi Gerganov 1e5ad50f8f bench : add rtx 5090 [no ci] 2025-09-30 13:58:15 +03:00
Georgi Gerganov 527ff158d0 ggml : bump version to 0.9.4 (ggml/1363) 2025-09-30 13:54:08 +03:00
Georgi Gerganov e4bf87b0e9 bench : update [no ci] 2025-09-30 12:51:25 +03:00
Georgi Gerganov b57b9d3a27 sync : ggml 2025-09-30 12:31:08 +03:00
Georgi Gerganov 78f85f2b92 metal : dynamic simdgroups for MV kernels (llama/16340) 
* metal : dynamic simdgroups for MV kernels

* cont : minor
 2025-09-30 12:31:04 +03:00
Georgi GerganovandGitHub 32be14f8eb bench : update [no ci] (#3439) 2025-09-29 17:42:38 +03:00
Georgi GerganovandGitHub a77d11d91e bench : warm-up all kernels (#3438) 2025-09-29 17:27:53 +03:00
Georgi Gerganov 22c12ee86d ggml : remove oboslete files (#0) 2025-09-29 16:47:30 +03:00
Georgi GerganovandGitHub d8cdcce884 ci : add self-hosted workflows (#3437) 
* ci : add self-hosted workflows

* cont : fail workflow if there is an error
 2025-09-29 16:42:39 +03:00
Georgi GerganovandGitHub b4909a6c78 whisper : remove ggml_mul_mat padding (#3436) 2025-09-29 16:42:08 +03:00
Georgi Gerganov fcf0181ee2 talk-llama : sync llama.cpp 2025-09-29 15:18:41 +03:00
Georgi Gerganov 404a93114c sync : ggml 2025-09-29 15:18:18 +03:00
Georgi Gerganov 3201382792 cmake : remove metal flag (llama/0) 2025-09-29 15:18:13 +03:00
Georgi Gerganov 7ce0a7bcd0 ggml : fix dependencies for ggml_set_rows (llama/16318) 2025-09-29 15:18:12 +03:00
Georgi Gerganov 0102733cca metal : extend mat-mat multiplication support (llama/16225) 
* metal : support mul_mm with src1->type == GGML_TYPE_F16

* metal : support mul_mm_id with src1->type == GGML_TYPE_F16

[no ci]

* metal : mul_mm support ne00 % 32 != 0

* metal : support mul_mm_id with ne00 % 32 != 0

* cont : remove unnecessary unrolls

* cont : simplify data loading

* metal : optimize mul_mm when output bounds checks are not needed
 2025-09-29 15:18:12 +03:00
Georgi Gerganov 45976f2857 metal : fuse non-sequential nodes (llama/16102) 
* metal : fuse non-sequential nodes

* cont : add comment

* cont : simplify bounds checks
 2025-09-29 15:18:12 +03:00
Georgi Gerganov 670d54ef5d metal : report OOM errors (llama/16274) 2025-09-29 15:18:11 +03:00
Georgi Gerganov ac678efb35 metal : fuse NORM + MUL + ADD, support non-multiples of 4 (llama/16220) 
* metal : fuse NORM + MUL + ADD

* metal : support norms of non-multiple of 4

* cont : fix comment [no ci]
 2025-09-29 15:18:10 +03:00
Georgi Gerganov 268f1c961b metal : relax reorder conditions (llama/16216) 2025-09-29 15:18:10 +03:00
Georgi Gerganov 0a5b811f2e metal : restore im2col perf (llama/16219) 2025-09-29 15:18:10 +03:00
Georgi Gerganov df672c6372 ggml : extend ggml_can_fuse to work with non-sequential nodes (llama/16123) 
* ggml : extend ggml_can_fuse to work with non-sequential nodes in the graph

* cont : fix wrong bounds check condition

* cont : remove unnecessary overload
 2025-09-29 15:18:09 +03:00
Georgi Gerganov 973054a8cd ggml : add ggml_op_is_empty (llama/16122) 
* ggml : add ggml_op_is_empty

* ggml : move to ggml-impl.h
 2025-09-29 15:18:09 +03:00
Georgi Gerganov 8d10ded025 ggml : prepare for development of 0.9.2-dev 2025-09-29 15:18:08 +03:00
Georgi Gerganov d89164a08d ggml : bump version to 0.9.1 2025-09-29 15:18:05 +03:00
Georgi Gerganov 36778bd8b8 talk-llama : sync llama.cpp 2025-09-20 13:58:28 +03:00
Georgi Gerganov 66ad624d5b sync : ggml 2025-09-20 13:46:41 +03:00
Georgi Gerganov 960aaa9904 metal : use function constants for mul_mv_ext kernels (llama/16074) 
* metal : use function constants for mul_mv_ext kernels

ggml-ci

* metal : remove NW template argument

ggml-ci

* metal : adjust constants

ggml-ci
 2025-09-20 13:46:38 +03:00
Georgi Gerganov d37f590a77 metal : improve F32, F16 and BF16 mat-vec multiplication (llama/16057) 
* metal : improve F32, F16 and BF16 mat-vec multiplication

ggml-ci

* metal : make the NSG a function constant in mul_mv kernels

ggml-ci
 2025-09-20 13:46:38 +03:00
Georgi Gerganov 1f24b1df4d metal : handle nil cv during pipeline creation (llama/16065) 
ggml-ci
 2025-09-20 13:46:38 +03:00
Georgi Gerganov eb2c01f92e metal : refactor + optimize v2 (llama/15995) 2025-09-20 13:46:10 +03:00
Georgi Gerganov 6458bac4c1 sync : ggml 2025-09-20 13:45:32 +03:00
Georgi Gerganov 82a8c141ea metal : remove memory pools (llama/15966) 
* metal : remove mem pool usage

ggml-ci

* metal : remove mem pool implementation

ggml-ci

* metal : take into account the actual allocated memory of the tensor

ggml-ci

* cont : use ggml_backend_buft_get_alloc_size

ggml-ci

* cont : improve, comments

ggml-ci

* cont : add functions for the extra tensor sizes

* metal : add comments

ggml-ci

* metal : implement .get_alloc_size for the rest of the buffer types

ggml-ci

* metal : remove ggml_metal_heap

ggml-ci
 2025-09-20 13:45:29 +03:00
Georgi Gerganov 2d3f15607f metal : fix kernel requirements (llama/15983) 
* metal : fix kernel requirements

ggml-ci

* cont : fix supports_op

* cont : fix supports_op for ARGMAX
 2025-09-20 13:45:29 +03:00
Georgi Gerganov 2caf15d68a metal : refactor kernel loading (llama/15964) 
* metal : refactor bin kernels loading

ggml-ci

* metal : refactor rms kernel loading

ggml-ci

* ci : try to add memory leaks check

ggml-ci

* ci : try to enable memory leak detection for Mac

* cont : seems to be working
 2025-09-20 13:45:29 +03:00
Georgi Gerganov 0d36ba9e1a metal : allow ops to run concurrently (llama/15929) 
* metal : run graphs ops concurrently

ggml-ci

* cont : add flags for debugging and disabling concurrency

ggml-ci

* cont : refactor and handle fusing

ggml-ci

* cont : simplify - no need to use GPU address

ggml-ci

* cont : prepare mem ranges for reuse + add ggml-metal-common.cpp

ggml-ci

* cont : avoid redundant keywords in cpp [no ci]

* metal : reorder graph for better concurrency

ggml-ci

* metal : fix race on mem pool buffers

ggml-ci

* cont : add env GGML_METAL_GRAPH_OPTIMIZE_DISABLE

ggml-ci

* cont : refactor, optimize, add comments

ggml-ci

* cont : refactor ggml-metal.m

ggml-ci

* minor : update logs [no ci]
 2025-09-20 13:45:29 +03:00
Georgi Gerganov 20a930ec94 metal : fix memory leaks (llama/15962) 
ggml-ci
 2025-09-20 13:45:28 +03:00
Georgi Gerganov c974f63057 sync : ggml 2025-09-20 13:44:48 +03:00