* ci : add ccache clear action
* ci : split self-hosted GPU jobs into build-self-hosted.yml
Extract self-hosted runner jobs from build.yml into a dedicated
build-self-hosted.yml following the llama.cpp pattern:
- gpu-cuda (NVIDIA Linux)
- gpu-vulkan-nvidia-cm (NVIDIA Linux)
- gpu-vulkan-nvidia-cm2 (NVIDIA Linux + COOPMAT2)
- gpu-metal (macOS ARM64)
- gpu-vulkan (macOS ARM64)
GitHub-hosted CPU jobs remain in build.yml.
Assisted-by: llama.cpp:local pi
* ci : split release jobs into release.yml
Extract release-related jobs from build.yml into a dedicated
release.yml following the llama.cpp pattern:
- determine-tag
- windows (Win32/x64, SDL2)
- windows-blas (Win32/x64, OpenBLAS)
- windows-cublas (x64, CUDA 11.8/12.4)
- ios-xcode-build
- bindings-java (depends on windows)
- release (artifact aggregation + GitHub release)
CoreML job stays in build.yml with its own local tag calculation.
Assisted-by: llama.cpp:local pi
* ci : remove bindings-java job from release.yml
Assisted-by: llama.cpp:local pi
* cont : add manual trigger for build.yml
* cont : remove obsolete ifs
* ci : extract sanitizer job to bild-sanitize.yml
* ci : extract linux jobs into build-linux.yml
* ci : extract macos jobs to build-macos.yml
* ci : extract gcc jobs to build-gcc.yml
* ci : extract clang jobs to build-clang.yml
* ci : extract sycl jobs to build-sycl.yml
* ci : extract windows jobs to build-windows.yml
* ci : extract emscripten job to build-wasm.yml
* ci : extract android jobs into build-android.yml
* ci : extract quantize job to quantize.yml
* ci : extract coreml job into coreml.yml
* ci : extract vad job to vad.yml
* ci : extract cpu jobs to build-cpu.yml
* ci : make naming of yml files consistent
* ci : add --fail to curl download and propagate
This commit adds the --fail option to the model download scripts so that
if the model download returns a server error this is picked up. This is
then detected in run.sh and a error message is displayed and the script
stops and returns an error.
The motivation for this is that currently it is possible for the model
download to fail but this script proceeds and instead of a model file
the contents will be an html page probably with the error. This will
then cause the model to not be able to load due to a missing magic
number. I'm not sure we can do much about the downloading failing,
perhaps a retry but at least this will give a clearer error message.
Refs: https://github.com/danbev/whisper.cpp/actions/runs/26866349389/job/79230794512
* ci : enable command traces to see download command in use
* ci : add retry functionality to download model script
This commit adds curl retry options to the model download script.
The motivation is that currently when CI jobs run huggingface rate limit
the requests and return:
```console
curl: (22) The requested URL returned error: 429
```
This is an attempt to work around this and if it does not work then we
can an authorization token.
* ci : extract freebsd job to build-freebsd.yml
This job has been commented out as it has been flaky in the past. I'll
monitor this and if it continues to be unreliable we can disable it in
the github actions GUI instead of commenting it out like we did before.
* ci : add ccache to jobs (non-docker builds)
The ccache will only be saved on pushed to master.
* ci : bump ccache-action version to v1.2.21
The motivation for this is that the save parameter does not seem to work
with the current version.
* ci : add ccache to docker jobs in build-linux.yml
* ci : add debug statements to linux docker build
* ci : set CCACHE_DIR for build-linux.yml
* ci : add ccache to the remaining docker jobs
* ci : remove build-linux.yml
This commit remove build-linux.yml as the same jobs are also run by
build-gcc.yml, with the exception that build-gcc.yml also run ctest).
So keeping build-gcc.yml and removing the redundant build-linux.yml.
* ci : add linux build artifacts to release
* ci : revert to hendrikmuhs/ccache-action for win job
This is currently causing the following failure:
```console
sccache C:\PROGRA~1\NVIDIA~1\CUDA\v\bin\nvcc.exe -forward-unknown-to-host-compiler -DGGML_BACKEND_BUILD -DGGML_BACKEND_SHARED -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 -Dggml_cuda_EXPORTS -DCMAKE_INTDIR=\"Release\" -ID:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\.. -ID:\a\whisper.cpp\whisper.cpp\ggml\src\..\include -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v\include" -Xcompiler="-MD -O2 -Ob2" -DNDEBUG -std=c++17 -arch=native -use_fast_math -extended-lambda -Xcompiler /Zc:preprocessor -MD -MT ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -MF ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj.d -x cu -c D:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\allreduce.cu -o ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -Xcompiler=-Fdggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\,-FS
sccache: encountered fatal error
sccache: error: Could not parse shell line
sccache: caused by: Could not parse shell line
```
Refs: https://github.com/danbev/whisper.cpp/actions/runs/26883673904/job/79290017353
* ci : make static linux artifacts
* ci : make linux release artifact names consistent
This commit removes the tag form the linux release artifacts to be
consistent with the existing artifacts.
If we want to include the tag then we can do that in a follow-up PR.
* ci : fix linux zip files to have a directory
* ci : add HF_TOKEN secret for HF download authorization
This is to avoid the HR rate limiting when downloading model.
---------
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
On busybox-based systems like Alpine Linux, wget does not have
certain CLI flags such as '--no-config'. Thus, search for the
existence of 'curl' first in the PATH before wget. wget2 is
still the preferred download tool.
* whisper : improve whisper-cli executable path detection in model download shell scripts
If whisper-cli is found on the path, do not suggest invoking from build directory. This improves flexibility and usability for distribution and packaging scenarios.
* whisper : enhance Windows model download batch script to have comparable functionality and behaviour as shell scripts
* Download models to the current directory if the script is executed from the \bin\ directory (for future distribution scenarios where the script is in the \bin\ subdirectory of a Windows build)
* Add model_path command line argument
* If whisper-cli is found on the path, do not suggest invoking from build directory
* whisper : resolve compiler warning by removing duplicate definition of NOMINMAX in whisper-cli code
This change ensures that when the script is packaged and distributed, models are downloaded to the current directory instead of the script's location, preventing conflicts with system directories. This improves flexibility and usability for distribution and packaging scenarios.
* build: add dockerfile for ci
* ci: add action to build/push docker image
* fix: lowercase repository to fix ci
* ci: update cuBLAS flag
* build: install curl and ffmped in image
* docs: add docker section
* fix: improve args check when download model
* add HuggingFace mirror to download ggml model
* support tdrz via simple hack overriding solm tokens
* fix incorrect translate/transcribe token_ids that are not static const
* add apollo 13 sample for tdrz demo
* render [SPEAKER TURN] consistently in all terminal output using vocab.id_to_token
* extend whisper_segment with speaker_turn_next field and save in json output
* fix failing go build
* slipped in some python syntax whoops
* whisper : finalize tinydiarize support (add flag + fixes)
* whisper : tdrz support for word-level timestamps (respect max_len)
* java : try to fix tests after adding tdrz_enable flag
* main : remove TODO leftover
* java : fix params order list after adding "tdrz_enable"
* whisper : fix solm and add nosp token
* main : print tinydiarize help
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit modifies the `get_script_path` function to correctly handle
spaces and special characters in directory paths. The fix involves adding
double quotes around variables and commands where needed to ensure proper
parsing of paths with spaces and special characters.
The old "large" model is now renamed "large-v1".
If you have been using it, make sure to rename it and download the new
"large" model for best results.
