mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-08-12 22:29:39 +04:00

Files

T

NexesenexandGitHub c3b075f069 Chores : tidy up more typos project wide (ggml directory excluded), new -ptcall alias (#2237 )

* common: fix coding mistakes (typos in identifiers, flags and log strings)

Fix misspelled identifiers and user-facing strings across common, server
and model loading:

- allow_ruless -> allow_rules (misspelled identifier used in the allowlist
  CLI parsing and the server slot/context code)
- get_formated_timings/get_formated_generation -> get_formatted_*
- 'termionated' -> 'terminated' in the fit-margin assert message
- 'defaulr' -> 'default' in the YAML dump
- 'overriden' -> 'overridden' in tensor buffer type override logs
- 'becausee' -> 'because' in the output-tensor split log
- 'etected NaNs' -> 'detected NaNs' in the imatrix error message

* common: fix comment typos across src, common, include and examples

Fix misspelled words in code comments:

- llama.h: 'typy' -> 'type', 'transfrom' -> 'transform', 'ecoder' ->
  'encoder', 'indicies' -> 'indices', 'Intializes' -> 'Initializes'
- common.h: 'embendings' -> 'embeddings', 'pr' -> 'or' in the
  fused-indexer-topk comment
- chat.cpp: 'overridde' -> 'override'
- ngram-map: 'occurences' -> 'occurrences', 'stastistics' -> 'statistics'
- speculative.cpp: 'dont'/'inehit' -> 'don't'/'inherit'
- llama-mmap.cpp: 'dont't' -> 'don't'
- llama-model.h: 'hcurrently andle' -> 'currently handle'
- build_gemma3/4.cpp: 'emdeddings' -> 'embeddings'
- examples: 'quantizuation', 'logprobe', 'throught', 'retrun', 'swich',
  'convinient', 'temporally' (-> 'temporary'), 'temproal', 'preceed'

* common: remove duplicate definitions and duplicate help entries

- clip-impl.h: drop the second, identical #define TN_FFN_GATE
- common.cpp: remove the duplicate '-t, --threads N' help entry that was
  misplaced in the export-lora section (already listed in the general
  section)
- common.cpp: merge the two 'embedding' help groups into a single group
  so the embedding options are listed together
- llama.cpp: remove the redundant LLAMA_MAX_LAYERS define (llama-hparams.h
  already defines the same value and is included by llama.cpp)

* common: fix remaining typos (accomodate, recommanded, occurences, occassionally)

- accomodate -> accommodate in src/llama.cpp comment
- recommanded -> recommended in quantize.cpp user-facing output
- occurences -> occurrences in test-chat.cpp JSON string
- occassionally -> occasionally in vendor/stb/stb_image_resize2.h comment

Note: tokenizer.ggml.seperator_token_id kept as-is to match GGUF spec

* common: remove duplicate help entries

- remove the duplicate '--reasoning-budget N' help entry that was repeated
  in the main section (introduced in e0596bf614 'Autoparser - complete
  refactoring of parser architecture (PR 1376)')
- remove the second '--parallel-tool-calls' help entry that advertised the
  '-ptc' short flag, which belongs to '--print-token-count' (introduced in
  e0596bf614 'Autoparser - complete refactoring of parser architecture
  (PR 1376)'); the '-ptc' alias was non-functional for '--parallel-tool-calls'
  because the parser only binds it to '--print-token-count'

The canonical help entries are kept:
- '--reasoning-budget N' is listed once
- '--parallel-tool-calls' is listed once (without the conflicting '-ptc' alias)

* common: remove duplicate LOG_ENABLE define

- the '#undef LOG_ENABLE / #define LOG_ENABLE() // dummy stub' pair was
  repeated verbatim inside the LOG_DISABLE_LOGS section
- remove the second occurrence (introduced in a2588b53e1 'main : log
  file (PR 2748)')

* llama-bench: align MLA and attention-max-batch flags with common tools

llama-bench used '--mla-attn' and '--attn-max-batch' while the common
CLI parsing (common/common.cpp) uses '--mla-use' and
'--attention-max-batch' for the same features. This made the flags
inconsistent across tools.

- update the help text to advertise the canonical names
  '--mla-use' and '--attention-max-batch'
- keep the old '--mla-attn' and '--attn-max-batch' spellings working
  as aliases so existing scripts are not broken

The divergent names were introduced in 3e536b95b0 'Add optional MLA
(PR 188)'.

* fix typos in comments and user-facing strings

- ngram-map.cpp: 'Do we haven a existing' -> 'Do we have an existing'
  (introduced in 1cb7e1bf39 'spec : add self speculative decoding,
  ngram and refactor (PR 1261)')
- build_mamba.cpp: 'weigth' -> 'weight' (introduced in 8befd92ea5
  'Refactor model compute graphs (PR 1651)')
- gguf-split.cpp: 'one of splits have 0 tensors' -> 'one of the splits
  has 0 tensors' (introduced in 75b580db0a 'split: allow
  --split-max-size option (PR 6343)')
- gguf-split.cpp: 'merged from %d split' -> 'merged from %d splits'
  (introduced in 1b5523dc79 'gguf-split: split and merge gguf per
  batch of tensors (PR 6135)')
- convert-llama2c-to-ggml.cpp: missing opening quote in the help line,
  '(default %s\\')' -> '(default '%s\\')' (introduced in bb9ebb4394
  'Adding support for llama2.c models (PR 2559)')

* harmonize British and American spelling to American English

The codebase uses American English (e.g. --embd-normalize, --color),
but a few strings/comments still used British spellings. Unify them:

- 'normalisation' -> 'normalization' in common.h, common.cpp help text
  and code comment, and llama-build-context.cpp comment
- 'colorise' -> 'colorize' in the --color help text (common.cpp)
- 'behaviour' -> 'behavior' in a chat.cpp warning and a llama.cpp comment
- also fix 'openai' -> 'OpenAI' capitalization in the embedding help
  text and common.h comment (embedding output format is OpenAI-style)

* common: fix help text formatting inconsistencies

- '-smf16'/'--split-mode-f16' and '-smf32'/'--split-mode-f32' help
  entries displayed hardcoded 'true'/'false' as the default value;
  show the actual state derived from params.reduce_type instead
- '-no-mmad' help entry had 'fused_mmad?' without a space before the
  ternary operator
- '--reasoning-tokens' help continuation lines used tab characters for
  indentation while the sibling '--reasoning-format' entry uses spaces;
  convert to consistent space indentation

* common: revert smf16/smf32 help text default display change

Revert the '-smf16'/'--split-mode-f16' and '-smf32'/'--split-mode-f32'
help entries back to their original hardcoded 'true'/'false' default
display. The change to derive the default from params.reduce_type was
not desired; the split-mode options are legacy and the hardcoded
defaults reflect their intended meaning.

The other formatting fixes in the same area (fused_mmad ternary
spacing and the reasoning-tokens tab-to-space indentation) are kept.

* llama-bench: fix help text column alignment

The --mla-use and --attention-max-batch help lines introduced by the
flag alignment landed one column off from the sibling entries
((default: at column 51 instead of 50). Adjust the padding so all
help lines align.

* common: fix help text defaults for graph-reduce-type and log-format

Mismatch 1: -grt, --graph-reduce-type help shows default "f32", but actual default (common.h:463) is "f16" and llama.cpp uses GGML_TYPE_F16.

Mismatch 2: --log-format help shows default "json", but actual default (common.h:536 log_json=false) is text.

* common: add -ptcall short flag for --parallel-tool-calls

* typo

2026-08-03 08:01:18 +03:00

deps

Merge mainline llama.cpp (#3 )

2024-07-27 07:55:01 +02:00

CMakeLists.txt

Merge mainline llama.cpp (#3 )

2024-07-27 07:55:01 +02:00

gguf-hash.cpp

Chores : tidy up more typos project wide (ggml directory excluded), new -ptcall alias (#2237 )

2026-08-03 08:01:18 +03:00

README.md

Merge mainline llama.cpp (#3 )

2024-07-27 07:55:01 +02:00

README.md

llama-gguf-hash

CLI to hash GGUF files to detect difference on a per model and per tensor level.

Command line options:

--help: display help message
--xxh64: use xhash 64bit hash mode (default)
--sha1: use sha1
--uuid: use uuid
--sha256: use sha256
--all: use all hash
--no-layer: exclude per layer hash
--uuid: generate UUIDv5 ID
-c, --check <manifest>: verify against a manifest

About

While most POSIX systems already have hash checking programs like sha256sum, it is designed to check entire files. This is not ideal for our purpose if we want to check for consistency of the tensor data even if the metadata content of the gguf KV store has been updated.

This program is designed to hash a gguf tensor payload on a 'per tensor layer' in addition to a 'entire tensor model' hash. The intent is that the entire tensor layer can be checked first but if there is any detected inconsistencies, then the per tensor hash can be used to narrow down the specific tensor layer that has inconsistencies.

For Maintainers:

Detection of tensor inconsistency during development and automated tests
- This is served by xxh64 which is fast
- This is also served by having per tensor layer to assist in narrowing down the location of the faulty tensor layer
- This is also served by sha1 which is much slower but more widely supported

For Model Creators:

Optional consistent UUID generation based on model tensor content
- This is served by UUIDv5 which is useful for databases keys
  - llama.cpp UUIDv5 Namespace: ef001206-dadc-5f6d-a15f-3359e577d4e5
    - Made via UUIDv5 URL namespace of en.wikipedia.org/wiki/Llama.cpp

For Model Users:

Assurance of tensor layer integrity even if metadata was updated
- This is served by sha256 which is still considered very secure as of 2024

Design Note

The default behavior of this program if no arguments is provided is to hash using xxhash's xxh32 mode because it is very fast and is primarily targeted towards maintainers who may want to use this in automated tests.
xxhash support xxh32 and xxh128 for 32bit hash and 128bit hash respectively however we picked 64bit xxhash as most computers are 64bit as of 2024 and thus would have a better affinity to calculating hash that is 64bit in size.

Compile Example

cmake -B build -DCMAKE_BUILD_TYPE=Debug -DLLAMA_FATAL_WARNINGS=ON
make -C build clean
make -C build llama-gguf-hash VERBOSE=1
./build/bin/llama-gguf-hash test.gguf
./build/bin/llama-gguf-hash --xxh64 test.gguf
./build/bin/llama-gguf-hash --sha1 test.gguf
./build/bin/llama-gguf-hash --uuid test.gguf
./build/bin/llama-gguf-hash --sha256 test.gguf

Generation and Verification Example

To generate we may use this command

./llama-gguf-hash --all test.gguf > test.gguf.manifest

Which would generate a manifest that looks like below, which contains multiple hash type and per tensor layer hashes as well (This excludes UUID as that is an ID not a hash)

xxh64     f66e9cd66a4396a0  test.gguf:tensor_0
sha1      59f79ecefd8125a996fdf419239051a7e99e5f20  test.gguf:tensor_0
sha256    c0510d38fa060c46265e0160a85c7243096b01dd31c2f355bdbb5516b20de1bd  test.gguf:tensor_0
xxh64     7d3a1f9ac04d0537  test.gguf:tensor_1
sha1      4765f592eacf096df4628ba59476af94d767080a  test.gguf:tensor_1
sha256    8514cbcc73692a2c56bd7a33a022edd5ff819614bd23b19915d7224387f397a7  test.gguf:tensor_1
xxh64     a0af5d700049693b  test.gguf:tensor_2
sha1      25cbfbad4513cc348e2c95ebdee69d6ff2fd8753  test.gguf:tensor_2
sha256    947e6b36e20f2cc95e1d2ce1c1669d813d574657ac6b5ac5196158d454d35180  test.gguf:tensor_2
xxh64     e83fddf559d7b6a6  test.gguf:tensor_3
sha1      a9cba73e2d90f2ee3dae2548caa42bef3fe6a96c  test.gguf:tensor_3
sha256    423b044e016d8ac73c39f23f60bf01bedef5ecb03c0230accd824c91fe86f1a1  test.gguf:tensor_3
xxh64     1257733306b7992d  test.gguf:tensor_4
sha1      d7bc61db93bb685ce9d598da89717c66729b7543  test.gguf:tensor_4
sha256    79737cb3912d4201384cf7f16a1a37ff7823f23ea796cb205b6ca361ab9e3ebf  test.gguf:tensor_4
xxh64     d238d16ba4711e58  test.gguf:tensor_5
sha1      0706566c198fe1072f37e0a5135b4b5f23654c52  test.gguf:tensor_5
sha256    60949be8298eced0ecdde64487643d018407bd261691e061d9e9c3dbc9fd358b  test.gguf:tensor_5
xxh64     3fbc3b65ab8c7f39  test.gguf:tensor_6
sha1      73922a0727226a409049f6fc3172a52219ca6f00  test.gguf:tensor_6
sha256    574f4c46ff384a3b9a225eb955d2a871847a2e8b3fa59387a8252832e92ef7b0  test.gguf:tensor_6
xxh64     c22021c29854f093  test.gguf:tensor_7
sha1      efc39cece6a951188fc41e354c73bbfe6813d447  test.gguf:tensor_7
sha256    4c0410cd3c500f078ae5b21e8dc9eb79e29112713b2ab58a882f82a3868d4d75  test.gguf:tensor_7
xxh64     936df61f5d64261f  test.gguf:tensor_8
sha1      c2490296d789a4f34398a337fed8377d943d9f06  test.gguf:tensor_8
sha256    c4401313feeba0261275c3b25bd2d8fe40ce04e0f440c2980ed0e9674c30ff01  test.gguf:tensor_8
xxh64     93fd20c64421c081  test.gguf:tensor_9
sha1      7047ce1e78437a6884337a3751c7ee0421918a65  test.gguf:tensor_9
sha256    23d57cf0d7a6e90b0b3616b41300e0cd354781e812add854a5f95aa55f2bc514  test.gguf:tensor_9
xxh64     5a54d3aad816f302  test.gguf
sha1      d15be52c4ff213e823cb6dd13af7ee2f978e7042  test.gguf
sha256    7dd641b32f59b60dbd4b5420c4b0f6321ccf48f58f6ae201a3dbc4a58a27c6e4  test.gguf

We can then use the normal check command which will by default check for the highest security strength hash and verify against that:

$ ./llama-gguf-hash --check test.gguf.manifest test.gguf
manifest  test.gguf.manifest  sha256  sha1  xxh64
sha256    c0510d38fa060c46265e0160a85c7243096b01dd31c2f355bdbb5516b20de1bd  test.gguf:tensor_0  -  Ok
sha256    8514cbcc73692a2c56bd7a33a022edd5ff819614bd23b19915d7224387f397a7  test.gguf:tensor_1  -  Ok
sha256    947e6b36e20f2cc95e1d2ce1c1669d813d574657ac6b5ac5196158d454d35180  test.gguf:tensor_2  -  Ok
sha256    423b044e016d8ac73c39f23f60bf01bedef5ecb03c0230accd824c91fe86f1a1  test.gguf:tensor_3  -  Ok
sha256    79737cb3912d4201384cf7f16a1a37ff7823f23ea796cb205b6ca361ab9e3ebf  test.gguf:tensor_4  -  Ok
sha256    60949be8298eced0ecdde64487643d018407bd261691e061d9e9c3dbc9fd358b  test.gguf:tensor_5  -  Ok
sha256    574f4c46ff384a3b9a225eb955d2a871847a2e8b3fa59387a8252832e92ef7b0  test.gguf:tensor_6  -  Ok
sha256    4c0410cd3c500f078ae5b21e8dc9eb79e29112713b2ab58a882f82a3868d4d75  test.gguf:tensor_7  -  Ok
sha256    c4401313feeba0261275c3b25bd2d8fe40ce04e0f440c2980ed0e9674c30ff01  test.gguf:tensor_8  -  Ok
sha256    23d57cf0d7a6e90b0b3616b41300e0cd354781e812add854a5f95aa55f2bc514  test.gguf:tensor_9  -  Ok
sha256    7dd641b32f59b60dbd4b5420c4b0f6321ccf48f58f6ae201a3dbc4a58a27c6e4  test.gguf  -  Ok

Verification results for test.gguf.manifest - Success

Or we may explicitly ask for a faster hash like:

$ ./llama-gguf-hash --check test.gguf.manifest --xxh64 test.gguf
manifest  test.gguf.manifest  sha256  sha1  xxh64
xxh64     f66e9cd66a4396a0  test.gguf:tensor_0  -  Ok
xxh64     7d3a1f9ac04d0537  test.gguf:tensor_1  -  Ok
xxh64     a0af5d700049693b  test.gguf:tensor_2  -  Ok
xxh64     e83fddf559d7b6a6  test.gguf:tensor_3  -  Ok
xxh64     1257733306b7992d  test.gguf:tensor_4  -  Ok
xxh64     d238d16ba4711e58  test.gguf:tensor_5  -  Ok
xxh64     3fbc3b65ab8c7f39  test.gguf:tensor_6  -  Ok
xxh64     c22021c29854f093  test.gguf:tensor_7  -  Ok
xxh64     936df61f5d64261f  test.gguf:tensor_8  -  Ok
xxh64     93fd20c64421c081  test.gguf:tensor_9  -  Ok
xxh64     5a54d3aad816f302  test.gguf  -  Ok

Verification results for test.gguf.manifest - Success

Or maybe we want to just check that all the hash is valid:

$./llama-gguf-hash --check test.gguf.manifest --all test.gguf.manifest
manifest  test.gguf.manifest  sha256  sha1  xxh64
xxh64     f66e9cd66a4396a0  test.gguf:tensor_0  -  Ok
sha1      59f79ecefd8125a996fdf419239051a7e99e5f20  test.gguf:tensor_0  -  Ok
sha256    c0510d38fa060c46265e0160a85c7243096b01dd31c2f355bdbb5516b20de1bd  test.gguf:tensor_0  -  Ok
xxh64     7d3a1f9ac04d0537  test.gguf:tensor_1  -  Ok
sha1      4765f592eacf096df4628ba59476af94d767080a  test.gguf:tensor_1  -  Ok
sha256    8514cbcc73692a2c56bd7a33a022edd5ff819614bd23b19915d7224387f397a7  test.gguf:tensor_1  -  Ok
xxh64     a0af5d700049693b  test.gguf:tensor_2  -  Ok
sha1      25cbfbad4513cc348e2c95ebdee69d6ff2fd8753  test.gguf:tensor_2  -  Ok
sha256    947e6b36e20f2cc95e1d2ce1c1669d813d574657ac6b5ac5196158d454d35180  test.gguf:tensor_2  -  Ok
xxh64     e83fddf559d7b6a6  test.gguf:tensor_3  -  Ok
sha1      a9cba73e2d90f2ee3dae2548caa42bef3fe6a96c  test.gguf:tensor_3  -  Ok
sha256    423b044e016d8ac73c39f23f60bf01bedef5ecb03c0230accd824c91fe86f1a1  test.gguf:tensor_3  -  Ok
xxh64     1257733306b7992d  test.gguf:tensor_4  -  Ok
sha1      d7bc61db93bb685ce9d598da89717c66729b7543  test.gguf:tensor_4  -  Ok
sha256    79737cb3912d4201384cf7f16a1a37ff7823f23ea796cb205b6ca361ab9e3ebf  test.gguf:tensor_4  -  Ok
xxh64     d238d16ba4711e58  test.gguf:tensor_5  -  Ok
sha1      0706566c198fe1072f37e0a5135b4b5f23654c52  test.gguf:tensor_5  -  Ok
sha256    60949be8298eced0ecdde64487643d018407bd261691e061d9e9c3dbc9fd358b  test.gguf:tensor_5  -  Ok
xxh64     3fbc3b65ab8c7f39  test.gguf:tensor_6  -  Ok
sha1      73922a0727226a409049f6fc3172a52219ca6f00  test.gguf:tensor_6  -  Ok
sha256    574f4c46ff384a3b9a225eb955d2a871847a2e8b3fa59387a8252832e92ef7b0  test.gguf:tensor_6  -  Ok
xxh64     c22021c29854f093  test.gguf:tensor_7  -  Ok
sha1      efc39cece6a951188fc41e354c73bbfe6813d447  test.gguf:tensor_7  -  Ok
sha256    4c0410cd3c500f078ae5b21e8dc9eb79e29112713b2ab58a882f82a3868d4d75  test.gguf:tensor_7  -  Ok
xxh64     936df61f5d64261f  test.gguf:tensor_8  -  Ok
sha1      c2490296d789a4f34398a337fed8377d943d9f06  test.gguf:tensor_8  -  Ok
sha256    c4401313feeba0261275c3b25bd2d8fe40ce04e0f440c2980ed0e9674c30ff01  test.gguf:tensor_8  -  Ok
xxh64     93fd20c64421c081  test.gguf:tensor_9  -  Ok
sha1      7047ce1e78437a6884337a3751c7ee0421918a65  test.gguf:tensor_9  -  Ok
sha256    23d57cf0d7a6e90b0b3616b41300e0cd354781e812add854a5f95aa55f2bc514  test.gguf:tensor_9  -  Ok
xxh64     5a54d3aad816f302  test.gguf  -  Ok
sha1      d15be52c4ff213e823cb6dd13af7ee2f978e7042  test.gguf  -  Ok
sha256    7dd641b32f59b60dbd4b5420c4b0f6321ccf48f58f6ae201a3dbc4a58a27c6e4  test.gguf  -  Ok

Verification results for test.gguf.manifest - Success

Crypto/Hash Libraries Used

These micro c libraries dependencies was installed via the clib c package manager