mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-08-12 22:29:39 +04:00
NexesenexandGitHub
f8b5664c18
* ggml: fix typos in comments across ggml/src Corrects misspellings found in source comments (no code/logic changes): CUDA: - quantize.cu / quantize_id.cu: 'memroy' -> 'memory', stray 'Exchange' word in reduction comment - fattn-mma-f16.cuh / fattn-new-mma.cu: 'synchonization' -> 'synchronization' - fattn-new-mma.cu / fattn-vec-common.cuh: 'at lease' -> 'at least' - fattn-vec-f32.cuh: 'Currenlty'/'dose' -> 'Currently'/'does' - mmq_id_common.cuh: 'alays' -> 'always' - softmax.cu: 'noncontigous' -> 'noncontiguous' CPU / quantization: - ggml-quants.c: 'At leat' -> 'At least' - ggml.c: 'repeatition' -> 'repetition' CANN: - aclnn_ops.cpp: 'alloced' -> 'allocated', 'contigous' -> 'contiguous' - kernels/dup.cpp: 'contigous' -> 'contiguous' IQK: - iqk_gemm_1bit.cpp: 'explicitely' -> 'explicitly' - iqk_gemm_ktquants.cpp: 'nn AVX2' -> 'in AVX2' Vulkan: - ggml-vulkan.cpp: duplicated 'in in' -> 'in' * ggml: fix less-common typos in comments (spellchecker pass) Second sweep using a spell-check pass with edit-distance filtering to catch typos missed by the common-misspelling list. Comment/comment-context fixes only, no code changes: - aclnn_ops.cpp: 'performace' -> 'performance' - acl_tensor.cpp: 'shoule in' -> 'should be in' - common.h: 'opertors'/'available' -> 'operators'/'available' - ggml-cuda.cu: 'resepctive' -> 'respective', 'resinding' -> 'residing' - conv2d.cu: 'ouptut_chanles' -> 'output_channels' - scale.cu: 'Whehn' -> 'When' - mmq_id_common.cuh: 'renameing' -> 'renaming' - solve_tri.cu: 'supress' -> 'suppress' - ggml-quants.c: 'ptoducts' -> 'products', 'quckly' -> 'quickly', 'Acummulate' -> 'Accumulate' - ggml-sycl.cpp: 'solutino'/'walkaroud' -> 'solution'/'workaround' - ggml-vulkan.cpp: 'aross' -> 'across' - ggml.c unified base: signficantly -> significantly (recorded in iqk too) Also fixed duplicate word 'get get' in the quckly comments (line 14272/14488). * Fix typo: correct hard-to-count words in comments/docs across common/examples Spellcheckedtypos across common/, examples/, tests/ and include/ (from typos2.txt): preserve->preserving, replacement->replacemnt, enhance->enchance, imatrix/ima->imatrix, correct->corerct, parameter->parmeter, utilizing->utilitizing, backward->backwrad, manipulate->manupulate, together->togather, incomplete->parial, sentence->dentence, retrieval->retie, prepared->prepa, partial->parial, randomly->Randonly. Comment/prose only, no code changes. * fixing typos (public_simplechat example) * fixing typos (examples subdirs)
2.2 KiB
2.2 KiB
llama.cpp/example/embedding
This example demonstrates generate high-dimensional embedding vector of a given text with llama.cpp.
Quick Start
To get started right away, run the following command, making sure to use the correct path for the model you have:
Unix-based systems (Linux, macOS, etc.):
./llama-embedding -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>/dev/null
Windows:
llama-embedding.exe -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>$null
The above command will output space-separated float values.
extra parameters
--embd-normalize integer
integer |
description | formula |
|---|---|---|
-1 |
none | |
0 |
max absolute int16 | \Large{{32760 * x_i} \over\max \lvert x_i\rvert} |
1 |
taxicab | \Large{x_i \over\sum \lvert x_i\rvert} |
2 |
euclidean (default) | \Large{x_i \over\sqrt{\sum x_i^2}} |
>2 |
p-norm | \Large{x_i \over\sqrt[p]{\sum \lvert x_i\rvert^p}} |
--embd-output-format 'string'
'string' |
description | |
|---|---|---|
| '' | same as before | (default) |
| 'array' | single embeddings | [[x_1,...,x_n]] |
| multiple embeddings | [[x_1,...,x_n],[x_1,...,x_n],...,[x_1,...,x_n]] |
|
| 'json' | openai style | |
| 'json+' | add cosine similarity matrix |
--embd-separator "string"
"string" |
|
|---|---|
| "\n" | (default) |
| "<#embSep#>" | for example |
| "<#sep#>" | other example |
examples
Unix-based systems (Linux, macOS, etc.):
./llama-embedding -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2 --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null
Windows:
llama-embedding.exe -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2 --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null