e0a5b08cdc
ggml : generalize quantize_fns for simpler FP16 handling ( #1237 )
...
* Generalize quantize_fns for simpler FP16 handling
* Remove call to ggml_cuda_mul_mat_get_wsize
* ci : disable FMA for mac os actions
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-07-05 19:13:06 +03:00
Stephan Walter and GitHub
2ac41ed3ee
Remove unused n_parts parameter ( #1509 )
2023-05-17 22:12:01 +00:00
Stephan Walter and GitHub
8ceb2442b7
Various fixes to mat_mul benchmark ( #1253 )
2023-04-30 12:32:37 +00:00
Stephan Walter and GitHub
4aeea780bc
Remove Q4_3 which is no better than Q5 ( #1218 )
2023-04-28 23:10:43 +00:00
Stephan Walter and GitHub
e05d16c8c7
ggml : slightly faster AVX2 implementation for Q5 ( #1197 )
2023-04-26 23:26:42 +03:00
994f7e33aa
Update SHA256SUMS after quantization change ( #1181 )
...
Co-authored-by: Pavol Rusnak <pavol@rusnak.io >
2023-04-25 23:41:56 +02:00
Stephan Walter and GitHub
c62ac69921
Fix build for gcc 8 and test in CI ( #1154 )
2023-04-24 15:38:26 +00:00
Stephan Walter and GitHub
fe240a9faf
Improve AVX2 for vec_dot_q4_3_q8_0 ( #1138 )
2023-04-23 11:01:03 +00:00
Stephan Walter and GitHub
90da0b75a3
ci : trigger CI for drafts, but not most PR actions ( #1125 )
2023-04-22 16:12:29 +03:00
Stephan Walter and GitHub
949ca5ce05
Fix CI: ARM NEON, quantization unit tests, editorconfig ( #1122 )
2023-04-22 10:54:13 +00:00
Stephan Walter and GitHub
9cecc39408
ggml : AVX2 optimization for vec_dot_q4_3_q8_0 and refactoring ( #1099 )
...
* AVX2 optimization for vec_dot_q4_3_q8_0 and refactoring
* finish AVX vectorization of quantize_row_q8_0
* Rename hsum_int_8 to hsum_i32_8
2023-04-22 10:37:05 +03:00
fe9564f09e
Add ggml-model-*.bin checksums for 7B, 13B, 30B, 65B ( #1088 )
...
* Add ggml-model-*.bin checksums for 7B, 13B, 30B
* Add ggml-model-*.bin checksums for 65B
---------
Co-authored-by: Pavol Rusnak <pavol@rusnak.io >
2023-04-20 23:56:44 +02:00
Stephan Walter and GitHub
091a53228c
AVX2 optimization for vec_dot_q4_2_q8_0 ( #1068 )
2023-04-20 08:45:41 +02:00
Stephan Walter and GitHub
ec0e355be1
ggml : Q4 cleanup - remove 4-bit dot product code ( #1061 )
...
* Q4 cleanup
* Remove unused AVX512 Q4_0 code
2023-04-19 19:06:37 +03:00
Stephan Walter and GitHub
0363b17de2
Fix potential int8 overflow in non-SIMD vec_dot ( #986 )
2023-04-15 18:28:56 +00:00
Stephan Walter and GitHub
378ffbab0e
Refactor ggml.c for future tensor types ( #1001 )
2023-04-15 16:25:38 +00:00
Stephan Walter and GitHub
50879f9f5b
make : fix dependencies, use auto variables ( #983 )
2023-04-14 22:39:48 +03:00
Stephan Walter and GitHub
2bf3e1346e
ggml : optimize non-SIMD Q4_0 vector dot product ( #703 )
2023-04-13 17:59:50 +03:00
Stephan Walter and GitHub
2ed5c65183
Don't crash on ftype (formerly f16) == 4 ( #917 )
2023-04-12 15:06:16 +00:00
Stephan Walter and GitHub
c8296315db
Add enum llama_ftype, sync ggml_type to model files ( #709 )
2023-04-11 15:03:51 +00:00
Stephan Walter and GitHub
aa6766c975
llama : do not allocate KV cache for "vocab_only == true" ( #682 )
...
Fixes sanitizer CI
2023-04-02 10:18:53 +03:00
Stephan Walter and GitHub
f7ea9fa785
Enable -std= for cmake builds, fix warnings ( #598 )
2023-03-31 19:19:16 +00:00
Stephan Walter and GitHub
f27e27c590
cmake : properly invoke CTest ( #629 )
2023-03-30 20:56:59 +03:00
223cad655e
ggml : refactor quantized processing functions ( #509 )
...
* Refactor quantized processing functions
* ggml : minor
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-03-28 20:13:01 +03:00
188fb59d88
all : be more strict about converting float to double ( #458 )
...
* Be more strict about converting float to double
* Test equivalence of round, SILU implementations
Test module is commented out in CMakeLists.txt because the tests may
take a long time, depending on how much the compiler optimizes.
* Fix softmax in perplexity.cpp
* all : prefer float over double where appropriate
* perplexity : add <cmath>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-03-28 19:48:20 +03:00
884f88402f
ggml : introduce structs for the q4 data blocks ( #356 )
...
* Introduce structs for the q4 data blocks
* ggml : rename quant struct variables + fix ARM_NEON
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-03-28 18:56:03 +03:00
Stephan Walter and GitHub
180198d957
Fix undefined variables in debug build, remove unused variables ( #531 )
2023-03-26 15:34:02 +00:00
Stephan Walter and GitHub
3b8b2c584a
Update README and comments for standalone perplexity tool ( #525 )
2023-03-26 16:14:01 +03:00
3ebb023fb2
Revert "Delete SHA256SUMS for now" ( #429 )
...
* Revert "Delete SHA256SUMS for now (#416 )"
This reverts commit 8eea5ae0e5pavol@rusnak.io >
2023-03-23 15:15:48 +01:00
43a021a260
Deduplicate q4 quantization functions ( #383 )
...
* Deduplicate q4 quantization functions
* Use const; add basic test
* Re-enable quantization test
* Disable AVX2 flags in CI
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-03-22 19:29:06 +02:00
Stephan Walter and GitHub
a5d8a1a459
Add tqdm to Python requirements ( #293 )
...
* Add tqdm to Python requirements
* Remove torchvision torchaudio, add requests
2023-03-20 09:24:11 +01:00
Stephan Walter and GitHub
45113b2f42
Don't tell users to use a bad number of threads ( #243 )
...
The readme tells people to use the command line option "-t 8", causing 8
threads to be started. On systems with fewer than 8 cores, this causes a
significant slowdown. Remove the option from the example command lines
and use /proc/cpuinfo on Linux to determine a sensible default.
2023-03-17 19:47:35 +02:00
