Howard Su and GitHub
2d2e6da9c6
Revert "Support using mmap when applying LoRA ( #2095 )" ( #2206 )
...
Has perf regression when mlock is used.
This reverts commit 2347463201
2023-07-13 21:58:25 +08:00
Howard Su and GitHub
32f1dba0a5
Fix compile error on Windows CUDA ( #2207 )
2023-07-13 21:58:09 +08:00
cf1fc25386
FP16 is supported in CM=6.0 ( #2177 )
...
* FP16 is supported in CM=6.0
* Building PTX code for both of 60 and 61
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
2023-07-12 20:18:40 +08:00
Howard Su and GitHub
cc7216222c
Support using mmap when applying LoRA ( #2095 )
...
* Support using mmap when applying LoRA
* Fix Linux
* Update comment to reflect the support lora with mmap
2023-07-11 22:37:01 +08:00
Howard Su and GitHub
b29cd6ba6d
Fix opencl by wrap #if-else-endif with \n ( #2086 )
2023-07-07 05:34:18 +02:00
Howard Su and GitHub
928a2061d8
llama: Don't double count the sampling time ( #2107 )
2023-07-05 18:31:23 +08:00
Howard Su and GitHub
b4acd69f66
Fix crash of test-tokenizer-0 under Debug build ( #2064 )
...
* Fix crash of test-tokenizer-0 under Debug build
* Change per comment
2023-07-03 20:43:55 +02:00
Howard Su and GitHub
1a58c36ca8
[llama] No need to check file version when loading vocab score ( #2079 )
2023-07-03 19:58:58 +08:00
41ce2a335b
Use unsigned for random seed ( #2006 )
...
* Use unsigned for random seed. Keep -1 as the value to use a time based seed.
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-06-29 06:15:15 -07:00
Howard Su and GitHub
6d8b07691b
llama : remove shards weight file support ( #2000 )
...
* Remove multiple shards
* Remove multiple file loaders
* Remove llama_load_tensor_shard class
* Simplify load logic
* Remove dead code guess_n_parts function
* Remove vocab_only from constructor of llama_model_loader
* Remove alignment_prevents_mmap which is not more needed.
* Remove useless check
2023-06-28 20:13:02 +03:00
Howard Su and GitHub
0dacdf3f52
baby-llama : fix build after ggml_rope change ( #2016 )
2023-06-27 08:07:13 +03:00
Howard Su and GitHub
c2c3a4c9f5
Fix assert when free invalid cuda pointer ( #2005 )
...
Fix assert via initializing extra structure always.
CUDA error 1 at C:\GPT\llama.cpp\ggml-cuda.cu:2536: invalid argument
2023-06-26 23:15:47 +08:00
25cc6bb618
cmake : fix build shared ggml when CUDA is enabled ( #1929 )
...
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-06-19 18:10:37 +03:00
Howard Su and GitHub
9b0d4f1fc7
cmake : add CUDA_ARCHITECTURES to new target ggml_static ( #1917 )
2023-06-18 07:29:47 +03:00
Howard Su and GitHub
49fdf2d578
ggml : fix warnings under MSVC ( #1908 )
2023-06-17 18:46:15 +03:00
Howard Su and GitHub
9af138537f
Fix the validation of main device ( #1872 )
2023-06-15 19:29:59 +02:00
df687e822c
Leverage mmap for offloading tensors to GPU ( #1597 )
...
* Rebase to latest
* Show progress
* Add assert to make sure we only allocate temp buffer for non-CPU backend tensor
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
2023-06-12 14:44:16 +02:00
Howard Su and GitHub
139240d596
opencl : no need to allocate cl_mem on heap ( #1612 )
2023-05-28 20:13:36 +03:00
Howard Su and GitHub
b4e11a1e94
opencl : use strstr to check if fp16 supported ( #1611 )
...
* Use strstr to check if fp16 supported
* Ensure ext_buffer is null terminated
2023-05-28 20:09:56 +03:00
Howard Su and GitHub
f9a61db2de
cmake : fix build under Windows when enable BUILD_SHARED_LIBS ( #1100 )
...
* Fix build under Windows when enable BUILD_SHARED_LIBS
* Make AVX512 test on Windows to build the shared libs
2023-04-22 11:18:20 +03:00
Howard Su and GitHub
e0dbf8218f
ggml : optimize rope function to avoid call powf in the tight loop ( #807 )
2023-04-14 09:24:52 +03:00
