Consistify polling frequency of llama-bench with args

llama-bench differs from llama-server in 2 ways:
1. It attaches a persistent threadpool across cgraph invocations
2. It has a polling scheme of 50 (as opposed to 0 of args)

Together, this causes a significant perf regression in the CUDA backend
on Windows when the CPU backend is built with OpenMP disabled.
Why is the CPU backend affecting the CUDA backend?
Typically, we have a LUT on the CPU backend where we map Token_ID
-> Embedding to save on VRAM budget.
This commit is contained in:
Oliver Simons
2026-08-07 20:59:38 +02:00
parent fc6545d322
commit ae86645c28
+1 -1
View File
@@ -380,7 +380,7 @@ static const cmd_params cmd_params_defaults = {
/* n_threads */ { common_cpu_get_num_math() },
/* cpu_mask */ { "0x0" },
/* cpu_strict */ { false },
/* poll */ { 50 },
/* poll */ { 0 },
/* n_gpu_layers */ { -1 },
/* n_cpu_moe */ { 0 },
/* split_mode */ { LLAMA_SPLIT_MODE_LAYER },