3 people dd1ea52433 llama : support multi-output backend sampling (#25532) ...

* Enable backend sampling with token speculation

* Clamp the mask sum before converting it into the sampled index

* Add a numeric context parameter declaring the maximum outputs one sequence

* More fixes

* Don't reuse memory for output views.

* Match dist between CPU and GPU

* Fix CPU and backend sampling mismatches

* Simpify some of the changes

* Fix tests on Vulkan

* More test fixes

* Rebase changes

* Rebase and address review comments

* Address review comments

* Address review comments

* Update src/llama-sampler.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2026-08-10 16:58:56 +03:00

..

jinja

chat : add new template for DeepSeek V4 Flash 0731 (#26398)

2026-08-03 17:59:11 -05:00

arg.cpp

server: add more tool isolation support (ssh remote + podman rootless) (#26774)

2026-08-10 13:31:09 +02:00

arg.h

mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) (#26254)

2026-08-04 17:26:15 +02:00

base64.hpp

llava : expose as a shared library for downstream projects (#3613)

2023-11-07 00:36:23 +03:00

build-info.cpp.in

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

build-info.h

libs : rename libcommon -> libllama-common (#21936)

2026-04-17 11:11:46 +03:00

chat-auto-parser-generator.cpp

Add support for Laguna XS.2 & M.1 (#25165)

2026-07-22 09:54:08 +08:00

chat-auto-parser-helpers.cpp

server: fix checkpoints creation (#22929)

2026-05-25 08:56:18 +03:00

chat-auto-parser-helpers.h

chat : avoid including json in chat.h (#21306)

2026-04-03 09:07:59 +03:00

chat-auto-parser.h

Add support for Laguna XS.2 & M.1 (#25165)

2026-07-22 09:54:08 +08:00

chat-diff-analyzer.cpp

Add support for Laguna XS.2 & M.1 (#25165)

2026-07-22 09:54:08 +08:00

chat-peg-parser.cpp

chat : add qwen3 specialized parser (#26252)

2026-08-02 04:13:20 -05:00

chat-peg-parser.h

chat : add qwen3 specialized parser (#26252)

2026-08-02 04:13:20 -05:00

chat.cpp

model: Muse Glimmer Support (#26841)

2026-08-10 13:07:27 +02:00

chat.h

common/chat: add specialized minimax m3 parser (#26210)

2026-07-28 04:27:20 -05:00

CMakeLists.txt

common: add subproc.h wrapper, disabled on android/ios (#26102)

2026-07-26 20:54:25 +02:00

common.cpp

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

common.h

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

console.cpp

cli: fix stripping of \n in multiline input (#21485)

2026-04-06 20:54:06 +02:00

console.h

cli : add command and file auto-completion (#19985)

2026-03-05 10:47:28 +01:00

debug.cpp

common: fix missing exports in llama-common (#22340)

2026-04-27 08:06:39 +03:00

debug.h

common: fix missing exports in llama-common (#22340)

2026-04-27 08:06:39 +03:00

download.cpp

common: support the DSpark sidecar resolution (#26458)

2026-08-02 19:25:27 +02:00

download.h

common: support the DSpark sidecar resolution (#26458)

2026-08-02 19:25:27 +02:00

fit.cpp

fit: Fix memory allocation for MTP layers (#26605)

2026-08-05 13:29:45 +02:00

fit.h

fit : wrap llama_device_memory_data (#24522)

2026-06-13 08:09:52 +03:00

hf-cache.cpp

server: (router) add model management API (#23976)

2026-06-17 18:04:58 +02:00

hf-cache.h

server: (router) add model management API (#23976)

2026-06-17 18:04:58 +02:00

http.h

cli : move to HTTP-based implementation (#24948)

2026-07-08 14:52:43 +02:00

imatrix-loader.cpp

Move duplicated imatrix code into single common imatrix-loader.cpp (#22445)

2026-06-04 17:45:40 +02:00

imatrix-loader.h

Move duplicated imatrix code into single common imatrix-loader.cpp (#22445)

2026-06-04 17:45:40 +02:00

json-schema-to-grammar.cpp

common/json-schema-to-grammar : align spacing rules with parsers (#24835)

2026-06-20 17:43:04 -05:00

json-schema-to-grammar.h

common : add nemotron 3 parsing (#18077)

2025-12-16 04:05:23 -06:00

llguidance.cpp

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

log.cpp

common: update logging to enforce max_capacity and optimize queue resizing (#24490)

2026-06-17 09:19:11 +03:00

log.h

logs : reduce (#23021)

2026-05-14 13:05:52 +03:00

ngram-cache.cpp

spec : add self‑speculative decoding (no draft model required) + refactor (#18471)

2026-01-28 19:42:42 +02:00

ngram-cache.h

spec : add self‑speculative decoding (no draft model required) + refactor (#18471)

2026-01-28 19:42:42 +02:00

ngram-map.cpp

speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936)

2026-07-07 10:25:04 +03:00

ngram-map.h

fix: correct misspellings in code comments (#21217)

2026-03-31 13:50:51 +02:00

ngram-mod.cpp

ngram-mod : Add missing include (#23857)

2026-05-29 09:21:37 +03:00

ngram-mod.h

ngram-mod : fix build [no ci] (#19216)

2026-01-30 21:27:27 +02:00

peg-parser.cpp

common : add support for multiple end sequences in the reasoning budget sampler (#25544)

2026-07-25 11:58:09 +02:00

peg-parser.h

common/peg : implement ac parser for stricter grammar generation (#24869)

2026-06-21 16:20:58 -05:00

preset.cpp

common : skip empty implicit default preset (#25643)

2026-07-25 21:15:27 +02:00

preset.h

server: (router) rework -hf preset repo (#24739)

2026-06-18 12:45:23 +02:00

reasoning-budget.cpp

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

reasoning-budget.h

common : add support for multiple end sequences in the reasoning budget sampler (#25544)

2026-07-25 11:58:09 +02:00

sampling.cpp

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

sampling.h

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

speculative.cpp

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

speculative.h

llama : support multi-output backend sampling (#25532)

2026-08-10 16:58:56 +03:00

subproc.cpp

common: add subproc.h wrapper, disabled on android/ios (#26102)

2026-07-26 20:54:25 +02:00

subproc.h

common: add subproc.h wrapper, disabled on android/ios (#26102)

2026-07-26 20:54:25 +02:00

trie.cpp

common : add support for multiple end sequences in the reasoning budget sampler (#25544)

2026-07-25 11:58:09 +02:00

trie.h

common : add support for multiple end sequences in the reasoning budget sampler (#25544)

2026-07-25 11:58:09 +02:00

unicode.cpp

common/parser: handle reasoning budget (#20297)

2026-03-11 10:26:12 +01:00

unicode.h

common/parser: handle reasoning budget (#20297)

2026-03-11 10:26:12 +01:00