* base : @-mention picker foundation - glob search, picker nav, highlight
* feat : @-mention file/folder picker and mention badges in message bubbles
* fix: Imports
* feat : wire the @-mention picker into the chat form
* fix: Bound the glob-search result cache key and prune stale entries
* webui: load the model selected via ?model=
Opening the WebUI with ?model= selects the model but doesn't load it. The load only starts when you send your first message, so you wait for it then.
This loads it as soon as the page opens, while you're still typing your prompt. It's what the model dropdown already does, and it isn't awaited, so the UI still works while the model loads.
This is the path the Llama macOS app uses to open the WebUI, so it's a common way in.
* webui: gate the load behind ?load=true
Loading on landing is opt-in, so a plain ?model= link behaves as before and doesn't allocate memory on its own.
* webui: name the chat URL params
Collects the query params the chat routes read into a URL_PARAMS constant, instead of repeating the literals across three files. NEW_CHAT_PARAM folds into it.
The reference implementation has two mutually exclusive prompt layouts.
In non streaming mode the prefill carries the whole utterance text plus
tts_eos summed with codec_pad, and the trailing text hidden collapses to
a single tts_pad row. In streaming mode the prefill carries only the
first text token and the trailing rows stream the rest of the text
followed by tts_eos.
The pipeline built the non streaming prefill but the streaming overlay,
so the talker saw the utterance a second time during generation and read
it twice before emitting codec_eos.
The overlay is now the single tts_pad row that matches the prefill.
* ui: read model modalities from the router model list
The router advertises input modalities for every model, loaded or not.
Reading them at list build time lets the UI accept image and audio
uploads for a model selected through ?model=, which has no /props yet.
* enum
* ggml: add ggml_build_forward_order
ggml_build_forward_expand marks the tensor and all its ancestors for
compute, so using it as a pure ordering hint (keeping q, k and v
together) defeats ggml_build_forward_select: the unselected branch is
forced to run with inputs that were never uploaded. In the mtmd audio
graph this makes GEN_WAV calls execute the GEN_CODE branch with a
stale inp_code0, hitting the get_rows bound assert on CPU.
Add ggml_build_forward_order, which inserts nodes without the compute
flag; the flag is restored when the branch is actually selected.
Switch the q/k/v hints in clip_graph::build_attn to it.
* nit: reduce comments (AGENTS.md)
* server: don't walk Windows junctions in file_glob_search
std::filesystem reports a junction as a plain directory, so the symlink
guard misses it and a junction pointing back at an ancestor is walked
until the path length gives out
read the reparse tag and treat a symlink and a mount point as links,
leaving any other reparse point walkable so cloud placeholders and dedup
stubs still get searched
look junk directory names up case insensitively on Windows, where NTFS
makes Build the same directory as build
test that a junk directory stays selectable while its contents stay out
of search results
* server: report a directory the walk could not read
a directory that fails to open or to iterate was skipped in silence, so
a caller got a listing that looked complete while a whole subtree was
missing: a path over the platform limit, a volume going away, a name the
filesystem rejects
skip_permission_denied never reaches this path, so an error here is an
incomplete answer rather than a deliberate omission, and it now sets the
truncated flag
* server: simplify the file_glob_search listing plumbing
return a small result struct instead of two out params and a caller path
that only fed an error string, taking list_entries from six parameters
down to three
scope the error code to the directory being read, act on the status code
the entry lookups already returned, and treat an unreadable link state as
a link so the walk never descends on a guess
check the deadline when a directory is popped, not only per entry, so a
tree of empty directories cannot outlive the budget
read the path parameter once, and reject an invalid limit the way an
invalid type is already rejected, instead of silently falling back
normalize the resolved path, so a "." or ".." a caller typed reaches
neither git nor the client, and return the generic path form with '/'
separators on every platform, so the base sent to clients no longer needs
a local fixup
* ui: expire cached picker searches
the cache grew for the lifetime of the component: entries went stale
after the TTL but were never removed, so every distinct query typed in a
session stayed in memory
drop expired entries when a new result is stored
* server: address review from @ngxson
trim comments to one line each, and drop two that restate the code
rename junk_lookup_name to get_effective_name, and move it and the link
check to private static members next to junk_dir_names
merge the Windows and Linux link checks into one is_link, so symlinks are
checked everywhere and junctions only add to it on Windows
* server: convert tool paths as UTF-8 on Windows
a narrow path uses the active code page there, so a file name came back
mangled and a path with an accent could not be opened at all
convert explicitly at every crossing between a std::string, which always
carries UTF-8 here, and fs::path
read the home directory through the wide environment, since the narrow
one returns the profile path in the active code page too
the walker no longer normalizes separators by hand, since paths now come
back in generic form
* server: fold the platform branch inside console_output_to_utf8
match the shape of the other helpers, one definition with the #if inside,
instead of two definitions wrapped in #if and #else
inline the single caller helper and trim the comment
* mtmd: support multi-row batching for deepseek-ocr
* mtmd: weave deepseek-ocr rows in one shot instead of per row (#26615)
---------
Co-authored-by: Saba Fallah <sabafallah@gmail.com>
a child process writes in the OEM code page, which is not UTF-8 on a
western Windows install, so accented output reaches the JSON layer as
invalid bytes and gets replaced there, silently losing the characters
run() spawns without a console, so the child never inherits the console
code page and GetOEMCP is the one that applies
decode with MB_ERR_INVALID_CHARS so a wrong code page returns the text
untouched instead of emitting replacement characters, and pass text that
already decodes as UTF-8 through so a child emitting UTF-8 is never
decoded twice
the check drops an incomplete trailing sequence before validating, since
a streamed chunk can end in the middle of a multi-byte character
* Resolve -1 to 1024 instead of ctx-len for samplers
Because of backend-sampling we initialize samplers before the complete
llama_context is there. Therefore, we cannot infer the resolved context
length yet at the time we construct the samplers.
* Shared default of 64 for history-based samplers, remove context_size
* server : extend file_glob_search for UI pickers
* ui : add per-conversation working directory with picker
* ui : add path navigation and search scope to cwd picker
Treat path-like queries (starting with / or ~) as directory navigation
instead of glob-matching the whole query: search the parent for the last
segment, and descend into an exactly-typed directory by listing its
children. Show the effective search scope in the footer and auto-search
on open so the current directory and its siblings appear immediately.
Assisted-by: Claude
* db : persist per-call tool cwd on tool result messages
* ui : abbreviate tool paths under home with a tilde
* ui : show the per-call cwd on exec shell rows
* ui : clarify the synthetic cwd message for the model
* ui : reuse the trailing cwd row on a repeated pick
* ui : don't jump when a cwd row is injected mid-chat
* chore: Formatting
* refactor: Cleanup comments
* ui : unify working directory naming and add a synthetic-message flag
* ui : render synthetic cwd rows without a scroll jump
* ui : decouple the working directory picker into utils and sub-components
* ui : add get_info tool call block
* chore: Formatting
* refactor: Cleanup
* refactor: Cleanup
* refactor: Cleanup
* fix: UI
* server : harden file_glob_search listing (kind enum, timeout, symlink guard, absolute base)
* ui : use persisted isSynthetic flag for cwd rows, drop legacy formats
* ui : cache picker search, fail visibly on native resolve
* ui : escape glob metacharacters in picker search glob
* ui : simplify auto-scroll pin
* chore: Format
* fix: Use `SvelteMap`
* refactor: Post-review fixes
* ui: accept Windows roots in the working directory picker
recognize a drive root (C:) and a UNC share (//host/share) as path
navigation, alongside the POSIX root and ~, so a query like D:\repos
lists that directory instead of glob-matching it under the home dir
split below the root, so a bare drive resolves to its root rather than
to a drive-relative prefix
rewrite backslashes into forward slashes only when the query carries a
Windows root, since a backslash is a legal POSIX filename character
paths keep travelling with forward slashes, which is what the server
returns and what Windows accepts
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* server: add get_info tool
* fix --rpc in docs
* server: harden get_info probe result handling
Report the OS as unknown when the probe process fails to spawn or times
out, so the diagnostic text from run() is never returned as an OS name.
Strip the probe output on both ends, which also drops the blank line
that ver prints before the version on Windows. Name the output and
timeout limits, and report an unreadable working directory as unknown
instead of an empty string.
* server: simplify get_info result handling
Drop the named limits and the working directory error branch, keeping
the probe result handling to a single expression.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* sampling: enhance penalty handling in common_sampler_init
- Set default value for penalty_last_n based on model context if not specified.
- Ensure penalty_last_n and n_prev are non-negative.
- Update llama_sampler_penalties structure to inherit from llama_sampler_backend and add backend input handling for penalties.
- Implement backend initialization and application logic for penalties, including frequency and presence adjustments.
* tests: add backend penalties sampling tests and utility functions
- Introduced `accept_prompt` and `unique_prompt_tokens` functions to handle prompt acceptance and token uniqueness.
- Implemented `compare_penalties_logits` to compare logits from backend and CPU samplers with penalties.
- Added `test_backend_penalties_sampling` to validate backend penalties with various configurations.
- Enhanced the test suite for better coverage of penalty handling in sampling.
* sampling: add support for top-k penalties in backend sampling
* sampling: add fix to ensure stable numerical results. Preserve masked logits as -Inf and no longer generate NaN.
* sampling: enhance penalty comparison tests with masking penalties logic
* add comments on padding
* sampling: add comments on modifications
* add the unit test to cover masked-out token as -INF
* validate repeat penalty to ensure it is finite and greater than 0; add tests for invalid values
* refactor: test functions to share logic and be less verbose
* add test to cover case where previously penalized token is not part of candidates
* remove comments
* remove redundant penalty_last_n initialization and validation in common_sampler_init
* add support for penalties in sampler chain with configurable positions
* add validation for penalty parameters and enhance tests for non-finite values
* add context parameter to common_sampler_init and set default for penalty_last_n
* add llama_n_ctx parameter to common_sampler_init for improved sampler initialization
* replace penalty_last_n x n_candidates comparison matrix with a vocabulary-sized count tensor
* add tests for backend penalties sampling without filler entries , token_count.size() == n_active == n_max == 64
* add test for backend penalties sampling after top-p with large history window
* remove as unused
* add is_disabled method, tensor logits reshape, add rest review suggestions
* clarify comment
* cli : persist reasoning_content in chat history
llama-cli collected reasoning from the stream for display but only
stored assistant content in messages, so --reasoning-preserve could
not re-inject prior thoughts on later turns.
* fix: single-flight conversations store init
* refactor: remove unused legacy-migration util
* fix: make createSystemMessage transactional
* fix: delete message branches cascading on edit/regenerate
* fix: stop stamping lastModified on conversation metadata updates
* fix: count cascaded forks in bulk delete toast, bulkify deleteAll
* refactor: drop redundant conversation list respreads
* refactor: create conversation in a single write
* fix: use table constant in toggleConversationPin
* fix: keep the system message placeholder out of the edit form
* fix: keep focus in the system message editor after opening it
* fix: focus the main chat form after submitting a system message
* fix: update timestamp of the correct conversation on stream completion
Adds trace logging in server-context.cpp for slot similarity checking
during prompt cache slot selection, including skip reasons and similarity
calculation details.
Assisted-by: llama.cpp:Qwen3.6-27B
* mtmd : add Nemotron 3 Nano Omni support (parakeet)
This commit adds support for the subsampling and encoder part of
Nemotron Nemo 3 omni model.
The Parakeet subsampling/encoder were taken from parakeet.cpp which
is currently a pull request against whisper.cpp. I've tried to copy the
code a close as possible to hopefully enable easy patching between the
these two project later.
Refs: https://github.com/ggml-org/whisper.cpp/pull/3735
* mtmd : generate rel pos tensor in graph instead of in conversion [no ci]
This commit removes the generation of the relative positional tensor in
the model conversion script and instead computes it in the encoder
graph. This is only done for the window of positions required for the
current audio sample.
* mtmd : add clip_get_model to clip API [no ci]
This commit adds a function to get access to the clip_model. It also
removes the two functions clip_get_mel_filter_tensor, and
clip_get_window_tensor(const struct clip_ctx * ctx) which can now use
clip_get_model to access the model tensors that it needs.
* mtmd : read mel_filters and window into hparams
* mtmd : use set_input_f32 lambda [no ci]
* mtmd : add better asserts for mel_filters and hann window [no ci]
* mtmd : add missing size_t cast
* mtmd : change type of pad to size_t
* mtmd : zero initialize samples_padded
* mtmd : remove unsued ctx member from parakeet preprocessor
* mtmd : make log_mel_spectrogram_parakeet_worker_thread private static
* mtmd : sync/update parakeeet impl with latest whisper.cpp
This commit updates the parakeet code in mtmd to reflect the latest
updates to parakeet.cpp in whisper.cpp.
A follow up commit will address the currently hardcoded dw_pad and see
if we can add n_conv_kernel as a model metadata field.
* mtmd : add audio_conv_kernel_size to model conversion
This commit updates the model conversion to read the conv_kernel_size
field from the sound_config section of the models config.json file.
It then uses this field instead of the hardcoded values in parakeet.cpp.
* mtmd : cleanup [no ci]
* conversion : call super().filter_tensors [no ci]
* do not discard result of super filter_tensors
* mtmd : use build_mm instead of ggml_mul_mat
* mtmd : use build_ffn
* mtmd : move and reuse get_vector lambda
* mtmd : use build_inp_raw for parakeet
* mtmd : throw exception in get_scalar instead of assert
* mtmd : fix std::min call
* mtmt : use .c_str in throw clause in get_vector
* mtmd : check for F32 type and non-empty tensor in get_vector
The get_vector lambda is used by get_scalar but also standalone to read
in the mel_filters and the window data. Therefor we are not checking
for 1D tensors but allowing multiple dimensions. We do have a check in
get_scalar to verify the size of the vector.
* mtmd : replace hardcoded 1101 for n_tokens_real
* mtmd : assert subsampling_factor is 8
This commit adds an assert of the parakeet subsampling factor to check
that it is 8.
The motivation for this is that this model currently has three
convolutions with a stride of 2. If the underlying model updates the
subsampling factor these convolution operations will need to be updated
and this will produce and error if this occurs.
* mtmd : remove unused ggml_tensors attn_pos_w and mm_norm_w
* mtmd : remove single thread path
This commit removes the single thread path which was a left over from
the original parakeet.cpp where n_threads is configurable.
* fix some security issues
---------
Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* spec: add DSpark speculative decoding
DSpark (DeepSpec, 2026) on top of the merged DFlash drafter. It reuses the
DFlash encoder/decoder graph, target feature extraction and KV-cache injection,
and the verify/accept path unchanged; the draft model is a new "dspark" arch
adding a low-rank Markov head (markov_w1/w2) and an optional (unused here)
confidence head. No new public APIs.
The proposal is the only change: the block is anchor-first (position 0 already
predicts the first draft) and the decoder graph applies a semi-autoregressive,
previous-token conditioned logit bias in-graph, chained per block position:
logits'(i) = logits(i) + markov_w2 . markov_w1[prev(i)]
prev(0) = the block's anchor token, prev(i>0) = argmax(logits'(i-1))
vectorized across all blocks in the batch; the anchors are fed through a
dedicated graph input (token 0 of every block). Greedy stays lossless
(verify unchanged, same as DFlash).
- new arch "dspark" (llama_model_dspark : llama_model_dflash, reuses the graph,
loads the markov/confidence tensors; shares the target's embed/lm_head).
- Qwen3DSparkModel converter.
- new spec type "draft-dspark" (common_speculative_impl_draft_dspark :
common_speculative_impl_draft_dflash, overrides draft() only: submits whole
anchor-first blocks and greedily reads back the biased logits).
* spec: read draft block size in the dflash impl
* docs: add DSpark section to speculative.md
* spec: keep dspark block size read in the dspark impl
* dspark : add TODOs for incomplete parts
- confidence head is loaded but not used yet
- confidence-scheduled prefix pruning is not implemented
- the in-graph Markov chain is greedy-only
- only Qwen3 backbones are supported for now (also noted in docs)
* spec: fold DSpark into the DFlash arch
Address review: drop LLM_ARCH_DSPARK and the dspark.block_size /
markov_rank GGUF keys. A DSpark draft now converts to a DFlash GGUF;
the Markov head tensors are detected by presence (like eagle3 d2t),
block_size is read from the existing dflash.block_size key, and the
block anchors are taken as a strided view of the decoder's token
input instead of a separate graph input.
* spec: add confidence-based draft pruning for DSpark
The DSpark confidence head predicts per-position acceptance of the
drafted block. --spec-draft-conf-min truncates the block at the first
position below the threshold (default 0 = disabled).
* fold the dspark impl into dflash, selected by spec type
* address review comments
* dspark: clean up and improve naming
* update readme
* remove trailing whitespace
* dflash: draft full n_max blocks, defer dp.n_max to the central truncation
The DSpark markov head views the draft batch as a uniform [n_seqs x block]
grid, but the per-seq dp.n_max clamp could produce blocks of different
sizes, silently corrupting the strided views and the resulting logits.
Drop the clamp and always draft the full n_max block for every sequence:
dp.n_max is already enforced by the central truncation in
common_speculative_draft(), the same way eagle3 handles it.
Co-authored-by: Zaire404 <3147879462@qq.com>
* dflash: assert the markov head block-uniformity invariant, require the conf head
With the draft batch always submitting equal-size n_max blocks, a
non-divisible token count can only mean the batch was split across
ubatches or a caller broke the layout - fail loudly instead of silently
dropping the markov bias. The block_drafts > block_size early return
stays: worst-case graph reserve passes legitimately build with
n_seq_tokens > block_size.
Also make conf_proj required when the markov head is present: the
confidence head is part of the DSpark checkpoint format, and a missing
head would otherwise leave --spec-draft-conf-min silently reading stale
embeddings instead of confidences.
Co-authored-by: Zaire404 <3147879462@qq.com>
* dspark: fold conf_min into p_min
p_min and conf_min express the same thing - the minimum predicted
survival probability for a drafted position - differing only in how the
estimate is obtained: token probability for regular drafters, the
trained confidence head for DSpark. The DSpark readback never used
p_min, so reuse it for the confidence threshold and drop the separate
--spec-draft-conf-min flag. Both defaulted to 0 (disabled), so behavior
is unchanged.
Co-authored-by: Zaire404 <3147879462@qq.com>
* dflash: note the confidence broadcast workaround
Requested in review: the ggml_repeat only adapts the [1, n_tok]
confidences to the n_embd-wide embd_nextn transport so that
llama_get_embeddings_nextn can be reused - not a placeholder.
Co-authored-by: Zaire404 <3147879462@qq.com>
* cont : clarify
[no ci]
---------
Co-authored-by: Ruixiang Wang <wangruixiang07@outlook.com>
Co-authored-by: Zaire404 <3147879462@qq.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* server + ui: refactor resumable stream routes to query string conv_id
The conversation id can embed a model name containing slashes
(ggml-org/...) in router mode, which the decoded path splits before the
:conv_id param is captured, so stop and resume never matched the
session. Move the id to the conv_id query string on the public routes
and on the internal router -> child hop, where slashes survive
encoding. Handlers are unchanged since query and path params land in
the same map. Add a regression test with a slashed model name.
* server: move stream route docs to server-stream.h
Address review: ngxson wants the main server.cpp registration code kept
clean and simple, with route-level explanations living in the header.
Move the query string rationale and the lookup ownership note next to
the handler declarations in server-stream.h, and shorten the wiring
comment to a pointer.
* server: cancel a pending request when its stream is stopped during model load
The conversation was registered in the conv map only after the blocking
autoload wait, so a stop issued while the model loaded found nothing to
cancel and the request went on to generate an orphan once the load
ended. Register the conversation before the wait and give the entry a
ticket: a stop erases the entry, and the parked request checks its
ticket after the wait and aborts with 400 instead of starting. A newer
request on the same conversation replaces the entry, so only the
stopped request is cancelled. Add a regression test that stops during
the load window.
* server + ui: resume a stream after a page reload during model load
A pending request died with the client socket when the page was
reloaded while its model was loading, so no session ever existed and
the conversation had nothing to recover. A session request that waited
for a load now detaches from the client socket and reaches the child
regardless, the session buffer receives the generation, and the resume
route answers 503 while the owner is loading so the client retries
instead of dropping its state. The WebUI persists the pending stream at
send time, quietly polls on 503, and attaches once the session exists.
Add a regression test that drops the client during the load window.
* ui: show the model load progress again after a page refresh
The resume wait was invisible, so a conversation refreshed while its
model was loading showed nothing until the first byte. On a 503 from
the resume probe, mark the conversation as loading again so the
assistant row persisted at send time renders the processing info, and
target the model frozen in the persisted stream state for the
progress, since the row has no model yet and the dropdown may not be
restored.
* fix CI
* fix CI bis
