LLM-Inference/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-08-12 22:29:39 +04:00
Code Issues Packages Projects Releases Wiki Activity

webui update (#1003)

Browse Source 
webui: add system message in export conversation, support upload conversation with system message
Webui: show upload only when in new conversation
Webui: Add model name
webui: increase height of chat message window when clicking editing
Webui: autoclose settings dialog dropdown and maximze screen width when zoom in
webui: fix date issues and add more dates
webui: change error to toast.error.
server: add n_past and slot_id in props_simple
webui: add cache tokens, context and prompt speed in chat
webui: modernize ui
webui: change welcome message
webui: change speed display
webui: change run python icon
webui: add config to use server defaults for sampler
webui: put speed on left and context on right

webui: recognize AsciiDoc files as valid text files (#16850)

* webui: recognize AsciiDoc files as valid text files

* webui: add an updated static webui build

* webui: add the updated dependency list

* webui: re-add an updated static webui build

Add a setting to display message generation statistics (#16901)

* feat: Add setting to display message generation statistics

* chore: build static webui output

webui: add HTML/JS preview support to MarkdownContent with sandboxed iframe (#16757)

* webui: add HTML/JS preview support to MarkdownContent with sandboxed iframe dialog

Extended MarkdownContent to flag previewable code languages,
add a preview button alongside copy controls, manage preview
dialog state, and share styling for the new button group

Introduced CodePreviewDialog.svelte, a sandboxed iframe modal
for rendering HTML/JS previews with consistent dialog controls

* webui: fullscreen HTML preview dialog using bits-ui

* Update tools/server/webui/src/lib/components/app/misc/CodePreviewDialog.svelte

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

* Update tools/server/webui/src/lib/components/app/misc/MarkdownContent.svelte

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

* webui: pedantic style tweak for CodePreviewDialog close button

* webui: remove overengineered preview language logic

* chore: update webui static build

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

webui: auto-refresh /props on inference start to resync model metadata (#16784)

* webui: auto-refresh /props on inference start to resync model metadata

- Add no-cache headers to /props and /slots
- Throttle slot checks to 30s
- Prevent concurrent fetches with promise guard
- Trigger refresh from chat streaming for legacy and ModelSelector
- Show dynamic serverWarning when using cached data

* fix: restore proper legacy behavior in webui by using unified /props refresh

Updated assistant message bubbles to show each message's stored model when available,
falling back to the current server model only when the per-message value is missing

When the model selector is disabled, now fetches /props and prioritizes that model name
over chunk metadata, then persists it with the streamed message so legacy mode properly
reflects the backend configuration

* fix: detect first valid SSE chunk and refresh server props once

* fix: removed the slots availability throttle constant and state

* webui: purge ai-generated cruft

* chore: update webui static build

feat(webui): improve LaTeX rendering with currency detection (#16508)

* webui : Revised LaTeX formula recognition

* webui : Further examples containg amounts

* webui : vitest for maskInlineLaTeX

* webui: Moved preprocessLaTeX to lib/utils

* webui: LaTeX in table-cells

* chore: update webui build output (use theirs)

* webui: backslash in LaTeX-preprocessing

* chore: update webui build output

* webui: look-behind backslash-check

* chore: update webui build output

* Apply suggestions from code review

Code maintenance (variable names, code formatting, string handling)

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

* webui: Moved constants to lib/constants.

* webui: package woff2 inside base64 data

* webui: LaTeX-line-break in display formula

* chore: update webui build output

* webui: Bugfix (font embedding)

* webui: Bugfix (font embedding)

* webui: vite embeds assets

* webui: don't suppress 404 (fonts)

* refactor: KaTeX integration with SCSS

Moves KaTeX styling to SCSS for better customization and font embedding.

This change includes:
- Adding `sass` as a dev dependency.
- Introducing a custom SCSS file to override KaTeX variables and disable TTF/WOFF fonts, relying solely on WOFF2 for embedding.
- Adjusting the Vite configuration to resolve `katex-fonts` alias and inject SCSS variables.

* fix: LaTeX processing within blockquotes

* webui: update webui build output

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

server : add props.model_alias (#16943)

* server : add props.model_alias

webui: fix keyboard shortcuts for new chat & edit chat title (#17007)

Better UX for handling multiple attachments in WebUI (#17246)

webui: add OAI-Compat Harmony tool-call streaming visualization and persistence in chat UI (#16618)

* webui: add OAI-Compat Harmony tool-call live streaming visualization and persistence in chat UI

- Purely visual and diagnostic change, no effect on model context, prompt
  construction, or inference behavior

- Captured assistant tool call payloads during streaming and non-streaming
  completions, and persisted them in chat state and storage for downstream use

- Exposed parsed tool call labels beneath the assistant's model info line
  with graceful fallback when parsing fails

- Added tool call badges beneath assistant responses that expose JSON tooltips
  and copy their payloads when clicked, matching the existing model badge styling

- Added a user-facing setting to toggle tool call visibility to the Developer
  settings section directly under the model selector option

* webui: remove scroll listener causing unnecessary layout updates (model selector)

* Update tools/server/webui/src/lib/components/app/chat/ChatMessages/ChatMessageAssistant.svelte

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

* Update tools/server/webui/src/lib/components/app/chat/ChatMessages/ChatMessageAssistant.svelte

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

* chore: npm run format & update webui build output

* chore: update webui build output

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>

webui: Fix clickability around chat processing statistics UI (#17278)

* fix: Better pointer events handling in chat processing info elements

* chore: update webui build output

Fix merge error

webui: Add a "Continue" Action for Assistant Message (#16971)

* feat: Add "Continue" action for assistant messages

* feat: Continuation logic & prompt improvements

* chore: update webui build output

* feat: Improve logic for continuing the assistant message

* chore: update webui build output

* chore: Linting

* chore: update webui build output

* fix: Remove synthetic prompt logic, use the prefill feature by sending the conversation payload ending with assistant message

* chore: update webui build output

* feat: Enable "Continue" button based on config & non-reasoning model type

* chore: update webui build output

* chore: Update packages with `npm audit fix`

* fix: Remove redundant error

* chore: update webui build output

* chore: Update `.gitignore`

* fix: Add missing change

* feat: Add auto-resizing for Edit Assistant/User Message textareas

* chore: update webui build output

Improved file naming & structure for UI components (#17405)

* refactor: Component iles naming & structure

* chore: update webui build output

* refactor: Dialog titles + components namig

* chore: update webui build output

* refactor: Imports

* chore: update webui build output

webui: hide border of button

webui: update

webui: update

webui: update

add vision

webui: minor settings reorganization and add disable autoscroll option (#17452)

* webui: added a dedicated 'Display' settings section that groups visualization options

* webui: added a Display setting to toggle automatic chat scrolling

* chore: update webui build output

Co-authored-by: firecoperana <firecoperana>
This commit is contained in:
firecoperana
2025-11-24 07:03:45 +01:00
committed by GitHub
co-authored by firecoperana <firecoperana>
parent 920f424929
commit 07d08e15ad
91 changed files with 13516 additions and 11848 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/server/public/index.html.gz
BIN
View File
Binary file not shown.
examples/server/public_llamacpp/index_llamacpp.html
+98 -1039
View File
File diff suppressed because one or more lines are too long
examples/server/public_llamacpp/index_llamacpp.html.gz
BIN
View File
Binary file not shown.
examples/server/server.cpp
+45
View File
@@ -108,6 +108,8 @@ struct result_timings {
double predicted_ms;
double predicted_per_token_ms;
double predicted_per_second;
int32_t n_ctx = 0;
int32_t n_past = 0;
// Optional speculative metrics - only included when > 0
int32_t draft_n = 0;
@@ -124,6 +126,9 @@ struct result_timings {
{"predicted_ms", predicted_ms},
{"predicted_per_token_ms", predicted_per_token_ms},
{"predicted_per_second", predicted_per_second},
{"n_ctx", n_ctx},
{"n_past", n_past},
};
if (draft_n > 0) {
@@ -585,6 +590,13 @@ struct slot_params {
};
inline std::string get_model_name(std::string path)
{
std::string filename = path.substr(path.find_last_of("/\\") + 1);
return filename;
};
struct server_prompt_checkpoint {
llama_pos pos_min;
llama_pos pos_max;
@@ -988,6 +1000,9 @@ struct server_slot {
{"predicted_ms", t_token_generation},
{"predicted_per_token_ms", t_token_generation / n_decoded},
{"predicted_per_second", 1e3 / t_token_generation * n_decoded},
{"n_ctx", n_ctx},
{"n_past", n_past},
};
}
@@ -1003,6 +1018,10 @@ struct server_slot {
timings.predicted_per_token_ms = t_token_generation / n_decoded;
timings.predicted_per_second = 1e3 / t_token_generation * n_decoded;
timings.n_ctx = n_ctx;
timings.n_past = n_past;
// Add speculative metrics
if (n_draft_total > 0) {
timings.draft_n = n_draft_total;
@@ -4651,8 +4670,11 @@ int main(int argc, char ** argv) {
}
json data = {
{ "system_prompt", ctx_server.system_prompt.c_str() },
{ "model_alias", ctx_server.params.model_alias },
{ "model_path", ctx_server.params.model},
{ "default_generation_settings", ctx_server.default_generation_settings_for_props },
{ "total_slots", ctx_server.params.n_parallel },
{ "model_name", get_model_name(ctx_server.params.model)},
{ "chat_template", common_chat_templates_source(ctx_server.chat_templates.get()) },
{ "bos_token", llama_token_to_piece(ctx_server.ctx, llama_token_bos(ctx_server.model), /* special= */ true)},
{ "eos_token", llama_token_to_piece(ctx_server.ctx, llama_token_eos(ctx_server.model), /* special= */ true)},
@@ -4673,6 +4695,28 @@ int main(int argc, char ** argv) {
res.set_content(data.dump(), "application/json; charset=utf-8");
};
const auto handle_props_simple = [&ctx_server](const httplib::Request& req, httplib::Response& res) {
res.set_header("Access-Control-Allow-Origin", req.get_header_value("Origin"));
int n_past = 0;
int slot_id = 0;
for (server_slot& slot : ctx_server.slots) {
if (slot.n_past > n_past) {
n_past = slot.n_past;
slot_id = slot.id;
}
}
json data = {
{ "model_name", get_model_name(ctx_server.params.model)},
{ "model_path", ctx_server.params.model },
{ "modalities", json {
{"vision", ctx_server.oai_parser_opt.allow_image},
{"audio", ctx_server.oai_parser_opt.allow_audio},
} },
{ "n_ctx", ctx_server.n_ctx }
};
res.set_content(data.dump(), "application/json; charset=utf-8");
};
// handle completion-like requests (completion, chat, infill)
// we can optionally provide a custom format for partial results and final results
@@ -5411,6 +5455,7 @@ int main(int argc, char ** argv) {
svr->Get ("/health", handle_health);
svr->Get ("/metrics", handle_metrics);
svr->Get ("/props", handle_props);
svr->Get("/v1/props", handle_props_simple);
svr->Get ("/v1/models", handle_models);
svr->Post("/completion", handle_completions); // legacy
svr->Post("/completions", handle_completions); // legacy
examples/server/webui/dist/demo-conversation.json
Vendored
+3 -1
View File
@@ -16,7 +16,9 @@
"prompt_n": 1,
"prompt_ms": 28.923,
"predicted_n": 25,
"predicted_ms": 573.016
"predicted_ms": 573.016,
"n_ctx": 4048,
"n_past": 1024
}
},
{
examples/server/webui/dist/index.html
Vendored
+124 -123
View File
File diff suppressed because one or more lines are too long
examples/server/webui/index.html
+2 -1
View File
@@ -7,7 +7,8 @@
content="width=device-width, initial-scale=1, maximum-scale=1"
/>
<meta name="color-scheme" content="light dark" />
<title>🦙 ik_llama.cpp - chat</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>🦙</text></svg>">
<title>ik_llama.cpp</title>
</head>
<body>
<div id="root"></div>
examples/server/webui/package-lock.json
Generated
+90
View File
@@ -19,6 +19,7 @@
"dexie-export-import": "^4.0.11",
"highlight.js": "^11.10.0",
"katex": "^0.16.15",
"luxon": "^3.7.1",
"pdfjs-dist": "^5.2.133",
"postcss": "^8.4.49",
"react": "^18.3.1",
@@ -27,6 +28,7 @@
"react-hot-toast": "^2.5.2",
"react-markdown": "^9.0.3",
"react-router": "^7.1.5",
"react-textarea-autosize": "^8.5.9",
"rehype-highlight": "^7.0.2",
"rehype-katex": "^7.0.1",
"remark-breaks": "^4.0.0",
@@ -38,6 +40,7 @@
},
"devDependencies": {
"@eslint/js": "^9.17.0",
"@types/luxon": "^3.7.1",
"@types/markdown-it": "^14.1.2",
"@types/node": "^22.13.1",
"@types/react": "^18.3.18",
@@ -304,6 +307,15 @@
"@babel/core": "^7.0.0-0"
}
},
"node_modules/@babel/runtime": {
"version": "7.28.4",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.28.4.tgz",
"integrity": "sha512-Q/N6JNWvIvPnLDvjlE1OUBLPQHH6l3CltCEsHIujp45zQUSSh8K+gHnaEX45yAT1nyngnINhvWtzN+Nb9D8RAQ==",
"license": "MIT",
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/template": {
"version": "7.25.9",
"resolved": "https://registry.npmjs.org/@babel/template/-/template-7.25.9.tgz",
@@ -1855,6 +1867,13 @@
"dev": true,
"license": "MIT"
},
"node_modules/@types/luxon": {
"version": "3.7.1",
"resolved": "https://registry.npmjs.org/@types/luxon/-/luxon-3.7.1.tgz",
"integrity": "sha512-H3iskjFIAn5SlJU7OuxUmTEpebK6TKB8rxZShDslBMZJ5u9S//KM1sbdAisiSrqwLQncVjnpi2OK2J51h+4lsg==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/markdown-it": {
"version": "14.1.2",
"resolved": "https://registry.npmjs.org/@types/markdown-it/-/markdown-it-14.1.2.tgz",
@@ -3913,6 +3932,15 @@
"yallist": "^3.0.2"
}
},
"node_modules/luxon": {
"version": "3.7.1",
"resolved": "https://registry.npmjs.org/luxon/-/luxon-3.7.1.tgz",
"integrity": "sha512-RkRWjA926cTvz5rAb1BqyWkKbbjzCGchDUIKMCUvNi17j6f6j8uHGDV82Aqcqtzd+icoYpELmG3ksgGiFNNcNg==",
"license": "MIT",
"engines": {
"node": ">=12"
}
},
"node_modules/markdown-table": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/markdown-table/-/markdown-table-3.0.4.tgz",
@@ -5298,6 +5326,23 @@
}
}
},
"node_modules/react-textarea-autosize": {
"version": "8.5.9",
"resolved": "https://registry.npmjs.org/react-textarea-autosize/-/react-textarea-autosize-8.5.9.tgz",
"integrity": "sha512-U1DGlIQN5AwgjTyOEnI1oCcMuEr1pv1qOtklB2l4nyMGbHzWrI0eFsYK0zos2YWqAolJyG0IWJaqWmWj5ETh0A==",
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.20.13",
"use-composed-ref": "^1.3.0",
"use-latest": "^1.2.1"
},
"engines": {
"node": ">=10"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/rehype-highlight": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/rehype-highlight/-/rehype-highlight-7.0.2.tgz",
@@ -6334,6 +6379,51 @@
"punycode": "^2.1.0"
}
},
"node_modules/use-composed-ref": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/use-composed-ref/-/use-composed-ref-1.4.0.tgz",
"integrity": "sha512-djviaxuOOh7wkj0paeO1Q/4wMZ8Zrnag5H6yBvzN7AKKe8beOaED9SF5/ByLqsku8NP4zQqsvM2u3ew/tJK8/w==",
"license": "MIT",
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
}
}
},
"node_modules/use-isomorphic-layout-effect": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/use-isomorphic-layout-effect/-/use-isomorphic-layout-effect-1.2.1.tgz",
"integrity": "sha512-tpZZ+EX0gaghDAiFR37hj5MgY6ZN55kLiPkJsKxBMZ6GZdOSPJXiOzPM984oPYZ5AnehYx5WQp1+ME8I/P/pRA==",
"license": "MIT",
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
}
}
},
"node_modules/use-latest": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/use-latest/-/use-latest-1.3.0.tgz",
"integrity": "sha512-mhg3xdm9NaM8q+gLT8KryJPnRFOz1/5XPBhmDEVZK1webPzDjrPk7f/mbpeLqTgB9msytYWANxgALOCJKnLvcQ==",
"license": "MIT",
"dependencies": {
"use-isomorphic-layout-effect": "^1.1.1"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
}
}
},
"node_modules/varint": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/varint/-/varint-6.0.0.tgz",
examples/server/webui/package.json
+3
View File
@@ -22,6 +22,7 @@
"dexie-export-import": "^4.0.11",
"highlight.js": "^11.10.0",
"katex": "^0.16.15",
"luxon": "^3.7.1",
"pdfjs-dist": "^5.2.133",
"postcss": "^8.4.49",
"react": "^18.3.1",
@@ -30,6 +31,7 @@
"react-hot-toast": "^2.5.2",
"react-markdown": "^9.0.3",
"react-router": "^7.1.5",
"react-textarea-autosize": "^8.5.9",
"rehype-highlight": "^7.0.2",
"rehype-katex": "^7.0.1",
"remark-breaks": "^4.0.0",
@@ -41,6 +43,7 @@
},
"devDependencies": {
"@eslint/js": "^9.17.0",
"@types/luxon": "^3.7.1",
"@types/markdown-it": "^14.1.2",
"@types/node": "^22.13.1",
"@types/react": "^18.3.18",
examples/server/webui/public/demo-conversation.json
+3 -1
View File
@@ -16,7 +16,9 @@
"prompt_n": 1,
"prompt_ms": 28.923,
"predicted_n": 25,
"predicted_ms": 573.016
"predicted_ms": 573.016,
"n_ctx": 4048,
"n_past": 1024
}
},
{
examples/server/webui/src/App.tsx
+6 -3
View File
@@ -4,13 +4,14 @@ import Sidebar from './components/Sidebar';
import { AppContextProvider, useAppContext } from './utils/app.context';
import ChatScreen from './components/ChatScreen';
import SettingDialog from './components/SettingDialog';
import { Toaster } from 'react-hot-toast';
import { ModalProvider } from './components/ModalProvider';
function App() {
return (
<ModalProvider>
<HashRouter>
<div className="flex flex-row drawer lg:drawer-open">
<div className="flex flex-row drawer lg:drawer-open h-screen">
<AppContextProvider>
<Routes>
<Route element={<AppLayout />}>
@@ -30,19 +31,21 @@ function AppLayout() {
return (
<>
<Sidebar />
<div
<main
className="drawer-content grow flex flex-col h-screen mx-auto px-4 overflow-auto bg-base-100"
id="main-scroll"
>
<Header />
<Outlet />
</div>
</main>
{
<SettingDialog
show={showSettings}
onClose={() => setShowSettings(false)}
/>
}
<Toaster />
</>
);
}
examples/server/webui/src/Config.ts
+5 -3
View File
@@ -13,8 +13,9 @@ export const CONFIG_DEFAULT = {
// Do not use nested objects, keep it single level. Prefix the key if you need to group them.
apiKey: '',
systemMessage: 'You are a helpful assistant.',
showTokensPerSecond: false,
showThoughtInProgress: false,
showTokensPerSecond: false,
showThoughtInProgress: false,
useServerDefaults: false, // don't send defaults
excludeThoughtOnReq: true,
pasteLongTextToFileLen: 2500,
pdfAsImage: false,
@@ -51,7 +52,7 @@ export const CONFIG_INFO: Record<string, string> = {
pasteLongTextToFileLen:
'On pasting long text, it will be converted to a file. You can control the file length by setting the value of this parameter. Value 0 means disable.',
samplers:
'The order at which samplers are applied, in simplified way. Default is "dkypmxt": dry->top_k->typ_p->top_p->min_p->xtc->top_sigma->temperature',
'The order at which samplers are applied, in simplified way. Default is "dkypmxnt": dry->top_k->typ_p->top_p->min_p->xtc->top_sigma->temperature',
temperature:
'Controls the randomness of the generated text by affecting the probability distribution of the output tokens. Higher = more random, lower = more focused.',
dynatemp_range:
@@ -87,6 +88,7 @@ export const CONFIG_INFO: Record<string, string> = {
dry_penalty_last_n:
'DRY sampling reduces repetition in generated text even across long contexts. This parameter sets DRY penalty for the last n tokens.',
max_tokens: 'The maximum number of token per output.',
useServerDefaults: 'When enabled, skip sending WebUI defaults (e.g., temperature) and use the server\'s default values instead.',
custom: '', // custom json-stringified object
};
// config keys having numeric value (i.e. temperature, top_k, top_p, etc)
examples/server/webui/src/components/ChatMessage.tsx
+49 -15
View File
@@ -5,6 +5,7 @@ import { classNames } from '../utils/misc';
import MarkdownDisplay, { CopyButton } from './MarkdownDisplay';
import { ChevronLeftIcon, ChevronRightIcon, ArrowPathIcon, PencilSquareIcon } from '@heroicons/react/24/outline';
import ChatInputExtraContextItem from './ChatInputExtraContextItem';
import TextareaAutosize from 'react-textarea-autosize';
interface SplitMessage {
content: PendingMessage['content'];
@@ -34,7 +35,8 @@ export default function ChatMessage({
isPending?: boolean;
}) {
const { viewingChat, config } = useAppContext();
const [editingContent, setEditingContent] = useState<string | null>(null);
const [editingContent, setEditingContent] = useState<string | null>(null);
const timings = useMemo(
() =>
msg.timings
@@ -50,7 +52,6 @@ export default function ChatMessage({
);
const nextSibling = siblingLeafNodeIds[siblingCurrIdx + 1];
const prevSibling = siblingLeafNodeIds[siblingCurrIdx - 1];
// for reasoning model, we split the message into content and thought
// TODO: implement this as remark/rehype plugin in the future
const { content, thought, isThinking }: SplitMessage = useMemo(() => {
@@ -81,7 +82,8 @@ export default function ChatMessage({
}, [msg]);
if (!viewingChat) return null;
//const model_name = (timings?.model_name ??'')!== '' ? timings?.model_name: viewingChat.conv.model_name;
const model_name = viewingChat.conv.model_name;
return (
<div className="group"
id={id}
@@ -108,12 +110,14 @@ export default function ChatMessage({
{/* textarea for editing message */}
{editingContent !== null && (
<>
<textarea
<TextareaAutosize
dir="auto"
className="textarea textarea-bordered bg-base-100 text-base-content max-w-2xl w-[calc(90vw-8em)] h-24"
className="textarea textarea-bordered bg-base-100 text-base-content max-w-2xl w-[calc(90vw-8em)]"
value={editingContent}
onChange={(e) => setEditingContent(e.target.value)}
></textarea>
onChange={(e: React.ChangeEvent<HTMLTextAreaElement>) => setEditingContent(e.target.value)}
minRows={3}
maxRows={15}
/>
<br />
<button
className="btn btn-ghost mt-2 mr-2"
@@ -186,25 +190,48 @@ export default function ChatMessage({
)}
{/* render timings if enabled */}
{timings && config.showTokensPerSecond && (
<div className="dropdown dropdown-hover dropdown-top mt-2">
<div className="dropdown dropdown-hover dropdown-top ax-w-[900px] w-full mt-4">
<div
tabIndex={0}
role="button"
className="cursor-pointer font-semibold text-sm opacity-60"
>
Speed: {timings.predicted_per_second.toFixed(1)} t/s
<div className="font-bold text-xs">
{timings.n_ctx>0 && (
<div className="flex justify-between items-center">
<span className="whitespace-nowrap">
Token: {timings.predicted_per_second.toFixed(1)} t/s | Prompt: {timings.prompt_per_second.toFixed(1)} t/s
</span>
<span className="hidden lg:block pl-[200px] whitespace-nowrap">
Ctx: {timings.predicted_n+timings.prompt_n} / {timings.n_past} / {timings.n_ctx}
</span>
</div>
)}
{(timings.n_ctx==null || timings.n_ctx <=0) && (
<div>
Token: {timings.predicted_per_second.toFixed(1)} t/s | Prompt: {timings.prompt_per_second.toFixed(1)} t/s
</div>
)}
</div>
</div>
<div className="dropdown-content bg-base-100 z-10 w-64 p-2 shadow mt-4">
<p className="text-xs"><b>{model_name}</b></p>
<p className="text-sm">
<b>Prompt</b>
<br />- Tokens: {timings.prompt_n}
<br />- Time: {timings.prompt_ms} ms
<br />- Speed: {timings.prompt_per_second.toFixed(1)} t/s
<br />- Speed: {timings.prompt_per_second.toFixed(2)} t/s
<br />
<b>Generation</b>
<br />- Tokens: {timings.predicted_n}
<br />- Time: {timings.predicted_ms} ms
<br />- Speed: {timings.predicted_per_second.toFixed(1)} t/s
<br />- Speed: {timings.predicted_per_second.toFixed(2)} t/s
<br />
<b>Context</b>
<br />- n_ctx: {timings.n_ctx}
<br />- n_past: {timings.n_past}
<br />
</p>
</div>
</div>
)}
@@ -214,6 +241,13 @@ export default function ChatMessage({
</div>
{/* actions for each message */}
{msg.content !== null && !config.showTokensPerSecond && (
msg.role === 'assistant' &&(
<div className="badge border-none outline-none btn-mini show-on-hover mr-2">
<p className="text-xs">Model: {model_name}</p>
</div>
)
)}
{msg.content !== null && (
<div
className={classNames({
@@ -249,7 +283,7 @@ export default function ChatMessage({
{/* user message */}
{msg.role === 'user' && (
<button
className="badge btn-mini show-on-hover"
className="badge border-none outline-none btn-mini show-on-hover"
onClick={() => setEditingContent(msg.content)}
disabled={msg.content === null}
>
@@ -261,7 +295,7 @@ export default function ChatMessage({
<>
{!isPending && (
<button
className="badge btn-mini show-on-hover mr-2"
className="badge border-none outline-none btn-mini show-on-hover mr-2"
onClick={() => {
if (msg.content !== null) {
onRegenerateMessage(msg as Message);
@@ -274,7 +308,7 @@ export default function ChatMessage({
)}
{!isPending && (
<button
className="badge btn-mini show-on-hover"
className="badge border-none outline-none btn-mini show-on-hover"
onClick={() => setEditingContent(msg.content)}
disabled={msg.content === null}
>
@@ -284,7 +318,7 @@ export default function ChatMessage({
</>
)}
<CopyButton
className="badge btn-mini show-on-hover mr-2"
className="badge border-none outline-none btn-mini show-on-hover mr-2"
content={msg.content}
/>
</div>
examples/server/webui/src/components/ChatScreen.tsx
+49 -68
View File
@@ -1,12 +1,14 @@
import { ClipboardEvent, useEffect, useMemo, useState } from 'react';
import { useEffect, useMemo, useRef, useState } from 'react';
import toast from 'react-hot-toast';
import { CallbackGeneratedChunk, useAppContext } from '../utils/app.context';
import ChatMessage from './ChatMessage';
import { CanvasType, Message, PendingMessage } from '../utils/types';
import { classNames, cleanCurrentUrl, throttle } from '../utils/misc';
import { classNames, cleanCurrentUrl } from '../utils/misc';
import CanvasPyInterpreter from './CanvasPyInterpreter';
import StorageUtils from '../utils/storage';
import { useVSCodeContext } from '../utils/llama-vscode';
import { useChatTextarea, ChatTextareaApi } from './useChatTextarea.ts';
import { scrollToBottom, useChatScroll } from './useChatScroll.tsx';
import {
ArrowUpIcon,
StopIcon,
@@ -82,23 +84,6 @@ function getListMessageDisplay(
return res;
}
const scrollToBottom = throttle(
(requiresNearBottom: boolean, delay: number = 80) => {
const mainScrollElem = document.getElementById('main-scroll');
if (!mainScrollElem) return;
const spaceToBottom =
mainScrollElem.scrollHeight -
mainScrollElem.scrollTop -
mainScrollElem.clientHeight;
if (!requiresNearBottom || spaceToBottom < 50) {
setTimeout(
() => mainScrollElem.scrollTo({ top: mainScrollElem.scrollHeight }),
delay
);
}
},
80
);
export default function ChatScreen() {
const {
@@ -116,9 +101,10 @@ export default function ChatScreen() {
const extraContext = useChatExtraContext();
useVSCodeContext(textarea, extraContext);
//const { extraContext, clearExtraContext } = useVSCodeContext(textarea);
const msgListRef = useRef<HTMLDivElement>(null);
useChatScroll(msgListRef);
// TODO: improve this when we have "upload file" feature
// keep track of leaf node for rendering
const [currNodeId, setCurrNodeId] = useState<number>(-1);
const messages: MessageDisplay[] = useMemo(() => {
@@ -141,32 +127,44 @@ export default function ChatScreen() {
if (currLeafNodeId) {
setCurrNodeId(currLeafNodeId);
}
scrollToBottom(true);
//useChatScroll will handle the auto scroll
};
const sendNewMessage = async () => {
const lastInpMsg = textarea.value();
if (lastInpMsg.trim().length === 0 || isGenerating(currConvId ?? ''))
const lastInpMsg = textarea.value();
try {
const generate = isGenerating(currConvId ?? '');
console.log('IsGenerating', generate);
if (lastInpMsg.trim().length === 0 || generate)
return;
textarea.setValue('');
scrollToBottom(false);
setCurrNodeId(-1);
// get the last message node
const lastMsgNodeId = messages.at(-1)?.msg.id ?? null;
if (
!(await sendMessage(
const successSendMsg=await sendMessage(
currConvId,
lastMsgNodeId,
lastInpMsg,
extraContext.items,
onChunk
))
) {
);
console.log('Send msg success:', successSendMsg);
if (!successSendMsg)
{
// restore the input message if failed
textarea.setValue(lastInpMsg);
}
// OK
extraContext.clearItems();
}
catch (err) {
//console.error('Error sending message:', error);
toast.error(err instanceof Error ? err.message : String(err));
textarea.setValue(lastInpMsg); // Restore input on error
}
};
const handleEditMessage = async (msg: Message, content: string) => {
@@ -182,6 +180,7 @@ export default function ChatScreen() {
);
setCurrNodeId(-1);
scrollToBottom(false);
};
const handleRegenerateMessage = async (msg: Message) => {
@@ -197,9 +196,10 @@ export default function ChatScreen() {
);
setCurrNodeId(-1);
scrollToBottom(false);
};
const handleContinueMessage = async (msg: Message, content: string) => {
const handleContinueMessage = async (msg: Message, content: string) => {
if (!viewingChat || !continueMessageAndGenerate) return;
setCurrNodeId(msg.id);
scrollToBottom(false);
@@ -211,6 +211,7 @@ export default function ChatScreen() {
);
setCurrNodeId(-1);
scrollToBottom(false);
};
const hasCanvas = !!canvasData;
@@ -251,16 +252,29 @@ export default function ChatScreen() {
>
<div
className={classNames({
'flex flex-col w-full max-w-[900px] mx-auto': true,
'flex flex-col w-[75vw] mx-auto': true,
'hidden lg:flex': hasCanvas, // adapted for mobile
flex: !hasCanvas,
})}
>
<div className="flex items-center justify-center">
{viewingChat?.conv.model_name}
</div>
{/* chat messages */}
<div id="messages-list" className="grow">
<div id="messages-list" className="grow" ref={msgListRef}>
<div className="mt-auto flex justify-center">
{/* placeholder to shift the message to the bottom */}
{viewingChat ? '' : 'Send a message to start'}
<div>
{viewingChat ? '' : ''}
</div>
{viewingChat==null && (
<div className="w-full max-w-2xl px-4">
<div className="mb-8 text-center" >
<p className="text-1xl text-muted-foreground">How can I help you today?</p>
</div>
</div>
)}
</div>
{[...messages, ...pendingMsgDisplay].map((msgDisplay) => {
const actualMsgObject = msgDisplay.msg;
@@ -292,8 +306,7 @@ export default function ChatScreen() {
);
})}
</div>
{/* chat input */}
{/* chat input */}
<ChatInput
textarea={textarea}
extraContext={extraContext}
@@ -301,7 +314,7 @@ export default function ChatScreen() {
onStop={() => stopGenerating(currConvId ?? '')}
isGenerating={isGenerating(currConvId ?? '')}
/>
</div>
</div>
<div className="w-full sticky top-[7em] h-[calc(100vh-9em)]">
{canvasData?.type === CanvasType.PY_INTERPRETER && (
<CanvasPyInterpreter />
@@ -311,38 +324,6 @@ export default function ChatScreen() {
);
}
// function ServerInfo() {
// const { serverProps } = useAppContext();
// const modalities = [];
// if (serverProps?.modalities?.audio) {
// modalities.push('audio');
// }
// if (serverProps?.modalities?.vision) {
// modalities.push('vision');
// }
// return (
// <div
// className="card card-sm shadow-sm border-1 border-base-content/20 text-base-content/70 mb-6"
// tabIndex={0}
// aria-description="Server information"
// >
// <div className="card-body">
// <b>Server Info</b>
// <p>
// <b>Model</b>: {serverProps?.model_path?.split(/(\\|\/)/).pop()}
// <br />
// {modalities.length > 0 ? (
// <>
// <b>Supported modalities:</b> {modalities.join(', ')}
// </>
// ) : (
// ''
// )}
// </p>
// </div>
// </div>
// );
// }
function ChatInput({
textarea,
@@ -384,7 +365,7 @@ function ChatInput({
className="flex flex-col rounded-xl border-1 border-base-content/30 p-3 w-full"
// when a file is pasted to the input, we handle it here
// if a text is pasted, and if it is long text, we will convert it to a file
onPasteCapture={(e: ClipboardEvent<HTMLInputElement>) => {
onPasteCapture={(e: React.ClipboardEvent<HTMLInputElement>) => {
const text = e.clipboardData.getData('text/plain');
if (
text.length > 0 &&
examples/server/webui/src/components/Header.tsx
+22 -7
View File
@@ -62,11 +62,6 @@ export default function Header() {
if (newName && newName.trim().length > 0) {
StorageUtils.updateConversationName(viewingChat?.conv.id ?? '', newName);
}
//const importedConv = await StorageUtils.updateConversationName();
//if (importedConv) {
//console.log('Successfully imported:', importedConv.name);
// Refresh UI or navigate to conversation
//}
};
// at the top of your file, alongside ConversationExport:
@@ -75,13 +70,30 @@ export default function Header() {
if (importedConv) {
console.log('Successfully imported:', importedConv.name);
// Refresh UI or navigate to conversation
navigate(`/chat/${importedConv.id}`);
}
};
const downloadConversation = () => {
if (isCurrConvGenerating || !viewingChat) return;
const convId = viewingChat?.conv.id;
const conversationJson = JSON.stringify(viewingChat, null, 2);
// Get the current system message from config
const systemMessage = StorageUtils.getConfig().systemMessage;
// Clone the viewingChat object to avoid modifying the original
const exportData = {
conv: { ...viewingChat.conv },
messages: viewingChat.messages.map(msg => ({ ...msg }))
};
// Find the root message and update its content
const rootMessage = exportData.messages.find(m => m.type === 'root');
if (rootMessage) {
rootMessage.content = systemMessage;
}
const conversationJson = JSON.stringify(exportData, null, 2);
const blob = new Blob([conversationJson], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
@@ -116,9 +128,12 @@ export default function Header() {
{/* action buttons (top right) */}
<div className="flex items-center">
{viewingChat && (
{/* start */ }
{/*viewingChat && */ /* show options for new conversation as well */
(
<div className="dropdown dropdown-end">
{/* "..." button */}
<button
tabIndex={0}
role="button"
examples/server/webui/src/components/MarkdownDisplay.tsx
+1
View File
@@ -93,6 +93,7 @@ const CodeBlockButtons: React.ElementType<
);
};
export const CopyButton = ({
content,
className,
examples/server/webui/src/components/SettingDialog.tsx
+17 -8
View File
@@ -1,4 +1,4 @@
import { useState } from 'react';
import { useState, useRef} from 'react';
import { useAppContext } from '../utils/app.context';
import { CONFIG_DEFAULT, CONFIG_INFO } from '../Config';
import { isDev } from '../Config';
@@ -339,7 +339,7 @@ const SETTING_SECTIONS = (
{
type: SettingInputType.CHECKBOX,
label:
'Exclude thought process when sending requests to API (Recommended for DeepSeek-R1)',
'Exclude thought process when sending requests to API (Recommended for Reasoning Models like Deepseek R1)',
key: 'excludeThoughtOnReq',
},
],
@@ -361,7 +361,7 @@ const SETTING_SECTIONS = (
const demoConv = await res.json();
StorageUtils.remove(demoConv.id);
for (const msg of demoConv.messages) {
StorageUtils.appendMsg(demoConv.id, msg);
StorageUtils.appendMsg(demoConv.id, msg, msg.model_name);
}
};
return (
@@ -422,7 +422,7 @@ const SETTING_SECTIONS = (
toast.success('Import complete')
window.location.reload();
} catch (error) {
console.error('' + error);
//console.error('' + error);
toast.error('' + error);
}
};
@@ -452,9 +452,14 @@ const SETTING_SECTIONS = (
{
type: SettingInputType.CHECKBOX,
label: 'Show tokens per second',
label: 'Show generation stats (model name, context size, prompt and token per second)',
key: 'showTokensPerSecond',
},
{
type: SettingInputType.CHECKBOX,
label: 'Use server defaults for parameters (skip sending temp, top_k, top_p, min_p, typical p from WebUI)',
key: 'useServerDefaults',
},
{
type: SettingInputType.LONG_INPUT,
label: (
@@ -582,7 +587,8 @@ export default function SettingDialog({
return;
}
} else {
console.error(`Unknown default type for key ${key}`);
//console.error(`Unknown default type for key ${key}`);
toast.error(`Unknown default type for key ${key}`);
}
}
if (isDev) console.log('Saving config', newConfig);
@@ -595,6 +601,7 @@ export default function SettingDialog({
setLocalConfig({ ...localConfig, [key]: value });
};
const detailsRef = useRef<HTMLDetailsElement>(null); // <-- Add this line
return (
<dialog className={classNames({ modal: true, 'modal-open': show })}>
<div className="modal-box w-11/12 max-w-3xl">
@@ -619,7 +626,7 @@ export default function SettingDialog({
{/* Left panel, showing sections - Mobile version */}
<div className="md:hidden flex flex-row gap-2 mb-4">
<details className="dropdown">
<details className="dropdown" ref={detailsRef}>
<summary className="btn bt-sm w-full m-1">
{SETTING_SECTIONS_GENERATED[sectionIdx].title}
</summary>
@@ -631,7 +638,9 @@ export default function SettingDialog({
'btn btn-ghost justify-start font-normal': true,
'btn-active': sectionIdx === idx,
})}
onClick={() => setSectionIdx(idx)}
onClick={() => {setSectionIdx(idx);
detailsRef.current?.removeAttribute('open');
}}
dir="auto"
>
{section.title}
examples/server/webui/src/components/Sidebar.tsx
+34 -24
View File
@@ -16,6 +16,7 @@ import { BtnWithTooltips } from '../utils/common';
import { useAppContext } from '../utils/app.context';
import toast from 'react-hot-toast';
import { useModals } from './ModalProvider';
import {DateTime} from 'luxon'
// at the top of your file, alongside ConversationExport:
async function importConversation() {
@@ -114,7 +115,7 @@ export default function Sidebar() {
aria-label="New conversation"
>
<PencilSquareIcon className="w-5 h-5" />
New conversation
New Conversations
</button>
{/* list of conversations */}
@@ -251,11 +252,11 @@ function ConversationItem({
true,
'btn-soft': isCurrConv,
})}
onClick={onSelect}
>
<button
key={conv.id}
className="w-full overflow-hidden truncate text-start"
onClick={onSelect}
className="w-full overflow-hidden truncate text-start"
dir="auto"
>
{conv.name}
@@ -265,7 +266,7 @@ function ConversationItem({
// on mobile, we always show the ellipsis icon
// on desktop, we only show it when the user hovers over the conversation item
// we use opacity instead of hidden to avoid layout shift
className="cursor-pointer opacity-100 md:opacity-0 group-hover:opacity-100"
className="cursor-pointer opacity-100 xl:opacity-0 group-hover:opacity-100"
onClick={() => {}}
tooltipsContent="More"
>
@@ -318,23 +319,26 @@ export interface GroupedConversations {
// TODO @ngxson : add test for this function
// Group conversations by date
// - Yesterday
// - "Previous 7 Days"
// - "Previous 30 Days"
// - "Month Year" (e.g., "April 2023")
export function groupConversationsByDate(
conversations: Conversation[]
): GroupedConversations[] {
const now = new Date();
const today = new Date(now.getFullYear(), now.getMonth(), now.getDate()); // Start of today
const today=DateTime.now().startOf('day');
const yesterday = today.minus({ days: 1 });
const sevenDaysAgo = new Date(today);
sevenDaysAgo.setDate(today.getDate() - 7);
const yesterday2 = today.minus({ days: 2});
const thirtyDaysAgo = new Date(today);
thirtyDaysAgo.setDate(today.getDate() - 30);
const sevenDaysAgo = today.minus({ days: 7 });
const thirtyDaysAgo = today.minus({ days: 30 });
const groups: { [key: string]: Conversation[] } = {
Today: [],
Yesterday: [],
'Previous 2 Days': [],
'Previous 7 Days': [],
'Previous 30 Days': [],
};
@@ -347,17 +351,20 @@ export function groupConversationsByDate(
);
for (const conv of sortedConversations) {
const convDate = new Date(conv.lastModified);
const convDate=DateTime.fromMillis(conv.lastModified).setZone('America/Chicago');
if (convDate >= today) {
groups['Today'].push(conv);
} else if (convDate >= yesterday) {
groups['Yesterday'].push(conv);
} else if (convDate >= yesterday2) {
groups['Previous 2 Days'].push(conv);
} else if (convDate >= sevenDaysAgo) {
groups['Previous 7 Days'].push(conv);
} else if (convDate >= thirtyDaysAgo) {
groups['Previous 30 Days'].push(conv);
} else {
const monthName = convDate.toLocaleString('default', { month: 'long' });
const year = convDate.getFullYear();
const monthName = convDate.monthLong;
const year = convDate.year;
const monthYearKey = `${monthName} ${year}`;
if (!monthlyGroups[monthYearKey]) {
monthlyGroups[monthYearKey] = [];
@@ -374,20 +381,23 @@ export function groupConversationsByDate(
conversations: groups['Today'],
});
}
const timeRanges = [
{ key: 'Yesterday', display: 'Yesterday'},
{ key: 'Previous 2 Days', display: 'Previous 2 Days'},
{ key: 'Previous 7 Days', display: 'Previous 7 Days' },
{ key: 'Previous 30 Days', display: 'Previous 30 Days' },
if (groups['Previous 7 Days'].length > 0) {
// Add more ranges here if needed, e.g., 'Previous 90 Days'
];
for (const range of timeRanges) {
if (groups[range.key]?.length > 0) {
result.push({
title: 'Previous 7 Days',
conversations: groups['Previous 7 Days'],
});
}
if (groups['Previous 30 Days'].length > 0) {
result.push({
title: 'Previous 30 Days',
conversations: groups['Previous 30 Days'],
title: range.display,
conversations: groups[range.key]
});
}
}
// Sort monthly groups by date (most recent month first)
const sortedMonthKeys = Object.keys(monthlyGroups).sort((a, b) => {
examples/server/webui/src/components/useChatExtraContext.tsx
+1 -1
View File
@@ -231,7 +231,7 @@ async function convertPDFToImage(file: File): Promise<string[]> {
if (!ctx) {
throw new Error('Failed to get 2D context from canvas');
}
const task = page.render({ canvasContext: ctx, viewport: viewport });
const task = page.render({ canvasContext: ctx, canvas: canvas, viewport: viewport });
pages.push(
task.promise.then(() => {
return canvas.toDataURL();
examples/server/webui/src/components/useChatScroll.tsx
+34
View File
@@ -0,0 +1,34 @@
import React, { useEffect } from 'react';
import { throttle } from '../utils/misc';
export const scrollToBottom = (requiresNearBottom: boolean, delay?: number) => {
const mainScrollElem = document.getElementById('main-scroll');
if (!mainScrollElem) return;
const spaceToBottom =
mainScrollElem.scrollHeight -
mainScrollElem.scrollTop -
mainScrollElem.clientHeight;
if (!requiresNearBottom || spaceToBottom < 100) {
setTimeout(
() => mainScrollElem.scrollTo({ top: mainScrollElem.scrollHeight }),
delay ?? 80
);
}
};
const scrollToBottomThrottled = throttle(scrollToBottom, 80);
export function useChatScroll(msgListRef: React.RefObject<HTMLDivElement>) {
useEffect(() => {
if (!msgListRef.current) return;
const resizeObserver = new ResizeObserver((_) => {
scrollToBottomThrottled(true, 10);
});
resizeObserver.observe(msgListRef.current);
return () => {
resizeObserver.disconnect();
};
}, [msgListRef]);
}
examples/server/webui/src/index.scss
+6
View File
@@ -9,6 +9,7 @@ html {
scrollbar-gutter: auto;
}
.markdown {
h1,
h2,
@@ -31,6 +32,7 @@ html {
/* TODO: fix markdown table */
}
.show-on-hover {
@apply md:opacity-0 md:group-hover:opacity-100;
}
@@ -42,12 +44,16 @@ html {
}
.chat-bubble {
overflow-wrap: break-word;
word-break: break-word;
@apply break-words;
}
.chat-bubble-base-300 {
--tw-bg-opacity: 1;
--tw-text-opacity: 1;
overflow-wrap: break-word;
word-break: break-word;
@apply break-words bg-base-300 text-base-content;
}
examples/server/webui/src/utils/app.context.tsx
+45 -19
View File
@@ -13,7 +13,7 @@ import {
filterThoughtFromMsgs,
normalizeMsgsForAPI,
getSSEStreamAsync,
getServerProps
getServerProps,
} from './misc';
import { BASE_URL, CONFIG_DEFAULT, isDev } from '../Config';
import { matchPath, useLocation, useNavigate } from 'react-router';
@@ -110,8 +110,7 @@ export const AppContextProvider = ({
setServerProps(props);
})
.catch((err) => {
console.error(err);
toast.error('Failed to fetch server props');
console.error(err);
});
// eslint-disable-next-line
}, []);
@@ -216,6 +215,7 @@ export const AppContextProvider = ({
content: null,
parent: leafNodeId,
children: [],
model_name: '',
};
setPending(convId, pendingMsg as PendingMessage);
}
@@ -240,13 +240,8 @@ export const AppContextProvider = ({
cache_prompt: true,
reasoning_format: config.reasoning_format===''?'auto':config.reasoning_format,
samplers: config.samplers,
temperature: config.temperature,
dynatemp_range: config.dynatemp_range,
dynatemp_exponent: config.dynatemp_exponent,
top_k: config.top_k,
top_p: config.top_p,
min_p: config.min_p,
typical_p: config.typical_p,
xtc_probability: config.xtc_probability,
xtc_threshold: config.xtc_threshold,
top_n_sigma: config.top_n_sigma,
@@ -260,6 +255,13 @@ export const AppContextProvider = ({
dry_penalty_last_n: config.dry_penalty_last_n,
max_tokens: config.max_tokens,
timings_per_token: !!config.showTokensPerSecond,
...(config.useServerDefaults ? {} :{
temperature: config.temperature,
top_k: config.top_k,
top_p: config.top_p,
min_p: config.min_p,
typical_p: config.typical_p,
}),
...(config.custom.length ? JSON.parse(config.custom) : {}),
};
@@ -322,6 +324,8 @@ export const AppContextProvider = ({
prompt_ms: timings.prompt_ms,
predicted_n: timings.predicted_n,
predicted_ms: timings.predicted_ms,
n_ctx: timings.n_ctx,
n_past: timings.n_past,
};
}
setPending(convId, pendingMsg as PendingMessage);
@@ -333,10 +337,7 @@ export const AppContextProvider = ({
// user stopped the generation via stopGeneration() function
// we can safely ignore this error
} else {
console.error(err);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
alert((err as any)?.message ?? 'Unknown error');
//throw err; // rethrow
toast.error(err instanceof Error ? err.message : String(err));
}
}
finally {
@@ -344,7 +345,7 @@ export const AppContextProvider = ({
if (isContinuation) {
await StorageUtils.updateMessage(pendingMsg as Message);
} else if (pendingMsg.content.trim().length > 0) {
await StorageUtils.appendMsg(pendingMsg as Message, leafNodeId);
await StorageUtils.appendMsg(pendingMsg as Message, leafNodeId, '');
}
}
}
@@ -375,6 +376,16 @@ export const AppContextProvider = ({
const now = Date.now()+Timer.timercount;
Timer.timercount=Timer.timercount + 2;
const currMsgId = now;
let model_name:string='';
await getServerProps(BASE_URL)
.then((props) => {
console.debug('Server props:', props);
model_name = props.model_name;
})
.catch((err) => {
console.error(err);
});
StorageUtils.appendMsg(
{
id: currMsgId,
@@ -383,11 +394,13 @@ export const AppContextProvider = ({
convId,
role: 'user',
content,
model_name: model_name,
extra,
parent: leafNodeId,
children: [],
},
leafNodeId
leafNodeId,
model_name
);
onChunk(currMsgId);
@@ -415,9 +428,20 @@ export const AppContextProvider = ({
) => {
if (isGenerating(convId)) return;
if (content !== null) {
if (content !== null) {
const now = Date.now();
const currMsgId = now;
let model_name:string='';
await getServerProps(BASE_URL)
.then((props) => {
console.debug('Server props:', props);
model_name = props.model_name;
})
.catch((err) => {
console.error(err);
});
StorageUtils.appendMsg(
{
id: currMsgId,
@@ -426,11 +450,13 @@ export const AppContextProvider = ({
convId,
role: 'user',
content,
model_name:model_name,
extra,
parent: parentNodeId,
children: [],
},
parentNodeId
parentNodeId,
model_name
);
parentNodeId = currMsgId;
}
@@ -452,9 +478,9 @@ export const AppContextProvider = ({
messageIdToContinue
);
if (!existingMessage || existingMessage.role !== 'assistant') {
console.error(
'Cannot continue non-assistant message or message not found'
);
// console.error(
// 'Cannot continue non-assistant message or message not found'
// );
toast.error(
'Failed to continue message: Not an assistant message or not found.'
);
examples/server/webui/src/utils/misc.ts
+4 -4
View File
@@ -4,7 +4,6 @@ import { APIMessage, Message, LlamaCppServerProps, APIMessageContentPart } from
// ponyfill for missing ReadableStream asyncIterator on Safari
import { asyncIterator } from '@sec-ant/readable-stream/ponyfill/asyncIterator';
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export const isString = (x: any) => !!x.toLowerCase;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -177,19 +176,20 @@ export const getServerProps = async (
apiKey?: string
): Promise<LlamaCppServerProps> => {
try {
const response = await fetch(`${baseUrl}/props`, {
const response = await fetch(`${baseUrl}/v1/props`, {
headers: {
'Content-Type': 'application/json',
...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {}),
},
});
if (!response.ok) {
throw new Error('Failed to fetch server props');
//throw new Error('Failed to fetch server props');
}
const data = await response.json();
return data as LlamaCppServerProps;
} catch (error) {
console.error('Error fetching server props:', error);
//console.error('Error fetching server props:', error);
//toast.error('Error fetching server props:' +error);
throw error;
}
};
examples/server/webui/src/utils/storage.ts
+36 -11
View File
@@ -1,11 +1,12 @@
// coversations is stored in localStorage
// format: { [convId]: { id: string, lastModified: number, messages: [...] } }
import { CONFIG_DEFAULT } from '../Config';
//import { useState } from 'react';
import {BASE_URL, CONFIG_DEFAULT } from '../Config';
import { Conversation, Message, TimingReport, SettingsPreset } from './types';
import Dexie, { Table } from 'dexie';
import {getServerProps} from './misc'
import { exportDB as exportDexieDB } from 'dexie-export-import';
import toast from 'react-hot-toast';
const event = new EventTarget();
type CallbackConversationChanged = (convId: string) => void;
@@ -31,6 +32,12 @@ db.version(1).stores({
messages: '&id, convId, [convId+id], timestamp',
});
db.version(2).stores({
// Unlike SQL, you don’t need to specify all properties but only the one you wish to index.
conversations: '&id, lastModified, model_name',
messages: '&id, convId, [convId+id], timestamp',
});
// convId is a string prefixed with 'conv-'
const StorageUtils = {
@@ -118,11 +125,22 @@ const StorageUtils = {
async createConversation(name: string): Promise<Conversation> {
const now = Date.now();
const msgId = now;
let model_name:string = '';
//window.alert(BASE_URL);
await getServerProps(BASE_URL)
.then((props) => {
console.debug('Server props:', props);
model_name = props.model_name;
})
.catch((err) => {
console.error(err);
});
const conv: Conversation = {
id: `conv-${now}`,
lastModified: now,
currNode: msgId,
name,
model_name:model_name,
};
await db.conversations.add(conv);
// create a root node
@@ -133,6 +151,7 @@ const StorageUtils = {
timestamp: now,
role: 'system',
content: '',
model_name:conv.model_name,
parent: -1,
children: [],
});
@@ -143,7 +162,8 @@ const StorageUtils = {
*/
async appendMsg(
msg: Exclude<Message, 'parent' | 'children'>,
parentNodeId: Message['id']
parentNodeId: Message['id'],
model_name:string,
): Promise<void> {
if (msg.content === null) return;
const { convId } = msg;
@@ -161,9 +181,11 @@ const StorageUtils = {
`Parent message ID ${parentNodeId} does not exist in conversation ${convId}`
);
}
model_name = model_name!==''?model_name:conv.model_name;
await db.conversations.update(convId, {
lastModified: Date.now(),
currNode: msg.id,
model_name: model_name,
});
// update parent
await db.messages.update(parentNodeId, {
@@ -191,10 +213,10 @@ const StorageUtils = {
// event listeners
onConversationChanged(callback: CallbackConversationChanged) {
const fn = (e: Event) => callback((e as CustomEvent).detail.convId);
onConversationChangedHandlers.push([callback, fn]);
event.addEventListener('conversationChange', fn);
},
const fn = (e: Event) => callback((e as CustomEvent).detail.convId);
onConversationChangedHandlers.push([callback, fn]);
event.addEventListener('conversationChange', fn);
},
offConversationChanged(callback: CallbackConversationChanged) {
const fn = onConversationChangedHandlers.find(([cb, _]) => cb === callback);
if (fn) {
@@ -295,7 +317,6 @@ async importConversation(importedData: {
// Refresh the page to apply changes
window.location.reload();
return conversation;
},
/**
@@ -329,7 +350,7 @@ async importConversation(importedData: {
const conversation = await StorageUtils.importConversation(jsonData);
resolve(conversation);
} catch (error) {
console.error('Import failed:', error);
toast.error('Import failed:' +error);
alert(`Import failed: ${error instanceof Error ? error.message : 'Unknown error'}`);
resolve(null);
} finally {
@@ -367,7 +388,8 @@ async importConversation(importedData: {
try {
return JSON.parse(presetsJson);
} catch (e) {
console.error('Failed to parse presets', e);
toast.error('Failed to parse presets: '+ e);
return [];
}
},
@@ -444,6 +466,7 @@ async function migrationLStoIDB() {
lastModified,
currNode: lastMsg.id,
name,
model_name:'migrate_name'
});
const rootId = messages[0].id - 2;
await db.messages.add({
@@ -454,6 +477,7 @@ async function migrationLStoIDB() {
role: 'system',
content: '',
parent: -1,
model_name:'migrate_name',
children: [firstMsg.id],
});
for (let i = 0; i < messages.length; i++) {
@@ -465,6 +489,7 @@ async function migrationLStoIDB() {
timestamp: msg.id,
parent: i === 0 ? rootId : messages[i - 1].id,
children: i === messages.length - 1 ? [] : [messages[i + 1].id],
model_name:'',
});
}
migratedCount++;
examples/server/webui/src/utils/types.ts
+5
View File
@@ -3,6 +3,8 @@ export interface TimingReport {
prompt_ms: number;
predicted_n: number;
predicted_ms: number;
n_ctx: number;
n_past: number;
}
/**
@@ -42,6 +44,7 @@ export interface Message {
role: 'user' | 'assistant' | 'system';
content: string;
timings?: TimingReport;
model_name:string;
extra?: MessageExtra[];
// node based system for branching
parent: Message['id'];
@@ -103,6 +106,7 @@ export interface Conversation {
lastModified: number; // timestamp from Date.now()
currNode: Message['id']; // the current message node being viewed
name: string;
model_name: string;
}
export interface ViewingChat {
@@ -136,6 +140,7 @@ export interface SettingsPreset {
// a non-complete list of props, only contains the ones we need
export interface LlamaCppServerProps {
model_path: string;
model_name: string;
n_ctx: number;
modalities?: {
vision: boolean;
examples/server/webui/tsconfig.app.json
+3
View File
@@ -6,6 +6,9 @@
"lib": ["ES2021", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"sourceMap": false,
"inlineSources": false, // Add this
"inlineSourceMap": false, // Ensure this is false
/* Bundler mode */
"moduleResolution": "bundler",
examples/server/webui/tsconfig.json
+4 -1
View File
@@ -3,5 +3,8 @@
"references": [
{ "path": "./tsconfig.app.json" },
{ "path": "./tsconfig.node.json" }
]
],
"compilerOptions": {
"sourceMap": false
}
}
examples/server/webui/tsconfig.node.json
+2 -1
View File
@@ -5,7 +5,8 @@
"lib": ["ES2023"],
"module": "ESNext",
"skipLibCheck": true,
"sourceMap": false,
"inlineSources": false,
/* Bundler mode */
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
examples/server/webui/vite.config.ts
+1 -2
View File
@@ -7,7 +7,7 @@ import zlib from 'node:zlib';
/* eslint-disable */
const MAX_BUNDLE_SIZE = 2 * 1024 * 1024; // only increase when absolutely necessary
const MAX_BUNDLE_SIZE = 5 * 1024 * 1024; // only increase when absolutely necessary
const GUIDE_FOR_FRONTEND = `
<!--
@@ -66,7 +66,6 @@ const BUILD_PLUGINS = [
];
export default defineConfig({
// @ts-ignore
plugins: process.env.ANALYZE ? FRONTEND_PLUGINS : BUILD_PLUGINS,
server: {
proxy: {
examples/server/webui_llamacpp/.storybook/preview.ts
+12 -4
View File
@@ -5,16 +5,24 @@ import TooltipProviderDecorator from './TooltipProviderDecorator.svelte';
const preview: Preview = {
parameters: {
controls: {
controls: {
matchers: {
color: /(background|color)$/i,
date: /Date$/i
}
},
backgrounds: {
backgrounds: {
disable: true
}
},
},
a11y: {
// 'todo' - show a11y violations in the test UI only
// 'error' - fail CI on a11y violations
// 'off' - skip a11y checks entirely
test: 'todo'
}
},
decorators: [
(story) => ({
Component: ModeWatcherDecorator,
examples/server/webui_llamacpp/.storybook/vitest.setup.ts
+2 -1
View File
@@ -1,8 +1,9 @@
import * as a11yAddonAnnotations from "@storybook/addon-a11y/preview";
import { setProjectAnnotations } from '@storybook/sveltekit';
import * as previewAnnotations from './preview';
import { beforeAll } from 'vitest';
const project = setProjectAnnotations([previewAnnotations]);
const project = setProjectAnnotations([a11yAddonAnnotations, previewAnnotations]);
beforeAll(async () => {
if (project.beforeAll) {
examples/server/webui_llamacpp/package-lock.json
Generated
+9316 -9080
View File
File diff suppressed because it is too large Load Diff
examples/server/webui_llamacpp/package.json
+93 -92
View File
@@ -1,94 +1,95 @@
{
"name": "webui",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "bash scripts/dev.sh",
"build": "vite build",
"preview": "vite preview",
"prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
"reset": "rm -rf .svelte-kit node_modules",
"format": "prettier --write .",
"lint": "prettier --check . && eslint .",
"test": "npm run test:ui -- --run && npm run test:client -- --run && npm run test:server -- --run && npm run test:e2e",
"test:e2e": "playwright test",
"test:client": "vitest --project=client",
"test:server": "vitest --project=server",
"test:ui": "vitest --project=ui",
"test:unit": "vitest",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build",
"cleanup": "rm -rf .svelte-kit build node_modules test-results"
},
"devDependencies": {
"@chromatic-com/storybook": "^4.0.1",
"@eslint/compat": "^1.2.5",
"@eslint/js": "^9.18.0",
"@internationalized/date": "^3.8.2",
"@lucide/svelte": "^0.515.0",
"@playwright/test": "^1.49.1",
"@storybook/addon-a11y": "^9.0.17",
"@storybook/addon-docs": "^9.0.17",
"@storybook/addon-svelte-csf": "^5.0.7",
"@storybook/addon-vitest": "^9.0.17",
"@storybook/sveltekit": "^9.0.17",
"@sveltejs/adapter-static": "^3.0.8",
"@sveltejs/kit": "^2.22.0",
"@sveltejs/vite-plugin-svelte": "^6.0.0",
"@tailwindcss/forms": "^0.5.9",
"@tailwindcss/typography": "^0.5.15",
"@tailwindcss/vite": "^4.0.0",
"@types/node": "^22",
"@vitest/browser": "^3.2.3",
"bits-ui": "^2.8.11",
"clsx": "^2.1.1",
"dexie": "^4.0.11",
"eslint": "^9.18.0",
"eslint-config-prettier": "^10.0.1",
"eslint-plugin-storybook": "^9.0.17",
"eslint-plugin-svelte": "^3.0.0",
"fflate": "^0.8.2",
"globals": "^16.0.0",
"http-server": "^14.1.1",
"mdast": "^3.0.0",
"mdsvex": "^0.12.3",
"playwright": "^1.53.0",
"prettier": "^3.4.2",
"prettier-plugin-svelte": "^3.3.3",
"prettier-plugin-tailwindcss": "^0.6.11",
"rehype-katex": "^7.0.1",
"remark-math": "^6.0.0",
"storybook": "^9.0.17",
"svelte": "^5.0.0",
"svelte-check": "^4.0.0",
"tailwind-merge": "^3.3.1",
"tailwind-variants": "^1.0.0",
"tailwindcss": "^4.0.0",
"tw-animate-css": "^1.3.5",
"typescript": "^5.0.0",
"typescript-eslint": "^8.20.0",
"unified": "^11.0.5",
"uuid": "^13.0.0",
"vite": "^7.0.4",
"vite-plugin-devtools-json": "^0.2.0",
"vitest": "^3.2.3",
"vitest-browser-svelte": "^0.1.0"
},
"dependencies": {
"highlight.js": "^11.11.1",
"mode-watcher": "^1.1.0",
"pdfjs-dist": "^5.4.54",
"rehype-highlight": "^7.0.2",
"rehype-stringify": "^10.0.1",
"remark": "^15.0.1",
"remark-breaks": "^4.0.0",
"remark-gfm": "^4.0.1",
"remark-html": "^16.0.1",
"remark-rehype": "^11.1.2",
"svelte-sonner": "^1.0.5",
"unist-util-visit": "^5.0.0"
}
"name": "webui",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "bash scripts/dev.sh",
"build": "vite build",
"preview": "vite preview",
"prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
"reset": "rm -rf .svelte-kit node_modules",
"format": "prettier --write .",
"lint": "prettier --check . && eslint .",
"test": "npm run test:ui -- --run && npm run test:client -- --run && npm run test:server -- --run && npm run test:e2e",
"test:e2e": "playwright test",
"test:client": "vitest --project=client",
"test:server": "vitest --project=server",
"test:ui": "vitest --project=ui",
"test:unit": "vitest",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build",
"cleanup": "rm -rf .svelte-kit build node_modules test-results"
},
"devDependencies": {
"@chromatic-com/storybook": "^4.1.3",
"@eslint/compat": "^1.2.5",
"@eslint/js": "^9.18.0",
"@internationalized/date": "^3.8.2",
"@lucide/svelte": "^0.515.0",
"@playwright/test": "^1.49.1",
"@storybook/addon-a11y": "^10.0.8",
"@storybook/addon-docs": "^10.0.8",
"@storybook/addon-svelte-csf": "^5.0.10",
"@storybook/addon-vitest": "^10.0.8",
"@storybook/sveltekit": "^10.0.8",
"@sveltejs/adapter-static": "^3.0.8",
"@sveltejs/kit": "^2.22.0",
"@sveltejs/vite-plugin-svelte": "^6.0.0",
"@tailwindcss/forms": "^0.5.9",
"@tailwindcss/typography": "^0.5.15",
"@tailwindcss/vite": "^4.0.0",
"@types/node": "^22",
"@vitest/browser": "^3.2.3",
"bits-ui": "^2.8.11",
"clsx": "^2.1.1",
"dexie": "^4.0.11",
"eslint": "^9.18.0",
"eslint-config-prettier": "^10.0.1",
"eslint-plugin-storybook": "^10.0.8",
"eslint-plugin-svelte": "^3.0.0",
"fflate": "^0.8.2",
"globals": "^16.0.0",
"http-server": "^14.1.1",
"mdast": "^3.0.0",
"mdsvex": "^0.12.3",
"playwright": "^1.53.0",
"prettier": "^3.4.2",
"prettier-plugin-svelte": "^3.3.3",
"prettier-plugin-tailwindcss": "^0.6.11",
"rehype-katex": "^7.0.1",
"remark-math": "^6.0.0",
"sass": "^1.93.3",
"storybook": "^10.0.8",
"svelte": "^5.0.0",
"svelte-check": "^4.0.0",
"tailwind-merge": "^3.3.1",
"tailwind-variants": "^1.0.0",
"tailwindcss": "^4.0.0",
"tw-animate-css": "^1.3.5",
"typescript": "^5.0.0",
"typescript-eslint": "^8.20.0",
"unified": "^11.0.5",
"uuid": "^13.0.0",
"vite": "^7.0.4",
"vite-plugin-devtools-json": "^0.2.0",
"vitest": "^3.2.3",
"vitest-browser-svelte": "^0.1.0"
},
"dependencies": {
"highlight.js": "^11.11.1",
"mode-watcher": "^1.1.0",
"pdfjs-dist": "^5.4.54",
"rehype-highlight": "^7.0.2",
"rehype-stringify": "^10.0.1",
"remark": "^15.0.1",
"remark-breaks": "^4.0.0",
"remark-gfm": "^4.0.1",
"remark-html": "^16.0.1",
"remark-rehype": "^11.1.2",
"svelte-sonner": "^1.0.5",
"unist-util-visit": "^5.0.0"
}
}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentPreview.svelte
+273
View File
@@ -0,0 +1,273 @@
<script lang="ts">
import { FileText, Image, Music, FileIcon, Eye } from '@lucide/svelte';
import { FileTypeCategory, MimeTypeApplication } from '$lib/enums/files';
import { convertPDFToImage } from '$lib/utils/pdf-processing';
import { Button } from '$lib/components/ui/button';
import { getFileTypeCategory } from '$lib/utils/file-type';
interface Props {
// Either an uploaded file or a stored attachment
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
// For uploaded files
preview?: string;
name?: string;
type?: string;
textContent?: string;
}
let { uploadedFile, attachment, preview, name, type, textContent }: Props = $props();
let displayName = $derived(uploadedFile?.name || attachment?.name || name || 'Unknown File');
let displayPreview = $derived(
uploadedFile?.preview || (attachment?.type === 'imageFile' ? attachment.base64Url : preview)
);
let displayType = $derived(
uploadedFile?.type ||
(attachment?.type === 'imageFile'
? 'image'
: attachment?.type === 'textFile'
? 'text'
: attachment?.type === 'audioFile'
? attachment.mimeType || 'audio'
: attachment?.type === 'pdfFile'
? MimeTypeApplication.PDF
: type || 'unknown')
);
let displayTextContent = $derived(
uploadedFile?.textContent ||
(attachment?.type === 'textFile'
? attachment.content
: attachment?.type === 'pdfFile'
? attachment.content
: textContent)
);
let isAudio = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.AUDIO || displayType === 'audio'
);
let isImage = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.IMAGE || displayType === 'image'
);
let isPdf = $derived(displayType === MimeTypeApplication.PDF);
let isText = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.TEXT || displayType === 'text'
);
let IconComponent = $derived(() => {
if (isImage) return Image;
if (isText || isPdf) return FileText;
if (isAudio) return Music;
return FileIcon;
});
let pdfViewMode = $state<'text' | 'pages'>('pages');
let pdfImages = $state<string[]>([]);
let pdfImagesLoading = $state(false);
let pdfImagesError = $state<string | null>(null);
async function loadPdfImages() {
if (!isPdf || pdfImages.length > 0 || pdfImagesLoading) return;
pdfImagesLoading = true;
pdfImagesError = null;
try {
let file: File | null = null;
if (uploadedFile?.file) {
file = uploadedFile.file;
} else if (attachment?.type === 'pdfFile') {
// Check if we have pre-processed images
if (attachment.images && Array.isArray(attachment.images)) {
pdfImages = attachment.images;
return;
}
// Convert base64 back to File for processing
if (attachment.base64Data) {
const base64Data = attachment.base64Data;
const byteCharacters = atob(base64Data);
const byteNumbers = new Array(byteCharacters.length);
for (let i = 0; i < byteCharacters.length; i++) {
byteNumbers[i] = byteCharacters.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
file = new File([byteArray], displayName, { type: MimeTypeApplication.PDF });
}
}
if (file) {
pdfImages = await convertPDFToImage(file);
} else {
throw new Error('No PDF file available for conversion');
}
} catch (error) {
pdfImagesError = error instanceof Error ? error.message : 'Failed to load PDF images';
} finally {
pdfImagesLoading = false;
}
}
export function reset() {
pdfImages = [];
pdfImagesLoading = false;
pdfImagesError = null;
pdfViewMode = 'pages';
}
$effect(() => {
if (isPdf && pdfViewMode === 'pages') {
loadPdfImages();
}
});
</script>
<div class="space-y-4">
<div class="flex items-center justify-end gap-6">
{#if isPdf}
<div class="flex items-center gap-2">
<Button
variant={pdfViewMode === 'text' ? 'default' : 'outline'}
size="sm"
onclick={() => (pdfViewMode = 'text')}
disabled={pdfImagesLoading}
>
<FileText class="mr-1 h-4 w-4" />
Text
</Button>
<Button
variant={pdfViewMode === 'pages' ? 'default' : 'outline'}
size="sm"
onclick={() => {
pdfViewMode = 'pages';
loadPdfImages();
}}
disabled={pdfImagesLoading}
>
{#if pdfImagesLoading}
<div
class="mr-1 h-4 w-4 animate-spin rounded-full border-2 border-current border-t-transparent"
></div>
{:else}
<Eye class="mr-1 h-4 w-4" />
{/if}
Pages
</Button>
</div>
{/if}
</div>
<div class="flex-1 overflow-auto">
{#if isImage && displayPreview}
<div class="flex items-center justify-center">
<img
src={displayPreview}
alt={displayName}
class="max-h-full rounded-lg object-contain shadow-lg"
/>
</div>
{:else if isPdf && pdfViewMode === 'pages'}
{#if pdfImagesLoading}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<div
class="mx-auto mb-4 h-8 w-8 animate-spin rounded-full border-4 border-primary border-t-transparent"
></div>
<p class="text-muted-foreground">Converting PDF to images...</p>
</div>
</div>
{:else if pdfImagesError}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<FileText class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
<p class="mb-4 text-muted-foreground">Failed to load PDF images</p>
<p class="text-sm text-muted-foreground">{pdfImagesError}</p>
<Button class="mt-4" onclick={() => (pdfViewMode = 'text')}>View as Text</Button>
</div>
</div>
{:else if pdfImages.length > 0}
<div class="max-h-[70vh] space-y-4 overflow-auto">
{#each pdfImages as image, index (image)}
<div class="text-center">
<p class="mb-2 text-sm text-muted-foreground">Page {index + 1}</p>
<img
src={image}
alt="PDF Page {index + 1}"
class="mx-auto max-w-full rounded-lg shadow-lg"
/>
</div>
{/each}
</div>
{:else}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<FileText class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
<p class="mb-4 text-muted-foreground">No PDF pages available</p>
</div>
</div>
{/if}
{:else if (isText || (isPdf && pdfViewMode === 'text')) && displayTextContent}
<div
class="max-h-[60vh] overflow-auto rounded-lg bg-muted p-4 font-mono text-sm break-words whitespace-pre-wrap"
>
{displayTextContent}
</div>
{:else if isAudio}
<div class="flex items-center justify-center p-8">
<div class="w-full max-w-md text-center">
<Music class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
{#if attachment?.type === 'audioFile'}
<audio
controls
class="mb-4 w-full"
src="data:{attachment.mimeType};base64,{attachment.base64Data}"
>
Your browser does not support the audio element.
</audio>
{:else if uploadedFile?.preview}
<audio controls class="mb-4 w-full" src={uploadedFile.preview}>
Your browser does not support the audio element.
</audio>
{:else}
<p class="mb-4 text-muted-foreground">Audio preview not available</p>
{/if}
<p class="text-sm text-muted-foreground">
{displayName}
</p>
</div>
</div>
{:else}
<div class="flex items-center justify-center p-8">
<div class="text-center">
{#if IconComponent}
<IconComponent class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
{/if}
<p class="mb-4 text-muted-foreground">Preview not available for this file type</p>
</div>
</div>
{/if}
</div>
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentPreviewDialog.svelte
-305
View File
@@ -1,305 +0,0 @@
<script lang="ts">
import * as Dialog from '$lib/components/ui/dialog';
import { FileText, Image, Music, FileIcon, Eye } from '@lucide/svelte';
import { FileTypeCategory, MimeTypeApplication } from '$lib/enums/files';
import { convertPDFToImage } from '$lib/utils/pdf-processing';
import { Button } from '$lib/components/ui/button';
import { getFileTypeCategory } from '$lib/utils/file-type';
import { formatFileSize } from '$lib/utils/file-preview';
interface Props {
open: boolean;
// Either an uploaded file or a stored attachment
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
// For uploaded files
preview?: string;
name?: string;
type?: string;
size?: number;
textContent?: string;
}
let {
open = $bindable(),
uploadedFile,
attachment,
preview,
name,
type,
size,
textContent
}: Props = $props();
let displayName = $derived(uploadedFile?.name || attachment?.name || name || 'Unknown File');
let displayPreview = $derived(
uploadedFile?.preview || (attachment?.type === 'imageFile' ? attachment.base64Url : preview)
);
let displayType = $derived(
uploadedFile?.type ||
(attachment?.type === 'imageFile'
? 'image'
: attachment?.type === 'textFile'
? 'text'
: attachment?.type === 'audioFile'
? attachment.mimeType || 'audio'
: attachment?.type === 'pdfFile'
? MimeTypeApplication.PDF
: type || 'unknown')
);
let displaySize = $derived(uploadedFile?.size || size);
let displayTextContent = $derived(
uploadedFile?.textContent ||
(attachment?.type === 'textFile'
? attachment.content
: attachment?.type === 'pdfFile'
? attachment.content
: textContent)
);
let isAudio = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.AUDIO || displayType === 'audio'
);
let isImage = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.IMAGE || displayType === 'image'
);
let isPdf = $derived(displayType === MimeTypeApplication.PDF);
let isText = $derived(
getFileTypeCategory(displayType) === FileTypeCategory.TEXT || displayType === 'text'
);
let IconComponent = $derived(() => {
if (isImage) return Image;
if (isText || isPdf) return FileText;
if (isAudio) return Music;
return FileIcon;
});
let pdfViewMode = $state<'text' | 'pages'>('pages');
let pdfImages = $state<string[]>([]);
let pdfImagesLoading = $state(false);
let pdfImagesError = $state<string | null>(null);
async function loadPdfImages() {
if (!isPdf || pdfImages.length > 0 || pdfImagesLoading) return;
pdfImagesLoading = true;
pdfImagesError = null;
try {
let file: File | null = null;
if (uploadedFile?.file) {
file = uploadedFile.file;
} else if (attachment?.type === 'pdfFile') {
// Check if we have pre-processed images
if (attachment.images && Array.isArray(attachment.images)) {
pdfImages = attachment.images;
return;
}
// Convert base64 back to File for processing
if (attachment.base64Data) {
const base64Data = attachment.base64Data;
const byteCharacters = atob(base64Data);
const byteNumbers = new Array(byteCharacters.length);
for (let i = 0; i < byteCharacters.length; i++) {
byteNumbers[i] = byteCharacters.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
file = new File([byteArray], displayName, { type: MimeTypeApplication.PDF });
}
}
if (file) {
pdfImages = await convertPDFToImage(file);
} else {
throw new Error('No PDF file available for conversion');
}
} catch (error) {
pdfImagesError = error instanceof Error ? error.message : 'Failed to load PDF images';
} finally {
pdfImagesLoading = false;
}
}
$effect(() => {
if (open && isPdf && pdfViewMode === 'pages') {
loadPdfImages();
}
});
</script>
<Dialog.Root bind:open>
<Dialog.Content class="grid max-h-[90vh] max-w-5xl overflow-hidden !p-10 sm:w-auto sm:max-w-6xl">
<Dialog.Header class="flex-shrink-0">
<div class="flex items-center justify-between">
<div class="flex items-center gap-3">
{#if IconComponent}
<IconComponent class="h-5 w-5 text-muted-foreground" />
{/if}
<div>
<Dialog.Title class="text-left">{displayName}</Dialog.Title>
<div class="flex items-center gap-2 text-sm text-muted-foreground">
<span>{displayType}</span>
{#if displaySize}
<span>•</span>
<span>{formatFileSize(displaySize)}</span>
{/if}
</div>
</div>
</div>
{#if isPdf}
<div class="flex items-center gap-2">
<Button
variant={pdfViewMode === 'text' ? 'default' : 'outline'}
size="sm"
onclick={() => (pdfViewMode = 'text')}
disabled={pdfImagesLoading}
>
<FileText class="mr-1 h-4 w-4" />
Text
</Button>
<Button
variant={pdfViewMode === 'pages' ? 'default' : 'outline'}
size="sm"
onclick={() => {
pdfViewMode = 'pages';
loadPdfImages();
}}
disabled={pdfImagesLoading}
>
{#if pdfImagesLoading}
<div
class="mr-1 h-4 w-4 animate-spin rounded-full border-2 border-current border-t-transparent"
></div>
{:else}
<Eye class="mr-1 h-4 w-4" />
{/if}
Pages
</Button>
</div>
{/if}
</div>
</Dialog.Header>
<div class="flex-1 overflow-auto">
{#if isImage && displayPreview}
<div class="flex items-center justify-center">
<img
src={displayPreview}
alt={displayName}
class="max-h-full rounded-lg object-contain shadow-lg"
/>
</div>
{:else if isPdf && pdfViewMode === 'pages'}
{#if pdfImagesLoading}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<div
class="mx-auto mb-4 h-8 w-8 animate-spin rounded-full border-4 border-primary border-t-transparent"
></div>
<p class="text-muted-foreground">Converting PDF to images...</p>
</div>
</div>
{:else if pdfImagesError}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<FileText class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
<p class="mb-4 text-muted-foreground">Failed to load PDF images</p>
<p class="text-sm text-muted-foreground">{pdfImagesError}</p>
<Button class="mt-4" onclick={() => (pdfViewMode = 'text')}>View as Text</Button>
</div>
</div>
{:else if pdfImages.length > 0}
<div class="max-h-[70vh] space-y-4 overflow-auto">
{#each pdfImages as image, index (image)}
<div class="text-center">
<p class="mb-2 text-sm text-muted-foreground">Page {index + 1}</p>
<img
src={image}
alt="PDF Page {index + 1}"
class="mx-auto max-w-full rounded-lg shadow-lg"
/>
</div>
{/each}
</div>
{:else}
<div class="flex items-center justify-center p-8">
<div class="text-center">
<FileText class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
<p class="mb-4 text-muted-foreground">No PDF pages available</p>
</div>
</div>
{/if}
{:else if (isText || (isPdf && pdfViewMode === 'text')) && displayTextContent}
<div
class="max-h-[60vh] overflow-auto rounded-lg bg-muted p-4 font-mono text-sm break-words whitespace-pre-wrap"
>
{displayTextContent}
</div>
{:else if isAudio}
<div class="flex items-center justify-center p-8">
<div class="w-full max-w-md text-center">
<Music class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
{#if attachment?.type === 'audioFile'}
<audio
controls
class="mb-4 w-full"
src="data:{attachment.mimeType};base64,{attachment.base64Data}"
>
Your browser does not support the audio element.
</audio>
{:else if uploadedFile?.preview}
<audio controls class="mb-4 w-full" src={uploadedFile.preview}>
Your browser does not support the audio element.
</audio>
{:else}
<p class="mb-4 text-muted-foreground">Audio preview not available</p>
{/if}
<p class="text-sm text-muted-foreground">
{displayName}
</p>
</div>
</div>
{:else}
<div class="flex items-center justify-center p-8">
<div class="text-center">
{#if IconComponent}
<IconComponent class="mx-auto mb-4 h-16 w-16 text-muted-foreground" />
{/if}
<p class="mb-4 text-muted-foreground">Preview not available for this file type</p>
</div>
</div>
{/if}
</div>
</Dialog.Content>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentFilePreview.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentThumbnailFile.svelte
+19 -29
View File
@@ -1,6 +1,5 @@
<script lang="ts">
import { X } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import { RemoveButton } from '$lib/components/app';
import { formatFileSize, getFileTypeLabel, getPreviewText } from '$lib/utils/file-preview';
import { FileTypeCategory, MimeTypeText } from '$lib/enums/files';
@@ -66,17 +65,15 @@
</button>
{:else}
<!-- Non-readonly mode (ChatForm) -->
<div class="relative rounded-lg border border-border bg-muted p-3 {className} w-64">
<Button
type="button"
variant="ghost"
size="sm"
class="absolute top-2 right-2 h-6 w-6 bg-white/20 p-0 hover:bg-white/30"
onclick={() => onRemove?.(id)}
aria-label="Remove file"
>
<X class="h-3 w-3" />
</Button>
<button
class="group relative rounded-lg border border-border bg-muted p-3 {className} {textContent
? 'max-h-24 max-w-72'
: 'max-w-36'} cursor-pointer text-left"
onclick={onClick}
>
<div class="absolute top-2 right-2 opacity-0 transition-opacity group-hover:opacity-100">
<RemoveButton {id} {onRemove} />
</div>
<div class="pr-8">
<span class="mb-3 block truncate text-sm font-medium text-foreground">{name}</span>
@@ -85,7 +82,7 @@
<div class="relative">
<div
class="overflow-hidden font-mono text-xs leading-relaxed break-words whitespace-pre-wrap text-muted-foreground"
style="max-height: 3.6em; line-height: 1.2em;"
style="max-height: 3rem; line-height: 1.2em;"
>
{getPreviewText(textContent)}
</div>
@@ -98,11 +95,11 @@
</div>
{/if}
</div>
</div>
</button>
{/if}
{:else}
<button
class="flex items-center gap-2 gap-3 rounded-lg border border-border bg-muted p-3 {className}"
class="group flex items-center gap-3 rounded-lg border border-border bg-muted p-3 {className} relative"
onclick={onClick}
>
<div
@@ -112,7 +109,9 @@
</div>
<div class="flex flex-col gap-1">
<span class="max-w-36 truncate text-sm font-medium text-foreground md:max-w-72">
<span
class="max-w-24 truncate text-sm font-medium text-foreground group-hover:pr-6 md:max-w-32"
>
{name}
</span>
@@ -122,18 +121,9 @@
</div>
{#if !readonly}
<Button
type="button"
variant="ghost"
size="sm"
class="h-6 w-6 p-0"
onclick={(e) => {
e.stopPropagation();
onRemove?.(id);
}}
>
<X class="h-3 w-3" />
</Button>
<div class="absolute top-2 right-2 opacity-0 transition-opacity group-hover:opacity-100">
<RemoveButton {id} {onRemove} />
</div>
{/if}
</button>
{/if}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentImagePreview.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentThumbnailImage.svelte
+5 -14
View File
@@ -1,6 +1,5 @@
<script lang="ts">
import { X } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import { RemoveButton } from '$lib/components/app';
interface Props {
id: string;
@@ -26,12 +25,12 @@
class: className = '',
// Default to small size for form previews
width = 'w-auto',
height = 'h-24',
height = 'h-16',
imageClass = ''
}: Props = $props();
</script>
<div class="relative overflow-hidden rounded-lg border border-border bg-muted {className}">
<div class="group relative overflow-hidden rounded-lg border border-border bg-muted {className}">
{#if onClick}
<button
type="button"
@@ -55,17 +54,9 @@
{#if !readonly}
<div
class="absolute top-1 right-1 flex items-center justify-center opacity-0 transition-opacity hover:opacity-100"
class="absolute top-1 right-1 flex items-center justify-center opacity-0 transition-opacity group-hover:opacity-100"
>
<Button
type="button"
variant="ghost"
size="sm"
class="h-6 w-6 bg-white/20 p-0 text-white hover:bg-white/30"
onclick={() => onRemove?.(id)}
>
<X class="h-3 w-3" />
</Button>
<RemoveButton {id} {onRemove} class="text-white" />
</div>
{/if}
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentsList.svelte
+144 -62
View File
@@ -1,11 +1,15 @@
<script lang="ts">
import { ChatAttachmentImagePreview, ChatAttachmentFilePreview } from '$lib/components/app';
import { ChatAttachmentThumbnailImage, ChatAttachmentThumbnailFile } from '$lib/components/app';
import { Button } from '$lib/components/ui/button';
import { ChevronLeft, ChevronRight } from '@lucide/svelte';
import { FileTypeCategory } from '$lib/enums/files';
import { getFileTypeCategory } from '$lib/utils/file-type';
import ChatAttachmentPreviewDialog from './ChatAttachmentPreviewDialog.svelte';
import { DialogChatAttachmentPreview, DialogChatAttachmentsViewAll } from '$lib/components/app';
import type { ChatAttachmentDisplayItem, ChatAttachmentPreviewItem } from '$lib/types/chat';
interface Props {
class?: string;
style?: string;
// For ChatMessage - stored attachments
attachments?: DatabaseMessageExtra[];
readonly?: boolean;
@@ -16,10 +20,13 @@
imageClass?: string;
imageHeight?: string;
imageWidth?: string;
// Limit display to single row with "+ X more" button
limitToSingleRow?: boolean;
}
let {
class: className = '',
style = '',
attachments = [],
readonly = false,
onFileRemove,
@@ -27,36 +34,23 @@
// Default to small size for form previews
imageClass = '',
imageHeight = 'h-24',
imageWidth = 'w-auto'
imageWidth = 'w-auto',
limitToSingleRow = false
}: Props = $props();
let displayItems = $derived(getDisplayItems());
// Preview dialog state
let canScrollLeft = $state(false);
let canScrollRight = $state(false);
let isScrollable = $state(false);
let previewDialogOpen = $state(false);
let previewItem = $state<{
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
preview?: string;
name?: string;
type?: string;
size?: number;
textContent?: string;
} | null>(null);
let previewItem = $state<ChatAttachmentPreviewItem | null>(null);
let scrollContainer: HTMLDivElement | undefined = $state();
let showViewAll = $derived(limitToSingleRow && displayItems.length > 0 && isScrollable);
let viewAllDialogOpen = $state(false);
function getDisplayItems() {
const items: Array<{
id: string;
name: string;
size?: number;
preview?: string;
type: string;
isImage: boolean;
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
attachmentIndex?: number;
textContent?: string;
}> = [];
function getDisplayItems(): ChatAttachmentDisplayItem[] {
const items: ChatAttachmentDisplayItem[] = [];
// Add uploaded files (ChatForm)
for (const file of uploadedFiles) {
@@ -127,14 +121,12 @@
}
}
return items;
return items.reverse();
}
function openPreview(item: (typeof displayItems)[0], event?: Event) {
if (event) {
event.preventDefault();
event.stopPropagation();
}
function openPreview(item: ChatAttachmentDisplayItem, event?: MouseEvent) {
event?.stopPropagation();
event?.preventDefault();
previewItem = {
uploadedFile: item.uploadedFile,
@@ -147,43 +139,123 @@
};
previewDialogOpen = true;
}
function scrollLeft(event?: MouseEvent) {
event?.stopPropagation();
event?.preventDefault();
if (!scrollContainer) return;
scrollContainer.scrollBy({ left: scrollContainer.clientWidth * -0.67, behavior: 'smooth' });
}
function scrollRight(event?: MouseEvent) {
event?.stopPropagation();
event?.preventDefault();
if (!scrollContainer) return;
scrollContainer.scrollBy({ left: scrollContainer.clientWidth * 0.67, behavior: 'smooth' });
}
function updateScrollButtons() {
if (!scrollContainer) return;
const { scrollLeft, scrollWidth, clientWidth } = scrollContainer;
canScrollLeft = scrollLeft > 0;
canScrollRight = scrollLeft < scrollWidth - clientWidth - 1;
isScrollable = scrollWidth > clientWidth;
}
$effect(() => {
if (scrollContainer && displayItems.length) {
scrollContainer.scrollLeft = 0;
setTimeout(() => {
updateScrollButtons();
}, 0);
}
});
</script>
{#if displayItems.length > 0}
<div class="flex flex-wrap items-start {readonly ? 'justify-end' : ''} gap-3 {className}">
{#each displayItems as item (item.id)}
{#if item.isImage && item.preview}
<ChatAttachmentImagePreview
class="cursor-pointer"
id={item.id}
name={item.name}
preview={item.preview}
{readonly}
onRemove={onFileRemove}
height={imageHeight}
width={imageWidth}
{imageClass}
onClick={(event) => openPreview(item, event)}
/>
{:else}
<ChatAttachmentFilePreview
class="cursor-pointer"
id={item.id}
name={item.name}
type={item.type}
size={item.size}
{readonly}
onRemove={onFileRemove}
textContent={item.textContent}
onClick={(event) => openPreview(item, event)}
/>
{/if}
{/each}
<div class={className} {style}>
<div class="relative">
<button
class="absolute top-1/2 left-4 z-10 flex h-6 w-6 -translate-y-1/2 items-center justify-center rounded-full bg-foreground/15 shadow-md backdrop-blur-xs transition-opacity hover:bg-foreground/35 {canScrollLeft
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollLeft}
aria-label="Scroll left"
>
<ChevronLeft class="h-4 w-4" />
</button>
<div
class="scrollbar-hide flex items-start gap-3 overflow-x-auto"
bind:this={scrollContainer}
onscroll={updateScrollButtons}
>
{#each displayItems as item (item.id)}
{#if item.isImage && item.preview}
<ChatAttachmentThumbnailImage
class="flex-shrink-0 cursor-pointer {limitToSingleRow ? 'first:ml-4 last:mr-4' : ''}"
id={item.id}
name={item.name}
preview={item.preview}
{readonly}
onRemove={onFileRemove}
height={imageHeight}
width={imageWidth}
{imageClass}
onClick={(event) => openPreview(item, event)}
/>
{:else}
<ChatAttachmentThumbnailFile
class="flex-shrink-0 cursor-pointer {limitToSingleRow ? 'first:ml-4 last:mr-4' : ''}"
id={item.id}
name={item.name}
type={item.type}
size={item.size}
{readonly}
onRemove={onFileRemove}
textContent={item.textContent}
onClick={(event) => openPreview(item, event)}
/>
{/if}
{/each}
</div>
<button
class="absolute top-1/2 right-4 z-10 flex h-6 w-6 -translate-y-1/2 items-center justify-center rounded-full bg-foreground/15 shadow-md backdrop-blur-xs transition-opacity hover:bg-foreground/35 {canScrollRight
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollRight}
aria-label="Scroll right"
>
<ChevronRight class="h-4 w-4" />
</button>
</div>
{#if showViewAll}
<div class="mt-2 -mr-2 flex justify-end px-4">
<Button
type="button"
variant="ghost"
size="sm"
class="h-6 text-xs text-muted-foreground hover:text-foreground"
onclick={() => (viewAllDialogOpen = true)}
>
View all
</Button>
</div>
{/if}
</div>
{/if}
{#if previewItem}
<ChatAttachmentPreviewDialog
<DialogChatAttachmentPreview
bind:open={previewDialogOpen}
uploadedFile={previewItem.uploadedFile}
attachment={previewItem.attachment}
@@ -194,3 +266,13 @@
textContent={previewItem.textContent}
/>
{/if}
<DialogChatAttachmentsViewAll
bind:open={viewAllDialogOpen}
{uploadedFiles}
{attachments}
{readonly}
{onFileRemove}
imageHeight="h-64"
{imageClass}
/>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatAttachments/ChatAttachmentsViewAll.svelte
+190
View File
@@ -0,0 +1,190 @@
<script lang="ts">
import {
ChatAttachmentThumbnailImage,
ChatAttachmentThumbnailFile,
DialogChatAttachmentPreview
} from '$lib/components/app';
import { FileTypeCategory } from '$lib/enums/files';
import { getFileTypeCategory } from '$lib/utils/file-type';
import type { ChatAttachmentDisplayItem, ChatAttachmentPreviewItem } from '$lib/types/chat';
interface Props {
uploadedFiles?: ChatUploadedFile[];
attachments?: DatabaseMessageExtra[];
readonly?: boolean;
onFileRemove?: (fileId: string) => void;
imageHeight?: string;
imageWidth?: string;
imageClass?: string;
}
let {
uploadedFiles = [],
attachments = [],
readonly = false,
onFileRemove,
imageHeight = 'h-24',
imageWidth = 'w-auto',
imageClass = ''
}: Props = $props();
let previewDialogOpen = $state(false);
let previewItem = $state<ChatAttachmentPreviewItem | null>(null);
let displayItems = $derived(getDisplayItems());
let imageItems = $derived(displayItems.filter((item) => item.isImage));
let fileItems = $derived(displayItems.filter((item) => !item.isImage));
function getDisplayItems(): ChatAttachmentDisplayItem[] {
const items: ChatAttachmentDisplayItem[] = [];
for (const file of uploadedFiles) {
items.push({
id: file.id,
name: file.name,
size: file.size,
preview: file.preview,
type: file.type,
isImage: getFileTypeCategory(file.type) === FileTypeCategory.IMAGE,
uploadedFile: file,
textContent: file.textContent
});
}
for (const [index, attachment] of attachments.entries()) {
if (attachment.type === 'imageFile') {
items.push({
id: `attachment-${index}`,
name: attachment.name,
preview: attachment.base64Url,
type: 'image',
isImage: true,
attachment,
attachmentIndex: index
});
} else if (attachment.type === 'textFile') {
items.push({
id: `attachment-${index}`,
name: attachment.name,
type: 'text',
isImage: false,
attachment,
attachmentIndex: index,
textContent: attachment.content
});
} else if (attachment.type === 'context') {
// Legacy format from old webui - treat as text file
items.push({
id: `attachment-${index}`,
name: attachment.name,
type: 'text',
isImage: false,
attachment,
attachmentIndex: index,
textContent: attachment.content
});
} else if (attachment.type === 'audioFile') {
items.push({
id: `attachment-${index}`,
name: attachment.name,
type: attachment.mimeType || 'audio',
isImage: false,
attachment,
attachmentIndex: index
});
} else if (attachment.type === 'pdfFile') {
items.push({
id: `attachment-${index}`,
name: attachment.name,
type: 'application/pdf',
isImage: false,
attachment,
attachmentIndex: index,
textContent: attachment.content
});
}
}
return items.reverse();
}
function openPreview(item: (typeof displayItems)[0], event?: Event) {
if (event) {
event.preventDefault();
event.stopPropagation();
}
previewItem = {
uploadedFile: item.uploadedFile,
attachment: item.attachment,
preview: item.preview,
name: item.name,
type: item.type,
size: item.size,
textContent: item.textContent
};
previewDialogOpen = true;
}
</script>
<div class="space-y-4">
<div class="min-h-0 flex-1 space-y-6 overflow-y-auto px-1">
{#if fileItems.length > 0}
<div>
<h3 class="mb-3 text-sm font-medium text-foreground">Files ({fileItems.length})</h3>
<div class="flex flex-wrap items-start gap-3">
{#each fileItems as item (item.id)}
<ChatAttachmentThumbnailFile
class="cursor-pointer"
id={item.id}
name={item.name}
type={item.type}
size={item.size}
{readonly}
onRemove={onFileRemove}
textContent={item.textContent}
onClick={(event) => openPreview(item, event)}
/>
{/each}
</div>
</div>
{/if}
{#if imageItems.length > 0}
<div>
<h3 class="mb-3 text-sm font-medium text-foreground">Images ({imageItems.length})</h3>
<div class="flex flex-wrap items-start gap-3">
{#each imageItems as item (item.id)}
{#if item.preview}
<ChatAttachmentThumbnailImage
class="cursor-pointer"
id={item.id}
name={item.name}
preview={item.preview}
{readonly}
onRemove={onFileRemove}
height={imageHeight}
width={imageWidth}
{imageClass}
onClick={(event) => openPreview(item, event)}
/>
{/if}
{/each}
</div>
</div>
{/if}
</div>
</div>
{#if previewItem}
<DialogChatAttachmentPreview
bind:open={previewDialogOpen}
uploadedFile={previewItem.uploadedFile}
attachment={previewItem.attachment}
preview={previewItem.preview}
name={previewItem.name}
type={previewItem.type}
size={previewItem.size}
textContent={previewItem.textContent}
/>
{/if}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatForm/ChatForm.svelte
+7 -1
View File
@@ -232,7 +232,13 @@
onsubmit={handleSubmit}
class="{INPUT_CLASSES} border-radius-bottom-none mx-auto max-w-[48rem] overflow-hidden rounded-3xl backdrop-blur-md {className}"
>
<ChatAttachmentsList bind:uploadedFiles {onFileRemove} class="mb-3 px-5 pt-5" />
<ChatAttachmentsList
bind:uploadedFiles
{onFileRemove}
limitToSingleRow
class="py-5"
style="scroll-padding: 1rem;"
/>
<div
class="flex-column relative min-h-[48px] items-center rounded-3xl px-5 py-3 shadow-sm transition-all focus-within:shadow-md"
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatForm/ChatFormActionFileAttachments.svelte
-121
View File
@@ -1,121 +0,0 @@
<script lang="ts">
import { Paperclip, Image, FileText, File, Volume2 } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import * as DropdownMenu from '$lib/components/ui/dropdown-menu';
import * as Tooltip from '$lib/components/ui/tooltip';
import { TOOLTIP_DELAY_DURATION } from '$lib/constants/tooltip-config';
import { FileTypeCategory } from '$lib/enums/files';
import { supportsAudio, supportsVision } from '$lib/stores/server.svelte';
interface Props {
class?: string;
disabled?: boolean;
onFileUpload?: (fileType?: FileTypeCategory) => void;
}
let { class: className = '', disabled = false, onFileUpload }: Props = $props();
const fileUploadTooltipText = $derived.by(() => {
return !supportsVision()
? 'Text files and PDFs supported. Images, audio, and video require vision models.'
: 'Attach files';
});
function handleFileUpload(fileType?: FileTypeCategory) {
onFileUpload?.(fileType);
}
</script>
<div class="flex items-center gap-1 {className}">
<DropdownMenu.Root>
<DropdownMenu.Trigger name="Attach files">
<Tooltip.Root delayDuration={TOOLTIP_DELAY_DURATION}>
<Tooltip.Trigger>
<Button
class="file-upload-button h-8 w-8 rounded-full bg-transparent p-0 text-muted-foreground hover:bg-foreground/10 hover:text-foreground"
{disabled}
type="button"
>
<span class="sr-only">Attach files</span>
<Paperclip class="h-4 w-4" />
</Button>
</Tooltip.Trigger>
<Tooltip.Content>
<p>{fileUploadTooltipText}</p>
</Tooltip.Content>
</Tooltip.Root>
</DropdownMenu.Trigger>
<DropdownMenu.Content align="start" class="w-48">
<Tooltip.Root delayDuration={TOOLTIP_DELAY_DURATION}>
<Tooltip.Trigger class="w-full">
<DropdownMenu.Item
class="images-button flex cursor-pointer items-center gap-2"
disabled={!supportsVision()}
onclick={() => handleFileUpload(FileTypeCategory.IMAGE)}
>
<Image class="h-4 w-4" />
<span>Images</span>
</DropdownMenu.Item>
</Tooltip.Trigger>
{#if !supportsVision()}
<Tooltip.Content>
<p>Images require vision models to be processed</p>
</Tooltip.Content>
{/if}
</Tooltip.Root>
<Tooltip.Root delayDuration={TOOLTIP_DELAY_DURATION}>
<Tooltip.Trigger class="w-full">
<DropdownMenu.Item
class="audio-button flex cursor-pointer items-center gap-2"
disabled={!supportsAudio()}
onclick={() => handleFileUpload(FileTypeCategory.AUDIO)}
>
<Volume2 class="h-4 w-4" />
<span>Audio Files</span>
</DropdownMenu.Item>
</Tooltip.Trigger>
{#if !supportsAudio()}
<Tooltip.Content>
<p>Audio files require audio models to be processed</p>
</Tooltip.Content>
{/if}
</Tooltip.Root>
<DropdownMenu.Item
class="flex cursor-pointer items-center gap-2"
onclick={() => handleFileUpload(FileTypeCategory.TEXT)}
>
<FileText class="h-4 w-4" />
<span>Text Files</span>
</DropdownMenu.Item>
<Tooltip.Root delayDuration={TOOLTIP_DELAY_DURATION}>
<Tooltip.Trigger class="w-full">
<DropdownMenu.Item
class="flex cursor-pointer items-center gap-2"
onclick={() => handleFileUpload(FileTypeCategory.PDF)}
>
<File class="h-4 w-4" />
<span>PDF Files</span>
</DropdownMenu.Item>
</Tooltip.Trigger>
{#if !supportsVision()}
<Tooltip.Content>
<p>PDFs will be converted to text. Image-based PDFs may not work properly.</p>
</Tooltip.Content>
{/if}
</Tooltip.Root>
</DropdownMenu.Content>
</DropdownMenu.Root>
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatForm/ChatFormActionRecord.svelte
-49
View File
@@ -1,49 +0,0 @@
<script lang="ts">
import { Mic } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import * as Tooltip from '$lib/components/ui/tooltip';
import { supportsAudio } from '$lib/stores/server.svelte';
interface Props {
class?: string;
disabled?: boolean;
isLoading?: boolean;
isRecording?: boolean;
onMicClick?: () => void;
}
let {
class: className = '',
disabled = false,
isLoading = false,
isRecording = false,
onMicClick
}: Props = $props();
</script>
<div class="flex items-center gap-1 {className}">
<Tooltip.Root delayDuration={100}>
<Tooltip.Trigger>
<Button
class="h-8 w-8 rounded-full p-0 {isRecording
? 'animate-pulse bg-red-500 text-white hover:bg-red-600'
: 'bg-transparent text-muted-foreground hover:bg-foreground/10 hover:text-foreground'} {!supportsAudio()
? 'cursor-not-allowed opacity-50'
: ''}"
disabled={disabled || isLoading || !supportsAudio()}
onclick={onMicClick}
type="button"
>
<span class="sr-only">{isRecording ? 'Stop recording' : 'Start recording'}</span>
<Mic class="h-4 w-4" />
</Button>
</Tooltip.Trigger>
{#if !supportsAudio()}
<Tooltip.Content>
<p>Current model does not support audio</p>
</Tooltip.Content>
{/if}
</Tooltip.Root>
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatForm/ChatFormActions.svelte
-63
View File
@@ -1,63 +0,0 @@
<script lang="ts">
import { Square, ArrowUp } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import ChatFormActionFileAttachments from './ChatFormActionFileAttachments.svelte';
import ChatFormActionRecord from './ChatFormActionRecord.svelte';
import ChatFormModelSelector from './ChatFormModelSelector.svelte';
import { config } from '$lib/stores/settings.svelte';
import type { FileTypeCategory } from '$lib/enums/files';
interface Props {
canSend?: boolean;
class?: string;
disabled?: boolean;
isLoading?: boolean;
isRecording?: boolean;
onFileUpload?: (fileType?: FileTypeCategory) => void;
onMicClick?: () => void;
onStop?: () => void;
}
let {
canSend = false,
class: className = '',
disabled = false,
isLoading = false,
isRecording = false,
onFileUpload,
onMicClick,
onStop
}: Props = $props();
let currentConfig = $derived(config());
</script>
<div class="flex w-full items-center gap-2 {className}">
<ChatFormActionFileAttachments class="mr-auto" {disabled} {onFileUpload} />
{#if currentConfig.modelSelectorEnabled}
<ChatFormModelSelector class="shrink-0" />
{/if}
{#if isLoading}
<Button
type="button"
onclick={onStop}
class="h-8 w-8 bg-transparent p-0 hover:bg-destructive/20"
>
<span class="sr-only">Stop</span>
<Square class="h-8 w-8 fill-destructive stroke-destructive" />
</Button>
{:else}
<ChatFormActionRecord {disabled} {isLoading} {isRecording} {onMicClick} />
<Button
type="submit"
disabled={!canSend || disabled || isLoading}
class="h-8 w-8 rounded-full p-0"
>
<span class="sr-only">Send</span>
<ArrowUp class="h-12 w-12" />
</Button>
{/if}
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatForm/ChatFormModelSelector.svelte
+1 -7
View File
@@ -72,12 +72,6 @@
}
}
function handleScroll() {
if (isOpen) {
updateMenuPosition();
}
}
async function handleSelect(value: string | undefined) {
if (!value) return;
@@ -259,7 +253,7 @@
}
</script>
<svelte:window onresize={handleResize} onscroll={handleScroll} />
<svelte:window onresize={handleResize} />
<svelte:document onpointerdown={handlePointerDown} onkeydown={handleKeydown} />
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatMessages/ChatMessage.svelte
+48 -1
View File
@@ -2,6 +2,7 @@
import { getDeletionInfo } from '$lib/stores/chat.svelte';
import { copyToClipboard } from '$lib/utils/copy';
import { isIMEComposing } from '$lib/utils/is-ime-composing';
import type { ApiChatCompletionToolCall } from '$lib/types/api';
import ChatMessageAssistant from './ChatMessageAssistant.svelte';
import ChatMessageUser from './ChatMessageUser.svelte';
@@ -9,6 +10,7 @@
class?: string;
message: DatabaseMessage;
onCopy?: (message: DatabaseMessage) => void;
onContinueAssistantMessage?: (message: DatabaseMessage) => void;
onDelete?: (message: DatabaseMessage) => void;
onEditWithBranching?: (message: DatabaseMessage, newContent: string) => void;
onEditWithReplacement?: (
@@ -16,6 +18,7 @@
newContent: string,
shouldBranch: boolean
) => void;
onEditUserMessagePreserveResponses?: (message: DatabaseMessage, newContent: string) => void;
onNavigateToSibling?: (siblingId: string) => void;
onRegenerateWithBranching?: (message: DatabaseMessage) => void;
siblingInfo?: ChatMessageSiblingInfo | null;
@@ -25,9 +28,11 @@
class: className = '',
message,
onCopy,
onContinueAssistantMessage,
onDelete,
onEditWithBranching,
onEditWithReplacement,
onEditUserMessagePreserveResponses,
onNavigateToSibling,
onRegenerateWithBranching,
siblingInfo = null
@@ -54,6 +59,29 @@
return null;
});
let toolCallContent = $derived.by((): ApiChatCompletionToolCall[] | string | null => {
if (message.role === 'assistant') {
const trimmedToolCalls = message.toolCalls?.trim();
if (!trimmedToolCalls) {
return null;
}
try {
const parsed = JSON.parse(trimmedToolCalls);
if (Array.isArray(parsed)) {
return parsed as ApiChatCompletionToolCall[];
}
} catch {
// Harmony-only path: fall back to the raw string so issues surface visibly.
}
return trimmedToolCalls;
}
return null;
});
function handleCancelEdit() {
isEditing = false;
editedContent = message.content;
@@ -109,17 +137,33 @@
onRegenerateWithBranching?.(message);
}
function handleContinue() {
onContinueAssistantMessage?.(message);
}
function handleSaveEdit() {
if (message.role === 'user') {
// For user messages, trim to avoid accidental whitespace
onEditWithBranching?.(message, editedContent.trim());
} else {
onEditWithReplacement?.(message, editedContent.trim(), shouldBranchAfterEdit);
// For assistant messages, preserve exact content including trailing whitespace
// This is important for the Continue feature to work properly
onEditWithReplacement?.(message, editedContent, shouldBranchAfterEdit);
}
isEditing = false;
shouldBranchAfterEdit = false;
}
function handleSaveEditOnly() {
if (message.role === 'user') {
// For user messages, trim to avoid accidental whitespace
onEditUserMessagePreserveResponses?.(message, editedContent.trim());
}
isEditing = false;
}
function handleShowDeleteDialogChange(show: boolean) {
showDeleteDialog = show;
}
@@ -142,6 +186,7 @@
onEditedContentChange={handleEditedContentChange}
{onNavigateToSibling}
onSaveEdit={handleSaveEdit}
onSaveEditOnly={handleSaveEditOnly}
onShowDeleteDialogChange={handleShowDeleteDialogChange}
{showDeleteDialog}
{siblingInfo}
@@ -157,6 +202,7 @@
messageContent={message.content}
onCancelEdit={handleCancelEdit}
onConfirmDelete={handleConfirmDelete}
onContinue={handleContinue}
onCopy={handleCopy}
onDelete={handleDelete}
onEdit={handleEdit}
@@ -171,5 +217,6 @@
{showDeleteDialog}
{siblingInfo}
{thinkingContent}
{toolCallContent}
/>
{/if}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatMessages/ChatMessageActions.svelte
+13 -4
View File
@@ -1,7 +1,10 @@
<script lang="ts">
import { Edit, Copy, RefreshCw, Trash2 } from '@lucide/svelte';
import { ActionButton, ConfirmationDialog } from '$lib/components/app';
import ChatMessageBranchingControls from './ChatMessageBranchingControls.svelte';
import { Edit, Copy, RefreshCw, Trash2, ArrowRight } from '@lucide/svelte';
import {
ActionButton,
ChatMessageBranchingControls,
DialogConfirmation
} from '$lib/components/app';
interface Props {
role: 'user' | 'assistant';
@@ -18,6 +21,7 @@
onCopy: () => void;
onEdit?: () => void;
onRegenerate?: () => void;
onContinue?: () => void;
onDelete: () => void;
onConfirmDelete: () => void;
onNavigateToSibling?: (siblingId: string) => void;
@@ -31,6 +35,7 @@
onCopy,
onEdit,
onConfirmDelete,
onContinue,
onDelete,
onNavigateToSibling,
onShowDeleteDialogChange,
@@ -69,12 +74,16 @@
<ActionButton icon={RefreshCw} tooltip="Regenerate" onclick={onRegenerate} />
{/if}
{#if role === 'assistant' && onContinue}
<ActionButton icon={ArrowRight} tooltip="Continue" onclick={onContinue} />
{/if}
<ActionButton icon={Trash2} tooltip="Delete" onclick={onDelete} />
</div>
</div>
</div>
<ConfirmationDialog
<DialogConfirmation
bind:open={showDeleteDialog}
title="Delete Message"
description={deletionInfo && deletionInfo.totalCount > 1
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatMessages/ChatMessageAssistant.svelte
+193 -18
View File
@@ -2,8 +2,19 @@
import { ChatMessageThinkingBlock, MarkdownContent } from '$lib/components/app';
import { useProcessingState } from '$lib/hooks/use-processing-state.svelte';
import { isLoading } from '$lib/stores/chat.svelte';
import autoResizeTextarea from '$lib/utils/autoresize-textarea';
import { fade } from 'svelte/transition';
import { Check, Copy, Package, X } from '@lucide/svelte';
import {
Check,
Copy,
Package,
X,
Gauge,
Clock,
WholeWord,
ChartNoAxesColumn,
Wrench
} from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import { Checkbox } from '$lib/components/ui/checkbox';
import { INPUT_CLASSES } from '$lib/constants/input-classes';
@@ -12,6 +23,7 @@
import { config } from '$lib/stores/settings.svelte';
import { modelName as serverModelName } from '$lib/stores/server.svelte';
import { copyToClipboard } from '$lib/utils/copy';
import type { ApiChatCompletionToolCall } from '$lib/types/api';
interface Props {
class?: string;
@@ -28,6 +40,7 @@
onCancelEdit?: () => void;
onCopy: () => void;
onConfirmDelete: () => void;
onContinue?: () => void;
onDelete: () => void;
onEdit?: () => void;
onEditKeydown?: (event: KeyboardEvent) => void;
@@ -42,6 +55,7 @@
siblingInfo?: ChatMessageSiblingInfo | null;
textareaElement?: HTMLTextAreaElement;
thinkingContent: string | null;
toolCallContent: ApiChatCompletionToolCall[] | string | null;
}
let {
@@ -53,6 +67,7 @@
messageContent,
onCancelEdit,
onConfirmDelete,
onContinue,
onCopy,
onDelete,
onEdit,
@@ -67,17 +82,23 @@
shouldBranchAfterEdit = false,
siblingInfo = null,
textareaElement = $bindable(),
thinkingContent
thinkingContent,
toolCallContent = null
}: Props = $props();
const toolCalls = $derived(
Array.isArray(toolCallContent) ? (toolCallContent as ApiChatCompletionToolCall[]) : null
);
const fallbackToolCalls = $derived(typeof toolCallContent === 'string' ? toolCallContent : null);
const processingState = useProcessingState();
let currentConfig = $derived(config());
let serverModel = $derived(serverModelName());
let displayedModel = $derived((): string | null => {
if (!currentConfig.showModelInfo) return null;
if (currentConfig.modelSelectorEnabled) {
return message.model ?? null;
if (message.model) {
return message.model;
}
return serverModel;
@@ -88,6 +109,64 @@
void copyToClipboard(model ?? '');
}
$effect(() => {
if (isEditing && textareaElement) {
autoResizeTextarea(textareaElement);
}
});
function formatToolCallBadge(toolCall: ApiChatCompletionToolCall, index: number) {
const callNumber = index + 1;
const functionName = toolCall.function?.name?.trim();
const label = functionName || `Call #${callNumber}`;
const payload: Record<string, unknown> = {};
const id = toolCall.id?.trim();
if (id) {
payload.id = id;
}
const type = toolCall.type?.trim();
if (type) {
payload.type = type;
}
if (toolCall.function) {
const fnPayload: Record<string, unknown> = {};
const name = toolCall.function.name?.trim();
if (name) {
fnPayload.name = name;
}
const rawArguments = toolCall.function.arguments?.trim();
if (rawArguments) {
try {
fnPayload.arguments = JSON.parse(rawArguments);
} catch {
fnPayload.arguments = rawArguments;
}
}
if (Object.keys(fnPayload).length > 0) {
payload.function = fnPayload;
}
}
const formattedPayload = JSON.stringify(payload, null, 2);
return {
label,
tooltip: formattedPayload,
copyValue: formattedPayload
};
}
function handleCopyToolCall(payload: string) {
void copyToClipboard(payload, 'Tool call copied to clipboard');
}
</script>
<div
@@ -120,7 +199,10 @@
bind:value={editedContent}
class="min-h-[50vh] w-full resize-y rounded-2xl px-3 py-2 text-sm {INPUT_CLASSES}"
onkeydown={onEditKeydown}
oninput={(e) => onEditedContentChange?.(e.currentTarget.value)}
oninput={(e) => {
autoResizeTextarea(e.currentTarget);
onEditedContentChange?.(e.currentTarget.value);
}}
placeholder="Edit assistant message..."
></textarea>
@@ -160,22 +242,99 @@
</div>
{/if}
{#if displayedModel()}
<span class="mt-6 mb-4 inline-flex items-center gap-1 text-xs text-muted-foreground">
<Package class="h-3.5 w-3.5" />
<div class="info my-6 grid gap-4">
{#if displayedModel()}
<span class="inline-flex items-center gap-2 text-xs text-muted-foreground">
<span class="inline-flex items-center gap-1">
<Package class="h-3.5 w-3.5" />
<span>Model used:</span>
<span>Model used:</span>
</span>
<button
class="inline-flex cursor-pointer items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
onclick={handleCopyModel}
>
{displayedModel()}
<button
class="inline-flex cursor-pointer items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
onclick={handleCopyModel}
>
{displayedModel()}
<Copy class="ml-1 h-3 w-3 " />
</button>
</span>
{/if}
<Copy class="ml-1 h-3 w-3 " />
</button>
</span>
{/if}
{#if config().showToolCalls}
{#if (toolCalls && toolCalls.length > 0) || fallbackToolCalls}
<span class="inline-flex flex-wrap items-center gap-2 text-xs text-muted-foreground">
<span class="inline-flex items-center gap-1">
<Wrench class="h-3.5 w-3.5" />
<span>Tool calls:</span>
</span>
{#if toolCalls && toolCalls.length > 0}
{#each toolCalls as toolCall, index (toolCall.id ?? `${index}`)}
{@const badge = formatToolCallBadge(toolCall, index)}
<button
type="button"
class="tool-call-badge inline-flex cursor-pointer items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
title={badge.tooltip}
aria-label={`Copy tool call ${badge.label}`}
onclick={() => handleCopyToolCall(badge.copyValue)}
>
{badge.label}
<Copy class="ml-1 h-3 w-3" />
</button>
{/each}
{:else if fallbackToolCalls}
<button
type="button"
class="tool-call-badge tool-call-badge--fallback inline-flex cursor-pointer items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
title={fallbackToolCalls}
aria-label="Copy tool call payload"
onclick={() => handleCopyToolCall(fallbackToolCalls)}
>
{fallbackToolCalls}
<Copy class="ml-1 h-3 w-3" />
</button>
{/if}
</span>
{/if}
{/if}
{#if currentConfig.showMessageStats && message.timings && message.timings.predicted_n && message.timings.predicted_ms}
{@const tokensPerSecond = (message.timings.predicted_n / message.timings.predicted_ms) * 1000}
<span class="inline-flex items-center gap-2 text-xs text-muted-foreground">
<span class="inline-flex items-center gap-1">
<ChartNoAxesColumn class="h-3.5 w-3.5" />
<span>Statistics:</span>
</span>
<div class="inline-flex flex-wrap items-center gap-2 text-xs text-muted-foreground">
<span
class="inline-flex items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
>
<Gauge class="h-3 w-3" />
{tokensPerSecond.toFixed(2)} tokens/s
</span>
<span
class="inline-flex items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
>
<WholeWord class="h-3 w-3" />
{message.timings.predicted_n} tokens
</span>
<span
class="inline-flex items-center gap-1 rounded-sm bg-muted-foreground/15 px-1.5 py-0.75"
>
<Clock class="h-3 w-3" />
{(message.timings.predicted_ms / 1000).toFixed(2)}s
</span>
</div>
</span>
{/if}
</div>
{#if message.timestamp && !isEditing}
<ChatMessageActions
@@ -188,6 +347,9 @@
{onCopy}
{onEdit}
{onRegenerate}
onContinue={currentConfig.enableContinueGeneration && !thinkingContent
? onContinue
: undefined}
{onDelete}
{onConfirmDelete}
{onNavigateToSibling}
@@ -242,4 +404,17 @@
white-space: pre-wrap;
word-break: break-word;
}
.tool-call-badge {
max-width: 12rem;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.tool-call-badge--fallback {
max-width: 20rem;
white-space: normal;
word-break: break-word;
}
</style>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatMessages/ChatMessageUser.svelte
+29 -6
View File
@@ -1,10 +1,11 @@
<script lang="ts">
import { Check, X } from '@lucide/svelte';
import { Check, X, Send } from '@lucide/svelte';
import { Card } from '$lib/components/ui/card';
import { Button } from '$lib/components/ui/button';
import { ChatAttachmentsList, MarkdownContent } from '$lib/components/app';
import { INPUT_CLASSES } from '$lib/constants/input-classes';
import { config } from '$lib/stores/settings.svelte';
import autoResizeTextarea from '$lib/utils/autoresize-textarea';
import ChatMessageActions from './ChatMessageActions.svelte';
interface Props {
@@ -22,6 +23,7 @@
} | null;
onCancelEdit: () => void;
onSaveEdit: () => void;
onSaveEditOnly?: () => void;
onEditKeydown: (event: KeyboardEvent) => void;
onEditedContentChange: (content: string) => void;
onCopy: () => void;
@@ -43,6 +45,7 @@
deletionInfo,
onCancelEdit,
onSaveEdit,
onSaveEditOnly,
onEditKeydown,
onEditedContentChange,
onCopy,
@@ -58,6 +61,12 @@
let messageElement: HTMLElement | undefined = $state();
const currentConfig = config();
$effect(() => {
if (isEditing && textareaElement) {
autoResizeTextarea(textareaElement);
}
});
$effect(() => {
if (!messageElement || !message.content.trim()) return;
@@ -95,20 +104,34 @@
bind:value={editedContent}
class="min-h-[60px] w-full resize-none rounded-2xl px-3 py-2 text-sm {INPUT_CLASSES}"
onkeydown={onEditKeydown}
oninput={(e) => onEditedContentChange(e.currentTarget.value)}
oninput={(e) => {
autoResizeTextarea(e.currentTarget);
onEditedContentChange(e.currentTarget.value);
}}
placeholder="Edit your message..."
></textarea>
<div class="mt-2 flex justify-end gap-2">
<Button class="h-8 px-3" onclick={onCancelEdit} size="sm" variant="outline">
<Button class="h-8 px-3" onclick={onCancelEdit} size="sm" variant="ghost">
<X class="mr-1 h-3 w-3" />
Cancel
</Button>
<Button class="h-8 px-3" onclick={onSaveEdit} disabled={!editedContent.trim()} size="sm">
<Check class="mr-1 h-3 w-3" />
{#if onSaveEditOnly}
<Button
class="h-8 px-3"
onclick={onSaveEditOnly}
disabled={!editedContent.trim()}
size="sm"
variant="outline"
>
<Check class="mr-1 h-3 w-3" />
Save
</Button>
{/if}
<Button class="h-8 px-3" onclick={onSaveEdit} disabled={!editedContent.trim()} size="sm">
<Send class="mr-1 h-3 w-3" />
Send
</Button>
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatMessages/ChatMessages.svelte
+26 -2
View File
@@ -3,10 +3,12 @@
import { DatabaseStore } from '$lib/stores/database';
import {
activeConversation,
continueAssistantMessage,
deleteMessage,
navigateToSibling,
editMessageWithBranching,
editAssistantMessage,
editMessageWithBranching,
editUserMessagePreserveResponses,
navigateToSibling,
regenerateMessageWithBranching
} from '$lib/stores/chat.svelte';
import { getMessageSiblings } from '$lib/utils/branching';
@@ -93,6 +95,26 @@
refreshAllMessages();
}
async function handleContinueAssistantMessage(message: DatabaseMessage) {
onUserAction?.();
await continueAssistantMessage(message.id);
refreshAllMessages();
}
async function handleEditUserMessagePreserveResponses(
message: DatabaseMessage,
newContent: string
) {
onUserAction?.();
await editUserMessagePreserveResponses(message.id, newContent);
refreshAllMessages();
}
async function handleDeleteMessage(message: DatabaseMessage) {
await deleteMessage(message.id);
@@ -110,7 +132,9 @@
onNavigateToSibling={handleNavigateToSibling}
onEditWithBranching={handleEditWithBranching}
onEditWithReplacement={handleEditWithReplacement}
onEditUserMessagePreserveResponses={handleEditUserMessagePreserveResponses}
onRegenerateWithBranching={handleRegenerateWithBranching}
onContinueAssistantMessage={handleContinueAssistantMessage}
/>
{/each}
</div>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte
+46 -27
View File
@@ -5,13 +5,13 @@
ChatScreenHeader,
ChatScreenWarning,
ChatMessages,
ChatProcessingInfo,
EmptyFileAlertDialog,
ChatErrorDialog,
ChatScreenProcessingInfo,
DialogEmptyFileAlert,
DialogChatError,
ServerErrorSplash,
ServerInfo,
ServerLoadingSplash,
ConfirmationDialog
DialogConfirmation
} from '$lib/components/app';
import * as AlertDialog from '$lib/components/ui/alert-dialog';
import {
@@ -29,6 +29,7 @@
sendMessage,
stopGeneration
} from '$lib/stores/chat.svelte';
import { config } from '$lib/stores/settings.svelte';
import {
supportsVision,
supportsAudio,
@@ -47,6 +48,7 @@
let { showCenteredEmpty = false } = $props();
let disableAutoScroll = $derived(Boolean(config().disableAutoScroll));
let autoScrollEnabled = $state(true);
let chatScrollContainer: HTMLDivElement | undefined = $state();
let dragCounter = $state(0);
@@ -149,7 +151,7 @@
}
function handleScroll() {
if (!chatScrollContainer) return;
if (disableAutoScroll || !chatScrollContainer) return;
const { scrollTop, scrollHeight, clientHeight } = chatScrollContainer;
const distanceFromBottom = scrollHeight - scrollTop - clientHeight;
@@ -194,8 +196,10 @@
const extras = result?.extras;
// Enable autoscroll for user-initiated message sending
userScrolledUp = false;
autoScrollEnabled = true;
if (!disableAutoScroll) {
userScrolledUp = false;
autoScrollEnabled = true;
}
await sendMessage(message, extras);
scrollChatToBottom();
@@ -241,6 +245,8 @@
}
function scrollChatToBottom(behavior: ScrollBehavior = 'smooth') {
if (disableAutoScroll) return;
chatScrollContainer?.scrollTo({
top: chatScrollContainer?.scrollHeight,
behavior
@@ -248,14 +254,27 @@
}
afterNavigate(() => {
setTimeout(() => scrollChatToBottom('instant'), INITIAL_SCROLL_DELAY);
if (!disableAutoScroll) {
setTimeout(() => scrollChatToBottom('instant'), INITIAL_SCROLL_DELAY);
}
});
onMount(() => {
setTimeout(() => scrollChatToBottom('instant'), INITIAL_SCROLL_DELAY);
if (!disableAutoScroll) {
setTimeout(() => scrollChatToBottom('instant'), INITIAL_SCROLL_DELAY);
}
});
$effect(() => {
if (disableAutoScroll) {
autoScrollEnabled = false;
if (scrollInterval) {
clearInterval(scrollInterval);
scrollInterval = undefined;
}
return;
}
if (isCurrentConversationLoading && autoScrollEnabled) {
scrollInterval = setInterval(scrollChatToBottom, AUTO_SCROLL_INTERVAL);
} else if (scrollInterval) {
@@ -289,9 +308,11 @@
class="mb-16 md:mb-24"
messages={activeMessages()}
onUserAction={() => {
userScrolledUp = false;
autoScrollEnabled = true;
scrollChatToBottom();
if (!disableAutoScroll) {
userScrolledUp = false;
autoScrollEnabled = true;
scrollChatToBottom();
}
}}
/>
@@ -299,7 +320,7 @@
class="pointer-events-none sticky right-0 bottom-0 left-0 mt-auto"
in:slide={{ duration: 150, axis: 'y' }}
>
<ChatProcessingInfo />
<ChatScreenProcessingInfo />
{#if serverWarning()}
<ChatScreenWarning class="pointer-events-auto mx-auto max-w-[48rem] px-4" />
@@ -333,7 +354,7 @@
ondrop={handleDrop}
role="main"
>
<div class="w-full max-w-2xl px-4">
<div class="w-full max-w-[48rem] px-4">
<div class="mb-8 text-center" in:fade={{ duration: 300 }}>
<h1 class="mb-2 text-3xl font-semibold tracking-tight">llama.cpp</h1>
@@ -368,7 +389,7 @@
<AlertDialog.Portal>
<AlertDialog.Overlay />
<AlertDialog.Content class="max-w-md">
<AlertDialog.Content class="flex max-w-md flex-col">
<AlertDialog.Header>
<AlertDialog.Title>File Upload Error</AlertDialog.Title>
@@ -377,7 +398,7 @@
</AlertDialog.Description>
</AlertDialog.Header>
<div class="space-y-4">
<div class="!max-h-[50vh] min-h-0 flex-1 space-y-4 overflow-y-auto">
{#if fileErrorData.generallyUnsupported.length > 0}
<div class="space-y-2">
<h4 class="text-sm font-medium text-destructive">Unsupported File Types</h4>
@@ -398,8 +419,6 @@
{#if fileErrorData.modalityUnsupported.length > 0}
<div class="space-y-2">
<h4 class="text-sm font-medium text-destructive">Model Compatibility Issues</h4>
<div class="space-y-1">
{#each fileErrorData.modalityUnsupported as file (file.name)}
<div class="rounded-md bg-destructive/10 px-3 py-2">
@@ -415,14 +434,14 @@
</div>
</div>
{/if}
</div>
<div class="rounded-md bg-muted/50 p-3">
<h4 class="mb-2 text-sm font-medium">This model supports:</h4>
<div class="rounded-md bg-muted/50 p-3">
<h4 class="mb-2 text-sm font-medium">This model supports:</h4>
<p class="text-sm text-muted-foreground">
{fileErrorData.supportedTypes.join(', ')}
</p>
</div>
<p class="text-sm text-muted-foreground">
{fileErrorData.supportedTypes.join(', ')}
</p>
</div>
<AlertDialog.Footer>
@@ -434,7 +453,7 @@
</AlertDialog.Portal>
</AlertDialog.Root>
<ConfirmationDialog
<DialogConfirmation
bind:open={showDeleteDialog}
title="Delete Conversation"
description="Are you sure you want to delete this conversation? This action cannot be undone and will permanently remove all messages in this conversation."
@@ -446,7 +465,7 @@
onCancel={() => (showDeleteDialog = false)}
/>
<EmptyFileAlertDialog
<DialogEmptyFileAlert
bind:open={showEmptyFileDialog}
emptyFiles={emptyFileNames}
onOpenChange={(open) => {
@@ -456,7 +475,7 @@
}}
/>
<ChatErrorDialog
<DialogChatError
message={activeErrorDialog?.message ?? ''}
onOpenChange={handleErrorDialogOpenChange}
open={Boolean(activeErrorDialog)}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatScreen/ChatScreenHeader.svelte
+2 -2
View File
@@ -1,6 +1,6 @@
<script lang="ts">
import { Settings } from '@lucide/svelte';
import { ChatSettingsDialog } from '$lib/components/app';
import { DialogChatSettings } from '$lib/components/app';
import { Button } from '$lib/components/ui/button';
let settingsOpen = $state(false);
@@ -20,4 +20,4 @@
</div>
</header>
<ChatSettingsDialog open={settingsOpen} onOpenChange={(open) => (settingsOpen = open)} />
<DialogChatSettings open={settingsOpen} onOpenChange={(open) => (settingsOpen = open)} />
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatProcessingInfo.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatScreen/ChatScreenProcessingInfo.svelte
+2 -4
View File
@@ -76,10 +76,10 @@
});
</script>
<div class="chat-processing-info-container" class:visible={showSlotsInfo}>
<div class="chat-processing-info-container pointer-events-none" class:visible={showSlotsInfo}>
<div class="chat-processing-info-content">
{#each processingDetails as detail (detail)}
<span class="chat-processing-info-detail">{detail}</span>
<span class="chat-processing-info-detail pointer-events-auto">{detail}</span>
{/each}
</div>
</div>
@@ -92,7 +92,6 @@
padding: 1.5rem 1rem;
opacity: 0;
transform: translateY(50%);
pointer-events: none;
transition:
opacity 300ms ease-out,
transform 300ms ease-out;
@@ -100,7 +99,6 @@
.chat-processing-info-container.visible {
opacity: 1;
pointer-events: auto;
transform: translateY(0);
}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ChatSettingsDialog.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ChatSettings.svelte
+154 -159
View File
@@ -3,7 +3,6 @@
Settings,
Funnel,
AlertTriangle,
Brain,
Code,
Monitor,
Sun,
@@ -12,20 +11,21 @@
ChevronRight,
Database
} from '@lucide/svelte';
import { ChatSettingsFooter, ChatSettingsFields } from '$lib/components/app';
import ImportExportTab from './ImportExportTab.svelte';
import * as Dialog from '$lib/components/ui/dialog';
import {
ChatSettingsFooter,
ChatSettingsImportExportTab,
ChatSettingsFields
} from '$lib/components/app';
import { ScrollArea } from '$lib/components/ui/scroll-area';
import { config, updateMultipleConfig } from '$lib/stores/settings.svelte';
import { setMode } from 'mode-watcher';
import type { Component } from 'svelte';
interface Props {
onOpenChange?: (open: boolean) => void;
open?: boolean;
onSave?: () => void;
}
let { onOpenChange, open = false }: Props = $props();
let { onSave }: Props = $props();
const settingSections: Array<{
fields: SettingsFieldConfig[];
@@ -52,6 +52,43 @@
{ value: 'dark', label: 'Dark', icon: Moon }
]
},
{
key: 'pasteLongTextToFileLen',
label: 'Paste long text to file length',
type: 'input'
},
{
key: 'enableContinueGeneration',
label: 'Enable "Continue" button',
type: 'checkbox',
isExperimental: true
},
{
key: 'pdfAsImage',
label: 'Parse PDF as image',
type: 'checkbox'
},
{
key: 'askForTitleConfirmation',
label: 'Ask for confirmation before changing conversation title',
type: 'checkbox'
}
]
},
{
title: 'Display',
icon: Monitor,
fields: [
{
key: 'showThoughtInProgress',
label: 'Show thought in progress',
type: 'checkbox'
},
{
key: 'showMessageStats',
label: 'Show message generation statistics',
type: 'checkbox'
},
{
key: 'showTokensPerSecond',
label: 'Show tokens per second',
@@ -62,26 +99,16 @@
label: 'Keep stats visible after generation',
type: 'checkbox'
},
{
key: 'askForTitleConfirmation',
label: 'Ask for confirmation before changing conversation title',
type: 'checkbox'
},
{
key: 'pasteLongTextToFileLen',
label: 'Paste long text to file length',
type: 'input'
},
{
key: 'pdfAsImage',
label: 'Parse PDF as image',
type: 'checkbox'
},
{
key: 'showModelInfo',
label: 'Show model information',
type: 'checkbox'
},
{
key: 'disableAutoScroll',
label: 'Disable automatic scroll',
type: 'checkbox'
},
{
key: 'renderUserContentAsMarkdown',
label: 'Render user content as Markdown',
@@ -196,17 +223,6 @@
}
]
},
{
title: 'Reasoning',
icon: Brain,
fields: [
{
key: 'showThoughtInProgress',
label: 'Show thought in progress',
type: 'checkbox'
}
]
},
{
title: 'Import/Export',
icon: Database,
@@ -221,6 +237,11 @@
label: 'Enable model selector',
type: 'checkbox'
},
{
key: 'showToolCalls',
label: 'Show tool call labels',
type: 'checkbox'
},
{
key: 'disableReasoningFormat',
label: 'Show raw LLM output',
@@ -253,7 +274,6 @@
settingSections.find((section) => section.title === activeSection) || settingSections[0]
);
let localConfig: SettingsConfigType = $state({ ...config() });
let originalTheme: string = $state('');
let canScrollLeft = $state(false);
let canScrollRight = $state(false);
@@ -269,18 +289,10 @@
localConfig[key] = value;
}
function handleClose() {
if (localConfig.theme !== originalTheme) {
setMode(originalTheme as 'light' | 'dark' | 'system');
}
onOpenChange?.(false);
}
function handleReset() {
localConfig = { ...config() };
setMode(localConfig.theme as 'light' | 'dark' | 'system');
originalTheme = localConfig.theme as string;
}
function handleSave() {
@@ -331,7 +343,7 @@
}
updateMultipleConfig(processedConfig);
onOpenChange?.(false);
onSave?.();
}
function scrollToCenter(element: HTMLElement) {
@@ -367,14 +379,11 @@
canScrollRight = scrollLeft < scrollWidth - clientWidth - 1; // -1 for rounding
}
$effect(() => {
if (open) {
localConfig = { ...config() };
originalTheme = config().theme as string;
export function reset() {
localConfig = { ...config() };
setTimeout(updateScrollButtons, 100);
}
});
setTimeout(updateScrollButtons, 100);
}
$effect(() => {
if (scrollContainer) {
@@ -383,120 +392,106 @@
});
</script>
<Dialog.Root {open} onOpenChange={handleClose}>
<Dialog.Content
class="z-999999 flex h-[100dvh] max-h-[100dvh] min-h-[100dvh] flex-col gap-0 rounded-none p-0
md:h-[64vh] md:max-h-[64vh] md:min-h-0 md:rounded-lg"
style="max-width: 48rem;"
>
<div class="flex flex-1 flex-col overflow-hidden md:flex-row">
<!-- Desktop Sidebar -->
<div class="hidden w-64 border-r border-border/30 p-6 md:block">
<nav class="space-y-1 py-2">
<Dialog.Title class="mb-6 flex items-center gap-2">Settings</Dialog.Title>
<div class="flex h-full flex-col overflow-hidden md:flex-row">
<!-- Desktop Sidebar -->
<div class="hidden w-64 border-r border-border/30 p-6 md:block">
<nav class="space-y-1 py-2">
{#each settingSections as section (section.title)}
<button
class="flex w-full cursor-pointer items-center gap-3 rounded-lg px-3 py-2 text-left text-sm transition-colors hover:bg-accent {activeSection ===
section.title
? 'bg-accent text-accent-foreground'
: 'text-muted-foreground'}"
onclick={() => (activeSection = section.title)}
>
<section.icon class="h-4 w-4" />
{#each settingSections as section (section.title)}
<button
class="flex w-full cursor-pointer items-center gap-3 rounded-lg px-3 py-2 text-left text-sm transition-colors hover:bg-accent {activeSection ===
section.title
? 'bg-accent text-accent-foreground'
: 'text-muted-foreground'}"
onclick={() => (activeSection = section.title)}
>
<section.icon class="h-4 w-4" />
<span class="ml-2">{section.title}</span>
</button>
{/each}
</nav>
</div>
<span class="ml-2">{section.title}</span>
</button>
{/each}
</nav>
</div>
<!-- Mobile Header with Horizontal Scrollable Menu -->
<div class="flex flex-col md:hidden">
<div class="border-b border-border/30 py-4">
<!-- Horizontal Scrollable Category Menu with Navigation -->
<div class="relative flex items-center" style="scroll-padding: 1rem;">
<button
class="absolute left-2 z-10 flex h-6 w-6 items-center justify-center rounded-full bg-muted shadow-md backdrop-blur-sm transition-opacity hover:bg-accent {canScrollLeft
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollLeft}
aria-label="Scroll left"
>
<ChevronLeft class="h-4 w-4" />
</button>
<!-- Mobile Header with Horizontal Scrollable Menu -->
<div class="flex flex-col md:hidden">
<div class="border-b border-border/30 py-4">
<Dialog.Title class="mb-6 flex items-center gap-2 px-4">Settings</Dialog.Title>
<!-- Horizontal Scrollable Category Menu with Navigation -->
<div class="relative flex items-center" style="scroll-padding: 1rem;">
<button
class="absolute left-2 z-10 flex h-6 w-6 items-center justify-center rounded-full bg-muted shadow-md backdrop-blur-sm transition-opacity hover:bg-accent {canScrollLeft
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollLeft}
aria-label="Scroll left"
>
<ChevronLeft class="h-4 w-4" />
</button>
<div
class="scrollbar-hide overflow-x-auto py-2"
bind:this={scrollContainer}
onscroll={updateScrollButtons}
>
<div class="flex min-w-max gap-2">
{#each settingSections as section (section.title)}
<button
class="flex cursor-pointer items-center gap-2 rounded-lg px-3 py-2 text-sm whitespace-nowrap transition-colors first:ml-4 last:mr-4 hover:bg-accent {activeSection ===
section.title
? 'bg-accent text-accent-foreground'
: 'text-muted-foreground'}"
onclick={(e: MouseEvent) => {
activeSection = section.title;
scrollToCenter(e.currentTarget as HTMLElement);
}}
>
<section.icon class="h-4 w-4 flex-shrink-0" />
<span>{section.title}</span>
</button>
{/each}
</div>
</div>
<button
class="absolute right-2 z-10 flex h-6 w-6 items-center justify-center rounded-full bg-muted shadow-md backdrop-blur-sm transition-opacity hover:bg-accent {canScrollRight
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollRight}
aria-label="Scroll right"
>
<ChevronRight class="h-4 w-4" />
</button>
<div
class="scrollbar-hide overflow-x-auto py-2"
bind:this={scrollContainer}
onscroll={updateScrollButtons}
>
<div class="flex min-w-max gap-2">
{#each settingSections as section (section.title)}
<button
class="flex cursor-pointer items-center gap-2 rounded-lg px-3 py-2 text-sm whitespace-nowrap transition-colors first:ml-4 last:mr-4 hover:bg-accent {activeSection ===
section.title
? 'bg-accent text-accent-foreground'
: 'text-muted-foreground'}"
onclick={(e: MouseEvent) => {
activeSection = section.title;
scrollToCenter(e.currentTarget as HTMLElement);
}}
>
<section.icon class="h-4 w-4 flex-shrink-0" />
<span>{section.title}</span>
</button>
{/each}
</div>
</div>
<button
class="absolute right-2 z-10 flex h-6 w-6 items-center justify-center rounded-full bg-muted shadow-md backdrop-blur-sm transition-opacity hover:bg-accent {canScrollRight
? 'opacity-100'
: 'pointer-events-none opacity-0'}"
onclick={scrollRight}
aria-label="Scroll right"
>
<ChevronRight class="h-4 w-4" />
</button>
</div>
<ScrollArea class="max-h-[calc(100dvh-13.5rem)] flex-1 md:max-h-[calc(100vh-13.5rem)]">
<div class="space-y-6 p-4 md:p-6">
<div class="grid">
<div class="mb-6 flex hidden items-center gap-2 border-b border-border/30 pb-6 md:flex">
<currentSection.icon class="h-5 w-5" />
<h3 class="text-lg font-semibold">{currentSection.title}</h3>
</div>
{#if currentSection.title === 'Import/Export'}
<ImportExportTab />
{:else}
<div class="space-y-6">
<ChatSettingsFields
fields={currentSection.fields}
{localConfig}
onConfigChange={handleConfigChange}
onThemeChange={handleThemeChange}
/>
</div>
{/if}
</div>
<div class="mt-8 border-t pt-6">
<p class="text-xs text-muted-foreground">
Settings are saved in browser's localStorage
</p>
</div>
</div>
</ScrollArea>
</div>
</div>
<ChatSettingsFooter onReset={handleReset} onSave={handleSave} />
</Dialog.Content>
</Dialog.Root>
<ScrollArea class="max-h-[calc(100dvh-13.5rem)] flex-1 md:max-h-[calc(100vh-13.5rem)]">
<div class="space-y-6 p-4 md:p-6">
<div class="grid">
<div class="mb-6 flex hidden items-center gap-2 border-b border-border/30 pb-6 md:flex">
<currentSection.icon class="h-5 w-5" />
<h3 class="text-lg font-semibold">{currentSection.title}</h3>
</div>
{#if currentSection.title === 'Import/Export'}
<ChatSettingsImportExportTab />
{:else}
<div class="space-y-6">
<ChatSettingsFields
fields={currentSection.fields}
{localConfig}
onConfigChange={handleConfigChange}
onThemeChange={handleThemeChange}
/>
</div>
{/if}
</div>
<div class="mt-8 border-t pt-6">
<p class="text-xs text-muted-foreground">Settings are saved in browser's localStorage</p>
</div>
</div>
</ScrollArea>
</div>
<ChatSettingsFooter onReset={handleReset} onSave={handleSave} />
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ChatSettingsFields.svelte
+24 -8
View File
@@ -1,5 +1,5 @@
<script lang="ts">
import { RotateCcw } from '@lucide/svelte';
import { RotateCcw, FlaskConical } from '@lucide/svelte';
import { Checkbox } from '$lib/components/ui/checkbox';
import { Input } from '$lib/components/ui/input';
import Label from '$lib/components/ui/label/label.svelte';
@@ -9,7 +9,7 @@
import { supportsVision } from '$lib/stores/server.svelte';
import { getParameterInfo, resetParameterToServerDefault } from '$lib/stores/settings.svelte';
import { ParameterSyncService } from '$lib/services/parameter-sync';
import ParameterSourceIndicator from './ParameterSourceIndicator.svelte';
import { ChatSettingsParameterSourceIndicator } from '$lib/components/app';
import type { Component } from 'svelte';
interface Props {
@@ -55,11 +55,15 @@
})()}
<div class="flex items-center gap-2">
<Label for={field.key} class="text-sm font-medium">
<Label for={field.key} class="flex items-center gap-1.5 text-sm font-medium">
{field.label}
{#if field.isExperimental}
<FlaskConical class="h-3.5 w-3.5 text-muted-foreground" />
{/if}
</Label>
{#if isCustomRealTime}
<ParameterSourceIndicator />
<ChatSettingsParameterSourceIndicator />
{/if}
</div>
@@ -97,8 +101,12 @@
</p>
{/if}
{:else if field.type === 'textarea'}
<Label for={field.key} class="block text-sm font-medium">
<Label for={field.key} class="block flex items-center gap-1.5 text-sm font-medium">
{field.label}
{#if field.isExperimental}
<FlaskConical class="h-3.5 w-3.5 text-muted-foreground" />
{/if}
</Label>
<Textarea
@@ -129,11 +137,15 @@
})()}
<div class="flex items-center gap-2">
<Label for={field.key} class="text-sm font-medium">
<Label for={field.key} class="flex items-center gap-1.5 text-sm font-medium">
{field.label}
{#if field.isExperimental}
<FlaskConical class="h-3.5 w-3.5 text-muted-foreground" />
{/if}
</Label>
{#if isCustomRealTime}
<ParameterSourceIndicator />
<ChatSettingsParameterSourceIndicator />
{/if}
</div>
@@ -214,9 +226,13 @@
for={field.key}
class="cursor-pointer text-sm leading-none font-medium {isDisabled
? 'text-muted-foreground'
: ''}"
: ''} flex items-center gap-1.5"
>
{field.label}
{#if field.isExperimental}
<FlaskConical class="h-3.5 w-3.5 text-muted-foreground" />
{/if}
</label>
{#if field.help || SETTING_CONFIG_INFO[field.key]}
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ImportExportTab.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ChatSettingsImportExportTab.svelte
+3 -3
View File
@@ -1,7 +1,7 @@
<script lang="ts">
import { Download, Upload } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import ConversationSelectionDialog from './ConversationSelectionDialog.svelte';
import { DialogConversationSelection } from '$lib/components/app';
import { DatabaseStore } from '$lib/stores/database';
import type { ExportedConversations } from '$lib/types/database';
import { createMessageCountMap } from '$lib/utils/conversation-utils';
@@ -236,7 +236,7 @@
</div>
</div>
<ConversationSelectionDialog
<DialogConversationSelection
conversations={availableConversations}
{messageCountMap}
mode="export"
@@ -245,7 +245,7 @@
onConfirm={handleExportConfirm}
/>
<ConversationSelectionDialog
<DialogConversationSelection
conversations={availableConversations}
{messageCountMap}
mode="import"
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ParameterSourceIndicator.svelte → examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ChatSettingsParameterSourceIndicator.svelte
View File
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSettings/ConversationSelectionDialog.svelte
-249
View File
@@ -1,249 +0,0 @@
<script lang="ts">
import { Search, X } from '@lucide/svelte';
import * as Dialog from '$lib/components/ui/dialog';
import { Button } from '$lib/components/ui/button';
import { Input } from '$lib/components/ui/input';
import { Checkbox } from '$lib/components/ui/checkbox';
import { ScrollArea } from '$lib/components/ui/scroll-area';
import { SvelteSet } from 'svelte/reactivity';
interface Props {
conversations: DatabaseConversation[];
messageCountMap?: Map<string, number>;
mode: 'export' | 'import';
onCancel: () => void;
onConfirm: (selectedConversations: DatabaseConversation[]) => void;
open?: boolean;
}
let {
conversations,
messageCountMap = new Map(),
mode,
onCancel,
onConfirm,
open = $bindable(false)
}: Props = $props();
let searchQuery = $state('');
let selectedIds = $state.raw<SvelteSet<string>>(new SvelteSet(conversations.map((c) => c.id)));
let lastClickedId = $state<string | null>(null);
let filteredConversations = $derived(
conversations.filter((conv) => {
const name = conv.name || 'Untitled conversation';
return name.toLowerCase().includes(searchQuery.toLowerCase());
})
);
let allSelected = $derived(
filteredConversations.length > 0 &&
filteredConversations.every((conv) => selectedIds.has(conv.id))
);
let someSelected = $derived(
filteredConversations.some((conv) => selectedIds.has(conv.id)) && !allSelected
);
function toggleConversation(id: string, shiftKey: boolean = false) {
const newSet = new SvelteSet(selectedIds);
if (shiftKey && lastClickedId !== null) {
const lastIndex = filteredConversations.findIndex((c) => c.id === lastClickedId);
const currentIndex = filteredConversations.findIndex((c) => c.id === id);
if (lastIndex !== -1 && currentIndex !== -1) {
const start = Math.min(lastIndex, currentIndex);
const end = Math.max(lastIndex, currentIndex);
const shouldSelect = !newSet.has(id);
for (let i = start; i <= end; i++) {
if (shouldSelect) {
newSet.add(filteredConversations[i].id);
} else {
newSet.delete(filteredConversations[i].id);
}
}
selectedIds = newSet;
return;
}
}
if (newSet.has(id)) {
newSet.delete(id);
} else {
newSet.add(id);
}
selectedIds = newSet;
lastClickedId = id;
}
function toggleAll() {
if (allSelected) {
const newSet = new SvelteSet(selectedIds);
filteredConversations.forEach((conv) => newSet.delete(conv.id));
selectedIds = newSet;
} else {
const newSet = new SvelteSet(selectedIds);
filteredConversations.forEach((conv) => newSet.add(conv.id));
selectedIds = newSet;
}
}
function handleConfirm() {
const selected = conversations.filter((conv) => selectedIds.has(conv.id));
onConfirm(selected);
}
function handleCancel() {
selectedIds = new SvelteSet(conversations.map((c) => c.id));
searchQuery = '';
lastClickedId = null;
onCancel();
}
let previousOpen = $state(false);
$effect(() => {
if (open && !previousOpen) {
selectedIds = new SvelteSet(conversations.map((c) => c.id));
searchQuery = '';
lastClickedId = null;
} else if (!open && previousOpen) {
onCancel();
}
previousOpen = open;
});
</script>
<Dialog.Root bind:open>
<Dialog.Portal>
<Dialog.Overlay class="z-[1000000]" />
<Dialog.Content class="z-[1000001] max-w-2xl">
<Dialog.Header>
<Dialog.Title>
Select Conversations to {mode === 'export' ? 'Export' : 'Import'}
</Dialog.Title>
<Dialog.Description>
{#if mode === 'export'}
Choose which conversations you want to export. Selected conversations will be downloaded
as a JSON file.
{:else}
Choose which conversations you want to import. Selected conversations will be merged
with your existing conversations.
{/if}
</Dialog.Description>
</Dialog.Header>
<div class="space-y-4">
<div class="relative">
<Search class="absolute top-1/2 left-3 h-4 w-4 -translate-y-1/2 text-muted-foreground" />
<Input bind:value={searchQuery} placeholder="Search conversations..." class="pr-9 pl-9" />
{#if searchQuery}
<button
class="absolute top-1/2 right-3 -translate-y-1/2 text-muted-foreground hover:text-foreground"
onclick={() => (searchQuery = '')}
type="button"
>
<X class="h-4 w-4" />
</button>
{/if}
</div>
<div class="flex items-center justify-between text-sm text-muted-foreground">
<span>
{selectedIds.size} of {conversations.length} selected
{#if searchQuery}
({filteredConversations.length} shown)
{/if}
</span>
</div>
<div class="overflow-hidden rounded-md border">
<ScrollArea class="h-[400px]">
<table class="w-full">
<thead class="sticky top-0 z-10 bg-muted">
<tr class="border-b">
<th class="w-12 p-3 text-left">
<Checkbox
checked={allSelected}
indeterminate={someSelected}
onCheckedChange={toggleAll}
/>
</th>
<th class="p-3 text-left text-sm font-medium">Conversation Name</th>
<th class="w-32 p-3 text-left text-sm font-medium">Messages</th>
</tr>
</thead>
<tbody>
{#if filteredConversations.length === 0}
<tr>
<td colspan="3" class="p-8 text-center text-sm text-muted-foreground">
{#if searchQuery}
No conversations found matching "{searchQuery}"
{:else}
No conversations available
{/if}
</td>
</tr>
{:else}
{#each filteredConversations as conv (conv.id)}
<tr
class="cursor-pointer border-b transition-colors hover:bg-muted/50"
onclick={(e) => toggleConversation(conv.id, e.shiftKey)}
>
<td class="p-3">
<Checkbox
checked={selectedIds.has(conv.id)}
onclick={(e) => {
e.preventDefault();
e.stopPropagation();
toggleConversation(conv.id, e.shiftKey);
}}
/>
</td>
<td class="p-3 text-sm">
<div
class="max-w-[17rem] truncate"
title={conv.name || 'Untitled conversation'}
>
{conv.name || 'Untitled conversation'}
</div>
</td>
<td class="p-3 text-sm text-muted-foreground">
{messageCountMap.get(conv.id) ?? 0}
</td>
</tr>
{/each}
{/if}
</tbody>
</table>
</ScrollArea>
</div>
</div>
<Dialog.Footer>
<Button variant="outline" onclick={handleCancel}>Cancel</Button>
<Button onclick={handleConfirm} disabled={selectedIds.size === 0}>
{mode === 'export' ? 'Export' : 'Import'} ({selectedIds.size})
</Button>
</Dialog.Footer>
</Dialog.Content>
</Dialog.Portal>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/chat/ChatSidebar/ChatSidebar.svelte
+2 -2
View File
@@ -2,7 +2,7 @@
import { goto } from '$app/navigation';
import { page } from '$app/state';
import { Trash2 } from '@lucide/svelte';
import { ChatSidebarConversationItem, ConfirmationDialog } from '$lib/components/app';
import { ChatSidebarConversationItem, DialogConfirmation } from '$lib/components/app';
import ScrollArea from '$lib/components/ui/scroll-area/scroll-area.svelte';
import * as Sidebar from '$lib/components/ui/sidebar';
import * as AlertDialog from '$lib/components/ui/alert-dialog';
@@ -158,7 +158,7 @@
<div class="bottom-0 z-10 bg-sidebar bg-sidebar/50 px-4 py-4 backdrop-blur-lg md:sticky"></div>
</ScrollArea>
<ConfirmationDialog
<DialogConfirmation
bind:open={showDeleteDialog}
title="Delete Conversation"
description={selectedConversation
examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogChatAttachmentPreview.svelte
+78
View File
@@ -0,0 +1,78 @@
<script lang="ts">
import * as Dialog from '$lib/components/ui/dialog';
import { ChatAttachmentPreview } from '$lib/components/app';
import { formatFileSize } from '$lib/utils/file-preview';
interface Props {
open: boolean;
// Either an uploaded file or a stored attachment
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
// For uploaded files
preview?: string;
name?: string;
type?: string;
size?: number;
textContent?: string;
}
let {
open = $bindable(),
uploadedFile,
attachment,
preview,
name,
type,
size,
textContent
}: Props = $props();
let chatAttachmentPreviewRef: ChatAttachmentPreview | undefined = $state();
let displayName = $derived(uploadedFile?.name || attachment?.name || name || 'Unknown File');
let displayType = $derived(
uploadedFile?.type ||
(attachment?.type === 'imageFile'
? 'image'
: attachment?.type === 'textFile'
? 'text'
: attachment?.type === 'audioFile'
? attachment.mimeType || 'audio'
: attachment?.type === 'pdfFile'
? 'application/pdf'
: type || 'unknown')
);
let displaySize = $derived(uploadedFile?.size || size);
$effect(() => {
if (open && chatAttachmentPreviewRef) {
chatAttachmentPreviewRef.reset();
}
});
</script>
<Dialog.Root bind:open>
<Dialog.Content class="grid max-h-[90vh] max-w-5xl overflow-hidden sm:w-auto sm:max-w-6xl">
<Dialog.Header>
<Dialog.Title>{displayName}</Dialog.Title>
<Dialog.Description>
{displayType}
{#if displaySize}
• {formatFileSize(displaySize)}
{/if}
</Dialog.Description>
</Dialog.Header>
<ChatAttachmentPreview
bind:this={chatAttachmentPreviewRef}
{uploadedFile}
{attachment}
{preview}
{name}
{type}
{textContent}
/>
</Dialog.Content>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogChatAttachmentsViewAll.svelte
+51
View File
@@ -0,0 +1,51 @@
<script lang="ts">
import * as Dialog from '$lib/components/ui/dialog';
import { ChatAttachmentsViewAll } from '$lib/components/app';
interface Props {
open?: boolean;
uploadedFiles?: ChatUploadedFile[];
attachments?: DatabaseMessageExtra[];
readonly?: boolean;
onFileRemove?: (fileId: string) => void;
imageHeight?: string;
imageWidth?: string;
imageClass?: string;
}
let {
open = $bindable(false),
uploadedFiles = [],
attachments = [],
readonly = false,
onFileRemove,
imageHeight = 'h-24',
imageWidth = 'w-auto',
imageClass = ''
}: Props = $props();
let totalCount = $derived(uploadedFiles.length + attachments.length);
</script>
<Dialog.Root bind:open>
<Dialog.Portal>
<Dialog.Overlay />
<Dialog.Content class="flex !max-h-[90vh] !max-w-6xl flex-col">
<Dialog.Header>
<Dialog.Title>All Attachments ({totalCount})</Dialog.Title>
<Dialog.Description>View and manage all attached files</Dialog.Description>
</Dialog.Header>
<ChatAttachmentsViewAll
{uploadedFiles}
{attachments}
{readonly}
{onFileRemove}
{imageHeight}
{imageWidth}
{imageClass}
/>
</Dialog.Content>
</Dialog.Portal>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/dialogs/ChatErrorDialog.svelte → examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogChatError.svelte
View File
examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogChatSettings.svelte
+37
View File
@@ -0,0 +1,37 @@
<script lang="ts">
import * as Dialog from '$lib/components/ui/dialog';
import { ChatSettings } from '$lib/components/app';
interface Props {
onOpenChange?: (open: boolean) => void;
open?: boolean;
}
let { onOpenChange, open = false }: Props = $props();
let chatSettingsRef: ChatSettings | undefined = $state();
function handleClose() {
onOpenChange?.(false);
}
function handleSave() {
onOpenChange?.(false);
}
$effect(() => {
if (open && chatSettingsRef) {
chatSettingsRef.reset();
}
});
</script>
<Dialog.Root {open} onOpenChange={handleClose}>
<Dialog.Content
class="z-999999 flex h-[100dvh] max-h-[100dvh] min-h-[100dvh] flex-col gap-0 rounded-none p-0
md:h-[64vh] md:max-h-[64vh] md:min-h-0 md:rounded-lg"
style="max-width: 48rem;"
>
<ChatSettings bind:this={chatSettingsRef} onSave={handleSave} />
</Dialog.Content>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/dialogs/ConfirmationDialog.svelte → examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogConfirmation.svelte
View File
examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogConversationSelection.svelte
+68
View File
@@ -0,0 +1,68 @@
<script lang="ts">
import * as Dialog from '$lib/components/ui/dialog';
import { ConversationSelection } from '$lib/components/app';
interface Props {
conversations: DatabaseConversation[];
messageCountMap?: Map<string, number>;
mode: 'export' | 'import';
onCancel: () => void;
onConfirm: (selectedConversations: DatabaseConversation[]) => void;
open?: boolean;
}
let {
conversations,
messageCountMap = new Map(),
mode,
onCancel,
onConfirm,
open = $bindable(false)
}: Props = $props();
let conversationSelectionRef: ConversationSelection | undefined = $state();
let previousOpen = $state(false);
$effect(() => {
if (open && !previousOpen && conversationSelectionRef) {
conversationSelectionRef.reset();
} else if (!open && previousOpen) {
onCancel();
}
previousOpen = open;
});
</script>
<Dialog.Root bind:open>
<Dialog.Portal>
<Dialog.Overlay class="z-[1000000]" />
<Dialog.Content class="z-[1000001] max-w-2xl">
<Dialog.Header>
<Dialog.Title>
Select Conversations to {mode === 'export' ? 'Export' : 'Import'}
</Dialog.Title>
<Dialog.Description>
{#if mode === 'export'}
Choose which conversations you want to export. Selected conversations will be downloaded
as a JSON file.
{:else}
Choose which conversations you want to import. Selected conversations will be merged
with your existing conversations.
{/if}
</Dialog.Description>
</Dialog.Header>
<ConversationSelection
bind:this={conversationSelectionRef}
{conversations}
{messageCountMap}
{mode}
{onCancel}
{onConfirm}
/>
</Dialog.Content>
</Dialog.Portal>
</Dialog.Root>
examples/server/webui_llamacpp/src/lib/components/app/dialogs/ConversationTitleUpdateDialog.svelte → examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogConversationTitleUpdate.svelte
View File
examples/server/webui_llamacpp/src/lib/components/app/dialogs/EmptyFileAlertDialog.svelte → examples/server/webui_llamacpp/src/lib/components/app/dialogs/DialogEmptyFileAlert.svelte
View File
examples/server/webui_llamacpp/src/lib/components/app/index.ts
+37 -27
View File
@@ -1,53 +1,63 @@
// Chat
export { default as ChatAttachmentPreview } from './chat/ChatAttachments/ChatAttachmentPreview.svelte';
export { default as ChatAttachmentThumbnailFile } from './chat/ChatAttachments/ChatAttachmentThumbnailFile.svelte';
export { default as ChatAttachmentThumbnailImage } from './chat/ChatAttachments/ChatAttachmentThumbnailImage.svelte';
export { default as ChatAttachmentsList } from './chat/ChatAttachments/ChatAttachmentsList.svelte';
export { default as ChatAttachmentFilePreview } from './chat/ChatAttachments/ChatAttachmentFilePreview.svelte';
export { default as ChatAttachmentImagePreview } from './chat/ChatAttachments/ChatAttachmentImagePreview.svelte';
export { default as ChatAttachmentPreviewDialog } from './chat/ChatAttachments/ChatAttachmentPreviewDialog.svelte';
export { default as ChatAttachmentsViewAll } from './chat/ChatAttachments/ChatAttachmentsViewAll.svelte';
export { default as ChatForm } from './chat/ChatForm/ChatForm.svelte';
export { default as ChatFormTextarea } from './chat/ChatForm/ChatFormTextarea.svelte';
export { default as ChatFormActions } from './chat/ChatForm/ChatFormActions.svelte';
export { default as ChatFormActionFileAttachments } from './chat/ChatForm/ChatFormActionFileAttachments.svelte';
export { default as ChatFormActionRecord } from './chat/ChatForm/ChatFormActionRecord.svelte';
export { default as ChatFormModelSelector } from './chat/ChatForm/ChatFormModelSelector.svelte';
export { default as ChatFormHelperText } from './chat/ChatForm/ChatFormHelperText.svelte';
export { default as ChatFormActionFileAttachments } from './chat/ChatForm/ChatFormActions/ChatFormActionFileAttachments.svelte';
export { default as ChatFormActionRecord } from './chat/ChatForm/ChatFormActions/ChatFormActionRecord.svelte';
export { default as ChatFormActions } from './chat/ChatForm/ChatFormActions/ChatFormActions.svelte';
export { default as ChatFormFileInputInvisible } from './chat/ChatForm/ChatFormFileInputInvisible.svelte';
export { default as ChatFormHelperText } from './chat/ChatForm/ChatFormHelperText.svelte';
export { default as ChatFormModelSelector } from './chat/ChatForm/ChatFormModelSelector.svelte';
export { default as ChatFormTextarea } from './chat/ChatForm/ChatFormTextarea.svelte';
export { default as ChatMessage } from './chat/ChatMessages/ChatMessage.svelte';
export { default as ChatMessages } from './chat/ChatMessages/ChatMessages.svelte';
export { default as ChatMessageBranchingControls } from './chat/ChatMessages/ChatMessageBranchingControls.svelte';
export { default as ChatMessageThinkingBlock } from './chat/ChatMessages/ChatMessageThinkingBlock.svelte';
export { default as MessageBranchingControls } from './chat/ChatMessages/ChatMessageBranchingControls.svelte';
export { default as ChatProcessingInfo } from './chat/ChatProcessingInfo.svelte';
export { default as ChatScreenHeader } from './chat/ChatScreen/ChatScreenHeader.svelte';
export { default as ChatScreenWarning } from './chat/ChatScreen/ChatScreenWarning.svelte';
export { default as ChatScreen } from './chat/ChatScreen/ChatScreen.svelte';
export { default as ChatScreenHeader } from './chat/ChatScreen/ChatScreenHeader.svelte';
export { default as ChatScreenProcessingInfo } from './chat/ChatScreen/ChatScreenProcessingInfo.svelte';
export { default as ChatScreenWarning } from './chat/ChatScreen/ChatScreenWarning.svelte';
export { default as ChatSettingsDialog } from './chat/ChatSettings/ChatSettingsDialog.svelte';
export { default as ChatSettings } from './chat/ChatSettings/ChatSettings.svelte';
export { default as ChatSettingsFooter } from './chat/ChatSettings/ChatSettingsFooter.svelte';
export { default as ChatSettingsFields } from './chat/ChatSettings/ChatSettingsFields.svelte';
export { default as ImportExportTab } from './chat/ChatSettings/ImportExportTab.svelte';
export { default as ConversationSelectionDialog } from './chat/ChatSettings/ConversationSelectionDialog.svelte';
export { default as ParameterSourceIndicator } from './chat/ChatSettings/ParameterSourceIndicator.svelte';
export { default as ChatSettingsImportExportTab } from './chat/ChatSettings/ChatSettingsImportExportTab.svelte';
export { default as ChatSettingsParameterSourceIndicator } from './chat/ChatSettings/ChatSettingsParameterSourceIndicator.svelte';
export { default as ChatSidebar } from './chat/ChatSidebar/ChatSidebar.svelte';
export { default as ChatSidebarConversationItem } from './chat/ChatSidebar/ChatSidebarConversationItem.svelte';
export { default as ChatSidebarSearch } from './chat/ChatSidebar/ChatSidebarSearch.svelte';
export { default as ChatErrorDialog } from './dialogs/ChatErrorDialog.svelte';
export { default as EmptyFileAlertDialog } from './dialogs/EmptyFileAlertDialog.svelte';
export { default as ConversationTitleUpdateDialog } from './dialogs/ConversationTitleUpdateDialog.svelte';
// Dialogs
export { default as DialogChatAttachmentPreview } from './dialogs/DialogChatAttachmentPreview.svelte';
export { default as DialogChatAttachmentsViewAll } from './dialogs/DialogChatAttachmentsViewAll.svelte';
export { default as DialogChatError } from './dialogs/DialogChatError.svelte';
export { default as DialogChatSettings } from './dialogs/DialogChatSettings.svelte';
export { default as DialogConfirmation } from './dialogs/DialogConfirmation.svelte';
export { default as DialogConversationSelection } from './dialogs/DialogConversationSelection.svelte';
export { default as DialogConversationTitleUpdate } from './dialogs/DialogConversationTitleUpdate.svelte';
export { default as DialogEmptyFileAlert } from './dialogs/DialogEmptyFileAlert.svelte';
// Miscellanous
export { default as ActionButton } from './misc/ActionButton.svelte';
export { default as ActionDropdown } from './misc/ActionDropdown.svelte';
export { default as ConversationSelection } from './misc/ConversationSelection.svelte';
export { default as KeyboardShortcutInfo } from './misc/KeyboardShortcutInfo.svelte';
export { default as MarkdownContent } from './misc/MarkdownContent.svelte';
export { default as RemoveButton } from './misc/RemoveButton.svelte';
// Server
export { default as ServerStatus } from './server/ServerStatus.svelte';
export { default as ServerErrorSplash } from './server/ServerErrorSplash.svelte';
export { default as ServerLoadingSplash } from './server/ServerLoadingSplash.svelte';
export { default as ServerInfo } from './server/ServerInfo.svelte';
// Shared components
export { default as ActionButton } from './misc/ActionButton.svelte';
export { default as ActionDropdown } from './misc/ActionDropdown.svelte';
export { default as ConfirmationDialog } from './dialogs/ConfirmationDialog.svelte';
examples/server/webui_llamacpp/src/lib/components/app/misc/CodePreviewDialog.svelte
+93
View File
@@ -0,0 +1,93 @@
<script lang="ts">
import { Dialog as DialogPrimitive } from 'bits-ui';
import XIcon from '@lucide/svelte/icons/x';
interface Props {
open: boolean;
code: string;
language: string;
onOpenChange?: (open: boolean) => void;
}
let { open = $bindable(), code, language, onOpenChange }: Props = $props();
let iframeRef = $state<HTMLIFrameElement | null>(null);
$effect(() => {
if (!iframeRef) return;
if (open) {
iframeRef.srcdoc = code;
} else {
iframeRef.srcdoc = '';
}
});
function handleOpenChange(nextOpen: boolean) {
open = nextOpen;
onOpenChange?.(nextOpen);
}
</script>
<DialogPrimitive.Root {open} onOpenChange={handleOpenChange}>
<DialogPrimitive.Portal>
<DialogPrimitive.Overlay class="code-preview-overlay" />
<DialogPrimitive.Content class="code-preview-content">
<iframe
bind:this={iframeRef}
title="Preview {language}"
sandbox="allow-scripts"
class="code-preview-iframe"
></iframe>
<DialogPrimitive.Close
class="code-preview-close absolute top-4 right-4 border-none bg-transparent text-white opacity-70 mix-blend-difference transition-opacity hover:opacity-100 focus-visible:ring-0 focus-visible:ring-offset-0 focus-visible:outline-none disabled:pointer-events-none [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-8"
aria-label="Close preview"
>
<XIcon />
<span class="sr-only">Close preview</span>
</DialogPrimitive.Close>
</DialogPrimitive.Content>
</DialogPrimitive.Portal>
</DialogPrimitive.Root>
<style lang="postcss">
:global(.code-preview-overlay) {
position: fixed;
inset: 0;
background-color: transparent;
z-index: 100000;
}
:global(.code-preview-content) {
position: fixed;
inset: 0;
top: 0 !important;
left: 0 !important;
width: 100dvw;
height: 100dvh;
margin: 0;
padding: 0;
border: none;
border-radius: 0;
background-color: transparent;
box-shadow: none;
display: block;
overflow: hidden;
transform: none !important;
z-index: 100001;
}
:global(.code-preview-iframe) {
display: block;
width: 100dvw;
height: 100dvh;
border: 0;
}
:global(.code-preview-close) {
position: absolute;
z-index: 100002;
}
</style>
examples/server/webui_llamacpp/src/lib/components/app/misc/ConversationSelection.svelte
+205
View File
@@ -0,0 +1,205 @@
<script lang="ts">
import { Search, X } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
import { Input } from '$lib/components/ui/input';
import { Checkbox } from '$lib/components/ui/checkbox';
import { ScrollArea } from '$lib/components/ui/scroll-area';
import { SvelteSet } from 'svelte/reactivity';
interface Props {
conversations: DatabaseConversation[];
messageCountMap?: Map<string, number>;
mode: 'export' | 'import';
onCancel: () => void;
onConfirm: (selectedConversations: DatabaseConversation[]) => void;
}
let { conversations, messageCountMap = new Map(), mode, onCancel, onConfirm }: Props = $props();
let searchQuery = $state('');
let selectedIds = $state.raw<SvelteSet<string>>(new SvelteSet(conversations.map((c) => c.id)));
let lastClickedId = $state<string | null>(null);
let filteredConversations = $derived(
conversations.filter((conv) => {
const name = conv.name || 'Untitled conversation';
return name.toLowerCase().includes(searchQuery.toLowerCase());
})
);
let allSelected = $derived(
filteredConversations.length > 0 &&
filteredConversations.every((conv) => selectedIds.has(conv.id))
);
let someSelected = $derived(
filteredConversations.some((conv) => selectedIds.has(conv.id)) && !allSelected
);
function toggleConversation(id: string, shiftKey: boolean = false) {
const newSet = new SvelteSet(selectedIds);
if (shiftKey && lastClickedId !== null) {
const lastIndex = filteredConversations.findIndex((c) => c.id === lastClickedId);
const currentIndex = filteredConversations.findIndex((c) => c.id === id);
if (lastIndex !== -1 && currentIndex !== -1) {
const start = Math.min(lastIndex, currentIndex);
const end = Math.max(lastIndex, currentIndex);
const shouldSelect = !newSet.has(id);
for (let i = start; i <= end; i++) {
if (shouldSelect) {
newSet.add(filteredConversations[i].id);
} else {
newSet.delete(filteredConversations[i].id);
}
}
selectedIds = newSet;
return;
}
}
if (newSet.has(id)) {
newSet.delete(id);
} else {
newSet.add(id);
}
selectedIds = newSet;
lastClickedId = id;
}
function toggleAll() {
if (allSelected) {
const newSet = new SvelteSet(selectedIds);
filteredConversations.forEach((conv) => newSet.delete(conv.id));
selectedIds = newSet;
} else {
const newSet = new SvelteSet(selectedIds);
filteredConversations.forEach((conv) => newSet.add(conv.id));
selectedIds = newSet;
}
}
function handleConfirm() {
const selected = conversations.filter((conv) => selectedIds.has(conv.id));
onConfirm(selected);
}
function handleCancel() {
selectedIds = new SvelteSet(conversations.map((c) => c.id));
searchQuery = '';
lastClickedId = null;
onCancel();
}
export function reset() {
selectedIds = new SvelteSet(conversations.map((c) => c.id));
searchQuery = '';
lastClickedId = null;
}
</script>
<div class="space-y-4">
<div class="relative">
<Search class="absolute top-1/2 left-3 h-4 w-4 -translate-y-1/2 text-muted-foreground" />
<Input bind:value={searchQuery} placeholder="Search conversations..." class="pr-9 pl-9" />
{#if searchQuery}
<button
class="absolute top-1/2 right-3 -translate-y-1/2 text-muted-foreground hover:text-foreground"
onclick={() => (searchQuery = '')}
type="button"
>
<X class="h-4 w-4" />
</button>
{/if}
</div>
<div class="flex items-center justify-between text-sm text-muted-foreground">
<span>
{selectedIds.size} of {conversations.length} selected
{#if searchQuery}
({filteredConversations.length} shown)
{/if}
</span>
</div>
<div class="overflow-hidden rounded-md border">
<ScrollArea class="h-[400px]">
<table class="w-full">
<thead class="sticky top-0 z-10 bg-muted">
<tr class="border-b">
<th class="w-12 p-3 text-left">
<Checkbox
checked={allSelected}
indeterminate={someSelected}
onCheckedChange={toggleAll}
/>
</th>
<th class="p-3 text-left text-sm font-medium">Conversation Name</th>
<th class="w-32 p-3 text-left text-sm font-medium">Messages</th>
</tr>
</thead>
<tbody>
{#if filteredConversations.length === 0}
<tr>
<td colspan="3" class="p-8 text-center text-sm text-muted-foreground">
{#if searchQuery}
No conversations found matching "{searchQuery}"
{:else}
No conversations available
{/if}
</td>
</tr>
{:else}
{#each filteredConversations as conv (conv.id)}
<tr
class="cursor-pointer border-b transition-colors hover:bg-muted/50"
onclick={(e) => toggleConversation(conv.id, e.shiftKey)}
>
<td class="p-3">
<Checkbox
checked={selectedIds.has(conv.id)}
onclick={(e) => {
e.preventDefault();
e.stopPropagation();
toggleConversation(conv.id, e.shiftKey);
}}
/>
</td>
<td class="p-3 text-sm">
<div class="max-w-[17rem] truncate" title={conv.name || 'Untitled conversation'}>
{conv.name || 'Untitled conversation'}
</div>
</td>
<td class="p-3 text-sm text-muted-foreground">
{messageCountMap.get(conv.id) ?? 0}
</td>
</tr>
{/each}
{/if}
</tbody>
</table>
</ScrollArea>
</div>
<div class="flex justify-end gap-2">
<Button variant="outline" onclick={handleCancel}>Cancel</Button>
<Button onclick={handleConfirm} disabled={selectedIds.size === 0}>
{mode === 'export' ? 'Export' : 'Import'} ({selectedIds.size})
</Button>
</div>
</div>
examples/server/webui_llamacpp/src/lib/components/app/misc/MarkdownContent.svelte
+141 -56
View File
@@ -8,13 +8,15 @@
import rehypeKatex from 'rehype-katex';
import rehypeStringify from 'rehype-stringify';
import { copyCodeToClipboard } from '$lib/utils/copy';
import { preprocessLaTeX } from '$lib/utils/latex-protection';
import { browser } from '$app/environment';
import 'katex/dist/katex.min.css';
import '$styles/katex-custom.scss';
import githubDarkCss from 'highlight.js/styles/github-dark.css?inline';
import githubLightCss from 'highlight.js/styles/github.css?inline';
import { mode } from 'mode-watcher';
import { remarkLiteralHtml } from '$lib/markdown/literal-html';
import CodePreviewDialog from './CodePreviewDialog.svelte';
interface Props {
content: string;
@@ -25,6 +27,9 @@
let containerRef = $state<HTMLDivElement>();
let processedHtml = $state('');
let previewDialogOpen = $state(false);
let previewCode = $state('');
let previewLanguage = $state('text');
function loadHighlightTheme(isDark: boolean) {
if (!browser) return;
@@ -117,7 +122,6 @@
const rawCode = codeElement.textContent || '';
const codeId = `code-${Date.now()}-${index}`;
codeElement.setAttribute('data-code-id', codeId);
codeElement.setAttribute('data-raw-code', rawCode);
@@ -138,11 +142,30 @@
copyButton.setAttribute('type', 'button');
copyButton.innerHTML = `
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-copy-icon lucide-copy"><rect width="14" height="14" x="8" y="8" rx="2" ry="2"/><path d="M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2"/></svg>
`;
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-copy-icon lucide-copy"><rect width="14" height="14" x="8" y="8" rx="2" ry="2"/><path d="M4 16c-1.1 0-2-.9-2-2V4c0-1.1.9-2 2-2h10c1.1 0 2 .9 2 2"/></svg>
`;
const actions = document.createElement('div');
actions.className = 'code-block-actions';
actions.appendChild(copyButton);
if (language.toLowerCase() === 'html') {
const previewButton = document.createElement('button');
previewButton.className = 'preview-code-btn';
previewButton.setAttribute('data-code-id', codeId);
previewButton.setAttribute('title', 'Preview code');
previewButton.setAttribute('type', 'button');
previewButton.innerHTML = `
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-eye lucide-eye-icon"><path d="M2.062 12.345a1 1 0 0 1 0-.69C3.5 7.73 7.36 5 12 5s8.5 2.73 9.938 6.655a1 1 0 0 1 0 .69C20.5 16.27 16.64 19 12 19s-8.5-2.73-9.938-6.655"/><circle cx="12" cy="12" r="3"/></svg>
`;
actions.appendChild(previewButton);
}
header.appendChild(languageLabel);
header.appendChild(copyButton);
header.appendChild(actions);
wrapper.appendChild(header);
const clonedPre = pre.cloneNode(true) as HTMLElement;
@@ -154,19 +177,9 @@
return mutated ? tempDiv.innerHTML : html;
}
function normalizeMathDelimiters(text: string): string {
return text
.replace(/(^|[^\\])\\\[((?:\\.|[\s\S])*?)\\\]/g, (_, prefix: string, content: string) => {
return `${prefix}$$${content}$$`;
})
.replace(/(^|[^\\])\\\(((?:\\.|[\s\S])*?)\\\)/g, (_, prefix: string, content: string) => {
return `${prefix}$${content}$`;
});
}
async function processMarkdown(text: string): Promise<string> {
try {
const normalized = normalizeMathDelimiters(text);
let normalized = preprocessLaTeX(text);
const result = await processor().process(normalized);
const html = String(result);
const enhancedLinks = enhanceLinks(html);
@@ -180,49 +193,105 @@
}
}
function setupCopyButtons() {
function getCodeInfoFromTarget(target: HTMLElement) {
const wrapper = target.closest('.code-block-wrapper');
if (!wrapper) {
console.error('No wrapper found');
return null;
}
const codeElement = wrapper.querySelector<HTMLElement>('code[data-code-id]');
if (!codeElement) {
console.error('No code element found in wrapper');
return null;
}
const rawCode = codeElement.getAttribute('data-raw-code');
if (rawCode === null) {
console.error('No raw code found');
return null;
}
const languageLabel = wrapper.querySelector<HTMLElement>('.code-language');
const language = languageLabel?.textContent?.trim() || 'text';
return { rawCode, language };
}
async function handleCopyClick(event: Event) {
event.preventDefault();
event.stopPropagation();
const target = event.currentTarget as HTMLButtonElement | null;
if (!target) {
return;
}
const info = getCodeInfoFromTarget(target);
if (!info) {
return;
}
try {
await copyCodeToClipboard(info.rawCode);
} catch (error) {
console.error('Failed to copy code:', error);
}
}
function handlePreviewClick(event: Event) {
event.preventDefault();
event.stopPropagation();
const target = event.currentTarget as HTMLButtonElement | null;
if (!target) {
return;
}
const info = getCodeInfoFromTarget(target);
if (!info) {
return;
}
previewCode = info.rawCode;
previewLanguage = info.language;
previewDialogOpen = true;
}
function setupCodeBlockActions() {
if (!containerRef) return;
const copyButtons = containerRef.querySelectorAll('.copy-code-btn');
const wrappers = containerRef.querySelectorAll<HTMLElement>('.code-block-wrapper');
for (const button of copyButtons) {
button.addEventListener('click', async (e) => {
e.preventDefault();
e.stopPropagation();
for (const wrapper of wrappers) {
const copyButton = wrapper.querySelector<HTMLButtonElement>('.copy-code-btn');
const previewButton = wrapper.querySelector<HTMLButtonElement>('.preview-code-btn');
const target = e.currentTarget as HTMLButtonElement;
const codeId = target.getAttribute('data-code-id');
if (copyButton && copyButton.dataset.listenerBound !== 'true') {
copyButton.dataset.listenerBound = 'true';
copyButton.addEventListener('click', handleCopyClick);
}
if (!codeId) {
console.error('No code ID found on button');
return;
}
if (previewButton && previewButton.dataset.listenerBound !== 'true') {
previewButton.dataset.listenerBound = 'true';
previewButton.addEventListener('click', handlePreviewClick);
}
}
}
// Find the code element within the same wrapper
const wrapper = target.closest('.code-block-wrapper');
if (!wrapper) {
console.error('No wrapper found');
return;
}
function handlePreviewDialogOpenChange(open: boolean) {
previewDialogOpen = open;
const codeElement = wrapper.querySelector('code[data-code-id]');
if (!codeElement) {
console.error('No code element found in wrapper');
return;
}
const rawCode = codeElement.getAttribute('data-raw-code');
if (!rawCode) {
console.error('No raw code found');
return;
}
try {
await copyCodeToClipboard(rawCode);
} catch (error) {
console.error('Failed to copy code:', error);
}
});
if (!open) {
previewCode = '';
previewLanguage = 'text';
}
}
@@ -243,7 +312,7 @@
$effect(() => {
if (containerRef && processedHtml) {
setupCopyButtons();
setupCodeBlockActions();
}
});
</script>
@@ -253,6 +322,13 @@
{@html processedHtml}
</div>
<CodePreviewDialog
open={previewDialogOpen}
code={previewCode}
language={previewLanguage}
onOpenChange={handlePreviewDialogOpenChange}
/>
<style>
/* Base typography styles */
div :global(p:not(:last-child)) {
@@ -472,7 +548,14 @@
letter-spacing: 0.05em;
}
div :global(.copy-code-btn) {
div :global(.code-block-actions) {
display: flex;
align-items: center;
gap: 0.5rem;
}
div :global(.copy-code-btn),
div :global(.preview-code-btn) {
display: flex;
align-items: center;
justify-content: center;
@@ -483,11 +566,13 @@
transition: all 0.2s ease;
}
div :global(.copy-code-btn:hover) {
div :global(.copy-code-btn:hover),
div :global(.preview-code-btn:hover) {
transform: scale(1.05);
}
div :global(.copy-code-btn:active) {
div :global(.copy-code-btn:active),
div :global(.preview-code-btn:active) {
transform: scale(0.95);
}
examples/server/webui_llamacpp/src/lib/components/app/misc/RemoveButton.svelte
+26
View File
@@ -0,0 +1,26 @@
<script lang="ts">
import { X } from '@lucide/svelte';
import { Button } from '$lib/components/ui/button';
interface Props {
id: string;
onRemove?: (id: string) => void;
class?: string;
}
let { id, onRemove, class: className = '' }: Props = $props();
</script>
<Button
type="button"
variant="ghost"
size="sm"
class="h-6 w-6 bg-white/20 p-0 hover:bg-white/30 {className}"
onclick={(e) => {
e.stopPropagation();
onRemove?.(id);
}}
aria-label="Remove file"
>
<X class="h-3 w-3" />
</Button>
examples/server/webui_llamacpp/src/lib/constants/latex-protection.ts
+35
View File
@@ -0,0 +1,35 @@
/**
* Matches common Markdown code blocks to exclude them from further processing (e.g. LaTeX).
* - Fenced: ```...```
* - Inline: `...` (does NOT support nested backticks or multi-backtick syntax)
*
* Note: This pattern does not handle advanced cases like:
* `` `code with `backticks` `` or \\``...\\``
*/
export const CODE_BLOCK_REGEXP = /(```[\s\S]*?```|`[^`\n]+`)/g;
/**
* Matches LaTeX math delimiters \(...\) and \[...\] only when not preceded by a backslash (i.e., not escaped),
* while also capturing code blocks (```, `...`) so they can be skipped during processing.
*
* Uses negative lookbehind `(?<!\\)` to avoid matching \\( or \\[.
* Using the look‑behind pattern `(?<!\\)` we skip matches
* that are preceded by a backslash, e.g.
* `Definitions\\(also called macros)` (title of chapter 20 in The TeXbook)
* or `\\[4pt]` (LaTeX line-break).
*
* group 1: code-block
* group 2: square-bracket
* group 3: round-bracket
*/
export const LATEX_MATH_AND_CODE_PATTERN =
/(```[\S\s]*?```|`.*?`)|(?<!\\)\\\[([\S\s]*?[^\\])\\]|(?<!\\)\\\((.*?)\\\)/g;
/** Regex to capture the content of a $$...\\\\...$$ block (display-formula with line-break) */
export const LATEX_LINEBREAK_REGEXP = /\$\$([\s\S]*?\\\\[\s\S]*?)\$\$/;
/** map from mchem-regexp to replacement */
export const MHCHEM_PATTERN_MAP: readonly [RegExp, string][] = [
[/(\s)\$\\ce{/g, '$1$\\\\ce{'],
[/(\s)\$\\pu{/g, '$1$\\\\pu{']
] as const;
examples/server/webui_llamacpp/src/lib/constants/settings-config.ts
+14 -2
View File
@@ -6,12 +6,15 @@ export const SETTING_CONFIG_DEFAULT: Record<string, string | number | boolean> =
theme: 'system',
showTokensPerSecond: false,
showThoughtInProgress: false,
showToolCalls: false,
disableReasoningFormat: false,
keepStatsVisible: false,
showMessageStats: true,
askForTitleConfirmation: false,
pasteLongTextToFileLen: 2500,
pdfAsImage: false,
showModelInfo: false,
disableAutoScroll: false,
renderUserContentAsMarkdown: false,
modelSelectorEnabled: false,
// make sure these default values are in sync with `common.h`
@@ -36,7 +39,8 @@ export const SETTING_CONFIG_DEFAULT: Record<string, string | number | boolean> =
max_tokens: -1,
custom: '', // custom json-stringified object
// experimental features
pyInterpreterEnabled: false
pyInterpreterEnabled: false,
enableContinueGeneration: false
};
export const SETTING_CONFIG_INFO: Record<string, string> = {
@@ -79,16 +83,24 @@ export const SETTING_CONFIG_INFO: Record<string, string> = {
custom: 'Custom JSON parameters to send to the API. Must be valid JSON format.',
showTokensPerSecond: 'Display generation speed in tokens per second during streaming.',
showThoughtInProgress: 'Expand thought process by default when generating messages.',
showToolCalls:
'Display tool call labels and payloads from Harmony-compatible delta.tool_calls data below assistant messages.',
disableReasoningFormat:
'Show raw LLM output without backend parsing and frontend Markdown rendering to inspect streaming across different models.',
keepStatsVisible: 'Keep processing statistics visible after generation finishes.',
showMessageStats:
'Display generation statistics (tokens/second, token count, duration) below each assistant message.',
askForTitleConfirmation:
'Ask for confirmation before automatically changing conversation title when editing the first message.',
pdfAsImage: 'Parse PDF as image instead of text (requires vision-capable model).',
showModelInfo: 'Display the model name used to generate each message below the message content.',
disableAutoScroll:
'Disable automatic scrolling while messages stream so you can control the viewport position manually.',
renderUserContentAsMarkdown: 'Render user messages using markdown formatting in the chat.',
modelSelectorEnabled:
'Enable the model selector in the chat input to choose the inference model. Sends the associated model field in API requests.',
pyInterpreterEnabled:
'Enable Python interpreter using Pyodide. Allows running Python code in markdown code blocks.'
'Enable Python interpreter using Pyodide. Allows running Python code in markdown code blocks.',
enableContinueGeneration:
'Enable "Continue" button for assistant messages. Currently works only with non-reasoning models.'
};
examples/server/webui_llamacpp/src/lib/constants/supported-file-types.ts
+4
View File
@@ -69,6 +69,10 @@ export const TEXT_FILE_TYPES = {
extensions: [FileExtensionText.MD],
mimeTypes: [MimeTypeText.MARKDOWN]
},
[FileTypeText.ASCIIDOC]: {
extensions: [FileExtensionText.ADOC],
mimeTypes: [MimeTypeText.ASCIIDOC]
},
[FileTypeText.JAVASCRIPT]: {
extensions: [FileExtensionText.JS],
mimeTypes: [MimeTypeText.JAVASCRIPT, MimeTypeText.JAVASCRIPT_APP]
examples/server/webui_llamacpp/src/lib/enums/files.ts
+3
View File
@@ -33,6 +33,7 @@ export enum FileTypePdf {
export enum FileTypeText {
PLAIN_TEXT = 'plainText',
MARKDOWN = 'markdown',
ASCIIDOC = 'asciidoc',
JAVASCRIPT = 'javascript',
TYPESCRIPT = 'typescript',
JSX = 'jsx',
@@ -86,6 +87,7 @@ export enum FileExtensionPdf {
export enum FileExtensionText {
TXT = '.txt',
MD = '.md',
ADOC = '.adoc',
JS = '.js',
TS = '.ts',
JSX = '.jsx',
@@ -147,6 +149,7 @@ export enum MimeTypeImage {
export enum MimeTypeText {
PLAIN = 'text/plain',
MARKDOWN = 'text/markdown',
ASCIIDOC = 'text/asciidoc',
JAVASCRIPT = 'text/javascript',
JAVASCRIPT_APP = 'application/javascript',
TYPESCRIPT = 'text/typescript',
examples/server/webui_llamacpp/src/lib/services/chat.ts
+170 -18
View File
@@ -1,6 +1,25 @@
import { config } from '$lib/stores/settings.svelte';
import { selectedModelName } from '$lib/stores/models.svelte';
import { slotsService } from './slots';
import type {
ApiChatCompletionRequest,
ApiChatCompletionResponse,
ApiChatCompletionStreamChunk,
ApiChatCompletionToolCall,
ApiChatCompletionToolCallDelta,
ApiChatMessageData
} from '$lib/types/api';
import type {
DatabaseMessage,
DatabaseMessageExtra,
DatabaseMessageExtraAudioFile,
DatabaseMessageExtraImageFile,
DatabaseMessageExtraLegacyContext,
DatabaseMessageExtraPdfFile,
DatabaseMessageExtraTextFile
} from '$lib/types/database';
import type { ChatMessagePromptProgress, ChatMessageTimings } from '$lib/types/chat';
import type { SettingsChatServiceOptions } from '$lib/types/settings';
/**
* ChatService - Low-level API communication layer for llama.cpp server interactions
*
@@ -53,7 +72,9 @@ export class ChatService {
onComplete,
onError,
onReasoningChunk,
onToolCallChunk,
onModel,
onFirstValidChunk,
// Generation parameters
temperature,
max_tokens,
@@ -200,13 +221,21 @@ export class ChatService {
onComplete,
onError,
onReasoningChunk,
onToolCallChunk,
onModel,
onFirstValidChunk,
conversationId,
abortController.signal
);
return;
} else {
return this.handleNonStreamResponse(response, onComplete, onError, onModel);
return this.handleNonStreamResponse(
response,
onComplete,
onError,
onToolCallChunk,
onModel
);
}
} catch (error) {
if (error instanceof Error && error.name === 'AbortError') {
@@ -262,11 +291,14 @@ export class ChatService {
onComplete?: (
response: string,
reasoningContent?: string,
timings?: ChatMessageTimings
timings?: ChatMessageTimings,
toolCalls?: string
) => void,
onError?: (error: Error) => void,
onReasoningChunk?: (chunk: string) => void,
onToolCallChunk?: (chunk: string) => void,
onModel?: (model: string) => void,
onFirstValidChunk?: () => void,
conversationId?: string,
abortSignal?: AbortSignal
): Promise<void> {
@@ -279,10 +311,50 @@ export class ChatService {
const decoder = new TextDecoder();
let aggregatedContent = '';
let fullReasoningContent = '';
let hasReceivedData = false;
let aggregatedToolCalls: ApiChatCompletionToolCall[] = [];
let lastTimings: ChatMessageTimings | undefined;
let streamFinished = false;
let modelEmitted = false;
let firstValidChunkEmitted = false;
let toolCallIndexOffset = 0;
let hasOpenToolCallBatch = false;
const finalizeOpenToolCallBatch = () => {
if (!hasOpenToolCallBatch) {
return;
}
toolCallIndexOffset = aggregatedToolCalls.length;
hasOpenToolCallBatch = false;
};
const processToolCallDelta = (toolCalls?: ApiChatCompletionToolCallDelta[]) => {
if (!toolCalls || toolCalls.length === 0) {
return;
}
aggregatedToolCalls = this.mergeToolCallDeltas(
aggregatedToolCalls,
toolCalls,
toolCallIndexOffset
);
if (aggregatedToolCalls.length === 0) {
return;
}
hasOpenToolCallBatch = true;
const serializedToolCalls = JSON.stringify(aggregatedToolCalls);
if (!serializedToolCalls) {
return;
}
if (!abortSignal?.aborted) {
onToolCallChunk?.(serializedToolCalls);
}
};
try {
let chunk = '';
@@ -311,17 +383,26 @@ export class ChatService {
try {
const parsed: ApiChatCompletionStreamChunk = JSON.parse(data);
if (!firstValidChunkEmitted && parsed.object === 'chat.completion.chunk') {
firstValidChunkEmitted = true;
if (!abortSignal?.aborted) {
onFirstValidChunk?.();
}
}
const content = parsed.choices[0]?.delta?.content;
const reasoningContent = parsed.choices[0]?.delta?.reasoning_content;
const toolCalls = parsed.choices[0]?.delta?.tool_calls;
const timings = parsed.timings;
const promptProgress = parsed.prompt_progress;
const chunkModel = this.extractModelName(parsed);
if (chunkModel && !modelEmitted) {
modelEmitted = true;
onModel?.(chunkModel);
}
const content = parsed.choices[0]?.delta?.content;
const reasoningContent = parsed.choices[0]?.delta?.reasoning_content;
const timings = parsed.timings;
const promptProgress = parsed.prompt_progress;
if (timings || promptProgress) {
this.updateProcessingState(timings, promptProgress, conversationId);
if (timings) {
@@ -330,7 +411,7 @@ export class ChatService {
}
if (content) {
hasReceivedData = true;
finalizeOpenToolCallBatch();
aggregatedContent += content;
if (!abortSignal?.aborted) {
onChunk?.(content);
@@ -338,12 +419,14 @@ export class ChatService {
}
if (reasoningContent) {
hasReceivedData = true;
finalizeOpenToolCallBatch();
fullReasoningContent += reasoningContent;
if (!abortSignal?.aborted) {
onReasoningChunk?.(reasoningContent);
}
}
processToolCallDelta(toolCalls);
} catch (e) {
console.error('Error parsing JSON chunk:', e);
}
@@ -356,12 +439,17 @@ export class ChatService {
if (abortSignal?.aborted) return;
if (streamFinished) {
if (!hasReceivedData && aggregatedContent.length === 0) {
const noResponseError = new Error('No response received from server. Please try again.');
throw noResponseError;
}
finalizeOpenToolCallBatch();
onComplete?.(aggregatedContent, fullReasoningContent || undefined, lastTimings);
const finalToolCalls =
aggregatedToolCalls.length > 0 ? JSON.stringify(aggregatedToolCalls) : undefined;
onComplete?.(
aggregatedContent,
fullReasoningContent || undefined,
lastTimings,
finalToolCalls
);
}
} catch (error) {
const err = error instanceof Error ? error : new Error('Stream error');
@@ -374,6 +462,54 @@ export class ChatService {
}
}
private mergeToolCallDeltas(
existing: ApiChatCompletionToolCall[],
deltas: ApiChatCompletionToolCallDelta[],
indexOffset = 0
): ApiChatCompletionToolCall[] {
const result = existing.map((call) => ({
...call,
function: call.function ? { ...call.function } : undefined
}));
for (const delta of deltas) {
const index =
typeof delta.index === 'number' && delta.index >= 0
? delta.index + indexOffset
: result.length;
while (result.length <= index) {
result.push({ function: undefined });
}
const target = result[index]!;
if (delta.id) {
target.id = delta.id;
}
if (delta.type) {
target.type = delta.type;
}
if (delta.function) {
const fn = target.function ? { ...target.function } : {};
if (delta.function.name) {
fn.name = delta.function.name;
}
if (delta.function.arguments) {
fn.arguments = (fn.arguments ?? '') + delta.function.arguments;
}
target.function = fn;
}
}
return result;
}
/**
* Handles non-streaming response from the chat completion API.
* Parses the JSON response and extracts the generated content.
@@ -389,9 +525,11 @@ export class ChatService {
onComplete?: (
response: string,
reasoningContent?: string,
timings?: ChatMessageTimings
timings?: ChatMessageTimings,
toolCalls?: string
) => void,
onError?: (error: Error) => void,
onToolCallChunk?: (chunk: string) => void,
onModel?: (model: string) => void
): Promise<string> {
try {
@@ -411,17 +549,31 @@ export class ChatService {
const content = data.choices[0]?.message?.content || '';
const reasoningContent = data.choices[0]?.message?.reasoning_content;
const toolCalls = data.choices[0]?.message?.tool_calls;
if (reasoningContent) {
console.log('Full reasoning content:', reasoningContent);
}
if (!content.trim()) {
let serializedToolCalls: string | undefined;
if (toolCalls && toolCalls.length > 0) {
const mergedToolCalls = this.mergeToolCallDeltas([], toolCalls);
if (mergedToolCalls.length > 0) {
serializedToolCalls = JSON.stringify(mergedToolCalls);
if (serializedToolCalls) {
onToolCallChunk?.(serializedToolCalls);
}
}
}
if (!content.trim() && !serializedToolCalls) {
const noResponseError = new Error('No response received from server. Please try again.');
throw noResponseError;
}
onComplete?.(content, reasoningContent);
onComplete?.(content, reasoningContent, undefined, serializedToolCalls);
return content;
} catch (error) {
examples/server/webui_llamacpp/src/lib/stores/chat.svelte.ts
+350 -4
View File
@@ -1,6 +1,7 @@
import { DatabaseStore } from '$lib/stores/database';
import { chatService, slotsService } from '$lib/services';
import { config } from '$lib/stores/settings.svelte';
import { serverStore } from '$lib/stores/server.svelte';
import { normalizeModelName } from '$lib/utils/model-names';
import { filterByLeafNodeId, findLeafNode, findDescendantMessages } from '$lib/utils/branching';
import { browser } from '$app/environment';
@@ -204,6 +205,7 @@ class ChatStore {
type,
timestamp: Date.now(),
thinking: '',
toolCalls: '',
children: [],
extra: extras
},
@@ -359,12 +361,45 @@ class ChatStore {
): Promise<void> {
let streamedContent = '';
let streamedReasoningContent = '';
let streamedToolCallContent = '';
let resolvedModel: string | null = null;
let modelPersisted = false;
const currentConfig = config();
const preferServerPropsModel = !currentConfig.modelSelectorEnabled;
let serverPropsRefreshed = false;
let updateModelFromServerProps: ((persistImmediately?: boolean) => void) | null = null;
const recordModel = (modelName: string, persistImmediately = true): void => {
const normalizedModel = normalizeModelName(modelName);
const refreshServerPropsOnce = () => {
if (serverPropsRefreshed) {
return;
}
serverPropsRefreshed = true;
const hasExistingProps = serverStore.serverProps !== null;
serverStore
.fetchServerProps({ silent: hasExistingProps })
.then(() => {
updateModelFromServerProps?.(true);
})
.catch((error) => {
console.warn('Failed to refresh server props after streaming started:', error);
});
};
const recordModel = (modelName: string | null | undefined, persistImmediately = true): void => {
const serverModelName = serverStore.modelName;
const preferredModelSource = preferServerPropsModel
? (serverModelName ?? modelName ?? null)
: (modelName ?? serverModelName ?? null);
if (!preferredModelSource) {
return;
}
const normalizedModel = normalizeModelName(preferredModelSource);
if (!normalizedModel || normalizedModel === resolvedModel) {
return;
@@ -388,6 +423,20 @@ class ChatStore {
}
};
if (preferServerPropsModel) {
updateModelFromServerProps = (persistImmediately = true) => {
const currentServerModel = serverStore.modelName;
if (!currentServerModel) {
return;
}
recordModel(currentServerModel, persistImmediately);
};
updateModelFromServerProps(false);
}
slotsService.startStreaming();
slotsService.setActiveConversation(assistantMessage.convId);
@@ -396,6 +445,9 @@ class ChatStore {
{
...this.getApiOptions(),
onFirstValidChunk: () => {
refreshServerPropsOnce();
},
onChunk: (chunk: string) => {
streamedContent += chunk;
this.setConversationStreaming(
@@ -418,6 +470,20 @@ class ChatStore {
this.updateMessageAtIndex(messageIndex, { thinking: streamedReasoningContent });
},
onToolCallChunk: (toolCallChunk: string) => {
const chunk = toolCallChunk.trim();
if (!chunk) {
return;
}
streamedToolCallContent = chunk;
const messageIndex = this.findMessageIndex(assistantMessage.id);
this.updateMessageAtIndex(messageIndex, { toolCalls: streamedToolCallContent });
},
onModel: (modelName: string) => {
recordModel(modelName);
},
@@ -425,18 +491,21 @@ class ChatStore {
onComplete: async (
finalContent?: string,
reasoningContent?: string,
timings?: ChatMessageTimings
timings?: ChatMessageTimings,
toolCallContent?: string
) => {
slotsService.stopStreaming();
const updateData: {
content: string;
thinking: string;
toolCalls: string;
timings?: ChatMessageTimings;
model?: string;
} = {
content: finalContent || streamedContent,
thinking: reasoningContent || streamedReasoningContent,
toolCalls: toolCallContent || streamedToolCallContent,
timings: timings
};
@@ -449,7 +518,11 @@ class ChatStore {
const messageIndex = this.findMessageIndex(assistantMessage.id);
const localUpdateData: { timings?: ChatMessageTimings; model?: string } = {
const localUpdateData: {
timings?: ChatMessageTimings;
model?: string;
toolCalls?: string;
} = {
timings: timings
};
@@ -457,6 +530,10 @@ class ChatStore {
localUpdateData.model = updateData.model;
}
if (updateData.toolCalls !== undefined) {
localUpdateData.toolCalls = updateData.toolCalls;
}
this.updateMessageAtIndex(messageIndex, localUpdateData);
await DatabaseStore.updateCurrentNode(assistantMessage.convId, assistantMessage.id);
@@ -570,6 +647,7 @@ class ChatStore {
content: '',
timestamp: Date.now(),
thinking: '',
toolCalls: '',
children: [],
model: null
},
@@ -1393,6 +1471,7 @@ class ChatStore {
role: messageToEdit.role,
content: newContent,
thinking: messageToEdit.thinking || '',
toolCalls: messageToEdit.toolCalls || '',
children: [],
model: messageToEdit.model // Preserve original model info when branching
},
@@ -1407,6 +1486,10 @@ class ChatStore {
timestamp: Date.now()
});
// Ensure currNode points to the edited message to maintain correct path
await DatabaseStore.updateCurrentNode(this.activeConversation.id, messageToEdit.id);
this.activeConversation.currNode = messageToEdit.id;
this.updateMessageAtIndex(messageIndex, {
content: newContent,
timestamp: Date.now()
@@ -1420,6 +1503,69 @@ class ChatStore {
}
}
/**
* Edits a user message and preserves all responses below
* Updates the message content in-place without deleting or regenerating responses
*
* **Use Case**: When you want to fix a typo or rephrase a question without losing the assistant's response
*
* **Important Behavior:**
* - Does NOT create a branch (unlike editMessageWithBranching)
* - Does NOT regenerate assistant responses
* - Only updates the user message content in the database
* - Preserves the entire conversation tree below the edited message
* - Updates conversation title if this is the first user message
*
* @param messageId - The ID of the user message to edit
* @param newContent - The new content for the message
*/
async editUserMessagePreserveResponses(messageId: string, newContent: string): Promise<void> {
if (!this.activeConversation) return;
try {
const messageIndex = this.findMessageIndex(messageId);
if (messageIndex === -1) {
console.error('Message not found for editing');
return;
}
const messageToEdit = this.activeMessages[messageIndex];
if (messageToEdit.role !== 'user') {
console.error('Only user messages can be edited with this method');
return;
}
// Simply update the message content in-place
await DatabaseStore.updateMessage(messageId, {
content: newContent,
timestamp: Date.now()
});
this.updateMessageAtIndex(messageIndex, {
content: newContent,
timestamp: Date.now()
});
// Check if first user message for title update
const allMessages = await DatabaseStore.getConversationMessages(this.activeConversation.id);
const rootMessage = allMessages.find((m) => m.type === 'root' && m.parent === null);
const isFirstUserMessage =
rootMessage && messageToEdit.parent === rootMessage.id && messageToEdit.role === 'user';
if (isFirstUserMessage && newContent.trim()) {
await this.updateConversationTitleWithConfirmation(
this.activeConversation.id,
newContent.trim(),
this.titleUpdateConfirmationCallback
);
}
this.updateConversationTimestamp();
} catch (error) {
console.error('Failed to edit user message:', error);
}
}
/**
* Edits a message by creating a new branch with the edited content
* @param messageId - The ID of the message to edit
@@ -1468,6 +1614,7 @@ class ChatStore {
role: messageToEdit.role,
content: newContent,
thinking: messageToEdit.thinking || '',
toolCalls: messageToEdit.toolCalls || '',
children: [],
extra: messageToEdit.extra ? JSON.parse(JSON.stringify(messageToEdit.extra)) : undefined,
model: messageToEdit.model // Preserve original model info when branching
@@ -1539,6 +1686,7 @@ class ChatStore {
role: 'assistant',
content: '',
thinking: '',
toolCalls: '',
children: [],
model: null
},
@@ -1597,6 +1745,7 @@ class ChatStore {
role: 'assistant',
content: '',
thinking: '',
toolCalls: '',
children: [],
model: null
},
@@ -1614,6 +1763,200 @@ class ChatStore {
}
}
/**
* Continues generation for an existing assistant message
* @param messageId - The ID of the assistant message to continue
*/
async continueAssistantMessage(messageId: string): Promise<void> {
if (!this.activeConversation || this.isLoading) return;
try {
const messageIndex = this.findMessageIndex(messageId);
if (messageIndex === -1) {
console.error('Message not found for continuation');
return;
}
const messageToContinue = this.activeMessages[messageIndex];
if (messageToContinue.role !== 'assistant') {
console.error('Only assistant messages can be continued');
return;
}
// Race condition protection: Check if this specific conversation is already loading
// This prevents multiple rapid clicks on "Continue" from creating concurrent operations
if (this.isConversationLoading(this.activeConversation.id)) {
console.warn('Continuation already in progress for this conversation');
return;
}
this.errorDialogState = null;
this.setConversationLoading(this.activeConversation.id, true);
this.clearConversationStreaming(this.activeConversation.id);
// IMPORTANT: Fetch the latest content from the database to ensure we have
// the most up-to-date content, especially after a stopped generation
// This prevents issues where the in-memory state might be stale
const allMessages = await DatabaseStore.getConversationMessages(this.activeConversation.id);
const dbMessage = allMessages.find((m) => m.id === messageId);
if (!dbMessage) {
console.error('Message not found in database for continuation');
this.setConversationLoading(this.activeConversation.id, false);
return;
}
// Use content from database as the source of truth
const originalContent = dbMessage.content;
const originalThinking = dbMessage.thinking || '';
// Get conversation context up to (but not including) the message to continue
const conversationContext = this.activeMessages.slice(0, messageIndex);
const contextWithContinue = [
...conversationContext.map((msg) => {
if ('id' in msg && 'convId' in msg && 'timestamp' in msg) {
return msg as DatabaseMessage & { extra?: DatabaseMessageExtra[] };
}
return msg as ApiChatMessageData;
}),
{
role: 'assistant' as const,
content: originalContent
}
];
let appendedContent = '';
let appendedThinking = '';
let hasReceivedContent = false;
await chatService.sendMessage(
contextWithContinue,
{
...this.getApiOptions(),
onChunk: (chunk: string) => {
hasReceivedContent = true;
appendedContent += chunk;
// Preserve originalContent exactly as-is, including any trailing whitespace
// The concatenation naturally preserves any whitespace at the end of originalContent
const fullContent = originalContent + appendedContent;
this.setConversationStreaming(
messageToContinue.convId,
fullContent,
messageToContinue.id
);
this.updateMessageAtIndex(messageIndex, {
content: fullContent
});
},
onReasoningChunk: (reasoningChunk: string) => {
hasReceivedContent = true;
appendedThinking += reasoningChunk;
const fullThinking = originalThinking + appendedThinking;
this.updateMessageAtIndex(messageIndex, {
thinking: fullThinking
});
},
onComplete: async (
finalContent?: string,
reasoningContent?: string,
timings?: ChatMessageTimings
) => {
const fullContent = originalContent + (finalContent || appendedContent);
const fullThinking = originalThinking + (reasoningContent || appendedThinking);
const updateData: {
content: string;
thinking: string;
timestamp: number;
timings?: ChatMessageTimings;
} = {
content: fullContent,
thinking: fullThinking,
timestamp: Date.now(),
timings: timings
};
await DatabaseStore.updateMessage(messageToContinue.id, updateData);
this.updateMessageAtIndex(messageIndex, updateData);
this.updateConversationTimestamp();
this.setConversationLoading(messageToContinue.convId, false);
this.clearConversationStreaming(messageToContinue.convId);
slotsService.clearConversationState(messageToContinue.convId);
},
onError: async (error: Error) => {
if (this.isAbortError(error)) {
// User cancelled - save partial continuation if any content was received
if (hasReceivedContent && appendedContent) {
const partialContent = originalContent + appendedContent;
const partialThinking = originalThinking + appendedThinking;
await DatabaseStore.updateMessage(messageToContinue.id, {
content: partialContent,
thinking: partialThinking,
timestamp: Date.now()
});
this.updateMessageAtIndex(messageIndex, {
content: partialContent,
thinking: partialThinking,
timestamp: Date.now()
});
}
this.setConversationLoading(messageToContinue.convId, false);
this.clearConversationStreaming(messageToContinue.convId);
slotsService.clearConversationState(messageToContinue.convId);
return;
}
// Non-abort error - rollback to original content
console.error('Continue generation error:', error);
// Rollback: Restore original content in UI
this.updateMessageAtIndex(messageIndex, {
content: originalContent,
thinking: originalThinking
});
// Ensure database has original content (in case of partial writes)
await DatabaseStore.updateMessage(messageToContinue.id, {
content: originalContent,
thinking: originalThinking
});
this.setConversationLoading(messageToContinue.convId, false);
this.clearConversationStreaming(messageToContinue.convId);
slotsService.clearConversationState(messageToContinue.convId);
const dialogType = error.name === 'TimeoutError' ? 'timeout' : 'server';
this.showErrorDialog(dialogType, error.message);
}
},
messageToContinue.convId
);
} catch (error) {
if (this.isAbortError(error)) return;
console.error('Failed to continue message:', error);
if (this.activeConversation) {
this.setConversationLoading(this.activeConversation.id, false);
}
}
}
/**
* Public methods for accessing per-conversation states
*/
@@ -1661,8 +2004,11 @@ export const refreshActiveMessages = chatStore.refreshActiveMessages.bind(chatSt
export const navigateToSibling = chatStore.navigateToSibling.bind(chatStore);
export const editAssistantMessage = chatStore.editAssistantMessage.bind(chatStore);
export const editMessageWithBranching = chatStore.editMessageWithBranching.bind(chatStore);
export const editUserMessagePreserveResponses =
chatStore.editUserMessagePreserveResponses.bind(chatStore);
export const regenerateMessageWithBranching =
chatStore.regenerateMessageWithBranching.bind(chatStore);
export const continueAssistantMessage = chatStore.continueAssistantMessage.bind(chatStore);
export const deleteMessage = chatStore.deleteMessage.bind(chatStore);
export const getDeletionInfo = chatStore.getDeletionInfo.bind(chatStore);
export const updateConversationName = chatStore.updateConversationName.bind(chatStore);
examples/server/webui_llamacpp/src/lib/stores/database.ts
+2
View File
@@ -114,6 +114,7 @@ export class DatabaseStore {
...message,
id: uuid(),
parent: parentId,
toolCalls: message.toolCalls ?? '',
children: []
};
@@ -154,6 +155,7 @@ export class DatabaseStore {
content: '',
parent: null,
thinking: '',
toolCalls: '',
children: []
};
examples/server/webui_llamacpp/src/lib/stores/server.svelte.ts
+111 -62
View File
@@ -52,6 +52,7 @@ class ServerStore {
private _error = $state<string | null>(null);
private _serverWarning = $state<string | null>(null);
private _slotsEndpointAvailable = $state<boolean | null>(null);
private fetchServerPropsPromise: Promise<void> | null = null;
private readCachedServerProps(): ApiLlamaCppServerProps | null {
if (!browser) return null;
@@ -98,6 +99,9 @@ class ServerStore {
}
get modelName(): string | null {
if (this._serverProps?.model_alias) {
return this._serverProps.model_alias;
}
if (!this._serverProps?.model_path) return null;
return this._serverProps.model_path.split(/(\\|\/)/).pop() || null;
}
@@ -171,73 +175,65 @@ class ServerStore {
/**
* Fetches server properties from the server
*/
async fetchServerProps(): Promise<void> {
this._loading = true;
this._error = null;
this._serverWarning = null;
async fetchServerProps(options: { silent?: boolean } = {}): Promise<void> {
const { silent = false } = options;
const isSilent = silent && this._serverProps !== null;
try {
console.log('Fetching server properties...');
const props = await ChatService.getServerProps();
this._serverProps = props;
this.persistServerProps(props);
console.log('Server properties loaded:', props);
if (this.fetchServerPropsPromise) {
return this.fetchServerPropsPromise;
}
// Check slots endpoint availability after server props are loaded
await this.checkSlotsEndpointAvailability();
} catch (error) {
const hadCachedProps = this._serverProps !== null;
let errorMessage = 'Failed to connect to server';
let isOfflineLikeError = false;
let isServerSideError = false;
if (!isSilent) {
this._loading = true;
this._error = null;
this._serverWarning = null;
}
if (error instanceof Error) {
// Handle specific error types with user-friendly messages
if (error.name === 'TypeError' && error.message.includes('fetch')) {
errorMessage = 'Server is not running or unreachable';
isOfflineLikeError = true;
} else if (error.message.includes('ECONNREFUSED')) {
errorMessage = 'Connection refused - server may be offline';
isOfflineLikeError = true;
} else if (error.message.includes('ENOTFOUND')) {
errorMessage = 'Server not found - check server address';
isOfflineLikeError = true;
} else if (error.message.includes('ETIMEDOUT')) {
errorMessage = 'Request timed out - the server took too long to respond';
isOfflineLikeError = true;
} else if (error.message.includes('503')) {
errorMessage = 'Server temporarily unavailable - try again shortly';
isServerSideError = true;
} else if (error.message.includes('500')) {
errorMessage = 'Server error - check server logs';
isServerSideError = true;
} else if (error.message.includes('404')) {
errorMessage = 'Server endpoint not found';
} else if (error.message.includes('403') || error.message.includes('401')) {
errorMessage = 'Access denied';
const hadProps = this._serverProps !== null;
const fetchPromise = (async () => {
try {
const props = await ChatService.getServerProps();
this._serverProps = props;
this.persistServerProps(props);
this._error = null;
this._serverWarning = null;
await this.checkSlotsEndpointAvailability();
} catch (error) {
if (isSilent && hadProps) {
console.warn('Silent server props refresh failed, keeping cached data:', error);
return;
}
this.handleFetchServerPropsError(error, hadProps);
} finally {
if (!isSilent) {
this._loading = false;
}
this.fetchServerPropsPromise = null;
}
})();
let cachedProps: ApiLlamaCppServerProps | null = null;
this.fetchServerPropsPromise = fetchPromise;
if (!hadCachedProps) {
cachedProps = this.readCachedServerProps();
if (cachedProps) {
this._serverProps = cachedProps;
this._error = null;
await fetchPromise;
}
if (isOfflineLikeError || isServerSideError) {
this._serverWarning = errorMessage;
}
/**
* Handles fetch failures by attempting to recover cached server props and
* updating the user-facing error or warning state appropriately.
*/
private handleFetchServerPropsError(error: unknown, hadProps: boolean): void {
const { errorMessage, isOfflineLikeError, isServerSideError } = this.normalizeFetchError(error);
console.warn(
'Failed to refresh server properties, using cached values from localStorage:',
errorMessage
);
} else {
this._error = errorMessage;
}
} else {
let cachedProps: ApiLlamaCppServerProps | null = null;
if (!hadProps) {
cachedProps = this.readCachedServerProps();
if (cachedProps) {
this._serverProps = cachedProps;
this._error = null;
if (isOfflineLikeError || isServerSideError) {
@@ -245,14 +241,66 @@ class ServerStore {
}
console.warn(
'Failed to refresh server properties, continuing with cached values:',
'Failed to refresh server properties, using cached values from localStorage:',
errorMessage
);
} else {
this._error = errorMessage;
}
console.error('Error fetching server properties:', error);
} finally {
this._loading = false;
} else {
this._error = null;
if (isOfflineLikeError || isServerSideError) {
this._serverWarning = errorMessage;
}
console.warn(
'Failed to refresh server properties, continuing with cached values:',
errorMessage
);
}
console.error('Error fetching server properties:', error);
}
private normalizeFetchError(error: unknown): {
errorMessage: string;
isOfflineLikeError: boolean;
isServerSideError: boolean;
} {
let errorMessage = 'Failed to connect to server';
let isOfflineLikeError = false;
let isServerSideError = false;
if (error instanceof Error) {
const message = error.message || '';
if (error.name === 'TypeError' && message.includes('fetch')) {
errorMessage = 'Server is not running or unreachable';
isOfflineLikeError = true;
} else if (message.includes('ECONNREFUSED')) {
errorMessage = 'Connection refused - server may be offline';
isOfflineLikeError = true;
} else if (message.includes('ENOTFOUND')) {
errorMessage = 'Server not found - check server address';
isOfflineLikeError = true;
} else if (message.includes('ETIMEDOUT')) {
errorMessage = 'Request timed out - the server took too long to respond';
isOfflineLikeError = true;
} else if (message.includes('503')) {
errorMessage = 'Server temporarily unavailable - try again shortly';
isServerSideError = true;
} else if (message.includes('500')) {
errorMessage = 'Server error - check server logs';
isServerSideError = true;
} else if (message.includes('404')) {
errorMessage = 'Server endpoint not found';
} else if (message.includes('403') || message.includes('401')) {
errorMessage = 'Access denied';
}
}
return { errorMessage, isOfflineLikeError, isServerSideError };
}
/**
@@ -264,6 +312,7 @@ class ServerStore {
this._serverWarning = null;
this._loading = false;
this._slotsEndpointAvailable = null;
this.fetchServerPropsPromise = null;
this.persistServerProps(null);
}
}
examples/server/webui_llamacpp/src/lib/types/api.d.ts
Vendored
+20
View File
@@ -183,9 +183,27 @@ export interface ApiChatCompletionRequest {
samplers?: string[];
// Custom parameters (JSON string)
custom?: Record<string, unknown>;
timings_per_token?: boolean;
}
export interface ApiChatCompletionToolCallFunctionDelta {
name?: string;
arguments?: string;
}
export interface ApiChatCompletionToolCallDelta {
index?: number;
id?: string;
type?: string;
function?: ApiChatCompletionToolCallFunctionDelta;
}
export interface ApiChatCompletionToolCall extends ApiChatCompletionToolCallDelta {
function?: ApiChatCompletionToolCallFunctionDelta & { arguments?: string };
}
export interface ApiChatCompletionStreamChunk {
object?: string;
model?: string;
choices: Array<{
model?: string;
@@ -194,6 +212,7 @@ export interface ApiChatCompletionStreamChunk {
content?: string;
reasoning_content?: string;
model?: string;
tool_calls?: ApiChatCompletionToolCallDelta[];
};
}>;
timings?: {
@@ -215,6 +234,7 @@ export interface ApiChatCompletionResponse {
content: string;
reasoning_content?: string;
model?: string;
tool_calls?: ApiChatCompletionToolCallDelta[];
};
}>;
}
examples/server/webui_llamacpp/src/lib/types/chat.d.ts
Vendored
+23
View File
@@ -11,6 +11,29 @@ export interface ChatUploadedFile {
textContent?: string;
}
export interface ChatAttachmentDisplayItem {
id: string;
name: string;
size?: number;
preview?: string;
type: string;
isImage: boolean;
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
attachmentIndex?: number;
textContent?: string;
}
export interface ChatAttachmentPreviewItem {
uploadedFile?: ChatUploadedFile;
attachment?: DatabaseMessageExtra;
preview?: string;
name?: string;
type?: string;
size?: number;
textContent?: string;
}
export interface ChatMessageSiblingInfo {
message: DatabaseMessage;
siblingIds: string[];
examples/server/webui_llamacpp/src/lib/types/database.d.ts
Vendored
+1
View File
@@ -60,6 +60,7 @@ export interface DatabaseMessage {
content: string;
parent: string;
thinking: string;
toolCalls?: string;
children: string[];
extra?: DatabaseMessageExtra[];
timings?: ChatMessageTimings;
examples/server/webui_llamacpp/src/lib/types/settings.d.ts
Vendored
+10 -1
View File
@@ -7,6 +7,7 @@ export interface SettingsFieldConfig {
key: string;
label: string;
type: 'input' | 'textarea' | 'checkbox' | 'select';
isExperimental?: boolean;
help?: string;
options?: Array<{ value: string; label: string; icon?: typeof import('@lucide/svelte').Icon }>;
}
@@ -38,11 +39,19 @@ export interface SettingsChatServiceOptions {
samplers?: string | string[];
// Custom parameters
custom?: string;
timings_per_token?: boolean;
// Callbacks
onChunk?: (chunk: string) => void;
onReasoningChunk?: (chunk: string) => void;
onToolCallChunk?: (chunk: string) => void;
onModel?: (model: string) => void;
onComplete?: (response: string, reasoningContent?: string, timings?: ChatMessageTimings) => void;
onFirstValidChunk?: () => void;
onComplete?: (
response: string,
reasoningContent?: string,
timings?: ChatMessageTimings,
toolCalls?: string
) => void;
onError?: (error: Error) => void;
}
examples/server/webui_llamacpp/src/lib/utils/latex-protection.test.ts
+355
View File
@@ -0,0 +1,355 @@
/* eslint-disable no-irregular-whitespace */
import { describe, it, expect, test } from 'vitest';
import { maskInlineLaTeX, preprocessLaTeX } from './latex-protection';
describe('maskInlineLaTeX', () => {
it('should protect LaTeX $x + y$ but not money $3.99', () => {
const latexExpressions: string[] = [];
const input = 'I have $10, $3.99 and $x + y$ and $100x$. The amount is $2,000.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('I have $10, $3.99 and <<LATEX_0>> and <<LATEX_1>>. The amount is $2,000.');
expect(latexExpressions).toEqual(['$x + y$', '$100x$']);
});
it('should ignore money like $5 and $12.99', () => {
const latexExpressions: string[] = [];
const input = 'Prices are $12.99 and $5. Tax?';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('Prices are $12.99 and $5. Tax?');
expect(latexExpressions).toEqual([]);
});
it('should protect inline math $a^2 + b^2$ even after text', () => {
const latexExpressions: string[] = [];
const input = 'Pythagorean: $a^2 + b^2 = c^2$.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('Pythagorean: <<LATEX_0>>.');
expect(latexExpressions).toEqual(['$a^2 + b^2 = c^2$']);
});
it('should not protect math that has letter after closing $ (e.g. units)', () => {
const latexExpressions: string[] = [];
const input = 'The cost is $99 and change.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('The cost is $99 and change.');
expect(latexExpressions).toEqual([]);
});
it('should allow $x$ followed by punctuation', () => {
const latexExpressions: string[] = [];
const input = 'We know $x$, right?';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('We know <<LATEX_0>>, right?');
expect(latexExpressions).toEqual(['$x$']);
});
it('should work across multiple lines', () => {
const latexExpressions: string[] = [];
const input = `Emma buys cupcakes for $3 each.\nHow much is $x + y$?`;
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe(`Emma buys cupcakes for $3 each.\nHow much is <<LATEX_0>>?`);
expect(latexExpressions).toEqual(['$x + y$']);
});
it('should not protect $100 but protect $matrix$', () => {
const latexExpressions: string[] = [];
const input = '$100 and $\\mathrm{GL}_2(\\mathbb{F}_7)$ are different.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('$100 and <<LATEX_0>> are different.');
expect(latexExpressions).toEqual(['$\\mathrm{GL}_2(\\mathbb{F}_7)$']);
});
it('should skip if $ is followed by digit and alphanumeric after close (money)', () => {
const latexExpressions: string[] = [];
const input = 'I paid $5 quickly.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('I paid $5 quickly.');
expect(latexExpressions).toEqual([]);
});
it('should protect LaTeX even with special chars inside', () => {
const latexExpressions: string[] = [];
const input = 'Consider $\\alpha_1 + \\beta_2$ now.';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('Consider <<LATEX_0>> now.');
expect(latexExpressions).toEqual(['$\\alpha_1 + \\beta_2$']);
});
it('short text', () => {
const latexExpressions: string[] = ['$0$'];
const input = '$a$\n$a$ and $b$';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('<<LATEX_1>>\n<<LATEX_2>> and <<LATEX_3>>');
expect(latexExpressions).toEqual(['$0$', '$a$', '$a$', '$b$']);
});
it('empty text', () => {
const latexExpressions: string[] = [];
const input = '$\n$$\n';
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe('$\n$$\n');
expect(latexExpressions).toEqual([]);
});
it('LaTeX-spacer preceded by backslash', () => {
const latexExpressions: string[] = [];
const input = `\\[
\\boxed{
\\begin{aligned}
N_{\\text{att}}^{\\text{(MHA)}} &=
h \\bigl[\\, d_{\\text{model}}\\;d_{k} + d_{\\text{model}}\\;d_{v}\\, \\bigr] && (\\text{Q,K,V の重み})\\\\
&\\quad+ h(d_{k}+d_{k}+d_{v}) && (\\text{バイアス Q,K,V）}\\\\[4pt]
&\\quad+ (h d_{v})\\, d_{\\text{model}} && (\\text{出力射影 }W^{O})\\\\
&\\quad+ d_{\\text{model}} && (\\text{バイアス }b^{O})
\\end{aligned}}
\\]`;
const output = maskInlineLaTeX(input, latexExpressions);
expect(output).toBe(input);
expect(latexExpressions).toEqual([]);
});
});
describe('preprocessLaTeX', () => {
test('converts inline \\( ... \\) to $...$', () => {
const input =
'\\( \\mathrm{GL}_2(\\mathbb{F}_7) \\): Group of invertible matrices with entries in \\(\\mathbb{F}_7\\).';
const output = preprocessLaTeX(input);
expect(output).toBe(
'$ \\mathrm{GL}_2(\\mathbb{F}_7) $: Group of invertible matrices with entries in $\\mathbb{F}_7$.'
);
});
test("don't inline \\\\( ... \\) to $...$", () => {
const input =
'Chapter 20 of The TeXbook, in source "Definitions\\\\(also called Macros)", containst the formula \\((x_1,\\ldots,x_n)\\).';
const output = preprocessLaTeX(input);
expect(output).toBe(
'Chapter 20 of The TeXbook, in source "Definitions\\\\(also called Macros)", containst the formula $(x_1,\\ldots,x_n)$.'
);
});
test('preserves display math \\[ ... \\] and protects adjacent text', () => {
const input = `Some kernel of \\(\\mathrm{SL}_2(\\mathbb{F}_7)\\):
\\[
\\left\\{ \\begin{pmatrix} 1 & 0 \\\\ 0 & 1 \\end{pmatrix}, \\begin{pmatrix} -1 & 0 \\\\ 0 & -1 \\end{pmatrix} \\right\\} = \\{\\pm I\\}
\\]`;
const output = preprocessLaTeX(input);
expect(output).toBe(`Some kernel of $\\mathrm{SL}_2(\\mathbb{F}_7)$:
$$
\\left\\{ \\begin{pmatrix} 1 & 0 \\\\ 0 & 1 \\end{pmatrix}, \\begin{pmatrix} -1 & 0 \\\\ 0 & -1 \\end{pmatrix} \\right\\} = \\{\\pm I\\}
$$`);
});
test('handles standalone display math equation', () => {
const input = `Algebra:
\\[
x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}
\\]`;
const output = preprocessLaTeX(input);
expect(output).toBe(`Algebra:
$$
x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}
$$`);
});
test('does not interpret currency values as LaTeX', () => {
const input = 'I have $10, $3.99 and $x + y$ and $100x$. The amount is $2,000.';
const output = preprocessLaTeX(input);
expect(output).toBe('I have \\$10, \\$3.99 and $x + y$ and $100x$. The amount is \\$2,000.');
});
test('ignores dollar signs followed by digits (money), but keeps valid math $x + y$', () => {
const input = 'I have $10, $3.99 and $x + y$ and $100x$. The amount is $2,000.';
const output = preprocessLaTeX(input);
expect(output).toBe('I have \\$10, \\$3.99 and $x + y$ and $100x$. The amount is \\$2,000.');
});
test('handles real-world word problems with amounts and no math delimiters', () => {
const input =
'Emma buys 2 cupcakes for $3 each and 1 cookie for $1.50. How much money does she spend in total?';
const output = preprocessLaTeX(input);
expect(output).toBe(
'Emma buys 2 cupcakes for \\$3 each and 1 cookie for \\$1.50. How much money does she spend in total?'
);
});
test('handles decimal amounts in word problem correctly', () => {
const input =
'Maria has $20. She buys a notebook for $4.75 and a pack of pencils for $3.25. How much change does she receive?';
const output = preprocessLaTeX(input);
expect(output).toBe(
'Maria has \\$20. She buys a notebook for \\$4.75 and a pack of pencils for \\$3.25. How much change does she receive?'
);
});
test('preserves display math with surrounding non-ASCII text', () => {
const input = `1 kg の質量は
\\[
E = (1\\ \\text{kg}) \\times (3.0 \\times 10^8\\ \\text{m/s})^2 \\approx 9.0 \\times 10^{16}\\ \\text{J}
\\]
というエネルギーに相当します。これは約 21 百万トンの TNT が爆発したときのエネルギーに匹敵します。`;
const output = preprocessLaTeX(input);
expect(output).toBe(
`1 kg の質量は
$$
E = (1\\ \\text{kg}) \\times (3.0 \\times 10^8\\ \\text{m/s})^2 \\approx 9.0 \\times 10^{16}\\ \\text{J}
$$
というエネルギーに相当します。これは約 21 百万トンの TNT が爆発したときのエネルギーに匹敵します。`
);
});
test('LaTeX-spacer preceded by backslash', () => {
const input = `\\[
\\boxed{
\\begin{aligned}
N_{\\text{att}}^{\\text{(MHA)}} &=
h \\bigl[\\, d_{\\text{model}}\\;d_{k} + d_{\\text{model}}\\;d_{v}\\, \\bigr] && (\\text{Q,K,V の重み})\\\\
&\\quad+ h(d_{k}+d_{k}+d_{v}) && (\\text{バイアス Q,K,V）}\\\\[4pt]
&\\quad+ (h d_{v})\\, d_{\\text{model}} && (\\text{出力射影 }W^{O})\\\\
&\\quad+ d_{\\text{model}} && (\\text{バイアス }b^{O})
\\end{aligned}}
\\]`;
const output = preprocessLaTeX(input);
expect(output).toBe(
`$$
\\boxed{
\\begin{aligned}
N_{\\text{att}}^{\\text{(MHA)}} &=
h \\bigl[\\, d_{\\text{model}}\\;d_{k} + d_{\\text{model}}\\;d_{v}\\, \\bigr] && (\\text{Q,K,V の重み})\\\\
&\\quad+ h(d_{k}+d_{k}+d_{v}) && (\\text{バイアス Q,K,V）}\\\\[4pt]
&\\quad+ (h d_{v})\\, d_{\\text{model}} && (\\text{出力射影 }W^{O})\\\\
&\\quad+ d_{\\text{model}} && (\\text{バイアス }b^{O})
\\end{aligned}}
$$`
);
});
test('converts \\[ ... \\] even when preceded by text without space', () => {
const input = 'Some line ...\nAlgebra: \\[x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}\\]';
const output = preprocessLaTeX(input);
expect(output).toBe(
'Some line ...\nAlgebra: \n$$x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}$$\n'
);
});
test('converts \\[ ... \\] in table-cells', () => {
const input = `| ID | Expression |\n| #1 | \\[
x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}
\\] |`;
const output = preprocessLaTeX(input);
expect(output).toBe(
'| ID | Expression |\n| #1 | $x = \\frac{-b \\pm \\sqrt{\\,b^{2}-4ac\\,}}{2a}$ |'
);
});
test('escapes isolated $ before digits ($5 → \\$5), but not valid math', () => {
const input = 'This costs $5 and this is math $x^2$. $100 is money.';
const output = preprocessLaTeX(input);
expect(output).toBe('This costs \\$5 and this is math $x^2$. \\$100 is money.');
// Note: Since $x^2$ is detected as valid LaTeX, it's preserved.
// $5 becomes \$5 only *after* real math is masked — but here it's correct because the masking logic avoids treating $5 as math.
});
test('display with LaTeX-line-breaks', () => {
const input = String.raw`- Algebraic topology, Homotopy Groups of $\mathbb{S}^3$:
$$\pi_n(\mathbb{S}^3) = \begin{cases}
\mathbb{Z} & n = 3 \\
0 & n > 3, n \neq 4 \\
\mathbb{Z}_2 & n = 4 \\
\end{cases}$$`;
const output = preprocessLaTeX(input);
// If the formula contains '\\' the $$-delimiters should be in their own line.
expect(output).toBe(`- Algebraic topology, Homotopy Groups of $\\mathbb{S}^3$:
$$\n\\pi_n(\\mathbb{S}^3) = \\begin{cases}
\\mathbb{Z} & n = 3 \\\\
0 & n > 3, n \\neq 4 \\\\
\\mathbb{Z}_2 & n = 4 \\\\
\\end{cases}\n$$`);
});
test('handles mhchem notation safely if present', () => {
const input = 'Chemical reaction: \\( \\ce{H2O} \\) and $\\ce{CO2}$';
const output = preprocessLaTeX(input);
expect(output).toBe('Chemical reaction: $ \\ce{H2O} $ and $\\ce{CO2}$');
});
test('preserves code blocks', () => {
const input = 'Inline code: `sum $total` and block:\n```\ndollar $amount\n```\nEnd.';
const output = preprocessLaTeX(input);
expect(output).toBe(input); // Code blocks prevent misinterpretation
});
test('escape backslash in mchem ce', () => {
const input = 'mchem ce:\n$\\ce{2H2(g) + O2(g) -> 2H2O(l)}$';
const output = preprocessLaTeX(input);
// mhchem-escape would insert a backslash here.
expect(output).toBe('mchem ce:\n$\\ce{2H2(g) + O2(g) -> 2H2O(l)}$');
});
test('escape backslash in mchem pu', () => {
const input = 'mchem pu:\n$\\pu{-572 kJ mol^{-1}}$';
const output = preprocessLaTeX(input);
// mhchem-escape would insert a backslash here.
expect(output).toBe('mchem pu:\n$\\pu{-572 kJ mol^{-1}}$');
});
test('LaTeX in blockquotes with display math', () => {
const input =
'> **Definition (limit):** \n> \\[\n> \\lim_{x\\to a} f(x) = L\n> \\]\n> means that as \\(x\\) gets close to \\(a\\).';
const output = preprocessLaTeX(input);
// Blockquote markers should be preserved, LaTeX should be converted
expect(output).toContain('> **Definition (limit):**');
expect(output).toContain('$$');
expect(output).toContain('$x$');
expect(output).not.toContain('\\[');
expect(output).not.toContain('\\]');
expect(output).not.toContain('\\(');
expect(output).not.toContain('\\)');
});
test('LaTeX in blockquotes with inline math', () => {
const input =
"> The derivative \\(f'(x)\\) at point \\(x=a\\) measures slope.\n> Formula: \\(f'(a)=\\lim_{h\\to 0}\\frac{f(a+h)-f(a)}{h}\\)";
const output = preprocessLaTeX(input);
// Blockquote markers should be preserved, inline LaTeX converted to $...$
expect(output).toContain("> The derivative $f'(x)$ at point $x=a$ measures slope.");
expect(output).toContain("> Formula: $f'(a)=\\lim_{h\\to 0}\\frac{f(a+h)-f(a)}{h}$");
});
test('Mixed content with blockquotes and regular text', () => {
const input =
'Regular text with \\(x^2\\).\n\n> Quote with \\(y^2\\).\n\nMore text with \\(z^2\\).';
const output = preprocessLaTeX(input);
// All LaTeX should be converted, blockquote markers preserved
expect(output).toBe('Regular text with $x^2$.\n\n> Quote with $y^2$.\n\nMore text with $z^2$.');
});
});
examples/server/webui_llamacpp/src/lib/utils/latex-protection.ts
+267
View File
@@ -0,0 +1,267 @@
import {
CODE_BLOCK_REGEXP,
LATEX_MATH_AND_CODE_PATTERN,
LATEX_LINEBREAK_REGEXP,
MHCHEM_PATTERN_MAP
} from '$lib/constants/latex-protection';
/**
* Replaces inline LaTeX expressions enclosed in `$...$` with placeholders, avoiding dollar signs
* that appear to be part of monetary values or identifiers.
*
* This function processes the input line by line and skips `$` sequences that are likely
* part of money amounts (e.g., `$5`, `$100.99`) or code-like tokens (e.g., `var$`, `$var`).
* Valid LaTeX inline math is replaced with a placeholder like `<<LATEX_0>>`, and the
* actual LaTeX content is stored in the provided `latexExpressions` array.
*
* @param content - The input text potentially containing LaTeX expressions.
* @param latexExpressions - An array used to collect extracted LaTeX expressions.
* @returns The processed string with LaTeX replaced by placeholders.
*/
export function maskInlineLaTeX(content: string, latexExpressions: string[]): string {
if (!content.includes('$')) {
return content;
}
return content
.split('\n')
.map((line) => {
if (line.indexOf('$') == -1) {
return line;
}
let processedLine = '';
let currentPosition = 0;
while (currentPosition < line.length) {
const openDollarIndex = line.indexOf('$', currentPosition);
if (openDollarIndex == -1) {
processedLine += line.slice(currentPosition);
break;
}
// Is there a next $-sign?
const closeDollarIndex = line.indexOf('$', openDollarIndex + 1);
if (closeDollarIndex == -1) {
processedLine += line.slice(currentPosition);
break;
}
const charBeforeOpen = openDollarIndex > 0 ? line[openDollarIndex - 1] : '';
const charAfterOpen = line[openDollarIndex + 1];
const charBeforeClose =
openDollarIndex + 1 < closeDollarIndex ? line[closeDollarIndex - 1] : '';
const charAfterClose = closeDollarIndex + 1 < line.length ? line[closeDollarIndex + 1] : '';
let shouldSkipAsNonLatex = false;
if (closeDollarIndex == currentPosition + 1) {
// No content
shouldSkipAsNonLatex = true;
}
if (/[A-Za-z0-9_$-]/.test(charBeforeOpen)) {
// Character, digit, $, _ or - before first '$', no TeX.
shouldSkipAsNonLatex = true;
}
if (
/[0-9]/.test(charAfterOpen) &&
(/[A-Za-z0-9_$-]/.test(charAfterClose) || ' ' == charBeforeClose)
) {
// First $ seems to belong to an amount.
shouldSkipAsNonLatex = true;
}
if (shouldSkipAsNonLatex) {
processedLine += line.slice(currentPosition, openDollarIndex + 1);
currentPosition = openDollarIndex + 1;
continue;
}
// Treat as LaTeX
processedLine += line.slice(currentPosition, openDollarIndex);
const latexContent = line.slice(openDollarIndex, closeDollarIndex + 1);
latexExpressions.push(latexContent);
processedLine += `<<LATEX_${latexExpressions.length - 1}>>`;
currentPosition = closeDollarIndex + 1;
}
return processedLine;
})
.join('\n');
}
function escapeBrackets(text: string): string {
return text.replace(
LATEX_MATH_AND_CODE_PATTERN,
(
match: string,
codeBlock: string | undefined,
squareBracket: string | undefined,
roundBracket: string | undefined
): string => {
if (codeBlock != null) {
return codeBlock;
} else if (squareBracket != null) {
return `$$${squareBracket}$$`;
} else if (roundBracket != null) {
return `$${roundBracket}$`;
}
return match;
}
);
}
// Escape $\\ce{...} → $\\ce{...} but with proper handling
function escapeMhchem(text: string): string {
return MHCHEM_PATTERN_MAP.reduce((result, [pattern, replacement]) => {
return result.replace(pattern, replacement);
}, text);
}
const doEscapeMhchem = false;
/**
* Preprocesses markdown content to safely handle LaTeX math expressions while protecting
* against false positives (e.g., dollar amounts like $5.99) and ensuring proper rendering.
*
* This function:
* - Protects code blocks (```) and inline code (`...`)
* - Safeguards block and inline LaTeX: \(...\), \[...\], $$...$$, and selective $...$
* - Escapes standalone dollar signs before numbers (e.g., $5 → \$5) to prevent misinterpretation
* - Restores protected LaTeX and code blocks after processing
* - Converts \(...\) → $...$ and \[...\] → $$...$$ for compatibility with math renderers
* - Applies additional escaping for brackets and mhchem syntax if needed
*
* @param content - The raw text (e.g., markdown) that may contain LaTeX or code blocks.
* @returns The preprocessed string with properly escaped and normalized LaTeX.
*
* @example
* preprocessLaTeX("Price: $10. The equation is \\(x^2\\).")
* // → "Price: $10. The equation is $x^2$."
*/
export function preprocessLaTeX(content: string): string {
// See also:
// https://github.com/danny-avila/LibreChat/blob/main/client/src/utils/latex.ts
// Step 0: Temporarily remove blockquote markers (>) to process LaTeX correctly
// Store the structure so we can restore it later
const blockquoteMarkers: Map<number, string> = new Map();
const lines = content.split('\n');
const processedLines = lines.map((line, index) => {
const match = line.match(/^(>\s*)/);
if (match) {
blockquoteMarkers.set(index, match[1]);
return line.slice(match[1].length);
}
return line;
});
content = processedLines.join('\n');
// Step 1: Protect code blocks
const codeBlocks: string[] = [];
content = content.replace(CODE_BLOCK_REGEXP, (match) => {
codeBlocks.push(match);
return `<<CODE_BLOCK_${codeBlocks.length - 1}>>`;
});
// Step 2: Protect existing LaTeX expressions
const latexExpressions: string[] = [];
// Match \S...\[...\] and protect them and insert a line-break.
content = content.replace(/([\S].*?)\\\[([\s\S]*?)\\\](.*)/g, (match, group1, group2, group3) => {
// Check if there are characters following the formula (display-formula in a table-cell?)
if (group1.endsWith('\\')) {
return match; // Backslash before \[, do nothing.
}
const hasSuffix = /\S/.test(group3);
let optBreak;
if (hasSuffix) {
latexExpressions.push(`\\(${group2.trim()}\\)`); // Convert into inline.
optBreak = '';
} else {
latexExpressions.push(`\\[${group2}\\]`);
optBreak = '\n';
}
return `${group1}${optBreak}<<LATEX_${latexExpressions.length - 1}>>${optBreak}${group3}`;
});
// Match \(...\), \[...\], $$...$$ and protect them
content = content.replace(
/(\$\$[\s\S]*?\$\$|(?<!\\)\\\[[\s\S]*?\\\]|(?<!\\)\\\(.*?\\\))/g,
(match) => {
latexExpressions.push(match);
return `<<LATEX_${latexExpressions.length - 1}>>`;
}
);
// Protect inline $...$ but NOT if it looks like money (e.g., $10, $3.99)
content = maskInlineLaTeX(content, latexExpressions);
// Step 3: Escape standalone $ before digits (currency like $5 → \$5)
// (Now that inline math is protected, this will only escape dollars not already protected)
content = content.replace(/\$(?=\d)/g, '\\$');
// Step 4: Restore protected LaTeX expressions (they are valid)
content = content.replace(/<<LATEX_(\d+)>>/g, (_, index) => {
let expr = latexExpressions[parseInt(index)];
const match = expr.match(LATEX_LINEBREAK_REGEXP);
if (match) {
// Katex: The $$-delimiters should be in their own line
// if there are \\-line-breaks.
const formula = match[1];
const prefix = formula.startsWith('\n') ? '' : '\n';
const suffix = formula.endsWith('\n') ? '' : '\n';
expr = '$$' + prefix + formula + suffix + '$$';
}
return expr;
});
// Step 5: Restore code blocks
content = content.replace(/<<CODE_BLOCK_(\d+)>>/g, (_, index) => {
return codeBlocks[parseInt(index)];
});
// Step 6: Apply additional escaping functions (brackets and mhchem)
content = escapeBrackets(content);
if (doEscapeMhchem && (content.includes('\\ce{') || content.includes('\\pu{'))) {
content = escapeMhchem(content);
}
// Final pass: Convert \(...\) → $...$, \[...\] → $$...$$
content = content
// Using the look‑behind pattern `(?<!\\)` we skip matches
// that are preceded by a backslash, e.g.
// `Definitions\\(also called macros)` (title of chapter 20 in The TeXbook).
.replace(/(?<!\\)\\\((.+?)\\\)/g, '$$$1$') // inline
.replace(
// Using the look‑behind pattern `(?<!\\)` we skip matches
// that are preceded by a backslash, e.g. `\\[4pt]`.
/(?<!\\)\\\[([\s\S]*?)\\\]/g, // display, see also PR #16599
(_, prefix: string, content: string) => {
return `${prefix}$$${content}$$`;
}
);
// Step 7: Restore blockquote markers
if (blockquoteMarkers.size > 0) {
const finalLines = content.split('\n');
const restoredLines = finalLines.map((line, index) => {
const marker = blockquoteMarkers.get(index);
return marker ? marker + line : line;
});
content = restoredLines.join('\n');
}
return content;
}
examples/server/webui_llamacpp/src/routes/+layout.svelte
+4 -4
View File
@@ -1,7 +1,7 @@
<script lang="ts">
import '../app.css';
import { page } from '$app/state';
import { ChatSidebar, ConversationTitleUpdateDialog } from '$lib/components/app';
import { ChatSidebar, DialogConversationTitleUpdate } from '$lib/components/app';
import {
activeMessages,
isLoading,
@@ -44,12 +44,12 @@
}
}
if (isCtrlOrCmd && event.shiftKey && event.key === 'o') {
if (isCtrlOrCmd && event.shiftKey && event.key === 'O') {
event.preventDefault();
goto('?new_chat=true#/');
}
if (event.shiftKey && isCtrlOrCmd && event.key === 'e') {
if (event.shiftKey && isCtrlOrCmd && event.key === 'E') {
event.preventDefault();
if (chatSidebar?.editActiveConversation) {
@@ -150,7 +150,7 @@
<Toaster richColors />
<ConversationTitleUpdateDialog
<DialogConversationTitleUpdate
bind:open={titleUpdateDialogOpen}
currentTitle={titleUpdateCurrentTitle}
newTitle={titleUpdateNewTitle}
examples/server/webui_llamacpp/src/stories/ChatSettings.stories.svelte
+19
View File
@@ -0,0 +1,19 @@
<script module>
import { defineMeta } from '@storybook/addon-svelte-csf';
import { ChatSettings } from '$lib/components/app';
import { fn } from 'storybook/test';
const { Story } = defineMeta({
title: 'Components/ChatSettings',
component: ChatSettings,
parameters: {
layout: 'fullscreen'
},
args: {
onClose: fn(),
onSave: fn()
}
});
</script>
<Story name="Default" />
examples/server/webui_llamacpp/src/stories/ChatSettingsDialog.stories.svelte
-26
View File
@@ -1,26 +0,0 @@
<script module>
import { defineMeta } from '@storybook/addon-svelte-csf';
import { ChatSettingsDialog } from '$lib/components/app';
import { fn } from 'storybook/test';
const { Story } = defineMeta({
title: 'Components/ChatSettingsDialog',
component: ChatSettingsDialog,
parameters: {
layout: 'fullscreen'
},
argTypes: {
open: {
control: 'boolean',
description: 'Whether the dialog is open'
}
},
args: {
onOpenChange: fn()
}
});
</script>
<Story name="Open" args={{ open: true }} />
<Story name="Closed" args={{ open: false }} />
examples/server/webui_llamacpp/src/stories/fixtures/math-formulas.ts
+65
View File
@@ -1,3 +1,4 @@
/* eslint-disable no-irregular-whitespace */
// Math Formulas Content
export const MATH_FORMULAS_MD = String.raw`
# Mathematical Formulas and Expressions
@@ -150,6 +151,70 @@ $$\lim_{x \to 0} \frac{\sin x}{x} = 1$$
$$\lim_{n \to \infty} \left(1 + \frac{x}{n}\right)^n = e^x$$
## Further Bracket Styles and Amounts
- \( \mathrm{GL}_2(\mathbb{F}_7) \): Group of invertible matrices with entries in \(\mathbb{F}_7\).
- Some kernel of \(\mathrm{SL}_2(\mathbb{F}_7)\):
\[
\left\{ \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix} \right\} = \{\pm I\}
\]
- Algebra:
\[
x = \frac{-b \pm \sqrt{\,b^{2}-4ac\,}}{2a}
\]
- $100 and $12.99 are amounts, not LaTeX.
- I have $10, $3.99 and $x + y$ and $100x$. The amount is $2,000.
- Emma buys 2 cupcakes for $3 each and 1 cookie for $1.50. How much money does she spend in total?
- Maria has $20. She buys a notebook for $4.75 and a pack of pencils for $3.25. How much change does she receive?
- 1 kg の質量は
\[
E = (1\ \text{kg}) \times (3.0 \times 10^8\ \text{m/s})^2 \approx 9.0 \times 10^{16}\ \text{J}
\]
というエネルギーに相当します。これは約 21 百万トンの TNT が爆発したときのエネルギーに匹敵します。
- Algebra: \[
x = \frac{-b \pm \sqrt{\,b^{2}-4ac\,}}{2a}
\]
- Algebraic topology, Homotopy Groups of $\mathbb{S}^3$:
$$\pi_n(\mathbb{S}^3) = \begin{cases}
\mathbb{Z} & n = 3 \\
0 & n > 3, n \neq 4 \\
\mathbb{Z}_2 & n = 4 \\
\end{cases}$$
- Spacer preceded by backslash:
\[
\boxed{
\begin{aligned}
N_{\text{att}}^{\text{(MHA)}} &=
h \bigl[\, d_{\text{model}}\;d_{k} + d_{\text{model}}\;d_{v}\, \bigr] && (\text{Q,K,V の重み})\\
&\quad+ h(d_{k}+d_{k}+d_{v}) && (\text{バイアス Q,K,V）}\\[4pt]
&\quad+ (h d_{v})\, d_{\text{model}} && (\text{出力射影 }W^{O})\\
&\quad+ d_{\text{model}} && (\text{バイアス }b^{O})
\end{aligned}}
\]
## Formulas in a Table
| Area | Expression | Comment |
|------|------------|---------|
| **Algebra** | \[
x = \frac{-b \pm \sqrt{\,b^{2}-4ac\,}}{2a}
\] | Quadratic formula |
| | \[
(a+b)^{n} = \sum_{k=0}^{n}\binom{n}{k}\,a^{\,n-k}\,b^{\,k}
\] | Binomial theorem |
| | \(\displaystyle \prod_{k=1}^{n}k = n! \) | Factorial definition |
| **Geometry** | \( \mathbf{a}\cdot \mathbf{b} = \|\mathbf{a}\|\,\|\mathbf{b}\|\,\cos\theta \) | Dot product & angle |
## No math (but chemical)
Balanced chemical reaction with states:
\[
\ce{2H2(g) + O2(g) -> 2H2O(l)}
\]
The standard enthalpy change for the reaction is: $\Delta H^\circ = \pu{-572 kJ mol^{-1}}$.
---
*This document showcases various mathematical notation and formulas that can be rendered in markdown using LaTeX syntax.*
examples/server/webui_llamacpp/svelte.config.js
+3
View File
@@ -22,6 +22,9 @@ const config = {
}),
output: {
bundleStrategy: 'inline'
},
alias: {
$styles: 'src/styles'
}
},
examples/server/webui_llamacpp/vite.config.ts
+28 -3
View File
@@ -18,6 +18,15 @@ const GUIDE_FOR_FRONTEND = `
const MAX_BUNDLE_SIZE = 2 * 1024 * 1024;
/**
* the maximum size of an embedded asset in bytes,
* e.g. maximum size of embedded font (see node_modules/katex/dist/fonts/*.woff2)
*/
const MAX_ASSET_SIZE = 32000;
/** public/index.html.gz minified flag */
const ENABLE_JS_MINIFICATION = true;
function llamaCppBuildPlugin() {
return {
name: 'llamacpp:build',
@@ -75,12 +84,28 @@ function llamaCppBuildPlugin() {
}
export default defineConfig({
resolve: {
alias: {
'katex-fonts': resolve('node_modules/katex/dist/fonts')
}
},
build: {
chunkSizeWarningLimit: 3072
assetsInlineLimit: MAX_ASSET_SIZE,
chunkSizeWarningLimit: 3072,
minify: ENABLE_JS_MINIFICATION
},
css: {
preprocessorOptions: {
scss: {
additionalData: `
$use-woff2: true;
$use-woff: false;
$use-ttf: false;
`
}
}
},
plugins: [tailwindcss(), sveltekit(), devtoolsJson(), llamaCppBuildPlugin()],
test: {
projects: [
{