* log, server: make router child lines carry their own colors
The logger writes the color reset after the trailing newline, so the
reset opens the next line. On the shared pipe of a router child it lands
in front of the next state command, which the router then misses, and
the line break that works around it shows up as an empty log line on
every progress update.
The reset now goes before the trailing newlines, so every line is self
contained and the command goes back to its plain framing. The router
passes its effective color setting to its children, whose output ends
up in its terminal, and leaves that option out when comparing presets
on reload.
* log: enable ANSI colors on the Windows console
A Windows console renders ANSI sequences only in virtual terminal mode,
which nothing turns on for the logger, so llama-server prints raw escape
codes on the Windows 10 console while llama-cli, whose console code
enables it, shows colors. The logger now enables virtual terminal mode
on stdout and stderr when it turns colors on, and keeps colors off when
a console cannot render them. Pipes and files take the sequences as is.
* server: separate the router child commands from its logs
The child sent its state commands on the same pipe as its logs, so the
router had to pick them out of the log stream by a line prefix, and any
unterminated write in front of a command made the router miss it. This
resolves the TODO at the spawn that called for splitting stdout and
stderr.
The child now keeps stdout for the commands and points everything else
written to stdout at stderr, before anything is written. The router
reads both pipes, handles the commands from stdout and forwards stderr
as the log, and warns about any other line on the command pipe.
* server: address review from ngxson
The single server_child is now created first in the entry point and its
constructor keeps stdout for the commands, so the stream is a member of
the instance instead of a static, and init() is gone. The instance is
passed down to the server, while the CLI entry point creates its own.
* Update tools/server/server.cpp
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
A TOOL_ID node that arrives after TOOL_CLOSE wrote through `current_tool`,
which still pointed into the just-destroyed `pending_tool_call` optional
(use-after-free, then a second free of the id buffer). Clear the pointer on
reset.
This is part of the fs::path modernization series.
That was also the opportunity to remove fs_list().
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* chat : honor json_schema in Ling 3.0 parser
Ling 3.0 only built a grammar for tool calls and did not handle inputs.json_schema, so response_format requests were left unconstrained.
Add an eager response-format grammar path with precedence over tools, following the existing parser patterns. Require </think> before JSON when thinking is enabled and do not allow trailing prose after the JSON response.
Fixes#29652.
Assisted-by: Claude Opus 5.5
* chat : require Ling 3.0 think block for response formats
* Make the drafter probabilistic and the target verify by rejection sampling
* Drop stale spec_draft_q before drafting
* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.
* Support grammar-constrained requests in rejection sampling
* Fix - renormalize distribution after masking
* copy rng on sampler copy and re-accept drafted tokens on replay
* Fix draft sampler sharing the target's rng stream
* Simplify the rejection sampler's inputs and move replay to the server
* Truncate the draft candidates along with the draft
---------
Co-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>
Co-authored-by: Pranesh Gonegandla <pgonegandla@nvidia.com>
* init conversion
* convert: ok
* model loaded
* add server code
* improve conversion script
* support shared prompt prefix
* add docs, imorove UX a bit
* add vision support
* add openjev tiny model for testing
* add dev docs
* support lev & kev
* clean up
* fix lev noul
* fix py lint
* nits docs
* clarify about not supporting date_facts
LLM-jp-4.1 uses the GPT-OSS format, but its tokenizer decodes a space
after every special token and parallel tool calls are separated by
<|end|>. The GPT-OSS handler rejects this output, so add a dedicated
handler, selected by the chat_format=llm-jp-harmony-v1 declaration in
the chat template.
Assisted-by: Claude Fable 5.1
* cli: exit on stdin EOF and drop the console wide Ctrl+C broadcast
On Windows the simple input reader sends CTRL_C_EVENT to every process
attached to the console when stdin reaches EOF, killing unrelated
processes such as a supervising agent. The CLI only stopped on EOF
because of that self inflicted SIGINT; on POSIX, and with the advanced
reader, it spins forever printing prompts.
Drop the broadcast so both platforms just return an empty read, and
treat an empty read as EOF in the chat loop and the model selection,
since a submitted line always ends with a newline.
* cli: keep the newline of a trailing "/" and stop mtmd-cli on EOF
A lone "/" came back as an empty read and was taken for EOF, and
mtmd-cli only stopped on EOF through the removed broadcast.
- Check for buffered write errors when closing downloaded files.
- Use UTF-8 paths when writing ETag files on Windows.
- Write in binary mode on Windows.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* adapt common
* add common_batch
* wip
* wip: spec
* cont
* common_speculative_process
* server_batch to use common_batch
* rm some stale calls
Assisted-by: Claude Fable 5.1
* migrate mtmd
* handle imrope, handle return val of add()/add_embd()
* add spec zeros vector
* add warning on zero fill path
- register --rpc unconditionally and call llama_supports_rpc() only from its handler
- print server "initialization ..." log after args are parsed
Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
The original function was broken on Windows for some unicode paths
Paths without a trailing separator now create the last directory too,
matching the function name. All current callers already include a
trailing separator, so this change does not affect them.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Restore get_cache_directory() as fs::path as string() can be lossy on Windows
Partially reverts #29125
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* jinja : parse unary +/- before variables
Lexer already emits unary_operator for -n / +n, and runtime executes
unary -. Parse them at multiplicative precedence so slices like
items[:-n] and GigaChat indent[:-indent_factor] work.
* jinja : keep filters/tests outside unary operands
Unary +/- must bind only the primary/postfix operand so -n|abs is
(-n)|abs, not -(n|abs). Add unary + and filter/test regression coverage.
Signed-off-by: sinksilk <785976238@qq.com>
---------
Signed-off-by: sinksilk <785976238@qq.com>
Avoid returning references through lambdas that hold a local cast pointer, which triggers -Werror=dangling-reference in some CI compilers. Reuse the precomputed select_expr pointer directly.
Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp
* convert: add MiMo-V2.6 support
Hoist the K3 mxfp4 conversion repack into base.py so it can be reused
Remove decoder from mmproj convert
* Update conversion/mimo.py
* fix: use autoparser
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Piotr Wilkin <piotr.wilkin@syndatis.com>
Allow configuring --temp, --top-p, --min-p, --repeat-penalty,
--presence-penalty and --frequency-penalty via LLAMA_ARG_* so
llama-server can be fully controlled from an EnvironmentFile
(e.g. systemd on Debian).
Use `llama-gen-docs` to regenerate the readme files.
* Fixed json enum handling
Added common_json_value handling for enum values.
Added tests/test-json.cpp to cover testing of some aspects of common_json.
* Removed tests as requested.
* Applied recommended style and simplification
Simplified by delegating enum constructor to the constructor of the underlying type
Matched style of surrounding templating code
PascalandXuan-Son Nguyen
Anas
Pranesh GonegandlaandPranesh Gonegandla
Adrien Gallouët
Nik Bogatyrev
Yash Raj Pandey
Xuan-Son NguyenandSigbjørn Skjæret
Aman GuptaandGeorgi Gerganov
a u s t i n
e-mon
pr3pony
Ethan Guo
vaibhavdedhiaandAlde Rojas
Gaurav Garg
Si Chen
Emanuil RusevandXuan Son Nguyen
Nicolas Mowen
Mikolaj Kucharski
Silverside