Commit Graph
11579 Commits
Author SHA1 Message Date
Pratyush Kumar 8f574b0ad4 ci : build OpenVINO releases with GGML_NATIVE=OFF and CPU backend variants (#30324) 2026-10-13 10:22:05 +08:00
Pratyush Kumar a49f58dcc3 ci: fix OpenVINO download links (#30401) 2026-10-13 10:20:04 +08:00
Xuan-Son Nguyen cbe28bb3cb model: fix vllm d3 multi seq case (#30408)
* model: fix vllm d3 multi seq case

* add TODO
2026-10-13 00:23:01 +02:00
Pascal 358a420ec4 graph: pad the KV dimension of the no-cache attention to 256 (#30416)
The no-cache path builds a [n_tokens, n_tokens] mask and passes K and V
unpadded, so FLASH_ATTN_EXT sees K->ne[1] == n_tokens. The CUDA kernels
for head size 512 require a multiple of FATTN_KQ_STRIDE, so the op falls
back to the CPU for most lengths, and the -fa auto probe (1 token)
disables flash attention for such models.

Pad the mask like llama_kv_cache::get_n_kv and pad K and V with zeros up
to the KV dimension of the mask. clef builds its own causal mask and
pads it the same way.
2026-10-12 23:00:27 +02:00
48e9857567 metal: fix metal tensor kernel_mul_mm error for large tensor (#28748)
* metal: fix metal tensor kernel_mul_mm error for large tensor

Assisted-by: Codex

* Update ggml/src/ggml-metal/kernels/mul_mm.metal

Co-authored-by: YiChen Lv <63285796+forforever73@users.noreply.github.com>

---------

Co-authored-by: 向锡铭 <xiangximing@xiangximings-MacBook-Air.local>
Co-authored-by: YiChen Lv <63285796+forforever73@users.noreply.github.com>
2026-10-12 22:57:26 +03:00
8bb3cf0d21 hexagon: optimize CPY/CONCAT — round 3, short-rows edition (#30341)
* hexagon: move the converting CPY in blocks of rows per DMA descriptor so short rows are not DMA-latency bound

* hex-cpy: cleanup cpy cursor handling

* hex-cpy-concat: introduce threaded dma/hvx gather-row helper

Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com>

* hex-cpy: remove leftover rejections for extended buffers

* hex-cpy: relax the gate for fast-path

* hex-cpy: remove scalar divs and vtcm access and use explicit gather_sync

* hex-cpy: add proper support for mdev work splitting to fastpath

* hex-cpy: optimize dense gather-row tasks

* hex-cpy: unroll gather-rows loops

* hex-cpy: add fastpath for f16

* hex-cpy-concat: support for colapsing 3/4D dims

* hex-cpy: double-buffer support

* hex-cpy: factor out gather-rows into more of an op by itself

* hex-cpy-concat: dedup kernel-params

* hex-cpy-concat: step-down nr to fit instead of hard fail

* hex-gather: replace div with a mask

* hex-gather-rows: cleanup dma pipeline and remove the stalls

* hex-gather: specialize compute funcs

* hex-gather: reduce scalar reg.spills and func calls

* hex-gather: do not recompute rows-per-thread on the npu

* hex-build: add support for building/installing just hexagon

* hex-cpy-concat: remove redundant barrier in concat and move cpy barrier under fence

* hex-gather: annotations for dma-out

* hex-mdev: make all key vtcm_acquire sync state atomic (fixes random hangs in mdev setups)

---------

Co-authored-by: Yiwei Shao <yiwei@aizip.ai>
Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com>
2026-10-12 07:39:55 -07:00
Alex Morgan 0f701c8829 vulkan: pack FMA for f16 accumulation, no dot product op (#29877)
* vulkan: pack FMA for f16 accumulation, no dot product op

* vulkan: dot_product packs two rows per f16 FMA

* vulkan: rename dot_product to dot_product_v2
2026-10-12 16:19:20 +02:00
Xuan-Son Nguyen efe3626b11 model: support vllm d3 models (#30383)
* models: support pplx-decider

* vllm d2

* update

* clarify naming rule

* stricter validation

* d3
2026-10-12 15:01:52 +02:00
Xuan-Son Nguyen 097cf33d94 model: support vllm decision 2 model (#30378)
* models: support pplx-decider

* vllm d2

* update

* clarify naming rule

* stricter validation

* fix ci

* rm redundant filter_tensors
2026-10-12 13:39:55 +02:00
Georgi Gerganov ccdbe29053 tests : link each test against only the libraries it uses (#30299)
Give llama_build() and llama_build_and_test() a LINK keyword (default:
llama-common) so call sites can declare what a test actually needs:

- ggml-only tests now link just ggml: test-opt, test-metal-graph-optimize,
  test-backend-ops, test-tiled-mulmat, test-barrier, test-quantize-fns,
  test-quantize-perf, test-rope, test-col2im-1d, test-gguf, test-alloc
- llama-only tests drop llama-common: test-unicode, test-sampling,
  test-grammar-parser, test-llama-grammar, test-batch-alloc,
  test-gbnf-validator, test-quant-type-selection
- both mtmd tests link mtmd only, dropping the LLAMA_TEST_NAME /
  target_link_libraries workarounds

Both helpers now set cxx_std_17 explicitly, since the tests no longer
inherit it from llama-common's PUBLIC interface.

tests/testing.h no longer includes common.h; the only symbol it used from
there (string_join) is inlined.

Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
2026-10-12 13:32:56 +03:00
Georgi Gerganov 073b74034b metal : fix mul_mat_id with duplicate ids (#30373)
* metal : fix mul_mat_id limits and kernel name

* metal : fix mul_mat_id with duplicate ids

kernel_mul_mm_id_map0 assumed at most one match per token and a fixed
capacity of ne21 entries per expert, which produced wrong or dropped
output rows and could overflow into the slice of the next expert when
the ids contain duplicates.

build a compact per-expert list of the (token, slot) entries instead:

- pass 1 counts the entries of every expert, thread 0 computes the
  offsets of the expert slices (tpe now holds [base, count])
- pass 2 writes the entries with one unconditional store per token
  (conditional stores of divergent lanes are very slow), the dummy
  value of the tokens without a match lands on a padding slot of the
  slice that is never read - duplicates are handled by a rare branch
- kernel_mul_mm_id reads the base/count of its expert and loops over
  the entry tiles that do not fit into the grid

The ids buffer is sized as ne20*ne21 + ne02 (entries + padding).

883/883 test-backend-ops MUL_MAT_ID tests pass, llama-bench is within
run-to-run variance of the previous behavior.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD

* metal : make mul_mat_id map0 ne02 a function constant

use FOR_UNROLL for the id loops of kernel_mul_mm_id_map0 and turn ne02
into the function constant FC_mul_mm_id_map0_ne02 (FC_MUL_MM + 7), so
that the threadgroup offsets of cnt/bas/sids and the prefix-sum bound
fold at compile time.

the pipeline name now encodes ne02 (same convention as the ssm_conv
nc/silu constants) and the map0 pipeline is compiled with the constant
value set. ne02 is no longer part of ggml_metal_kargs_mul_mm_id_map0 -
the dispatch grid still comes from the tensor ne[2] on the host.

no measurable perf change: 883/883 test-backend-ops MUL_MAT_ID tests
pass, op-level and llama-bench results are within run variance.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD

* cont : clean-up + TODO

* metal : dispatch mul_mat_id entry tiles from the prepass

follow the approach of the vulkan implementation (#29998): instead of
launching a grid over experts x ne21 entry tiles and letting most of
the threadgroups early exit, kernel_mul_mm_id_map0 now also emits a
row map - the total tile count plus one (expert, first entry, valid
rows) descriptor per 32-entry tile that has work.

kernel_mul_mm_id is launched with a static upper bound on the number
of tiles (full tiles over all entries plus at most one partial tile
per expert), reads its descriptor and processes a single tile - the
tail early exits. This removes the round loop that was needed to cover
duplicate ids past the grid (register pressure: th_max 768 -> back to
the plain mul_mm level) and duplicates are now handled implicitly,
since they only produce more descriptors within the bound.

The grid shrinks by ~8-16x on typical shapes (e.g. 128 experts,
8 used, 512 tokens: 128 tiles instead of 2048 entry tiles, of which
94% only launched to exit) and ne21 is no longer needed by the mm
kernel kargs.

883/883 test-backend-ops MUL_MAT_ID tests pass. llama-bench
(Qwen3.6-35B-A3B Q8_0) is now faster than master on all pp sizes
(pp2048/4096/8192 +2%, pp512 +0.5%), tg32 unchanged.

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD

* cont : add smem assert
2026-10-12 13:31:41 +03:00
Daniel Bevenius 77757a3467 examples : fix draft sampling and rejection handling (#30095)
* examples : fix draft sampling and rejection handling

This commit fixes two issues in the speculative example which are
described in detail in the linked issue.

Resolves: https://github.com/ggml-org/llama.cpp/issues/29975

* examples : add kv-unified requirement for parallel > 1
2026-10-12 12:02:24 +02:00
Georgi Gerganov 972b5fcc19 llama : report compute graph reallocations during inference (#30329)
Add an optional ggml_backend_sched_realloc_callback that user code registers with
ggml_backend_sched_set_realloc_callback to be notified when the buffers reserved for a
graph are not sufficient and have to be reserved again. The callback reports whether the
backend assignments and the graph size changed since the previous scheduling.

llama_context registers the callback when creating the scheduler (wrapped into the new
llama_context::sched_new helper) and treats the reallocations that follow a change of the
backend assignments as expected, counting the rest by graph size - unchanged or changed
from the previously scheduled graph. Both counters are reported on one line in the perf
stats.

Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
2026-10-12 11:53:58 +03:00
shamnadxyz 2f160ba59a ui: fix text attachment preview overflow (#29898)
* ui: fix text attachment preview overflow

Set the max height to 80dvh and the max width to full on screens
below medium and 80vw from medium onward.

* style: run formatter
2026-10-12 10:20:16 +02:00
Frosty40 f9a95aeb20 sycl: guard DMMV reads at row tails (#29338) 2026-10-12 04:06:27 -04:00
Amadeus dyw 08ffba21e5 vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164)
* vulkan: specialize quant block size for CM2 matmul

* vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size

* vulkan: clarify CM2 quant block size comment
2026-10-12 08:00:26 +02:00
Christian Kastner a05e58c981 ggml: Change default visibility of symbols to hidden (#30014)
See discussion in #29997. This prevents a handful of internal symbols
leaking via GGML_API.
2026-10-12 08:50:12 +03:00
miyan b2f036c0f1 vulkan: fix buffer logging on 32-bit platforms after source split (#30326)
Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp.
2026-10-12 06:51:24 +02:00
Jhen-Jie Hong e937b17a2d hexagon: optimize q2_k bpw and activation (#30035) 2026-10-11 20:52:01 -07:00
kurquharandMax Krasnyansky b13a93a3d5 hexagon: add IM2COL_3D (#30305)
* hexagon: add IM2COL_3D support

Assisted-by: OpenCode

* hexagon: add DMA paths for IM2COL_3D

Assisted-by: OpenCode

* hexagon: avoid scalar fp16 conversions in IM2COL

Assisted-by: OpenCode

* remove in-loop spills in pre-v81 devices

Assisted-by: OpenCode

* hex-im2col: re-write to use canonical dma dbl-buff pipeline, remove divs and very inefficient fp conversions

* hex-im2col: move kernel_params and vtcm layout precompute into the host

* hex-im2col: remove explicit tracking of number of dma-pushes

* hex-im2col: remove incorrect uint64_t casts

* hex-im2col: no need for macros for dma loops and pass compute func ptr via local context

* hex-im2col: use cursor pattern, tracing and proper n_threads handling

* hex-im2col: support non-contigue views

* hex-im2col: minor reformatting

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-10-11 20:24:47 -07:00
lhezandHongqiang Wang f2cc7282ce opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266)
* opencl: enable Gemma-4 E4B GPU decode

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* opencl: extend Gemma-4 GPU decode to E2B

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* opencl: optimize DK64 GQA8 decode

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* opencl: optimize DK128 GQA4 decode

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

---------

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>
2026-10-11 11:49:25 -07:00
lhezandHongqiang Wang c83f3058b6 opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310)
* opencl: fallback when Q4_K weight images exceed device limits

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* opencl: avoid incomplete subgroup in rms_norm

* opencl: refine conditions for dense bin kernels and fix bcast for q4_0

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

---------

Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>
2026-10-11 09:41:29 -07:00
Georgi Gerganov 5c8854fd88 rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs (#29544)
* rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs

GGML_RPC_DEBUG is now parsed as a number: 0/unset disables debug logs,
1-3 emit increasingly detailed output (events, per-command trace,
transport detail). Non-numeric values fall back to 1.

The duplicated env/macro blocks in ggml-rpc.cpp and transport.cpp are
replaced by a shared log.h, which becomes the single choke point for
all logging of the RPC backend: LOG_ERROR/LOG_WARN/LOG_INFO for
unconditional severity logs and LOG_DBG/LOG_DBG2/LOG_DBG3 for the
verbosity-gated ones. The transport files no longer need ggml-impl.h,
and the server banner now also goes through the ggml logger (stderr).

Missing logs are added on both the client (handshake, buffer ops,
tensor transfers, graph computes, cache decisions) and the server
(per-command dispatch, graph nodes), including the negotiated
transport via the new socket_t::transport_name().

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL

* rpc : align log levels with the documented verbosity semantics

The first pass introduced GGML_RPC_DEBUG levels, but many call sites did not
follow the documented semantics and some logs bypassed the macros entirely:

- Route the remaining raw GGML_LOG_* call sites through the local LOG_* macros
  so that log.h stays the single choke point for RPC logging
- Emit per-tensor transfers and per-command handler traces at level 2, and
  one-shot lifecycle events (backend and buffer type creation, buffer
  allocation, graph compute, graph cache eviction) at level 1
- Drop the duplicated enqueue trace in rpc_dispatcher::send, work() already
  logs every dispatched command together with its round-trip timing
- Promote degraded-operation events to LOG_WARN: remote allocation failure,
  RDMA falling back to TCP, unexpected peer disconnect
- Log graph cache eviction on both sides, since client and server must stay
  in sync for the incremental graph update to be valid
- Distinguish Unknown command (opcode out of range) from Unhandled command
  (valid opcode with no handler)
- Fix format specifiers (%zu for size_t, %u for device ids, 0x for hex
  pointers) and a missing newline in an error log
- Document that llama.cpp applications drop ggml debug records unless the
  verbosity threshold is raised with -lv 5

Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
2026-10-11 17:43:35 +03:00
bri-prismandGeorgi Gerganov a9d27ac693 model : support for Prism Bonsai 2 27B (#29600)
* Runtime support for Prism Bonsai 2 27B

Assisted-by: Claude Code

* address prism hadamard runtime feedback

* move hadamard tensors into method, define folded weight

* load_* hadamard fixes

* cont : clean-up

* conversion cleanup

Assisted-by: Claude Code

* prism hadamard key and methods for converter

Assisted-by: Claude Code

* address review bot feedback

Assisted-by: Claude Code

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-11 14:39:16 +02:00
Ploo 1cf7767765 cpu: fix AVX2 CPUs not using F16C when available (#30270) 2026-10-11 15:24:41 +03:00
Jeff BolzandAnjielon 966baae76b vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998)
* vulkan: compute every row of an expert in mul_mm_id when ids repeat

* vulkan: handle mul_mat_id duplicates in the prepass rather than looping

Extend the "hoist row ids" optimization to always be enabled and to emit a
compact list of tile descriptions that need to run, and to emit multiple tiles
when needed. Then we can launch a tighter upper bound on the number of
workgroups and the tail can trivially early exit.

---------

Co-authored-by: Anjielon <[EMAIL_REDACTED]>
2026-10-11 14:02:57 +02:00
Xuan-Son Nguyen 62a6f74bab server: add component tests (#30312)
* server: add component tests

* nits

* add codeowners
2026-10-11 12:51:07 +02:00
Pascal 23b0202a18 server: leave a busy slot untouched when a request pins it (#30295)
A request asking for a busy id_slot still ran the prompt cache update
on that slot before being deferred. When the RAM cache held a better
match, it was loaded into the slot while another request was still
generating there, and that generation continued on the wrong context.

The busy slot is now returned as is and the request waits for it.
2026-10-10 21:25:38 +02:00
69f201a205 model : support MiniCPM-V 4.7 (#29416)
* mtmd : add MiniCPM-V 4.7 support

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* model : allow mrope time from an extra position slot

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* Update conversion/minicpm.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Slim down comments

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix for "do not hand-wrap comments"

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix ci

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* rm 3d repo for pr one

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* gguf: add rope.section_order metadata

* fix comments

* handle grid layout

* allow compat

---------

Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-10 20:41:59 +02:00
Aaron Teo abee0c8476 cmake(s390x): disable z17 target for unsupported compilers (#30297) 2026-10-10 20:00:56 +02:00
Mikolaj Kucharski aa94f20861 args: add LLAMA_ARG_SLOT_SAVE_PATH env for --slot-save-path (#30272)
Allow configuring --slot-save-path via LLAMA_ARG_SLOT_SAVE_PATH so
it can be set from an EnvironmentFile in a systemd unit file.
2026-10-10 15:34:40 +02:00
Xuan-Son Nguyen 781dbc5ac9 spec: properly handle mtmd input for mtp (#30257)
* spec: properly handle mtmd input for mtp

* nits
2026-10-10 11:22:38 +02:00
Xuan-Son Nguyen 0fd868cbca mtmd: add build_inp_attn_mask (#30259) 2026-10-10 11:22:12 +02:00
David Friehs 1623d8ce47 cuda: always use MMVQ for MUL_MAT_ID on sm_60 (#27828) 2026-10-10 09:15:42 +03:00
uvos 1bb2b9fcbe CUDA/HIP: fix race in flash_attn_ext_f16_process_tile when nbatch_combine != DKQ/2 (#30103)
Suggested-by: Johannes Gäßler <johannesg@5d6.de>
2026-10-10 09:14:51 +03:00
Aaron Teo 2bbca8f202 ggml-cpu: vectorize fp32 to fp16 conversion (#30157)
ggml-cpu(s390x): rename ulong to uint64_t sized types



ggml-cpu(s390x): rm comment

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-10-10 09:08:52 +03:00
Amadeus dyw 404f557b5b vulkan: use 4 rows for NVIDIA MUL_MAT_ID MMVQ (#29274)
Keep the existing RDNA3/4 policy unchanged and update only rm_id to use 4 rows for NVIDIA except pre-Turing.
2026-10-10 09:08:19 +03:00
Aaron Teo b797c82c7d ggml: fix s390x all cpu build (#30140)
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-10-10 09:07:52 +03:00
Shawn Gu f2918cabbf opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) 2026-10-09 21:02:11 -07:00
Captain-Tripps 1e6f04a75e sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) 2026-10-09 22:56:23 -04:00
Xuan-Son Nguyen 10a60cf303 vendor: apply deep nested json patch from upstream (#30253) 2026-10-10 00:07:44 +02:00
Dante 79e2e74eb1 CUDA: fix round issue, under MSVC the CPU and GPU agree (#30229) 2026-10-09 19:59:50 +02:00
Georgi Gerganov 8e2d31e0eb graph : reorder get_rows for embeddings (#30160)
* graph : reorder get_rows for embeddings

* cont : fix gemma4 and improve input embedding construction logic

* cont : add TODO for lora

* cont : fix raw embeddings path

* gemma4 : avoid ple cast in embeddings path
2026-10-09 20:43:05 +03:00
Aleksander GrygierandPascal baef3ed9a1 ui: Models Manager Follow-up Improvements (#30228)
* common : read a GGUF's trained context from its metadata

common_get_gguf_n_ctx_train opens only the file's metadata (no_alloc,
like common_get_decision_type) and reads <arch>.context_length, so a
caller can learn the trained context without loading the model. It
accepts both u32 and u64 values and returns 0 when the file is missing,
unreadable, invalid, or reports no context length.

Assisted-by: pi:zai-org/GLM-5.3-Flash

* server : report the trained context in the models listing

update_caps already resolves the model file offline to read its
modalities, so it now reads the trained context from the same GGUF
metadata, and GET /models reports it as context_length when it is
known. A router listing then carries the context without any Hub
request, which lets the UI sort and filter by it offline.

Assisted-by: pi:zai-org/GLM-5.3-Flash

* ui : take the trained context from the models listing

The router now reports context_length per model, so the option mapping
fills contextLength from it and the manager reads it before the Hub
record. The Context column, the context sort and the context filter
then work with the Hugging Face Hub API turned off. A browser suite
guards the sort and the search, the Hub-cache driven context filter and
the re-sort when details arrive after the sort was clicked.

Assisted-by: pi:zai-org/GLM-5.3-Flash

* ui : mark favorite models with a heart

A favorited model shows a rose heart in the selector even before its
row is hovered, and the crossed heart takes its place on hover, so
unfavoriting stays one hover away. The manager table marks its
favorited rows with the same heart after the badges and capabilities.

Assisted-by: pi:zai-org/GLM-5.3-Flash

* fix: UI text nit

* fix: UI nits

* fix: Favorite models grouping in models table

* feat: Remove sorting from Status column in Models Table

* server: read the GGUF metadata once per model

Read the decision type and the trained context in a single GGUF open,
accept only a UINT32 context length like the model loader, and reset
n_ctx_train with the other caps so a failed refresh drops it.

* fix: Post-review fixes

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-10-09 19:28:54 +02:00
Martin Emrich f39148a953 llama-bench: respect -fitc if bigger than required benchmark size (#28331)
Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B
2026-10-09 19:14:30 +02:00
Anav Prasad 50e3e3e480 CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807)
* CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan

* CUDA: remove redundant cuda copies with K==1 (non spec-dec) scenario as well
2026-10-09 18:23:28 +02:00
jingzhou 64df9183f5 opencl: fix kernel compilation for a6x GPUs (#30176)
* opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash

* The A6x compiler backend found in iot device with a623 (E031.50.31.01)
  cannot handle kernels with a large number of arguments. Skip this
  kernel for A6x to avoid compiler crash

* opencl: A6X constant-fold workaround for get_local_size in GEMV kernels

* opencl: add Adreno 623 to A6X GPU detection list
2026-10-09 09:19:20 -07:00
bosh 6184e92c57 model : use exact GELU for ModernBERT encoders (#30108)
* model : use exact GELU for ModernBERT encoders

Assisted-by: Codex

* model : keep tanh GELU aliases on ggml_geglu

Assisted-by: Claude Opus 5.5

* model : map gelu_python to ggml_geglu_erf

Assisted-by: Claude Opus 5.5
2026-10-09 23:26:56 +08:00
Aldehir Rojas 8b54361025 chat : refactor API (#30210) 2026-10-09 10:06:22 -05:00
Pascal a518119d30 llama: keep the backend sampling graph static across ubatches (#30223)
The reserve builds n_outputs_max_per_seq sampling chains per sampler,
while a decode built one per output row, so the graph changed its
topology after the reserve and GGML_SCHED_NO_REALLOC builds aborted on
the next same sized graph. Every sampler now builds
n_outputs_max_per_seq chains, the ones without a row of the ubatch on
the padding row and not selected, and graph_max_nodes counts them.
2026-10-09 17:20:25 +03:00