Xuan-Son Nguyen 2851b3bcbf mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629) ...

* mtmd: add mtmd_audio_stream + support voxtral-realtime

* address review

2026-10-16 01:46:42 +02:00

..

cogvlm.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

conformer.cpp

model : add LiquidAI/d1-omni-600M decision model (#30114)

2026-10-08 02:07:49 +02:00

deepseek4v.cpp

mtmd: support DeepSeek-V4-Flash-Vision-Exp (#28133)

2026-09-02 16:43:43 +02:00

deepseekocr2.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

deepseekocr.cpp

mtmd: deepseek-ocr SAM ggml_conv_2d with the im2col kept in F32 (#26727)

2026-08-18 19:05:19 +02:00

dots3note.cpp

mtmd: support dots3-note vision+audio (#27524)

2026-08-22 10:35:50 +02:00

dotsocr.cpp

mtmd: support dots.ocr (#17575)

2026-04-09 12:16:38 +02:00

exaone4_5.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

gemma4a.cpp

models: pad on the left with ggml_pad_ext (#29567)

2026-09-28 20:56:15 +02:00

gemma4ua.cpp

mtmd, model: allow skip build_vit() (#24077)

2026-06-03 17:10:35 +02:00

gemma4uv.cpp

mtmd, model: allow skip build_vit() (#24077)

2026-06-03 17:10:35 +02:00

gemma4v.cpp

mtmd: use ggml_rope_set_offset (#27521)

2026-08-22 16:33:47 +02:00

glm4v.cpp

add GLM-5.3-Flash (GLM5-Next) support (#27773)

2026-09-30 14:20:32 +08:00

granite4-vision.cpp

mtmd: fix Granite4 Vision image sequence assembly (#26653)

2026-08-15 00:25:26 +02:00

granite-speech.cpp

models: pad on the left with ggml_pad_ext (#29567)

2026-09-28 20:56:15 +02:00

hunyuanvl.cpp

mtmd, model : merge HunyuanOCR into HunyuanVL and fix OCR vision precision (#23329)

2026-05-21 00:35:37 +02:00

internvl.cpp

mtmd: add batching support for internvl (#24775)

2026-06-19 01:16:16 +02:00

kimik25.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

kimivl.cpp

clip: move model cgraphs into their own files (#17965)

2025-12-12 21:14:48 +01:00

ling3vl.cpp

model : add Ling 3.0 VL support (#29151)

2026-09-24 08:57:31 +02:00

llama4.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

llava.cpp

model: Granite Speech Plus (#24818)

2026-06-23 12:03:31 +02:00

mimo-audio.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

mimovl.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

minicpmv.cpp

model : support MiniCPM-V 4.7 (#29416)

2026-10-10 20:41:59 +02:00

minimax-m3.cpp

mtmd: use ggml_rope_set_offset (#27521)

2026-08-22 16:33:47 +02:00

mobilenetv5.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

models.h

mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629)

2026-10-16 01:46:42 +02:00

muse-glimmer.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

muse-spark-audio.cpp

model: add Muse Spark Open (text, vision and audio) (#30616)

2026-10-15 23:27:12 +02:00

nemotron-omni.cpp

models : add Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support (#30520)

2026-10-15 16:34:56 +03:00

nemotron-v2-vl.cpp

mtmd : Add Nemotron Nano 12B v2 VL support (#19547)

2026-02-14 14:07:00 +01:00

paddleocr.cpp

model: Add PaddleOCR-VL model support (#18825)

2026-02-19 17:05:25 +01:00

parakeet.cpp

models: pad on the left with ggml_pad_ext (#29567)

2026-09-28 20:56:15 +02:00

pixtral.cpp

mtmd: add more validations (#25013)

2026-06-26 08:43:29 +02:00

pockettts-gen.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

pockettts-seanet.cpp

mtmd: support pocket-tts (#26871)

2026-08-11 14:18:30 +02:00

pockettts-spkenc.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

qwen2vl.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

qwen3a.cpp

mtmd: add chunks and fix preproc for qwen3a (#23073)

2026-05-15 19:32:47 +02:00

qwen3tts-gen.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00

qwen3tts-spkenc.cpp

ggml : update ggml_prec specification (#26675)

2026-09-08 09:06:24 +03:00

qwen3vl.cpp

mtmd : use align_corners for qwen3vl vision position embedding interpolation (#25781)

2026-07-21 23:58:34 +02:00

siglip.cpp

model : add LiquidAI/d1-omni-600M decision model (#30114)

2026-10-08 02:07:49 +02:00

step3vl.cpp

model : support step3-vl-10b (#21287)

2026-04-08 09:51:31 +02:00

voxtral-rt.cpp

mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629)

2026-10-16 01:46:42 +02:00

whisper-enc.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

yasa2.cpp

mtmd: Add support for Reka Edge 2603 (#21616)

2026-04-21 20:02:49 +02:00

youtuvl.cpp

mtmd: add build_inp_attn_mask (#30259)

2026-10-10 11:22:12 +02:00