model: Add MiniCPM-OCR-Jev (#30980)

* model : add MiniCPM-OCR-Jev support

The model is a Qwen3.5-VL trunk with a value head read at the last token: it
scores an OCR text against an image rendered from it, it does not generate.

Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone.
The class scores are the pooled output of the sequence, read with last pooling,
so the conversion turns score.weight into cls.output and sets
embedding_length_out. The checkpoint has no LM head, so text generation is
rejected instead of using the output weights duplicated from the token
embeddings.

The task is fixed, so its prompt is entirely in the systemone template, the
same way h2o-lightning keeps its default there. The processor of the checkpoint
allows up to 16384 merged tokens per image, more than the Qwen-VL default of
mtmd, so the mmproj keeps the image budget of the preprocessor config. The
checkpoint declares MiniCPMOCRForSequenceClassification in its config.json, as
other models do.

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix code

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* use gguf meta classifier_output_labels

* remove redudant code changes

---------

Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
tc-mbandXuan Son Nguyen authored and GitHub committed 2026-10-27 11:49:12 +01:00
1 parent 276378515c
commit 7eed6b7685
13 files changed
+270 -10

No files matched your search

+13 -1
View File
@@ -2473,7 +2473,14 @@ private:
const auto & decision = slot.task->decision;
if (!decision.labels.empty()) {
if (decision.pooled) {
const float * embd = llama_get_embeddings_seq(slot.ctx_tgt, slot.id);
if (embd == nullptr) {
send_error(slot, "failed to get the class scores", ERROR_TYPE_SERVER);
return;
}
res->scores.assign(embd, embd + llama_model_n_embd_out(model_tgt));
} else if (!decision.labels.empty()) {
const float * logits = llama_get_logits_ith(slot.ctx_tgt, i_batch);
if (logits == nullptr) {
send_error(slot, "failed to get logits", ERROR_TYPE_SERVER);
@@ -2964,6 +2971,11 @@ private:
case SERVER_TASK_TYPE_TTS:
case SERVER_TASK_TYPE_DECISION:
{
if (task.need_sampling() && !decision.can_generate()) {
send_error(task, "This model is a decision model without text generation, use /v1/systemone", ERROR_TYPE_NOT_SUPPORTED);
break;
}
// special case: if input is provided via CLI, tokenize it first
// otherwise, no need to tokenize as it's already done inside the HTTP thread
if (task.cli) {