model: Add MiniCPM-OCR-Jev (#30980)
* model : add MiniCPM-OCR-Jev support The model is a Qwen3.5-VL trunk with a value head read at the last token: it scores an OCR text against an image rendered from it, it does not generate. Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone. The class scores are the pooled output of the sequence, read with last pooling, so the conversion turns score.weight into cls.output and sets embedding_length_out. The checkpoint has no LM head, so text generation is rejected instead of using the output weights duplicated from the token embeddings. The task is fixed, so its prompt is entirely in the systemone template, the same way h2o-lightning keeps its default there. The processor of the checkpoint allows up to 16384 merged tokens per image, more than the Qwen-VL default of mtmd, so the mmproj keeps the image budget of the preprocessor config. The checkpoint declares MiniCPMOCRForSequenceClassification in its config.json, as other models do. Signed-off-by: tc-mb <tianchi_cai@icloud.com> * fix code Signed-off-by: tc-mb <tianchi_cai@icloud.com> * use gguf meta classifier_output_labels * remove redudant code changes --------- Signed-off-by: tc-mb <tianchi_cai@icloud.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
1 parent
276378515c
commit
7eed6b7685
13 files changed
+270
-10
No files matched your search
@@ -2473,7 +2473,14 @@ private:
|
||||
|
||||
const auto & decision = slot.task->decision;
|
||||
|
||||
if (!decision.labels.empty()) {
|
||||
if (decision.pooled) {
|
||||
const float * embd = llama_get_embeddings_seq(slot.ctx_tgt, slot.id);
|
||||
if (embd == nullptr) {
|
||||
send_error(slot, "failed to get the class scores", ERROR_TYPE_SERVER);
|
||||
return;
|
||||
}
|
||||
res->scores.assign(embd, embd + llama_model_n_embd_out(model_tgt));
|
||||
} else if (!decision.labels.empty()) {
|
||||
const float * logits = llama_get_logits_ith(slot.ctx_tgt, i_batch);
|
||||
if (logits == nullptr) {
|
||||
send_error(slot, "failed to get logits", ERROR_TYPE_SERVER);
|
||||
@@ -2964,6 +2971,11 @@ private:
|
||||
case SERVER_TASK_TYPE_TTS:
|
||||
case SERVER_TASK_TYPE_DECISION:
|
||||
{
|
||||
if (task.need_sampling() && !decision.can_generate()) {
|
||||
send_error(task, "This model is a decision model without text generation, use /v1/systemone", ERROR_TYPE_NOT_SUPPORTED);
|
||||
break;
|
||||
}
|
||||
|
||||
// special case: if input is provided via CLI, tokenize it first
|
||||
// otherwise, no need to tokenize as it's already done inside the HTTP thread
|
||||
if (task.cli) {
|
||||
|
||||
Reference in new issue
Block a user