server: fix speculation after an image (#28715)

* server: fix speculation after an image

Pass the actual position to the drafter after an image, instead of the
token count. Affects every drafter, not just DFlash.

* rename draft n_past to pos0

n_past is used to denote number of tokens and this parameter is meant to be a position
This commit is contained in:
Jesus Gulfo authored and GitHub committed 2026-09-11 11:33:26 +03:00
1 parent 16378d93f9
commit b0dcb8192b
4 files changed
+13 -13

No files matched your search

+1 -1
View File
@@ -3028,7 +3028,7 @@ private:
common_speculative_get_draft_params(spec.get(), slot.id) = {
/* .drafting = */ true,
/* .n_max = */ n_draft_max,
/* .n_past = */ slot.prompt.n_tokens(),
/* .pos0 = */ slot.prompt.tokens.pos_next(),
/* .id_last = */ slot.sampled,
/* .prompt = */ &slot.spec_prompt,
/* .result = */ &slot.spec_draft,