wmantly/llama.cpp-xrip
1
0 Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity

sampling : use greedy selection for eligible temperature-zero chains (#29797)

Browse Source 
* sampling : use greedy selection for eligible temperature-zero chains

Assisted-by: OpenAI Codex

* Apply suggestion from @ggerganov

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* sampling: simplify zero-temperature greedy eligibility

Allow the same greedy selection on CPU and grammar/reasoning-budget paths.
Keep distribution sampling for dynamic temperature and requested probabilities.
Cover the common sampler selection and probability behavior in the existing sampler tests.

Assisted-by: OpenAI Codex

* sampling : use greedy selection after final top-k with k=1

Assisted-by: OpenAI Codex

* cont : clean-up

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Hrishith ThadicherlaandGeorgi Gerganov authored and GitHub committed 2026-10-07 13:34:18 +03:00
1 parent 42b021b4dc
commit d0b490f25e
3 files changed
+155 -48
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/llama-sampler.cpp
+6
View File
@@ -1086,6 +1086,12 @@ static void llama_sampler_greedy_backend_apply(
struct ggml_tensor * curl = ggml_argmax(ctx, logits);
ggml_set_name(curl, "greedy_argmax");
if (data->candidates != nullptr) {
struct ggml_tensor * candidates = ggml_reshape_2d(ctx, data->candidates, 1, ggml_nelements(data->candidates));
curl = ggml_get_rows(ctx, candidates, curl);
ggml_set_name(curl, "greedy_sampled_token");
}
data->sampled = curl;
}