server: refactor + correctness fixes for metrics (#26920)

Browse Source

* server: refactor metrics

* move most fields to server_slot_stats

* cont

* rm result_timings

* tie stats to batch

* cont

* nits: move place in code

* exclude first generated token

* more accurate batch metrics tracking

* n_predict --> n_gen

* metrics_on_prediction

* metrics_flush_idle

* metrics: seperate cache/processed prompt tokens

* refactor server_task_result_metrics

* add test

* nits

* fix flush before reset()

* cont

* rm dead code

* nits

...

This commit is contained in:

Xuan-Son Nguyen authored and GitHub committed 2026-08-13 10:02:01 +02:00

1 parent e79e4bf660

commit decaf508bb

6 files changed

+813 -475

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tools/server/server-context.cpp

+277 -360

File diff suppressed because it is too large. Load diff