server : add realtime conversation and transcription on /v1/realtime (#31269)

* server : add realtime conversation and transcription on /v1/realtime

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5

* examples : default the realtime client to the server port 9931

Assisted-by: Claude Opus 5.5

* server : sample realtime responses right after their own decode

Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs.

Assisted-by: Claude Opus 5.5

* move more stuff to server-realtime, making it self-contained

* examples : fix the type check of the realtime client

Assisted-by: Claude Opus 5.5

* examples : use the model card prompts in the realtime client

Assisted-by: Claude Opus 5.5

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
Tarek DakhranandXuan Son Nguyen authored and GitHub committed 2026-10-28 01:26:12 +01:00
1 parent d05706b4f3
commit 8ca4060b7e
10 files changed
+1423 -14

No files matched your search

+10
View File
@@ -1511,6 +1511,16 @@ json server_task_result_rerank::to_json() {
//
// server_task_result_tts
//
json server_task_result_realtime::to_json() {
return json {
{"text", text},
{"n_samples", audio.size()},
{"response_done", response_done},
{"cancelled", cancelled},
{"n_decoded", n_decoded},
};
}
json server_task_result_tts::to_json() {
return json {
{"sample_rate", sample_rate},