wmantly/llama.cpp-xrip
1
0 Fork 0
Code Issues Pull Requests Actions 32 Packages Projects Releases Wiki Activity

docs : clarify SM75 decode result

Browse Source 
Assisted-by: Codex
This commit is contained in:
xrip committed 2026-09-24 17:10:14 +03:00
1 parent ee90e619dd
commit 5031d1c935
1 file changed
+1 -1
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -19,7 +19,7 @@
## SM75 prefill test
On an NVIDIA CMP 50HX with the Qwen3.8-27B IQ2_XXS GGUF, `build-best.sh` (`GGML_CUDA_FORCE_CUBLAS=ON`, `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`) measured 764.48 ± 11.67 tokens/s for `pp1024`, versus 618.29 ± 4.88 tokens/s with forced MMQ (+23.6%). Both builds used `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`, 99 GPU layers, FlashAttention, a Q8_0 V cache, and batch and ubatch sizes of 1024. `tg128` was 23.21 tokens/s with cuBLAS and 23.39 with MMQ, so the gain was in prefill. Results may differ on other GPUs or models.
On an NVIDIA CMP 50HX with the Qwen3.8-27B IQ2_XXS GGUF, `build-best.sh` (`GGML_CUDA_FORCE_CUBLAS=ON`, `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`) measured 764.48 ± 11.67 tokens/s for `pp1024`, versus 618.29 ± 4.88 tokens/s with forced MMQ (+23.6%). Both builds used `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`, 99 GPU layers, FlashAttention, a Q8_0 V cache, and batch and ubatch sizes of 1024. Token generation stayed around 23-24 tokens/s in these runs; the gain was in prefill. Results may differ on other GPUs or models.
This result is for Turing (SM75); it is not a Blackwell result. The SM75 dispatch and Q8 conversion paths do not run on Blackwell.