diff --git a/README.md b/README.md index 177a54eff..9e4f82b5b 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ ## SM75 prefill test -On an NVIDIA CMP 50HX with the Qwen3.8-27B IQ2_XXS GGUF, `build-best.sh` (`GGML_CUDA_FORCE_CUBLAS=ON`, `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`) measured 764.48 ± 11.67 tokens/s for `pp1024`, versus 618.29 ± 4.88 tokens/s with forced MMQ (+23.6%). Both builds used `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`, 99 GPU layers, FlashAttention, a Q8_0 V cache, and batch and ubatch sizes of 1024. `tg128` was 23.21 tokens/s with cuBLAS and 23.39 with MMQ, so the gain was in prefill. Results may differ on other GPUs or models. +On an NVIDIA CMP 50HX with the Qwen3.8-27B IQ2_XXS GGUF, `build-best.sh` (`GGML_CUDA_FORCE_CUBLAS=ON`, `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`) measured 764.48 ± 11.67 tokens/s for `pp1024`, versus 618.29 ± 4.88 tokens/s with forced MMQ (+23.6%). Both builds used `GGML_CUDA_SM75_Q8_0_FAST_HALF=ON`, 99 GPU layers, FlashAttention, a Q8_0 V cache, and batch and ubatch sizes of 1024. Token generation stayed around 23-24 tokens/s in these runs; the gain was in prefill. Results may differ on other GPUs or models. This result is for Turing (SM75); it is not a Blackwell result. The SM75 dispatch and Q8 conversion paths do not run on Blackwell.