Files
llama.cpp-xrip/ggml
Rhonstin f56cf8ff73 CUDA: use radix TOP_K when CUB DeviceTopK is unavailable
CUDA builds with CCCL < 3.2 have no cub::DeviceTopK, so ggml_top_k falls back to a full argsort + copy (~12 bytes of temp per cell). For sparse-attention top-k over long contexts this OOMs small cards (200K cols x 128 rows ~= 190MB+ of temp per layer). Reuse the exact radix selection from #27466 (merged for HIP) on CUDA when CUB_TOP_K_AVAILABLE is off: O(nrows) temp, same selected set. Also gate next_power_of_2, unused on that path now.
2026-09-21 20:54:01 +03:00
..
2024-07-13 18:12:39 +02:00