wmantly/llama.cpp-xrip
1
0 Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity

ggml-cpu: add AVX1 Q4_K and IQ4_XS paths

Browse Source 
Add AVX1 CPU kernels for the Q4_K and IQ4_XS quantized matrix paths.

The Q4_K changes add panel repacking and prefill support. The IQ4_XS changes decode eight-row panels with AVX1 intrinsics and select the path only when AVX is available. Existing fallback paths remain in place for CPUs without AVX and for other quant types.
This commit is contained in:
xrip committed 2026-09-21 20:46:25 +03:00
1 parent 337be720fc
commit 16a020ce1e
1 file changed
-1
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-cpu/arch-fallback.h
-1
View File
@@ -90,7 +90,6 @@
#define ggml_vec_dot_q2_0_q8_0_generic ggml_vec_dot_q2_0_q8_0
// repack.cpp
#define ggml_quantize_mat_q8_0_4x4_generic ggml_quantize_mat_q8_0_4x4
#define ggml_quantize_mat_q8_K_4x4_generic ggml_quantize_mat_q8_K_4x4
#define ggml_gemv_q1_0_4x4_q8_0_generic ggml_gemv_q1_0_4x4_q8_0
#define ggml_gemv_q1_0_4x8_q8_0_generic ggml_gemv_q1_0_4x8_q8_0
#define ggml_gemv_q4_0_4x4_q8_0_generic ggml_gemv_q4_0_4x4_q8_0