Measured +0.27% at n=2..4 against a -0.07% noise floor, i.e. no gain, so the
change was reverted. Records why the IQ4_XS technique does not transfer:
iq2xxs_grid is 2 KB and L1-resident so the eliminated lookups were already
cheap, while the decoded state is ~2x larger and costs occupancy. Also records
that load widening is impossible for IQ2_XXS (66-byte block, qs at offset 2).
Records every measurement with the exact launch command, plus the hypotheses
that were tested and rejected so they are not retried. Key entries:
- FORCE_CUBLAS is a stock upstream option worth +24.9% dense prefill; it is not
fork code and upstream gets it too.
- Fork AVX1 Q4_K CPU path is worth +7.1% MoE tg (4 alternating loads per arm).
- OMP_PROC_BIND=close + --threads-batch 40 is a free +4.6% MoE prefill.
- OMP_WAIT_POLICY=passive costs -47% tg: the libgomp spinning is load-bearing.
- mmap costs 5-10% vs --load-mode none, because numactl --interleave only
stripes anonymous memory, not page-cache-backed mappings.
- Method: alternate arms across model LOADS; reps within one load are correlated
and produced two contradictory conclusions that had to be withdrawn.
xrip