diff --git a/README.md b/README.md index d2a2d6e..113c88b 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,8 @@ Total VRAM: 32 GB GDDR6 across 3 GPUs (Shared PCIe Root Complex) │ ├── llama-server.service # Systemd unit with NUMA socket pinning │ ├── ollama-proxy.service # Systemd unit for API proxy │ └── gpu-power-governor.service # Systemd unit for automated power governor +├── vbios/ +│ └── msi_cmp50hx_90.02.60.00.17.rom # Tuned MSI VBIOS ROM (31W idle & 25% min fan curve) └── docs/ ├── SETUP_GUIDE.md # Complete step-by-step setup guide on any Linux server ├── PROXMOX_LXC_GUIDE.md # GPU passthrough & NVML permissions in Proxmox LXC diff --git a/docs/HARDWARE_LEARNINGS.md b/docs/HARDWARE_LEARNINGS.md index c213f11..898a6e7 100644 --- a/docs/HARDWARE_LEARNINGS.md +++ b/docs/HARDWARE_LEARNINGS.md @@ -6,21 +6,24 @@ This document details the practical hardware behaviors, quirks, and engineering ## 1. NVIDIA CMP 50HX Mining GPUs: VBIOS & Power States -### Why CMP 50HX Mining Cards Idle at ~75W–85W Stock -* **No Display / Headless Architecture**: CMP 50HX mining cards lack display outputs and display engines. -* **Missing Deep P8 VBIOS Tables**: Standard GeForce VBIOSes drop to P8 state (405 MHz GDDR6 / ~10W) when no display is active. Mining VBIOSes lock the cards in P0 state (1,890 MHz core / 7,000 MHz GDDR6) by default. +### Why CMP 50HX Mining Cards Idle at ~75W–85W Stock (The 100% Video Engine Pinning Bug) +* **No Display / Headless Architecture**: CMP 50HX mining cards lack physical display outputs and raster display engines. +* **The 100% Video Engine Pinning Bug**: The `.17` (and newer `.1A`) CMP mining VBIOSes have a firmware bug where the GPU's hardware **Video Engine (NVDEC)** registers as 100% utilized at all times. This prevents the GPU power controller from automatically dropping into its lowest P8 sleep state, trapping stock idle power at roughly **75W–85W per card**. +* **The Fix**: + * **On Windows**: Use *NVIDIA Inspector's Multi-Display Power Saver*. Add AI backends (e.g., `ollama_server.exe` or `python.exe`) to the high-performance threshold list to force the cards into true ~5W–8W idle states when inactive. + * **On Linux**: Use our custom dynamic power governor ([`power-governor.py`](../scripts/power-governor.py)) with `nvidia-smi` script wrappers to manually clamp idle clocks to **300 MHz** (`-lgc 300,300`) and apply a **150W power limit** (`-pl 150`). -### OEM Reference VBIOS vs. MSI VBIOS -Comparing the hardware profiles of two CMP 50HX cards on the exact same rig: -* **GPU 2 (MSI Board `0x1462`, VBIOS `90.02.60.00.17`)**: - * Idles at **`~31.8W`** when core is locked to 300–600 MHz. - * Minimum fan speed: **`25%`**. +### OEM Reference VBIOS vs. MSI VBIOS (`90.02.60.00.17`) +Comparing the hardware profiles and the cross-flashing outcome: +* **GPU 2 (Original MSI Board `0x1462`, VBIOS `90.02.60.00.17`)**: + * Idles at **`~31.8W – 33.7W`** when core is locked to 300 MHz. + * Minimum fan speed floor: **`25%`**. * Aggressive P3 voltage gating tables. -* **GPU 1 (NVIDIA Reference OEM `0x10DE`, VBIOS `90.02.60.00.01`)**: - * Idles at **`~61.2W`** when core is locked to 300–600 MHz. - * Minimum fan speed: **`40%`** (locked in VBIOS). - * Higher static VRM voltage rail leakage. -* **Actionable Solution**: Cross-flash GPU 2's MSI VBIOS (`90.02.60.00.17`) onto GPU 1 using `nvflash -6` to cut GPU 1's idle draw by ~30W and match fan curves. +* **GPU 1 (Original NVIDIA Reference OEM `0x10DE`, VBIOS `90.02.60.00.01`)**: + * Idled at **`~61.2W – 61.8W`** stock (locked 40% fan curve floor, higher static VRM leakage). +* **Cross-Flashing Result**: + * Cross-flashed GPU 1 with the MSI VBIOS ROM (`vbios/msi_cmp50hx_90.02.60.00.17.rom`) using `nvflash -6`. + * **Verified Outcome**: GPU 1 idle power immediately dropped from **`61.8W` down to `31.89W`** (a **~30W direct reduction!**), and fan curve synchronized to 25% minimum floor across both cards. --- diff --git a/vbios/msi_cmp50hx_90.02.60.00.17.rom b/vbios/msi_cmp50hx_90.02.60.00.17.rom new file mode 100644 index 0000000..ab8d72c Binary files /dev/null and b/vbios/msi_cmp50hx_90.02.60.00.17.rom differ