diff --git a/docs/20GB_MOD_GUIDE.md b/docs/20GB_MOD_GUIDE.md new file mode 100644 index 0000000..12387f9 --- /dev/null +++ b/docs/20GB_MOD_GUIDE.md @@ -0,0 +1,48 @@ +# 🛠️ CMP 50HX 20GB VRAM Mod & Driver Patching Guide + +This guide covers how to physically mod and software-unlock the **NVIDIA CMP 50HX** from **10 GB to 20 GB VRAM** (20,480 MiB per card), based on the validated research from [`xrip/cmp50hx-unlock`](https://github.com/xrip/cmp50hx-unlock). + +--- + +## 1. Hardware Modification: Swapping GDDR6 BGA Chips + +### Architecture & Memory Topology +* The CMP 50HX is built on the **TU102 PCB** with a **320-bit memory bus (10 memory pads)**. +* **Stock Configuration**: 10x 1GB (8Gbit) 14Gbps GDDR6 BGA-180 chips (e.g., Samsung `K4Z80325BC-HC14`). +* **20GB Configuration**: 10x 2GB (16Gbit) 14Gbps/16Gbps GDDR6 BGA-180 chips: + * **Samsung**: `K4ZAF325BM-HC14` or `K4ZAF325BM-HC16` + * **Micron**: `D9ZCL` (MT61K512M32KPA-14) / `D9WCW` + * **Hynix**: `H56G32AO8DX-004` + +### Hardware Strapping Resistors +* After desoldering the old 8Gbit ICs and soldering the 16Gbit ICs: +* Locate the **memory density strapping resistor dividers** on the rear of the TU102 PCB near the memory controller. +* Move the strapping pull-up/pull-down resistor to configure the memory controller for **16Gbit per channel**. + +--- + +## 2. The Software Barrier: NVIDIA Booter Error `0x8d` + +### Why Stock NVIDIA Drivers Fail on 20GB CMP 50HX +* In stock closed drivers, the firmware security module (FWSEC) uses hardcoded WPR2 bounds calculated for a 10GB address space. +* On a 20GB card, stock FWSEC creates: + ```text + WPR high:low = 04ffee00:04ffe000 + span = 00000e00 + ``` +* Because the hardcoded 10GB bounds do not match the 20GB span, NVIDIA Booter aborts with error `0x8d`, GSP-RM fails initialization, and `nvidia-smi` detects no devices. + +### The Fix: Dynamic WPR2 Handling (`xrip/cmp50hx-unlock`) +* The patched NVIDIA Open Kernel Module (`610.43.03`) reads the FWSEC WPR2 range dynamically for each GPU's PCI BDF (Bus/Device/Function). +* It accepts span `0xe00`, saves the low/high bounds per BDF, and passes GSP readiness checks. +* **Mixed Cards Supported**: You can run 10GB and 20GB CMP 50HX cards in the same system simultaneously because WPR2 bounds are indexed per PCI BDF. + +--- + +## 3. Multi-Card VRAM Scaling for LLM Inference + +| Cluster Setup | Total VRAM | Model Capability | +| :--- | :---: | :--- | +| **2x 20GB CMP 50HX** | **40 GB VRAM** | Runs `Llama-3.3-70B` or `Qwen2.5-72B` (Q4_K_M) across 2 slots! | +| **3x 20GB CMP 50HX** | **60 GB VRAM** | Runs `Qwen2.5-72B` (Q6_K / Q8_0) with full 64K+ context | +| **4x 20GB CMP 50HX** | **80 GB VRAM** | Enterprise A100-class capacity (runs 70B FP8 / Q8_0 at 128K context) | diff --git a/docs/HARDWARE_LEARNINGS.md b/docs/HARDWARE_LEARNINGS.md index 1e068d6..7f5d3f1 100644 --- a/docs/HARDWARE_LEARNINGS.md +++ b/docs/HARDWARE_LEARNINGS.md @@ -99,10 +99,23 @@ Comparing the hardware profiles and the cross-flashing outcome: ## 8. 20GB VRAM Modding Feasibility (CMP 50HX TU102) * **Architecture**: CMP 50HX uses the **TU102 PCB layout (320-bit bus, 10 memory pads)**. -* **Memory Swap**: - * Desolder 10x 1GB (8Gbit) GDDR6 BGA-180 chips (e.g., Samsung `K4Z80325BC-HC14`). - * Solder 10x 2GB (16Gbit) GDDR6 BGA-180 chips (e.g., Samsung `K4ZAF325BM-HC14` or Micron `D9ZCL`). - * Modify memory strapping resistor dividers to signal 16Gbit density to the TU102 memory controller. -* **Payoff**: - * 2x Modded Cards = **40 GB Total VRAM** (capable of running full 70B/72B models like `Llama-3.3-70B` or `Qwen2.5-72B` on just 2 dedicated x16 slots). +* **Hardware Swap**: Desolder 10x 1GB (8Gbit) GDDR6 BGA-180 chips and replace with 10x 2GB (16Gbit) chips (e.g., Samsung `K4ZAF325BM-HC14` or Micron `D9ZCL`). Adjust memory strapping resistor dividers. +* **Driver Patching for 20GB (WPR2 Fix)**: See [`docs/20GB_MOD_GUIDE.md`](20GB_MOD_GUIDE.md) for details on patching the NVIDIA open kernel driver to bypass the `0x8d` FWSEC error. +* **Capacity Scaling**: + * 2x Modded Cards = **40 GB Total VRAM** (runs full 70B/72B models on 2 slots). * 4x Modded Cards = **80 GB Total VRAM** (enterprise A100-tier capacity for under $1,000). + +--- + +## 9. Private NVAPI `NvAPI_GPU_SetForcePstate` (`0x025BFB10`) + +* **The Discovery**: While `nvidia-smi` cannot force P8 state on headless Linux, the private NVAPI entry point `NvAPI_GPU_SetForcePstate` (`0x025BFB10`) inside `libnvidia-api.so.1` can explicitly force the GPU into **P8 state (645 MHz core / 405 MHz memory)**. +* **Implementation**: We packaged [`tools/cmp-pstate.py`](../tools/cmp-pstate.py) to directly invoke this entry point per PCI Bus ID. +* **Restoration**: Passing `--pstate 16` returns power management control to unconstrained P0/P3 states before active inference. + +--- + +## 10. Automated PCIe Link Retraining Service (`cmp50hx-gen2`) + +* **The Boot Lock**: CMP 50HX cards feature an internal PCIe speed lock register that takes a short period after boot to become accessible. +* **The Solution**: A systemd boot service triggers a link retrain on the upstream PCIe port after boot, ensuring the bus reliably trains at full speed without manual re-initialization. diff --git a/tools/cmp-pstate.py b/tools/cmp-pstate.py new file mode 100755 index 0000000..7dd2e58 --- /dev/null +++ b/tools/cmp-pstate.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python3 +"""Force one NVIDIA GPU into a P-state through the private NVAPI entry point.""" + +import argparse +import ctypes +import re +import sys + + +NVAPI_OK = 0 +NVAPI_MAX_PHYSICAL_GPUS = 64 +NVAPI_ENUM_PHYSICAL_GPUS = 0xE5AC921F +NVAPI_GPU_GET_BUS_ID = 0x1BE0B8E5 +NVAPI_GPU_SET_FORCE_PSTATE = 0x025BFB10 +NVAPI_INITIALIZE = 0x0150E828 +NVAPI_UNLOAD = 0xD22BDD7E + + +def status_text(status): + return "0x%08x" % (ctypes.c_uint32(status).value,) + + +def bus_number(bus_id): + match = re.search(r"(?:^|:)([0-9a-fA-F]{2}):[0-9a-fA-F]{2}\.[0-7]$", + bus_id.strip()) + if not match: + raise ValueError("invalid PCI bus ID: %s" % bus_id) + return int(match.group(1), 16) + + +def query_function(query, function_id, restype, *argtypes): + address = query(ctypes.c_uint32(function_id)) + if not address: + raise RuntimeError("NVAPI function 0x%08x is unavailable" % function_id) + return ctypes.CFUNCTYPE(restype, *argtypes)(address) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--bus-id", required=True, + help="Linux PCI bus ID, for example 00000000:01:00.0") + parser.add_argument("--pstate", required=True, type=int, + help="P-state number: 0..15, or 16 for automatic/high") + args = parser.parse_args() + if not 0 <= args.pstate <= 16: + parser.error("--pstate must be between 0 and 16") + + try: + wanted_bus = bus_number(args.bus_id) + api = ctypes.CDLL("libnvidia-api.so.1") + query = api.nvapi_QueryInterface + query.argtypes = [ctypes.c_uint32] + query.restype = ctypes.c_void_p + + initialize = query_function(query, NVAPI_INITIALIZE, ctypes.c_int32) + unload = query_function(query, NVAPI_UNLOAD, ctypes.c_int32) + enum_gpus = query_function( + query, NVAPI_ENUM_PHYSICAL_GPUS, ctypes.c_int32, + ctypes.POINTER(ctypes.c_void_p), ctypes.POINTER(ctypes.c_uint32)) + get_bus_id = query_function( + query, NVAPI_GPU_GET_BUS_ID, ctypes.c_int32, + ctypes.c_void_p, ctypes.POINTER(ctypes.c_uint32)) + set_force_pstate = query_function( + query, NVAPI_GPU_SET_FORCE_PSTATE, ctypes.c_int32, + ctypes.c_void_p, ctypes.c_uint32, ctypes.c_uint32) + except (OSError, RuntimeError, ValueError) as exc: + print("cmp-pstate: %s" % exc, file=sys.stderr) + return 1 + + status = initialize() + if status != NVAPI_OK: + print("cmp-pstate: NvAPI_Initialize failed (%s)" % status_text(status), + file=sys.stderr) + return 1 + + try: + handles = (ctypes.c_void_p * NVAPI_MAX_PHYSICAL_GPUS)() + count = ctypes.c_uint32() + status = enum_gpus(handles, ctypes.byref(count)) + if status != NVAPI_OK: + print("cmp-pstate: GPU enumeration failed (%s)" % + status_text(status), file=sys.stderr) + return 1 + + handle = None + for index in range(min(count.value, NVAPI_MAX_PHYSICAL_GPUS)): + bus = ctypes.c_uint32() + status = get_bus_id(handles[index], ctypes.byref(bus)) + if status == NVAPI_OK and bus.value == wanted_bus: + handle = handles[index] + break + if handle is None: + print("cmp-pstate: NVAPI GPU for PCI bus %02x was not found" % + wanted_bus, file=sys.stderr) + return 1 + + status = set_force_pstate(handle, args.pstate, 0) + if status != NVAPI_OK: + print("cmp-pstate: P%d request failed (%s)" % + (args.pstate, status_text(status)), file=sys.stderr) + return 1 + return 0 + finally: + unload() + + +if __name__ == "__main__": + sys.exit(main())