Initial commit: Complete deployment scripts, power governor, systemd units, and architecture documentation for Turing multi-GPU LLM rig

2026-09-01 01:55:19 +00:00
commit aeb72cecfa
13 changed files with 1925 additions and 0 deletions
@@ -0,0 +1,64 @@
+# 📦 Proxmox VE LXC Container GPU Passthrough & Hardware Permissions
+
+Running multi-GPU AI inference and hardware power management inside a Proxmox LXC container requires specific device mappings, cgroup permissions, and capability flags.
+
+---
+
+## 1. Proxmox Host Configuration (`/etc/pve/lxc/<CTID>.conf`)
+
+Add the following lines to your container configuration file on the Proxmox host:
+
+```ini
+# /etc/pve/lxc/<CTID>.conf
+
+# 1. Unconfined AppArmor profile (Required for NVML clock/power limit modification)
+lxc.apparmor.profile: unconfined
+
+# 2. Grant SYS_ADMIN capability for hardware clock management
+lxc.cap.keep: sys_admin sys_rawio
+
+# 3. Allow all NVIDIA device cgroups
+lxc.cgroup2.devices.allow: c 195:* rwm
+lxc.cgroup2.devices.allow: c 235:* rwm
+lxc.cgroup2.devices.allow: c 510:* rwm
+lxc.cgroup2.devices.allow: c 511:* rwm
+
+# 4. Pass-through NVIDIA character device nodes
+lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
+lxc.mount.entry: /dev/nvidia1 dev/nvidia1 none bind,optional,create=file
+lxc.mount.entry: /dev/nvidia2 dev/nvidia2 none bind,optional,create=file
+lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
+lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
+lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
+lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
+```
+
+---
+
+## 2. Verifying Permissions Inside the Container
+
+Restart the container, then run:
+
+```bash
+# Verify all GPUs are visible
+nvidia-smi
+
+# Test NVML Persistence Mode (Requires CAP_SYS_ADMIN)
+nvidia-smi -pm 1
+
+# Test Power Capping
+nvidia-smi -i 1,2 -pl 150
+
+# Test Dynamic Clock Locking
+nvidia-smi -i 1,2 -lgc 600,600
+nvidia-smi -i 1,2 -rgc
+```
+
+If all four commands return with code `0` and "All done", your container has full hardware rights.
+
+---
+
+## 3. Important Systemd Service Security Flags
+
+When running services that execute `nvidia-smi` hardware commands inside systemd:
+* Do **NOT** set `NoNewPrivileges=true` in `gpu-power-governor.service`. `NoNewPrivileges=true` blocks processes from acquiring permissions to execute privileged NVML clock-locking calls.