wmantly/turing-multi-gpu-llm-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: remove container-side power governor in favor of host-level cmp-tune

Browse Source
This commit is contained in:
wmantly
2026-09-02 21:01:52 +00:00
parent e037f7a122
commit 5daafe24e8
4 changed files with 6 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+6 -7
View File
@@ -36,18 +36,17 @@ Total VRAM: 32 GB GDDR6 across 3 GPUs (Shared PCIe Root Complex)
├── scripts/
│ ├── build-nccl-llama.sh # Compiles llama.cpp with CUDA 12 + NCCL + sm_75 optimizations
│ ├── ollama-proxy.py # High-performance Ollama/OpenAI/Anthropic proxy with thinking control
│ ├── power-governor.py # Dynamic GPU power & clock governor for CMP 50HX cards
│ ├── start-server.sh # Production start script for llama-server
│ └── tune-gpus.sh # Helper script for power limits & persistence mode
│ └── start-server.sh # Production start script for llama-server
├── systemd/
│ ├── llama-server.service # Systemd unit with NUMA socket pinning
│ ├── ollama-proxy.service # Systemd unit for API proxy
│ └── gpu-power-governor.service # Systemd unit for automated power governor
│ └── ollama-proxy.service # Systemd unit for API proxy
├── tuning/
│ └── cmp-tune.conf # Host-level CMP 50HX tuning profiles (stable-fast, efficient, insane)
├── vbios/
│ └── msi_cmp50hx_90.02.60.00.17.rom # Tuned MSI VBIOS ROM (31W idle & 25% min fan curve)
└── docs/
├── SETUP_GUIDE.md # Complete step-by-step setup guide on any Linux server
├── PROXMOX_LXC_GUIDE.md # GPU passthrough & NVML permissions in Proxmox LXC
├── PROXMOX_LXC_GUIDE.md # GPU passthrough, CPU socket pinning & host power management
├── HARDWARE_LEARNINGS.md # Deep dive: CMP VBIOS, PCIe Gen1 vs Gen3, riser cables, 20GB mods
├── 20GB_MOD_GUIDE.md # Hardware strapping & driver patching for 20GB TU102 mods
├── CMP_ECOSYSTEM_AND_COMMUNITY.md # Master ecosystem guide: projects, techniques & community
@@ -67,7 +66,7 @@ bash scripts/build-nccl-llama.sh /opt/llama.cpp
```bash
sudo cp systemd/*.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now llama-server.service ollama-proxy.service gpu-power-governor.service
sudo systemctl enable --now llama-server.service ollama-proxy.service
```
### 3. Connect from OpenWebUI