fkrebs/docs
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

2026-05-23: PVE optimizations, LXC watchdog, KSM, startup order

Browse Source
This commit is contained in:
Florian Krebs
2026-05-23 11:19:21 +02:00
parent 7927390359
commit 3de4d5db63
3 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
7.8
View File
CHANGELOG.md
+7
View File
@@ -60,6 +60,13 @@ All notable infrastructure / service / doc changes. Newest first.
- **oom_score_adj=-300 set on HAOS KVM PID** (protects it from OOM killer — minimum score = hardest to kill).
- **Persistence via systemd timer** (`protect-haos-kvm.timer` on nuc): fires 60s after boot then every 5 min. Checks if KVM PID for VM 100 has oom_score_adj=-300; sets it if not.
### PVE host optimizations
- **LXC startup order set for all CTs**: CT103 (backrest), CT104 (docker), CT105 (nextcloud), CT112 (infisical), CT113 (db) had no startup order defined. Assigned: CT113=4, CT112=6, CT104=7, CT103=8, CT105=9. Full boot order now: DNS(1) → Zoraxy(2) → Pocket-ID(3) → DB(4) → Ops(5) → Infisical(6) → Docker(7) → Backrest(8) → Nextcloud(9) → HAOS(10) → Dev(20). Also set CT109 `onboot=1` (was missing).
- **KSM (Kernel Samepage Merging) enabled**: `echo 1 > /sys/kernel/mm/ksm/run` active now; persisted via `/etc/systemd/system/ksm-enable.service`. Deduplicates identical pages across LXC containers — effective for shared libc/runtime pages. Zero-risk, online change.
- **5-minute critical watchdog timer added**: `homelab-health-quick.timer` on nuc runs `QUICK=1 homelab-health` every 5 min. Checks: all LXC status (auto-starts if stopped), HAOS KVM (notify only), critical containers (Pocket-ID, Prometheus, Grafana, LiteLLM, MCP gateway). Complements the hourly full sweep.
- **Gitea issues filed for backlog PVE opts** (`fkrebs/docs` #2-4): NFS v3→v4 upgrade, ZFS ARC increase to 16 GiB (blocked on CT101 right-sizing), hugepages for HAOS KVM.
### n8n flow tracking
- **Gitea repo `fkrebs/n8n-flows` created**: exports all n8n workflows as individual JSON files. Daily cron (CT 104, 03:30) syncs via `n8n export:workflow --all` → split → commit + push.
RESUME.md
+7
View File
@@ -38,6 +38,13 @@ Last updated: 2026-05-23. Update this at the end of every session.
- [x] ~~**Arcane + Dozzle → CT 109**~~ — done 2026-05-23. Headless agents on all 7 Docker hosts.
- [x] ~~**crawl4ai SSE StreamConsumed**~~ — resolved 2026-05-22. Errors were from old code in log history; clean restart confirms no errors. Health: ok (7 tools), lobe-sync: 27/27 servers.
## Recently completed (2026-05-23, session continued ×2)
- **PVE LXC startup order fixed**: CT103/104/105/112/113 had no startup order. Set: CT113=4, CT112=6, CT104=7, CT103=8, CT105=9. Full deterministic boot sequence now in place. Also enabled CT109 onboot=1 (was missing).
- **KSM enabled on PVE host**: kernel samepage merging active + persisted via `ksm-enable.service`. Zero-cost memory deduplication across LXC containers.
- **5-minute critical watchdog deployed**: `homelab-health-quick.timer` on nuc; `QUICK=1` mode checks LXC status + critical containers every 5 min. Auto-starts stopped LXCs, notifies Gotify for HAOS KVM issues.
- **PVE backlog issues filed**: `fkrebs/docs` #2 (NFS v3→v4), #3 (ZFS ARC 16 GiB), #4 (hugepages for KVM).
## Recently completed (2026-05-23, session continued)
- **HAOS KVM OOM root-caused and fixed**: karakeep_chrome on CT 104 triggered host OOM → killed HAOS KVM twice. Applied: `shm_size: '512m'` to chrome, `oom_score_adj=-300` on KVM PID, systemd timer `protect-haos-kvm.timer` (every 5 min) for persistence.