Skip to content

Proxmox Memory Audit

Last audited: 2026-05-22

Host physical resources

Resource Total Used (idle) Available
RAM 62 GiB ~33 GiB ~28 GiB
Swap 31 GiB 0 GiB 31 GiB

CT memory allocations

CT Name Allocated (MiB) Swap (MiB) Typical use Notes
101 shepard 32,768 8,192 ~4 GB Heavy stack: Mongo, Neo4j, TimescaleDB, Keycloak
102 dns 1,024 512 ~100 MB AdGuard Home
103 backrest 4,096 1,024 ~200 MB Restic scheduler — bumped 2026-05-23 for 822 GB initial backup OOM fix
104 docker 49,152 32,000 6–22 GB Main Docker host; FLUX spikes to ~22 GB
105 nextcloud 8,196 8,196 ~2 GB Nextcloud AIO
108 zoraxy 2,048 512 ~300 MB Reverse proxy
109 ops 4,096 0 ~1.5 GB Prometheus + Grafana + Loki + Arcane + Dozzle + Homarr
110 id 1,024 512 ~200 MB Pocket-ID
111 dev 32,768 8,192 ~3 GB Coder + Gitea workspaces
112 secrets 4,096 512 ~600 MB Infisical
113 db 4,096 0 ~800 MB Postgres 17 + WAL-G
Sum 141,316 MiB (138 GiB) 2.2× overprovisioned vs physical RAM

Key findings

Overprovisioning is safe — until it isn't

Proxmox uses balloon drivers so CTs only consume what they actually use. At idle the host sits at ~33 GB used with 28 GB available. This is healthy. However, two specific CTs represent risk:

  • CT 104 (49 GiB limit) + CT 111 (32 GiB limit) together could claim 81 GiB — well over the 62 GiB physical. If both hit peak simultaneously the host would start swapping heavily.
  • CT 104's ComfyUI (FLUX generation) spikes from 6 GB baseline to ~22 GB. That 16 GB spike on top of the 33 GB idle baseline = 49 GB total, within physical RAM but tight.

Why 40G Docker container limit destabilised the system

Setting ComfyUI's container limit to 40 G was the trigger. At generation time: - FLUX model + activations: ~22 GB in the container - Other ~65 Docker containers on CT 104: ~7 GB
- CT 104 OS + kernel: ~1 GB - Other CTs idle: ~26 GB
- Total: ~56 GB → host started swapping, degrading all services

ComfyUI XPU memory accounting gap

ComfyUI's get_free_memory() for Intel XPU queries torch.xpu.get_device_properties().total_memory = 58 GB (the full shared memory pool — Arc shares system RAM). It has no awareness of the Docker cgroup limit. Smart memory management (free_memory()) calculates memory_required - get_free_memory() which is always hugely negative → never evicts models. This makes --disable-smart-memory irrelevant for XPU; smart memory is already broken.

Consequence: ComfyUI will always try to load the full model into XPU memory regardless of container limit. The cgroup OOM killer is the only backstop.

Container limit recommendation for ComfyUI (CT 104)

Scenario Limit Safe?
--lowvram (original) 20 G ✅ Safe but slow (231 s/image)
No --lowvram, FLUX only 24 G ✅ Fits FLUX peak (~22 GB) + 2 GB headroom
No --lowvram + img2img after FLUX 26 G ✅ FLUX stays resident, SD1.5 loads on top
28 G ⚠️ Marginal — OOM triggered in testing
40 G ❌ Destabilises host when generating

Peak host usage at 24 G container limit during FLUX generation: 24 + 7 (other containers) + 26 (other CTs idle) ≈ 57 GB — stays under 62 GB physical.

Recommendations

  1. ComfyUI container limit: set to 24 G when running without --lowvram. Current revert to 20 G + --lowvram is stable but slower.
  2. CT 104 LXC allocation (49 GiB): appropriately sized given Docker workload, but is by far the largest single consumer. Do not raise further without measuring host impact.
  3. CT 111 (dev, 32 GiB): Coder workspaces could spike if users run heavy jobs. Consider adding a per-workspace memory limit in the Coder template.
  4. Watch list: CT 101 (Shepard, 32 GiB) + CT 104 simultaneously at peak = 54 GB → host would need to swap. Unlikely in practice but possible during CI runs on CT 111 + FLUX generation on CT 104.
  5. ~~Long-term: when CT 109 (ops) is built, run Prometheus node_exporter on the PVE host and alert when host available RAM drops below 8 GiB.~~ Done 2026-05-23 — CT 109 live, node_exporter scraping PVE host via pve-exporter.

ComfyUI memory optimisation log

Date Change Effect
2026-05-22 Removed --lowvram + raised limit to 28 G OOM at 28 G (XPU DRM buffers counted against cgroup)
2026-05-22 Raised limit to 40 G Host destabilised — reverted
2026-05-22 Reverted to --lowvram + 20 G Stable, slow (231 s/image)
2026-05-22 Downloaded t5-v1_1-xxl-encoder-Q4_K_S.gguf (2.6 GB vs 3.2 GB Q5_K_M) Saves 600 MB at load time
2026-05-22 Kept Q5_K_M T5 (Q4_K_S degrades prompt following per city96); removed --lowvram; set limit to 24 G; added --async-offload --force-fp16 ~7–30 s generation, safe within host memory budget
2026-05-22 Removed --async-offload Flag incompatible with GGUF img2img on XPU — caused full CPU fallback (3.5 min/step) and pure-noise output. Removed; XPU generation now correct at ~7 s/step for txt2img. Current CLI: --listen 0.0.0.0 --enable-cors-header --use-pytorch-cross-attention --disable-smart-memory --force-fp16