Skip to content

# Proxmox Memory Audit

Last audited: 2026-05-22

## Host physical resources

Resource | Total | Used (idle) | Available
--- | --- | --- | ---
RAM | 62 GiB | ~33 GiB | ~28 GiB
Swap | 31 GiB | 0 GiB | 31 GiB

## CT memory allocations

CT | Name | Allocated (MiB) | Swap (MiB) | Typical use | Notes
--- | --- | --- | --- | --- | ---
101 | shepard | 32,768 | 8,192 | ~4 GB | Heavy stack: Mongo, Neo4j, TimescaleDB, Keycloak
102 | dns | 1,024 | 512 | ~100 MB | AdGuard Home
103 | backrest | 4,096 | 1,024 | ~200 MB | Restic scheduler — bumped 2026-05-23 for 822 GB initial backup OOM fix
104 | docker | 49,152 | 32,000 | 6–22 GB | Main Docker host; FLUX spikes to ~22 GB
105 | nextcloud | 8,196 | 8,196 | ~2 GB | Nextcloud AIO
108 | zoraxy | 2,048 | 512 | ~300 MB | Reverse proxy
109 | ops | 4,096 | 0 | ~1.5 GB | Prometheus + Grafana + Loki + Arcane + Dozzle + Homarr
110 | id | 1,024 | 512 | ~200 MB | Pocket-ID
111 | dev | 32,768 | 8,192 | ~3 GB | Coder + Gitea workspaces
112 | secrets | 4,096 | 512 | ~600 MB | Infisical
113 | db | 4,096 | 0 | ~800 MB | Postgres 17 + WAL-G
Sum |  | 141,316 MiB (138 GiB) |  |  | 2.2× overprovisioned vs physical RAM

## Key findings

### Overprovisioning is safe — until it isn't

Proxmox uses balloon drivers so CTs only consume what they actually use. At idle the host sits at ~33 GB used with 28 GB available. This is healthy. However, two specific CTs represent risk:

- CT 104 (49 GiB limit) + CT 111 (32 GiB limit) together could claim 81 GiB — well over the 62 GiB physical. If both hit peak simultaneously the host would start swapping heavily.
- CT 104's ComfyUI (FLUX generation) spikes from 6 GB baseline to ~22 GB. That 16 GB spike on top of the 33 GB idle baseline = 49 GB total, within physical RAM but tight.

### Why 40G Docker container limit destabilised the system

Setting ComfyUI's container limit to 40 G was the trigger. At generation time: - FLUX model + activations: ~22 GB in the container - Other ~65 Docker containers on CT 104: ~7 GB
- CT 104 OS + kernel: ~1 GB - Other CTs idle: ~26 GB
- Total: ~56 GB → host started swapping, degrading all services

### ComfyUI XPU memory accounting gap

ComfyUI's `get_free_memory()` for Intel XPU queries `torch.xpu.get_device_properties().total_memory` = 58 GB (the full shared memory pool — Arc shares system RAM). It has no awareness of the Docker cgroup limit. Smart memory management (`free_memory()`) calculates `memory_required - get_free_memory()` which is always hugely negative → never evicts models. This makes `--disable-smart-memory` irrelevant for XPU; smart memory is already broken.

Consequence: ComfyUI will always try to load the full model into XPU memory regardless of container limit. The cgroup OOM killer is the only backstop.

### Container limit recommendation for ComfyUI (CT 104)

Scenario | Limit | Safe?
--- | --- | ---
`--lowvram` (original) | 20 G | ✅ Safe but slow (231 s/image)
No `--lowvram`, FLUX only | 24 G | ✅ Fits FLUX peak (~22 GB) + 2 GB headroom
No `--lowvram` + img2img after FLUX | 26 G | ✅ FLUX stays resident, SD1.5 loads on top
28 G | ⚠️ | Marginal — OOM triggered in testing
40 G | ❌ | Destabilises host when generating

Peak host usage at 24 G container limit during FLUX generation: `24 + 7 (other containers) + 26 (other CTs idle) ≈ 57 GB` — stays under 62 GB physical.

## Recommendations

1. ComfyUI container limit: set to 24 G when running without `--lowvram`. Current revert to 20 G + `--lowvram` is stable but slower.
2. CT 104 LXC allocation (49 GiB): appropriately sized given Docker workload, but is by far the largest single consumer. Do not raise further without measuring host impact.
3. CT 111 (dev, 32 GiB): Coder workspaces could spike if users run heavy jobs. Consider adding a per-workspace memory limit in the Coder template.
4. Watch list: CT 101 (Shepard, 32 GiB) + CT 104 simultaneously at peak = 54 GB → host would need to swap. Unlikely in practice but possible during CI runs on CT 111 + FLUX generation on CT 104.
5. ~~Long-term: when CT 109 (ops) is built, run Prometheus `node_exporter` on the PVE host and alert when host available RAM drops below 8 GiB.~~ Done 2026-05-23 — CT 109 live, node_exporter scraping PVE host via pve-exporter.

## ComfyUI memory optimisation log

Date | Change | Effect
--- | --- | ---
2026-05-22 | Removed `--lowvram` + raised limit to 28 G | OOM at 28 G (XPU DRM buffers counted against cgroup)
2026-05-22 | Raised limit to 40 G | Host destabilised — reverted
2026-05-22 | Reverted to `--lowvram` + 20 G | Stable, slow (231 s/image)
2026-05-22 | Downloaded `t5-v1_1-xxl-encoder-Q4_K_S.gguf` (2.6 GB vs 3.2 GB Q5_K_M) | Saves 600 MB at load time
2026-05-22 | Kept Q5_K_M T5 (Q4_K_S degrades prompt following per city96); removed `--lowvram`; set limit to 24 G; added `--async-offload --force-fp16` | ~7–30 s generation, safe within host memory budget
2026-05-22 | Removed `--async-offload` | Flag incompatible with GGUF img2img on XPU — caused full CPU fallback (3.5 min/step) and pure-noise output. Removed; XPU generation now correct at ~7 s/step for txt2img. Current CLI: `--listen 0.0.0.0 --enable-cors-header --use-pytorch-cross-attention --disable-smart-memory --force-fp16`