docs: top-10 fixes (PORTMAP, storage truth, banners, AdGuard scrub, zoraxy UCI removed, CT 111 section, Daytona purge)
This commit is contained in:
+2
-2
@@ -31,14 +31,14 @@ Do not start these without checking with the operator first.
|
||||
|
||||
## MCP gateway leaves, MCP child containers stay (planned)
|
||||
|
||||
When CT 109 "observe" lands, **`/opt/stacks/ai/mcp-gateway/` moves to CT 109** (it's a control plane: OIDC, agent scheduling, token store, usage stats). The ~20 MCP child server containers in this CT (`mcp-searxng`, `mcp-gotify`, `mcp-immich`, `mcp-fetch`, `mcp-time`, `comfyui-mcp`, `daytona-mcp`, `kroki-mcp`, etc.) **stay here on CT 104**.
|
||||
When CT 109 "observe" lands, **`/opt/stacks/ai/mcp-gateway/` moves to CT 109** (it's a control plane: OIDC, agent scheduling, token store, usage stats). The ~20 MCP child server containers in this CT (`mcp-searxng`, `mcp-gotify`, `mcp-immich`, `mcp-fetch`, `mcp-time`, `comfyui-mcp`, `coder-mcp`, `kroki-mcp`, etc.) **stay here on CT 104**.
|
||||
|
||||
The migrated gateway will reach this CT's docker daemon over the planned docker-socket-proxy (see `/docs/proxmox-optimizations.md` §16). Until CT 109 exists, the gateway runs locally and uses the bind-mounted socket. Do not piecemeal-move the gateway; it's tied to the CT 109 build.
|
||||
|
||||
## What is safe to do here
|
||||
|
||||
- Anything to do with the actively-running ~65 containers (Immich, n8n, Karakeep,
|
||||
ComfyUI, Daytona, Vaultwarden, Paperless, LobeChat, LiteLLM, MCP servers, etc.).
|
||||
ComfyUI, Vaultwarden, Paperless, LobeChat, LiteLLM, MCP servers, etc.). Daytona was decommissioned 2026-05-20 — compose files renamed `.DECOMMISSIONED-2026-05-20`; do not restart.
|
||||
- Image pulls + rebuilds for the active stacks (mind the n8n version pin — see its
|
||||
`docker-compose.yaml` comment).
|
||||
- Rotate exposed secrets in `.env` files (compose YAMLs themselves are git-tracked).
|
||||
|
||||
+35
-12
@@ -40,8 +40,9 @@
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [11000](http://192.168.1.40:11000) | Pocket ID | `pocket-id` | [id.nuclide.systems](https://id.nuclide.systems) | OIDC provider |
|
||||
| [11001](http://192.168.1.40:11001) | Vaultwarden | `vaultwarden` | [vault.nuclide.systems](https://vault.nuclide.systems) | Password manager |
|
||||
| [11001](http://192.168.1.40:11001) | Vaultwarden | `vaultwarden` | [vault.nuclide.systems](https://vault.nuclide.systems) | Password manager · **no Pocket-ID client yet — gap** |
|
||||
|
||||
> Pocket-ID **migrated from this CT to LXC 110** on 2026-05-20. See the [External Services](#external-services-not-on-nuc-docker) table below.
|
||||
|
||||
---
|
||||
|
||||
@@ -115,8 +116,8 @@ All arr-stack services run behind `vpn_gluetun` container network.
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [18000](http://192.168.1.40:18000) | Daytona API | `daytona-minimal-api-1` | [daytona.nuclide.systems](https://daytona.nuclide.systems) | Dev sandbox |
|
||||
| [18001](http://192.168.1.40:18001) | Daytona SSH | `daytona-minimal-ssh-gateway-1` | — | SSH gateway |
|
||||
<!-- Daytona DECOMMISSIONED 2026-05-20; replaced by Coder on CT 111. Compose renamed *.DECOMMISSIONED-2026-05-20. -->
|
||||
|
||||
|
||||
---
|
||||
|
||||
@@ -134,17 +135,38 @@ _Traccar moved to 15000–15001 (Documents range). 19000–19001 now free._
|
||||
|
||||
---
|
||||
|
||||
## LXC 111 — Dev (`192.168.1.42`)
|
||||
|
||||
CT 111 hosts the self-hosted dev platform. Same Pocket-ID SSO as the rest.
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [7080](http://192.168.1.42:7080) | Coder | `coder` | [dev.nuclide.systems](https://dev.nuclide.systems) | Workspace orchestrator; OIDC via Pocket-ID; password auth disabled |
|
||||
| [3000](http://192.168.1.42:3000) | Gitea | `gitea` | [git.nuclide.systems](https://git.nuclide.systems) | Self-hosted Git; OIDC; password form disabled |
|
||||
| [222](http://192.168.1.42:222) | Gitea SSH | `gitea` | — | `ssh -p 222 git@git.nuclide.systems` |
|
||||
| 5432 (internal) | coder-db | `coder-db` | — | Postgres 16 for Coder |
|
||||
| 5432 (internal) | gitea-db | `gitea-db` | — | Postgres 16 for Gitea |
|
||||
| [13080](http://192.168.1.42:13080) | docs site | `docs-server` | — (LAN-only) | mkdocs Material; auto-rebuilds from `fkrebs/docs` every 5 min |
|
||||
| (no port) | coder-fkrebs-dev | (workspace) | (Coder app proxy) | Active Coder workspace |
|
||||
| (no port) | act-runner | `act-runner` | — | Gitea Actions runner (`ct111-runner`) |
|
||||
|
||||
---
|
||||
|
||||
## External Services (Not on NUC Docker)
|
||||
|
||||
Zoraxy routes to these external backends:
|
||||
|
||||
| Domain | Target | Host | Service |
|
||||
|---|---|---|---|
|
||||
| [ha.nuclide.systems](https://ha.nuclide.systems) | `192.168.1.60:8123` | Home Assistant VM | Home automation |
|
||||
| [nc.nuclide.systems](https://nc.nuclide.systems) | `192.168.1.41:11000` | Nextcloud LXC | Cloud storage |
|
||||
| [ocpp.nuclide.systems](https://ocpp.nuclide.systems) | `192.168.1.60:8887` | Home Assistant VM | EV charger OCPP |
|
||||
| [shepard.nuclide.systems](https://shepard.nuclide.systems) | `192.168.1.49:80` | Docker host | Shepard |
|
||||
| [shepard-api.nuclide.systems](https://shepard-api.nuclide.systems) | `192.168.1.49:8080` | Docker host | Shepard API |
|
||||
| [ha.nuclide.systems](https://ha.nuclide.systems) | `192.168.1.60:8123` | Home Assistant VM 100 | Home automation |
|
||||
| [nc.nuclide.systems](https://nc.nuclide.systems) | `192.168.1.41:11000` | Nextcloud LXC 105 | Cloud storage |
|
||||
| [ocpp.nuclide.systems](https://ocpp.nuclide.systems) | `192.168.1.60:8887` | Home Assistant VM 100 | EV charger OCPP |
|
||||
| [shepard.nuclide.systems](https://shepard.nuclide.systems) | `192.168.1.49:80` | Shepard LXC 101 | Shepard |
|
||||
| [shepard-api.nuclide.systems](https://shepard-api.nuclide.systems) | `192.168.1.49:8080` | Shepard LXC 101 | Shepard API |
|
||||
| [id.nuclide.systems](https://id.nuclide.systems) | `192.168.1.5:11000` | Pocket-ID LXC 110 | OIDC IdP (migrated 2026-05-20) |
|
||||
| [git.nuclide.systems](https://git.nuclide.systems) | `192.168.1.42:3000` | Dev LXC 111 | Gitea (self-hosted Git) |
|
||||
| [dev.nuclide.systems](https://dev.nuclide.systems) | `192.168.1.42:7080` | Dev LXC 111 | Coder (workspace orchestrator) |
|
||||
| Gitea SSH | `192.168.1.42:222` | Dev LXC 111 | `ssh -p 222 git@git.nuclide.systems` |
|
||||
|
||||
---
|
||||
|
||||
@@ -157,7 +179,6 @@ Zoraxy routes to these external backends:
|
||||
| [mcp.nuclide.systems](https://mcp.nuclide.systems) | mcp-gateway | 8080 |
|
||||
| [arcane.nuclide.systems](https://arcane.nuclide.systems) | arcane | 10002 |
|
||||
| [gotify.nuclide.systems](https://gotify.nuclide.systems) | gotify | 10003 |
|
||||
| [id.nuclide.systems](https://id.nuclide.systems) | pocket-id | 11000 |
|
||||
| [vault.nuclide.systems](https://vault.nuclide.systems) | vaultwarden | 11001 |
|
||||
| [immich.nuclide.systems](https://immich.nuclide.systems) | immich_server | 12000 |
|
||||
| [immich-tools.nuclide.systems](https://immich-tools.nuclide.systems) | immich_power_tools | (container-internal) |
|
||||
@@ -165,7 +186,6 @@ Zoraxy routes to these external backends:
|
||||
| [n8n.nuclide.systems](https://n8n.nuclide.systems) | n8n | 16000 |
|
||||
| [memos.nuclide.systems](https://memos.nuclide.systems) | memos | 17000 |
|
||||
| [hoarder.nuclide.systems](https://hoarder.nuclide.systems) | karakeep | 17001 |
|
||||
| [daytona.nuclide.systems](https://daytona.nuclide.systems) | daytona-api | 18000 |
|
||||
| [traccar.nuclide.systems](https://traccar.nuclide.systems) | traccar | 15000 |
|
||||
| [dozzle.nuclide.systems](https://dozzle.nuclide.systems) | dozzle | 10001 |
|
||||
|
||||
@@ -192,7 +212,10 @@ Zoraxy routes to these external backends:
|
||||
| `62bf4e0d` | memos | `https://memos.nuclide.systems/auth/callback` |
|
||||
| `d92f82b0` | karakeep | `https://hoarder.nuclide.systems/api/auth/callback/custom` |
|
||||
| `a14b8076` | nextcloud | `https://nc.nuclide.systems/apps/user_oidc/code` |
|
||||
| `798a367f` | daytona | `https://daytona.nuclide.systems` |
|
||||
| `0aee4280` | Coder | `https://dev.nuclide.systems/api/v2/users/oidc/callback` |
|
||||
| `9444609e` | Gitea | `https://git.nuclide.systems/user/oauth2/pocket-id/callback` |
|
||||
| `38469e7e` | Proxmox VE | `https://192.168.1.20:8006` |
|
||||
| `798a367f` | ~~daytona~~ | **ORPHAN — Daytona decommissioned 2026-05-20, remove** |
|
||||
| `cbbf20d5` | Audiobookshelf | `https://abs.nuclide.systems/…` + `*` |
|
||||
| `d8733fcc` | shelfarr | `*` |
|
||||
| `78c78998` | Claude MCP | `https://claude.ai/api/mcp/auth_callback` + `https://mcp.nuclide.systems/mcp/auth/callback` |
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: STALE — many claims (PVE version, Pocket-ID port, Dockge as docker manager) no longer accurate. Source of truth is /CLAUDE.md and /docs/services/. This file kept for the original nexa-stack design notes only.**
|
||||
|
||||
---
|
||||
|
||||
# Instructions for Claude (and other agents)
|
||||
|
||||
This file tells future automated runs what they need to know about this repo.
|
||||
|
||||
+70
-383
@@ -1,409 +1,96 @@
|
||||
# Storage Layout — NUC 14 Docker Stacks
|
||||
# Storage layout
|
||||
|
||||
## Data Location Architecture
|
||||
> Verified state as of 2026-05-21. Source of truth for which service lives on which storage class.
|
||||
> Earlier revisions of this doc were stale on multiple items (Garage, n8n, Arcane, Karakeep, Pocket-ID, arr-stack, Nextcloud's storage protocol, missing Coder/Gitea). Reconciled via live audit.
|
||||
|
||||
All persistent service data is being consolidated to `/mnt/pve/unas/services/{service}/`
|
||||
on the UNAS NFS share, with media content in `/mnt/pve/unas/media/`.
|
||||
## Storage classes
|
||||
|
||||
**PostgreSQL databases stay on local NVMe** — NFS ownership checks (`stat(2)` UID match) prevent
|
||||
PostgreSQL from running on network filesystems. PostgreSQL is backed up via `pg_dump` to UNAS instead.
|
||||
|
||||
| Mount | Source | Content |
|
||||
|---|---|---|
|
||||
| `/mnt/pve/unas/services/` | UNAS NFS | SQLite DBs, configs, files, S3 data |
|
||||
| `/mnt/pve/unas/media/` | UNAS NFS | User media (photos, docs, books, audio) |
|
||||
| `/mnt/pve/unas/backup/` | UNAS NFS | Service backups |
|
||||
| `/var/lib/docker/volumes/` | Local NVMe | PostgreSQL data, Redis data |
|
||||
| `/opt/stacks/` | Local NVMe | Compose files, small configs |
|
||||
|
||||
## Migration Status
|
||||
|
||||
### ✅ Already on UNAS — `/mnt/pve/unas/services/`
|
||||
|
||||
| Service | UNAS Path(s) | Container Mount | Data Type |
|
||||
| Class | Substrate | Where | Typical use |
|
||||
|---|---|---|---|
|
||||
| gluetun | `gluetun/data/` | `/gluetun` | VPN config |
|
||||
| immich | `immich/{encoded-video,profile,thumbs}/` | `/usr/src/app/upload/{encoded-video,profile,thumbs}` | App derived data |
|
||||
| karakeep | `karakeep/data/`, `karakeep/meilisearch/` | `/data`, `/meili_data` | Bookmarks DB + Meilisearch index |
|
||||
| ntfy | `ntfy/{config,cache}/` | `/etc/ntfy`, `/var/lib/ntfy` | Notification config + cache |
|
||||
| paperless-ai | `paperless-ai/` | `/app/data` | AI processing data |
|
||||
| pocketid | `pocketid/` | `/app/data` | Auth data (SQLite) |
|
||||
| stremio | `stremio/` | `/root/.stremio-server` | Streaming server data |
|
||||
| traccar | `traccar/{data,logs,traccar.xml}` | `/opt/traccar/{data,logs,conf/traccar.xml}` | GPS tracking data |
|
||||
| vaultwarden | `vaultwarden/` | `/data` | Password vault (SQLite) |
|
||||
| garage | `shared-db/garage/{data,meta}/` | `/var/lib/garage/{data,meta}` | S3 object storage (Garage) |
|
||||
| Local zfs (rootfs) | NVMe in nuc | `/opt/stacks/<stack>/` on each LXC | Tier-1 hot state: databases, SQLite, secret stores, anything latency-sensitive |
|
||||
| Local zfs (subvol) | NVMe in nuc | `rpool/data/subvol-<NNN>-disk-0` per LXC | Container rootfs, image cache |
|
||||
| Docker named volume | NVMe in nuc | `/var/lib/docker/volumes/...` on each Docker CT | Per-container persistent state managed by Docker |
|
||||
| UNAS over **NFSv3** | `192.168.1.31` | `/mnt/pve/unas` on CTs 101, 103, 104, 111 | Bulk media, workspace home dirs, document storage |
|
||||
| UNAS over **CIFS/SMB 3.1.1** | `192.168.1.31` | `/mnt/pve/unas` on CT 105 (Nextcloud) | Nextcloud-only path — earlier docs mis-labelled this as NFS |
|
||||
|
||||
### 🆕 Shared Infrastructure — `/opt/stacks/shared-db/`
|
||||
## What lives where (verified 2026-05-21)
|
||||
|
||||
| Service | Hostname | Type | Data Location |
|
||||
|---|---|---|---|
|
||||
| shared-postgres | `shared-postgres` | PostgreSQL 16 (local volume) | `shared-db_shared-pgdata` volume |
|
||||
| garage | `garage` | S3-compatible object storage (on UNAS) | `/mnt/pve/unas/services/shared-db/garage/` |
|
||||
### UNAS NFS — `/mnt/pve/unas/services/...`
|
||||
|
||||
### ✅ Migrated (Phase 1) — SQLite to UNAS
|
||||
Bulk + media + non-latency-sensitive app data.
|
||||
|
||||
| Service | Old Path | New Path | Status |
|
||||
|---|---|---|---|
|
||||
| memos | `/opt/stacks/memos/data` | `/mnt/pve/unas/services/memos` | ✅ Healthy |
|
||||
| arcane | volume `arcane_arcane-data` | `/mnt/pve/unas/services/arcane` | ✅ Running |
|
||||
| n8n | volume `n8n_n8n_storage` | `/mnt/pve/unas/services/n8n` | ✅ Healthy |
|
||||
|
||||
### ⏳ Still Local — `/opt/stacks/` or Docker Volumes
|
||||
|
||||
| Service | Current Location | Data Type | Estimated Size | UNAS Target |
|
||||
| Service | CT | Path on UNAS → in container | Backed up? | Notes |
|
||||
|---|---|---|---|---|
|
||||
| arr-stack | `/opt/stacks/arr-stack/` | Configs + media | ~55 GB | `arr-stack/{config,audiobooks,ebooks,podcasts,torrents}` |
|
||||
| qdrant | `/opt/stacks/qdrant/qdrant_storage` | Vector DB | ~7 MB | `qdrant/` |
|
||||
| dozzle | `/opt/stacks/dozzle/dozzle_data` | Log viewer state | tiny | *(ephemeral, optional)* |
|
||||
| homepage | `/opt/stacks/homepage/{config,icons}` | Dashboard config | ~164 KB | *(keep in stack dir)* |
|
||||
| Immich (uploads, thumbs, derived) | 104 | `services/immich/{encoded-video,profile,thumbs}` + `media/images` + `backup/immich` | Immich own pg_dump → `media/images/db-dumps`; no off-host copy | |
|
||||
| Paperless-ngx (documents) | 104 | `media/documents/public/paperless-ngx/{consume,export,library}` | none | |
|
||||
| Paperless-AI | 104 | `services/paperless-ai` → `/app/data` | none | |
|
||||
| Vaultwarden (attachments + SQLite DB) | 104 | `services/vaultwarden` → `/data` | none | tier-1 DB on NFS — risky, plan to move local |
|
||||
| Traccar (logs, config) | 104 | `services/traccar/{logs,traccar.xml}` | none | data dir reverted to local `/opt/stacks/traccar/data` |
|
||||
| Memos | 104 | `services/memos` → `/var/opt/memos` | none | |
|
||||
| Arr-stack (Audiobookshelf, Prowlarr, RDTClient, Shelfarr) + media | 104 | `services/arr-stack/*` + `media/{audiobooks,ebooks,podcasts,Torrents}` | none | already migrated (older doc claimed "still local") |
|
||||
| Gluetun (VPN) | 104 | `services/gluetun/data` → `/gluetun` | none | |
|
||||
| Gitea | 111 | `services/gitea` → `/data` | none | new 2026-05-20 |
|
||||
| Coder workspace home dirs | 111 | `services/coder/<user>/<workspace>` → `/home/<user>` | none | new 2026-05-20 |
|
||||
| Backrest's jottacloud-mirrored data | 103 | `media/data-dir` only | **rclone → jottacloud (Backrest)** | only path with off-host backup |
|
||||
|
||||
### 🗄️ Databases Requiring Migration
|
||||
### Local zfs — `/opt/stacks/...`
|
||||
|
||||
| Service | Engine | Current Path | Target | Migration Method | Status |
|
||||
|---|---|---|---|---|---|---|
|
||||
| immich | PostgreSQL | `/opt/stacks/immich/postgres` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| litellm | PostgreSQL | `/opt/stacks/ai/postgres_data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| lobehub | PostgreSQL | `/opt/stacks/ai/lobehub/data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| daytona | PostgreSQL | volume `daytona-minimal_db_data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| paperless-ngx | PostgreSQL | volume `paperless-ngx_pgdata` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| memos | SQLite | `/opt/stacks/memos/data` | `/mnt/pve/unas/services/memos` | `cp` + `chown` | ✅ Done |
|
||||
| arcane | SQLite | volume `arcane_arcane-data` | `/mnt/pve/unas/services/arcane` | `cp` + `chown` | ✅ Done |
|
||||
| n8n | SQLite | volume `n8n_n8n_storage` | `/mnt/pve/unas/services/n8n` | `cp` + `chown` | ✅ Done |
|
||||
| pocketid | SQLite | `pocketid/` | *(already on UNAS)* | — | ✅ Done |
|
||||
| vaultwarden | SQLite | `vaultwarden/` | *(already on UNAS)* | — | ✅ Done |
|
||||
| karakeep | SQLite + Meilisearch | `karakeep/{data,meilisearch}` | *(already on UNAS)* | — | ✅ Done |
|
||||
Tier-1 state and anything that should NOT be NFS-backed.
|
||||
|
||||
### 🧠 ML Models & Caches (Regenerable — Can Stay Local)
|
||||
|
||||
| Resource | Path | Decision |
|
||||
|---|---|---|
|
||||
| Immich ML models | volume `immich_model-cache` | Cache — stay |
|
||||
| Paperless Redis | volume `paperless-ngx_redisdata` | Ephemeral — stay |
|
||||
| AI stack Redis | volume `nuc-ai-core_redis_data` | Ephemeral — stay |
|
||||
| SearXNG search cache | volume (hash) at `/var/lib/docker/volumes/bb5bb...` | Ephemeral — stay |
|
||||
| Daytona runner Docker | volume `daytona-minimal` runner data | Can stay |
|
||||
|
||||
## Media Content (Already on UNAS)
|
||||
|
||||
| Mount | Content |
|
||||
|---|---|
|
||||
| `/mnt/pve/unas/media/images/` | Immich photo library |
|
||||
| `/mnt/pve/unas/media/documents/public/paperless-ngx/` | Paperless documents (consume, export, library) |
|
||||
| `/mnt/pve/unas/media/audiobooks/` | Audiobookshelf media |
|
||||
| `/mnt/pve/unas/media/ebooks/` | Audiobookshelf ebooks |
|
||||
| `/mnt/pve/unas/media/podcasts/` | Audiobookshelf podcasts |
|
||||
| `/mnt/pve/unas/backup/immich/` | Immich DB backups |
|
||||
|
||||
## Prospective UNAS Layout
|
||||
|
||||
```
|
||||
/mnt/pve/unas/
|
||||
├── services/
|
||||
│ ├── gluetun/
|
||||
│ │ └── data/ ← VPN config
|
||||
│ ├── immich/
|
||||
│ │ ├── db/ ← PostgreSQL (to migrate)
|
||||
│ │ ├── encoded-video/ ← Already migrated
|
||||
│ │ ├── profile/ ← Already migrated
|
||||
│ │ └── thumbs/ ← Already migrated
|
||||
│ ├── karakeep/
|
||||
│ │ ├── data/ ← Already migrated
|
||||
│ │ └── meilisearch/ ← Already migrated
|
||||
│ ├── ntfy/
|
||||
│ │ ├── cache/ ← Already migrated
|
||||
│ │ └── config/ ← Already migrated
|
||||
│ ├── paperless-ai/ ← Already migrated
|
||||
│ ├── paperless-ngx/
|
||||
│ │ └── pgdata/ ← PostgreSQL (to migrate)
|
||||
│ ├── pocketid/ ← Already migrated
|
||||
│ ├── stremio/ ← Already migrated
|
||||
│ ├── traccar/
|
||||
│ │ ├── data/ ← Already migrated
|
||||
│ │ ├── logs/ ← Already migrated
|
||||
│ │ └── traccar.xml ← Already migrated
|
||||
│ ├── vaultwarden/ ← Already migrated
|
||||
│ ├── memos/ ← SQLite ✅ Done
|
||||
│ ├── qdrant/ ← Vector DB (to migrate)
|
||||
│ ├── arcane/ ← SQLite ✅ Done
|
||||
│ ├── n8n/ ← SQLite ✅ Done
|
||||
│ ├── shared-db/
|
||||
│ │ ├── postgres/ ← ⚠️ Local volume, NOT on UNAS
|
||||
│ │ └── garage/
|
||||
│ │ ├── data/ ← S3 data (on UNAS)
|
||||
│ │ └── meta/ ← S3 metadata (on UNAS)
|
||||
│ ├── arr-stack/
|
||||
│ │ ├── config/ ← Bind mounts (to migrate)
|
||||
│ │ ├── audiobooks/ ← Media (to migrate)
|
||||
│ │ ├── ebooks/ ← Media (to migrate)
|
||||
│ │ ├── podcasts/ ← Media (to migrate)
|
||||
│ │ └── torrents/ ← Downloads (to migrate)
|
||||
│
|
||||
├── media/
|
||||
│ ├── images/ ← Immich photo library
|
||||
│ └── documents/
|
||||
│ └── public/
|
||||
│ └── paperless-ngx/
|
||||
│ ├── consume/
|
||||
│ ├── export/
|
||||
│ └── library/
|
||||
│
|
||||
├── backups/
|
||||
│ └── immich/ ← Immich DB backups
|
||||
│
|
||||
└── (potential future: torrents/, cache/)
|
||||
```
|
||||
|
||||
## Garage S3 — `/opt/stacks/shared-db/`
|
||||
|
||||
S3-compatible object storage using [Garage](https://garagehq.deuxfleurs.fr/) v1.0.0.
|
||||
Replaces RustFS from the LobeHub AI stack. Single-node, replication_factor=1.
|
||||
|
||||
| Detail | Value |
|
||||
|---|---|
|
||||
| Container | `garage` (dxflrs/garage:v1.0.0) |
|
||||
| Data | `/mnt/pve/unas/services/shared-db/garage/{data,meta}` |
|
||||
| S3 API | `garage:3900` (internal, shared_backend) |
|
||||
| Admin API | `127.0.0.1:3901` (localhost only) |
|
||||
| Regional | No auth (internal network) |
|
||||
| Bucket | `lobe-files` |
|
||||
| Key ID | `GK55210bd6b56d92891db26fac` |
|
||||
| Secret Key | `d0af9e4eb69d7773ffa17950feee27925dbc70eb3efd4ee23a3e1324672c869a` |
|
||||
|
||||
## Shared PostgreSQL — `/opt/stacks/shared-db/`
|
||||
|
||||
Centralized PostgreSQL 16 instance for all non-immich services.
|
||||
|
||||
| Detail | Value |
|
||||
|---|---|
|
||||
| Container | `shared-postgres` (postgres:16) |
|
||||
| Data | `shared-db_shared-pgdata` (local Docker volume, NOT NFS) |
|
||||
| Networks | `shared_backend` |
|
||||
| Auth | `postgres` superuser with password from `.env` |
|
||||
|
||||
### Databases & Users (pre-created on init)
|
||||
|
||||
| Database | Owner | Password | Used By |
|
||||
|---|---|---|---|
|
||||
| `paperless` | `paperless` | `paperless` | paperless-ngx |
|
||||
| `daytona` | `user` | `pass` | daytona |
|
||||
| `litellm` | `litellm` | `litellm_password_here` | LiteLLM |
|
||||
| `lobechat` | `lobe` | `uWNZugjBqixf8dxC` | LobeHub |
|
||||
| `paradedb` | `lobe` | `uWNZugjBqixf8dxC` | LobeHub |
|
||||
|
||||
## Key Architectural Decision: PostgreSQL Stays Local
|
||||
|
||||
**Problem:** PostgreSQL checks data directory ownership via `stat(2)` and refuses to start
|
||||
if the process UID doesn't match the file owner UID. NFS with `all_squash` maps all UIDs
|
||||
to `65534` (nobody), breaking this check. `chown` also fails on NFS.
|
||||
|
||||
**Solution:** All PostgreSQL instances use local Docker volumes on NVMe:
|
||||
|
||||
```
|
||||
Local NVMe (/var/lib/docker/volumes/) UNAS NFS (/mnt/pve/unas/)
|
||||
├── shared-postgres data ├── pg_dump backups ← scheduled
|
||||
├── immich_postgres data ├── SQLite DBs (memos, arcane, n8n)
|
||||
├── paperless-ngx pgdata ├── Garage S3 data
|
||||
├── daytona-minimal db_data ├── Media files (immich, paperless)
|
||||
├── litellm-db / ai postgres_data ├── Configs / service data
|
||||
├── lobehub postgres data
|
||||
└── all Redis / cache volumes
|
||||
```
|
||||
|
||||
**Backup strategy:** `pg_dump` each database to UNAS on a schedule (e.g. daily cron).
|
||||
|
||||
## Migration Priority
|
||||
|
||||
| Priority | Service | Effort | Risk | Status | Reason |
|
||||
|---|---|---|---|---|---|
|
||||
| P0 | memos | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | n8n | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | arcane | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | qdrant | Minimal (file copy) | Low | ✅ Done | Fresh start on UNAS |
|
||||
| P1 | daytona → shared-postgres | Medium (pg_dump) | Low | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | litellm → shared-postgres | Medium (pg_dump) | Low | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | lobehub → shared-postgres | Medium (pg_dump) | Low | ⛔ Skipped | Uses ParadeDB (pg17+extensions), incompatible with shared-postgres (pg16) |
|
||||
| P1 | paperless-ngx → shared-postgres | Medium (pg_dump) | Medium | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | immich → shared-postgres | Medium (pg_dump) | Medium | ⏳ | Custom pgvector image |
|
||||
| P2 | arr-stack | Large (55 GB copy) | Low | ⏳ | Simple copy, time-consuming |
|
||||
|
||||
## Mail Archive (Potential Savings)
|
||||
|
||||
Unattached anonymous volumes from old/stopped containers:
|
||||
|
||||
| Volume | Size | Likely From | Action |
|
||||
|---|---|---|---|
|
||||
| `0b6d07ff...` | ? | old daytona runner | prune |
|
||||
| (100+ hex-named volumes) | ? | ephemeral build cache | prune |
|
||||
|
||||
Recommend: `docker system df` then `docker volume prune` to clean.
|
||||
|
||||
---
|
||||
|
||||
## Database Consolidation Plan
|
||||
|
||||
### Current: 10 Database Instances
|
||||
|
||||
| # | Container | Type | Image | Used By | Networks |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | `immich_postgres` | PostgreSQL | `immich-app/postgres:14-vectorchord` (pgvector) | immich | `immich_default` |
|
||||
| 2 | `paperless-ngx-db-1` | PostgreSQL | `postgres:16` | paperless-ngx | `paperless-ngx_default` |
|
||||
| 3 | `daytona-minimal-db-1` | PostgreSQL | `postgres:16-alpine` | daytona | `daytona-minimal_daytona-network` |
|
||||
| 4 | `nuc-ai-core-litellm-db-1` | PostgreSQL | `postgres:16-alpine` | litellm | `ai-internal` |
|
||||
| 5 | `lobe-postgres` | PostgreSQL | `postgres:16-alpine` | lobehub (lobechat, paradedb) | `ai-internal` |
|
||||
| 6 | `paperless-ngx-broker-1` | Redis 7 | `redis:7` | paperless broker | `paperless-ngx_default` |
|
||||
| 7 | `immich_redis` | Valkey 9 | `valkey/valkey:9` | immich cache | `immich_default` |
|
||||
| 8 | `daytona-minimal-redis-1` | Redis Alpine | `redis:alpine` | daytona | `daytona-minimal_daytona-network` |
|
||||
| 9 | `lobe-redis` | Redis 7 | `redis:7-alpine` | lobehub | `ai-internal` |
|
||||
| 10 | `redis-searxng` | Redis 7 | `redis:7-alpine` | searxng | `ai-internal` |
|
||||
|
||||
### Target: 6 Database Instances (remove 4)
|
||||
|
||||
| # | Container | Type | DBs Hosted | Saved |
|
||||
| Service | CT | Local path → in container | Backed up? | Notes |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `immich_postgres` | PostgreSQL immich-app (pgvector) | immich | 🔒 Must stay separate |
|
||||
| 2 | **`shared-postgres`** 🆕 | **PostgreSQL 16** | **paperless, daytona, litellm, lobechat, paradedb** | **−3 containers** |
|
||||
| 3 | `paperless-ngx-broker-1` | Redis 7 | paperless broker | 🔒 Must stay separate |
|
||||
| 4 | `immich_redis` | Valkey 9 | immich cache | 🔒 Must stay separate |
|
||||
| 5 | `daytona-minimal-redis-1` | Redis Alpine | daytona | 🔒 Stay separate |
|
||||
| 6 | `lobe-redis` | Redis 7 | lobehub | 🔒 Stay separate |
|
||||
| 7 | `redis-searxng` | Redis 7 | searxng | 🔒 Stay separate |
|
||||
| 7 | ~~`nuc-ai-core-litellm-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 8 | ~~`lobe-postgres`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 9 | ~~`paperless-ngx-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 10 | ~~`daytona-minimal-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| Pocket-ID | **110** | `/opt/stacks/pocketid/data` → `/app/data` | none | CT 110 has no NFS mount at all. Earlier doc said UNAS — incorrect. |
|
||||
| Garage (S3 meta + data) | 104 | `/opt/stacks/shared-db/garage/{data,meta}` → `/var/lib/garage/*` | none | **moved off NFS** 2026-05-19 after WAL-G outage. Stale copy may still exist on UNAS. |
|
||||
| Shared Postgres (multi-stack) | 104 | docker named volume `shared-db_shared-pgdata` | **WAL-G → local Garage** (configured; outage 2026-05-19 prompted move) | Garage itself is single-disk local — no off-host copy. |
|
||||
| n8n | 104 | `/opt/stacks/n8n/data` → `/home/node/.n8n` | none | **back to local** after PG migration attempt failed. Stale 408 MB `database.sqlite` left on UNAS (May 15) — clean up. n8n now uses `shared-postgres` as its actual DB. |
|
||||
| Arcane | 104 | `/opt/stacks/arcane/data` → `/app/data` | none | reverted from UNAS to local 2026-05-19 (undocumented before now) |
|
||||
| Karakeep + Meilisearch | 104 | `/opt/stacks/karakeep/localdata/{data,meili}` | none | reverted from UNAS to local |
|
||||
| Immich Postgres (pgvector) | 104 | `/opt/stacks/immich/postgres` → `/var/lib/postgresql/data` | Immich pg_dump → UNAS | tier-1; PG demands local disk |
|
||||
| Homepage | 104 | `/opt/stacks/homepage/{config,icons}` | none | |
|
||||
| Dozzle | 104 | `/opt/stacks/dozzle/dozzle_data` | none | |
|
||||
| LiteLLM config | 104 | `/opt/stacks/ai/litellm-config` | none | |
|
||||
| SearXNG config | 104 | `/opt/stacks/ai/searxng` | none | |
|
||||
| Flaresolverr | 104 | `/var/lib/flaresolver` | none | |
|
||||
| AdGuard / Zoraxy / DNS / Shepard / Backrest binaries | 102/108/103/101 | local zfs only | none | Backrest itself has no self-backup |
|
||||
| Nextcloud config + sidecars | 105 | local zfs (CT rootfs) | none | data path is CIFS — see below |
|
||||
|
||||
**Result: 10 → 6 instances, removing 4 containers, saving ~160MB RAM.**
|
||||
### Docker named volumes
|
||||
|
||||
### Why Redis Stays Separate
|
||||
Local on `/var/lib/docker/volumes/`. Mostly databases and caches.
|
||||
|
||||
| Reason | Detail |
|
||||
|---|---|
|
||||
| **Valkey ≠ Redis** | Immich uses Valkey 9 which has diverged from Redis 7 |
|
||||
| **`SELECT` collisions** | Apps may use specific DB numbers that clash |
|
||||
| **Password mismatch** | Each app has its own `requirepass` / `REDIS_PASSWORD` |
|
||||
| **Ephemeral = low value** | Caches/brokers rebuild on restart, savings don't justify risk |
|
||||
| **`FLUSHALL` risk** | One misconfigured app could wipe another's cache |
|
||||
|
||||
### Why PG Consolidation Works
|
||||
|
||||
| Reason | Detail |
|
||||
|---|---|
|
||||
| **Same major version** | All target DBs use PostgreSQL 16 |
|
||||
| **Native compatibility** | No conflicting extensions (only immich has pgvector) |
|
||||
| **Isolation via databases** | Each service gets its own database + user within shared PG |
|
||||
| **Simple restore** | `pg_dump` + `pg_restore` into shared instance |
|
||||
| **Single backup** | One `pg_dumpall` covers all 5 databases |
|
||||
|
||||
### Implementation: Shared PostgreSQL
|
||||
|
||||
Create `/opt/stacks/shared-db/docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
container_name: shared-postgres
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/pve/unas/services/shared-db:/var/lib/postgresql/data
|
||||
environment:
|
||||
POSTGRES_PASSWORD: ${SHARED_DB_PASSWORD}
|
||||
ports:
|
||||
- "127.0.0.1:5432:5432"
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "2.0"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "1.0"
|
||||
memory: 512M
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
networks:
|
||||
default:
|
||||
external: true
|
||||
name: shared_backend
|
||||
```
|
||||
|
||||
### Migration Order (Phase 2)
|
||||
|
||||
```
|
||||
Step 0: Create shared-postgres container
|
||||
Step 1: Dump ALL 4 databases from their current instances
|
||||
Step 2: Create users + databases on shared-postgres
|
||||
Step 3: Restore each dump into shared-postgres
|
||||
Step 4: One by one, update each app's compose to point to shared-postgres
|
||||
```
|
||||
|
||||
#### Detailed: Paperless-ngx
|
||||
|
||||
| Change | Before | After |
|
||||
| Volume | Service / CT | Backed up? |
|
||||
|---|---|---|
|
||||
| DB service | `db` (in paperless compose) | remove entirely |
|
||||
| `PAPERLESS_DBHOST` | `db` | `shared-postgres` |
|
||||
| DB credentials | defaults (paperless/paperless) | keep same |
|
||||
| Network | `paperless-ngx_default` | `shared_backend` |
|
||||
| `shared-db_shared-pgdata` | shared-postgres / 104 | WAL-G → local Garage |
|
||||
| `paperless-ngx_pgdata`, `_redisdata`, `_data` | paperless-ngx / 104 | none |
|
||||
| `lobe-postgres` + `lobe-redis` | lobehub / 104 | none |
|
||||
| `nuc-ai-core_rustfs-data` | lobehub stack / 104 | none |
|
||||
| `pgadmin-data` | pgadmin / 104 | none (regenerable) |
|
||||
| `immich_model-cache` | immich-ml / 104 | regenerable |
|
||||
| `coder-db` + `gitea-db` (docker volume `coder-db`) | / 111 | none |
|
||||
| (orphaned) `daytona-minimal_db_data` + runner anon vol | / 104 | none — clean up post-decommission |
|
||||
|
||||
#### Detailed: Daytona
|
||||
### Nextcloud (CT 105) — the CIFS outlier
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `db` (in daytona compose) | remove entirely |
|
||||
| `DB_HOST` | `db` | `shared-postgres` |
|
||||
| `DB_USERNAME` | `user` | keep same |
|
||||
| `DB_DATABASE` | `daytona` | keep same |
|
||||
| Network | `daytona-minimal_daytona-network` | `shared_backend` |
|
||||
Nextcloud AIO mounts `//192.168.1.31/storage` over **CIFS/SMB 3.1.1**, not NFS like every other CT. Application data path is under that mount. The Postgres + Redis sidecars stay on local docker volumes.
|
||||
|
||||
#### Detailed: LiteLLM
|
||||
## Backup reality
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `litellm-db` (in ai compose) | remove entirely |
|
||||
| `DATABASE_URL` | `postgresql://litellm:...@litellm-db:5432/litellm` | `postgresql://litellm:...@shared-postgres:5432/litellm` |
|
||||
| Network | `ai-internal` | `shared_backend` |
|
||||
**There is effectively no off-host backup of service data.** The only configured Backrest plan covers `/mnt/pve/unas/media/data-dir → jottacloud` — i.e., Backrest backs up *one specific UNAS path*, not the services that write to UNAS.
|
||||
|
||||
#### Detailed: LobeHub
|
||||
Coverage:
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `postgresql` (in lobehub.yml) | remove entirely |
|
||||
| `DATABASE_URL` | `postgresql://postgres:...@postgresql:5432/lobechat` | `postgresql://postgres:...@shared-postgres:5432/lobechat` |
|
||||
| Lobe also needs `paradedb` DB | created manually on initial setup | created via SQL script |
|
||||
| Network | `ai-internal` | `shared_backend` |
|
||||
- Immich pg_dump → UNAS (same filer; survives container loss, not filer loss)
|
||||
- shared-postgres WAL-G → Garage (same host; survives container loss, not disk loss)
|
||||
- Everything else: zero
|
||||
|
||||
### Total Impact
|
||||
This is a known gap. Plans:
|
||||
1. Extend Backrest plans to snapshot tier-1 paths (Pocket-ID sqlite, Vaultwarden data, Gitea repos, Coder workspace homes) to jottacloud nightly.
|
||||
2. Once the second NVMe lands (see `infra/proxmox-state.md` §6), mirror `rpool` so a single disk death doesn't take everything.
|
||||
|
||||
| Metric | Before | After |
|
||||
|---|---|---|
|
||||
| PostgreSQL instances | 5 | 2 |
|
||||
| Redis instances | 5 | 5 (unchanged) |
|
||||
| Total container count | 10 DBs | 7 DBs (−3) |
|
||||
| RAM saved (estimated) | — | ~160 MB |
|
||||
| Complexity | Fragmented | Centralized |
|
||||
| Backup burden | 5 pg_dumps | 2 pg_dumps |
|
||||
## Drift cleanup TODO
|
||||
|
||||
### Timing Estimates
|
||||
|
||||
| Service | Downtime | Cumulative |
|
||||
|---|---|---|
|
||||
| Setup shared-postgres + dump all | 0 (live dump) | 0 |
|
||||
| daytona → shared | ~2 min | 2 min |
|
||||
| litellm → shared | ~2 min | 4 min |
|
||||
| lobehub → shared | ~3 min | 7 min |
|
||||
| paperless-ngx → shared | ~5 min | 12 min |
|
||||
| immich DB → UNAS bind mount | ~10 min | 22 min |
|
||||
| **Total** | **~22 min** | |
|
||||
|
||||
### Notes
|
||||
|
||||
- All compose files are version-controlled in `/opt/stacks/{service}/docker-compose.yml`
|
||||
- Always test with `docker compose config` before `up -d`
|
||||
- PostgreSQL stays on local NVMe — backed up via `pg_dump` to UNAS
|
||||
- Garage S3 replaces RustFS from LobeHub stack (RustFS will be removed when LobeHub is switched)
|
||||
- LobeHub's `paradedb` extension requires `CREATE EXTENSION IF NOT EXISTS paradedb;` on the shared instance
|
||||
- All PostgreSQL dbs consolidated into shared-postgres except immich (pgvector requirement)
|
||||
- [ ] Remove stale `services/n8n/database.sqlite` (408 MB, last write 2026-05-15) from UNAS.
|
||||
- [ ] Remove stale `services/shared-db/garage/` copy from UNAS if present.
|
||||
- [ ] Remove orphaned `daytona-minimal_db_data` Docker volume on CT 104.
|
||||
- [ ] Vaultwarden `data/` on UNAS — plan to migrate to local zfs (latency + tier-1).
|
||||
- [ ] Add a Backrest plan for Pocket-ID DB and Gitea data → jottacloud.
|
||||
|
||||
Reference in New Issue
Block a user