docs: top-10 fixes (PORTMAP, storage truth, banners, AdGuard scrub, zoraxy UCI removed, CT 111 section, Daytona purge)
This commit is contained in:
@@ -1,3 +1,7 @@
|
||||
> **STATUS: NARRATIVE — historical writeup. Not authoritative for current state.**
|
||||
|
||||
---
|
||||
|
||||
# Case Study — The Nuclide Homelab, built with Claude
|
||||
|
||||
## Origin story
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: SUPERSEDED 2026-05-17 — current implementation lives in services/mcp-gateway.md. Kept for design-rationale history.**
|
||||
|
||||
---
|
||||
|
||||
# MCP Gateway — Reconstructed Design Spec (in-progress, "Phase 1")
|
||||
|
||||
> Reconstructed 2026-05-16 from code/configs/git history. The gateway is a
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: SNAPSHOT — frozen inventory from 2026-05-17. Reality has moved on; consult CHANGELOG.md + services/* for current state.**
|
||||
|
||||
---
|
||||
|
||||
# Scrubbing List — /opt/stacks (2026-05-17)
|
||||
|
||||
Read-only audit of unused/stale data. **Nothing here has been deleted** — review
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: ABANDONED 2026-05-16 — Zoraxy is the production reverse proxy. Kept for design-decision history.**
|
||||
|
||||
---
|
||||
|
||||
# Traefik Migration Guide Using Docker Labels
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: ABANDONED 2026-05-16 — Zoraxy is the production reverse proxy. Kept for design-decision history. See services/zoraxy.md for current setup.**
|
||||
|
||||
---
|
||||
|
||||
# Recommended: Traefik Proxy Replacement
|
||||
|
||||
## Why Traefik over Zoraxy?
|
||||
|
||||
@@ -5,7 +5,7 @@ AdGuard DNS at `192.168.1.2` can handle internal domain resolution, eliminating
|
||||
|
||||
## Current Setup
|
||||
- **AdGuard URL**: `http://192.168.1.2`
|
||||
- **Status**: Running on Proxmox VM 102
|
||||
- **Status**: Running on Proxmox **LXC 102** (`dns`, 192.168.1.2)
|
||||
- **Homepage widget**: Configured
|
||||
|
||||
## AI Service DNS Entries
|
||||
@@ -48,7 +48,8 @@ Add these static DNS entries in AdGuard to resolve AI services locally:
|
||||
# Get session token first
|
||||
curl -c /tmp/cookies.txt -k -X POST http://192.168.1.2:3000/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"username": "root", "password": "tapirnase"}'
|
||||
-d "{\"username\": \"root\", \"password\": \"${ADGUARD_PASSWORD}\"}"
|
||||
# Export ADGUARD_PASSWORD from your shell env or a `.env` file — never hardcode.
|
||||
|
||||
# Add static DNS entry
|
||||
curl -s -k -c /tmp/cookies.txt -b /tmp/cookies.txt \
|
||||
@@ -115,8 +116,8 @@ echo "✅ Done! Add these in AdGuard UI manually."
|
||||
- `pp-api.nuclide.systems` → `192.168.1.40:8000` (Paperless API)
|
||||
|
||||
### Home Assistant
|
||||
- `ha.nuclide.systems` → `192.168.1.10` (Home Assistant)
|
||||
- `homeassistant.local` → `192.168.1.10` (mDNS fallback)
|
||||
- `ha.nuclide.systems` → `192.168.1.60` (Home Assistant OS VM 100)
|
||||
- `homeassistant.local` → `192.168.1.60` (mDNS fallback)
|
||||
|
||||
## Internal DNS Server Setup
|
||||
|
||||
@@ -200,7 +201,7 @@ S3_ENDPOINT=https://s3.nuclide.systems
|
||||
|
||||
## Notes
|
||||
|
||||
- AdGuard runs on VM 102 at `192.168.1.2`
|
||||
- AdGuard runs on **LXC 102** at `192.168.1.2`
|
||||
- All AI services run on `192.168.1.40`
|
||||
- Using same IP for all domains is intentional (single endpoint)
|
||||
- SSL certs are issued by Zoraxy (public domain validation)
|
||||
|
||||
@@ -10,7 +10,7 @@ across ~23 compose stacks. Principle: **self-host everything**, OIDC SSO,
|
||||
`rpool/data/subvol-104-disk-0`. Mounts the UNAS NFS at `/mnt/pve/unas`.
|
||||
- **Proxmox VE** — host **`192.168.1.20:8006`**, single node **`nuc`**
|
||||
(Intel Core Ultra 7 155H, 22 threads, 64G RAM, PVE 9.1.11). API access via
|
||||
`root@pam!mcp` token (now `PVEAuditor`, read-only). **7 guests:**
|
||||
`root@pam!mcp` token (now `PVEAuditor`, read-only). **9 guests (+ 2 planned)** — canonical roster in [`ct-inventory.md`](../ct-inventory.md):
|
||||
- `100` qemu **haos** — Home Assistant OS VM (4c/16G) → `.60`
|
||||
- `101` lxc **shepard** — secondary Docker host (12c/16G/107G) → `.49`
|
||||
- `102` lxc **dns** — **AdGuard Home** (network DNS + ad/tracker blocking,
|
||||
@@ -22,6 +22,10 @@ across ~23 compose stacks. Principle: **self-host everything**, OIDC SSO,
|
||||
- `104` lxc **docker** — **the primary `/opt/stacks` host** (16c/32G/215G) → `.40`
|
||||
- `105` lxc **nextcloud** — Nextcloud (4c/8G/107G) → `.41`
|
||||
- `108` lxc **zoraxy** — reverse proxy (2c/2G) → `.4`
|
||||
- `110` lxc **id** — Pocket-ID OIDC IdP (1c/1G/4G) → `.5`; migrated off 104 on 2026-05-20
|
||||
- `111` lxc **dev** — Coder + Gitea (12c/32G/60G) → `.42`; new 2026-05-20
|
||||
- `109` lxc **observe** *(planned)* — Prometheus/Grafana/Arcane/Dozzle/Homarr/Tinyauth; see §16 of `infra/proxmox-state.md`
|
||||
- `113` lxc **db** *(planned)* — shared Postgres + pgbouncer once second NVMe lands
|
||||
- Storages: `local` (dir), `local-zfs` (zfspool, ~1.9T), `unas` (nfs, ~20T).
|
||||
- **Ubiquiti UniFi** — UDM "Home" (UDMA6A8, UCG Fiber) gateway + 4× U7-series
|
||||
APs. Controller at **`https://192.168.1.1`** (UniFi OS 5.0.16, SSO + MFA
|
||||
@@ -133,12 +137,12 @@ across ~23 compose stacks. Principle: **self-host everything**, OIDC SSO,
|
||||
- **Auth** — **PocketID** (`id.nuclide.systems`, OIDC, SQLite) is the
|
||||
**universal SSO/IdP for the entire ecosystem** — effectively every service
|
||||
authenticates via PocketID OIDC (LiteLLM, LobeChat, MCP gateway, Nextcloud,
|
||||
Daytona via the Keycloak→PocketID OIDC adapter, etc.). Single sign-on
|
||||
everywhere; one identity source to secure/audit.
|
||||
Coder, Gitea, Vaultwarden, etc.). Single sign-on everywhere; one identity
|
||||
source to secure/audit. Vaultwarden remains the lone gap as of 2026-05-20.
|
||||
|
||||
## Docker networks
|
||||
`ai-internal` (AI/MCP plane) · `shared_backend` (cross-stack DB/S3) ·
|
||||
plus per-stack: `arr-stack_default`, `daytona-minimal_daytona-network`,
|
||||
plus per-stack: `arr-stack_default`,
|
||||
`immich_default`, `karakeep_default`, `homepage_default`, `vpn_default`.
|
||||
The MCP gateway bridges `ai-internal` + `shared_backend`.
|
||||
|
||||
@@ -176,12 +180,15 @@ The MCP gateway bridges `ai-internal` + `shared_backend`.
|
||||
`mcp-gateway/config.json` (servers), `agents.json` + `prompts/` (agents).
|
||||
- **MCP servers** (gateway-managed): time, home-assistant (exact_upstream to
|
||||
`.60` ha-mcp), kroki (self-hosted: kroki+mermaid+excalidraw companions),
|
||||
daytona, fetch, sequential-thinking, ntfy, memos, docling, markitdown,
|
||||
crawl4ai, context7, wikipedia, papersearch, comfyui, bluesky (brianellin,
|
||||
rate-limit pending). Groups incl. `shepard-mcp`.
|
||||
- **Daytona** (`daytona.nuclide.systems`, daytona-minimal stack) — secure
|
||||
code sandboxes (Agent Operator deep-dives, shepard). Registry fixed to
|
||||
internal+host `:5000` (was misconfigured at 6000). Snapshot `sciviz-py`.
|
||||
**coder** (replaced daytona 2026-05-20), fetch, sequential-thinking, ntfy, memos,
|
||||
docling, markitdown, crawl4ai, context7, wikipedia, papersearch, comfyui,
|
||||
bluesky (brianellin, rate-limit pending). Groups incl. `shepard-mcp`.
|
||||
Gaps to close (community MCPs exist): gitea, paperless, karakeep, vaultwarden,
|
||||
proxmox-ve, audiobookshelf. See [`services/mcp-gateway.md`](mcp-gateway.md).
|
||||
- **Coder** (`dev.nuclide.systems`, CT 111) — replaced Daytona as the sandbox /
|
||||
dev-environment runtime. Templates: `python-uv` (persistent, GPU passthrough,
|
||||
baked LiteLLM env + Claude Code), `mcp-sandbox` (ephemeral, sci stack
|
||||
pre-baked). See [`services/dev-environment.md`](dev-environment.md).
|
||||
- **LobeChat** (`chat.nuclide.systems`, :14001) — chat UI; MCP plugins +
|
||||
agents in `lobechat` DB (lobe-postgres). Agents scriptable via
|
||||
`scripts/lobe_agent.py`. Native QStash scheduler NOT functional here →
|
||||
@@ -199,7 +206,7 @@ The MCP gateway bridges `ai-internal` + `shared_backend`.
|
||||
## Data layer
|
||||
- **shared-postgres** (pg16, `shared_backend`, data on local ZFS) — the
|
||||
**primary/standard DB for all deployments**. Tenants: LiteLLM, paperless,
|
||||
daytona, **memos** (migrated 2026-05-19). Superuser
|
||||
**memos** (migrated 2026-05-19), n8n (added 2026-05-19). Superuser
|
||||
`postgres` via unix socket; per-app dedicated role+db (role owns its db,
|
||||
password in the app stack's `.env`). Tuned 2026-05-19 (3G limit,
|
||||
shared_buffers 768M, max_conn 200).
|
||||
|
||||
+5
-60
@@ -1,5 +1,8 @@
|
||||
# Zoraxy Reverse Proxy Setup Guide
|
||||
|
||||
> **Zoraxy is a Go-based reverse proxy + ACME daemon, running as a systemd service in LXC 108. It is NOT OpenWRT. Earlier docs referenced \`uci\` commands — those are removed; configuration is via the admin UI or by editing JSON config files in \`/opt/zoraxy/conf/proxy/\` and restarting \`systemctl restart zoraxy\`.**
|
||||
|
||||
|
||||
## Overview
|
||||
Configure reverse proxies for AI service endpoints with automatic SSL certificates.
|
||||
|
||||
@@ -12,7 +15,7 @@ Configure reverse proxies for AI service endpoints with automatic SSL certificat
|
||||
| MCP Gateway | `mcp.nuclide.systems` | `192.168.1.40` | `8080` |
|
||||
| Garage S3 | `s3.nuclide.systems` | `192.168.1.40` | `10004` |
|
||||
|
||||
## Option A: Web UI (Easiest - No SSH Required)
|
||||
## Method A — Zoraxy admin UI (recommended)
|
||||
|
||||
### Step 1: Access Zoraxy Panel
|
||||
```
|
||||
@@ -74,65 +77,7 @@ Click **"Save"** at the bottom of the page.
|
||||
- Wait 1-2 minutes for certificates to be issued
|
||||
- Visual indicator: Green padlock icon in the UI
|
||||
|
||||
## Option B: SSH/UCI (Automated - Requires Root Access)
|
||||
|
||||
### Prerequisites
|
||||
- SSH access to Zoraxy with root credentials
|
||||
- Command-line access to Zoraxy's UCI config system
|
||||
|
||||
### Step 1: SSH into Zoraxy
|
||||
```bash
|
||||
ssh root@192.168.1.4
|
||||
```
|
||||
|
||||
### Step 2: Configure via UCI
|
||||
Run this command directly on Zoraxy:
|
||||
|
||||
```bash
|
||||
# Apply all rewrite rules
|
||||
uci set network.rewrite_litellm.enable='1'
|
||||
uci set network.rewrite_litellm.domain='litellm.nuclide.systems'
|
||||
uci set network.rewrite_litellm.target_type='0'
|
||||
uci set network.rewrite_litellm.target='192.168.1.40:14000'
|
||||
uci set network.rewrite_litellm.ssl='1'
|
||||
uci set network.rewrite_litellm.hsts='1'
|
||||
|
||||
uci set network.rewrite_lobehub.enable='1'
|
||||
uci set network.rewrite_lobehub.domain='chat.nuclide.systems'
|
||||
uci set network.rewrite_lobehub.target_type='0'
|
||||
uci set network.rewrite_lobehub.target='192.168.1.40:14001'
|
||||
uci set network.rewrite_lobehub.ssl='1'
|
||||
uci set network.rewrite_lobehub.hsts='1'
|
||||
|
||||
uci set network.rewrite_mcp.enable='1'
|
||||
uci set network.rewrite_mcp.domain='mcp.nuclide.systems'
|
||||
uci set network.rewrite_mcp.target_type='0'
|
||||
uci set network.rewrite_mcp.target='192.168.1.40:8080'
|
||||
uci set network.rewrite_mcp.ssl='1'
|
||||
uci set network.rewrite_mcp.hsts='1'
|
||||
|
||||
uci set network.rewrite_garage.enable='1'
|
||||
uci set network.rewrite_garage.domain='s3.nuclide.systems'
|
||||
uci set network.rewrite_garage.target_type='0'
|
||||
uci set network.rewrite_garage.target='192.168.1.40:10004'
|
||||
uci set network.rewrite_garage.ssl='1'
|
||||
uci set network.rewrite_garage.hsts='1'
|
||||
|
||||
# Commit and restart
|
||||
uci commit network
|
||||
/etc/init.d/network restart
|
||||
```
|
||||
|
||||
### Step 3: Verify
|
||||
```bash
|
||||
# Check if rewrite rules are loaded
|
||||
uci show network.rewrite_*
|
||||
|
||||
# Test connectivity
|
||||
curl -I http://litellm.nuclide.systems
|
||||
```
|
||||
|
||||
## Option C: Automated Script (Local)
|
||||
## Method B — automation via scripts/zoraxy_sync.py
|
||||
|
||||
Run from the control node:
|
||||
|
||||
|
||||
+2
-2
@@ -31,14 +31,14 @@ Do not start these without checking with the operator first.
|
||||
|
||||
## MCP gateway leaves, MCP child containers stay (planned)
|
||||
|
||||
When CT 109 "observe" lands, **`/opt/stacks/ai/mcp-gateway/` moves to CT 109** (it's a control plane: OIDC, agent scheduling, token store, usage stats). The ~20 MCP child server containers in this CT (`mcp-searxng`, `mcp-gotify`, `mcp-immich`, `mcp-fetch`, `mcp-time`, `comfyui-mcp`, `daytona-mcp`, `kroki-mcp`, etc.) **stay here on CT 104**.
|
||||
When CT 109 "observe" lands, **`/opt/stacks/ai/mcp-gateway/` moves to CT 109** (it's a control plane: OIDC, agent scheduling, token store, usage stats). The ~20 MCP child server containers in this CT (`mcp-searxng`, `mcp-gotify`, `mcp-immich`, `mcp-fetch`, `mcp-time`, `comfyui-mcp`, `coder-mcp`, `kroki-mcp`, etc.) **stay here on CT 104**.
|
||||
|
||||
The migrated gateway will reach this CT's docker daemon over the planned docker-socket-proxy (see `/docs/proxmox-optimizations.md` §16). Until CT 109 exists, the gateway runs locally and uses the bind-mounted socket. Do not piecemeal-move the gateway; it's tied to the CT 109 build.
|
||||
|
||||
## What is safe to do here
|
||||
|
||||
- Anything to do with the actively-running ~65 containers (Immich, n8n, Karakeep,
|
||||
ComfyUI, Daytona, Vaultwarden, Paperless, LobeChat, LiteLLM, MCP servers, etc.).
|
||||
ComfyUI, Vaultwarden, Paperless, LobeChat, LiteLLM, MCP servers, etc.). Daytona was decommissioned 2026-05-20 — compose files renamed `.DECOMMISSIONED-2026-05-20`; do not restart.
|
||||
- Image pulls + rebuilds for the active stacks (mind the n8n version pin — see its
|
||||
`docker-compose.yaml` comment).
|
||||
- Rotate exposed secrets in `.env` files (compose YAMLs themselves are git-tracked).
|
||||
|
||||
+35
-12
@@ -40,8 +40,9 @@
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [11000](http://192.168.1.40:11000) | Pocket ID | `pocket-id` | [id.nuclide.systems](https://id.nuclide.systems) | OIDC provider |
|
||||
| [11001](http://192.168.1.40:11001) | Vaultwarden | `vaultwarden` | [vault.nuclide.systems](https://vault.nuclide.systems) | Password manager |
|
||||
| [11001](http://192.168.1.40:11001) | Vaultwarden | `vaultwarden` | [vault.nuclide.systems](https://vault.nuclide.systems) | Password manager · **no Pocket-ID client yet — gap** |
|
||||
|
||||
> Pocket-ID **migrated from this CT to LXC 110** on 2026-05-20. See the [External Services](#external-services-not-on-nuc-docker) table below.
|
||||
|
||||
---
|
||||
|
||||
@@ -115,8 +116,8 @@ All arr-stack services run behind `vpn_gluetun` container network.
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [18000](http://192.168.1.40:18000) | Daytona API | `daytona-minimal-api-1` | [daytona.nuclide.systems](https://daytona.nuclide.systems) | Dev sandbox |
|
||||
| [18001](http://192.168.1.40:18001) | Daytona SSH | `daytona-minimal-ssh-gateway-1` | — | SSH gateway |
|
||||
<!-- Daytona DECOMMISSIONED 2026-05-20; replaced by Coder on CT 111. Compose renamed *.DECOMMISSIONED-2026-05-20. -->
|
||||
|
||||
|
||||
---
|
||||
|
||||
@@ -134,17 +135,38 @@ _Traccar moved to 15000–15001 (Documents range). 19000–19001 now free._
|
||||
|
||||
---
|
||||
|
||||
## LXC 111 — Dev (`192.168.1.42`)
|
||||
|
||||
CT 111 hosts the self-hosted dev platform. Same Pocket-ID SSO as the rest.
|
||||
|
||||
| Port | Service | Container | Public URL | Notes |
|
||||
|---|---|---|---|---|
|
||||
| [7080](http://192.168.1.42:7080) | Coder | `coder` | [dev.nuclide.systems](https://dev.nuclide.systems) | Workspace orchestrator; OIDC via Pocket-ID; password auth disabled |
|
||||
| [3000](http://192.168.1.42:3000) | Gitea | `gitea` | [git.nuclide.systems](https://git.nuclide.systems) | Self-hosted Git; OIDC; password form disabled |
|
||||
| [222](http://192.168.1.42:222) | Gitea SSH | `gitea` | — | `ssh -p 222 git@git.nuclide.systems` |
|
||||
| 5432 (internal) | coder-db | `coder-db` | — | Postgres 16 for Coder |
|
||||
| 5432 (internal) | gitea-db | `gitea-db` | — | Postgres 16 for Gitea |
|
||||
| [13080](http://192.168.1.42:13080) | docs site | `docs-server` | — (LAN-only) | mkdocs Material; auto-rebuilds from `fkrebs/docs` every 5 min |
|
||||
| (no port) | coder-fkrebs-dev | (workspace) | (Coder app proxy) | Active Coder workspace |
|
||||
| (no port) | act-runner | `act-runner` | — | Gitea Actions runner (`ct111-runner`) |
|
||||
|
||||
---
|
||||
|
||||
## External Services (Not on NUC Docker)
|
||||
|
||||
Zoraxy routes to these external backends:
|
||||
|
||||
| Domain | Target | Host | Service |
|
||||
|---|---|---|---|
|
||||
| [ha.nuclide.systems](https://ha.nuclide.systems) | `192.168.1.60:8123` | Home Assistant VM | Home automation |
|
||||
| [nc.nuclide.systems](https://nc.nuclide.systems) | `192.168.1.41:11000` | Nextcloud LXC | Cloud storage |
|
||||
| [ocpp.nuclide.systems](https://ocpp.nuclide.systems) | `192.168.1.60:8887` | Home Assistant VM | EV charger OCPP |
|
||||
| [shepard.nuclide.systems](https://shepard.nuclide.systems) | `192.168.1.49:80` | Docker host | Shepard |
|
||||
| [shepard-api.nuclide.systems](https://shepard-api.nuclide.systems) | `192.168.1.49:8080` | Docker host | Shepard API |
|
||||
| [ha.nuclide.systems](https://ha.nuclide.systems) | `192.168.1.60:8123` | Home Assistant VM 100 | Home automation |
|
||||
| [nc.nuclide.systems](https://nc.nuclide.systems) | `192.168.1.41:11000` | Nextcloud LXC 105 | Cloud storage |
|
||||
| [ocpp.nuclide.systems](https://ocpp.nuclide.systems) | `192.168.1.60:8887` | Home Assistant VM 100 | EV charger OCPP |
|
||||
| [shepard.nuclide.systems](https://shepard.nuclide.systems) | `192.168.1.49:80` | Shepard LXC 101 | Shepard |
|
||||
| [shepard-api.nuclide.systems](https://shepard-api.nuclide.systems) | `192.168.1.49:8080` | Shepard LXC 101 | Shepard API |
|
||||
| [id.nuclide.systems](https://id.nuclide.systems) | `192.168.1.5:11000` | Pocket-ID LXC 110 | OIDC IdP (migrated 2026-05-20) |
|
||||
| [git.nuclide.systems](https://git.nuclide.systems) | `192.168.1.42:3000` | Dev LXC 111 | Gitea (self-hosted Git) |
|
||||
| [dev.nuclide.systems](https://dev.nuclide.systems) | `192.168.1.42:7080` | Dev LXC 111 | Coder (workspace orchestrator) |
|
||||
| Gitea SSH | `192.168.1.42:222` | Dev LXC 111 | `ssh -p 222 git@git.nuclide.systems` |
|
||||
|
||||
---
|
||||
|
||||
@@ -157,7 +179,6 @@ Zoraxy routes to these external backends:
|
||||
| [mcp.nuclide.systems](https://mcp.nuclide.systems) | mcp-gateway | 8080 |
|
||||
| [arcane.nuclide.systems](https://arcane.nuclide.systems) | arcane | 10002 |
|
||||
| [gotify.nuclide.systems](https://gotify.nuclide.systems) | gotify | 10003 |
|
||||
| [id.nuclide.systems](https://id.nuclide.systems) | pocket-id | 11000 |
|
||||
| [vault.nuclide.systems](https://vault.nuclide.systems) | vaultwarden | 11001 |
|
||||
| [immich.nuclide.systems](https://immich.nuclide.systems) | immich_server | 12000 |
|
||||
| [immich-tools.nuclide.systems](https://immich-tools.nuclide.systems) | immich_power_tools | (container-internal) |
|
||||
@@ -165,7 +186,6 @@ Zoraxy routes to these external backends:
|
||||
| [n8n.nuclide.systems](https://n8n.nuclide.systems) | n8n | 16000 |
|
||||
| [memos.nuclide.systems](https://memos.nuclide.systems) | memos | 17000 |
|
||||
| [hoarder.nuclide.systems](https://hoarder.nuclide.systems) | karakeep | 17001 |
|
||||
| [daytona.nuclide.systems](https://daytona.nuclide.systems) | daytona-api | 18000 |
|
||||
| [traccar.nuclide.systems](https://traccar.nuclide.systems) | traccar | 15000 |
|
||||
| [dozzle.nuclide.systems](https://dozzle.nuclide.systems) | dozzle | 10001 |
|
||||
|
||||
@@ -192,7 +212,10 @@ Zoraxy routes to these external backends:
|
||||
| `62bf4e0d` | memos | `https://memos.nuclide.systems/auth/callback` |
|
||||
| `d92f82b0` | karakeep | `https://hoarder.nuclide.systems/api/auth/callback/custom` |
|
||||
| `a14b8076` | nextcloud | `https://nc.nuclide.systems/apps/user_oidc/code` |
|
||||
| `798a367f` | daytona | `https://daytona.nuclide.systems` |
|
||||
| `0aee4280` | Coder | `https://dev.nuclide.systems/api/v2/users/oidc/callback` |
|
||||
| `9444609e` | Gitea | `https://git.nuclide.systems/user/oauth2/pocket-id/callback` |
|
||||
| `38469e7e` | Proxmox VE | `https://192.168.1.20:8006` |
|
||||
| `798a367f` | ~~daytona~~ | **ORPHAN — Daytona decommissioned 2026-05-20, remove** |
|
||||
| `cbbf20d5` | Audiobookshelf | `https://abs.nuclide.systems/…` + `*` |
|
||||
| `d8733fcc` | shelfarr | `*` |
|
||||
| `78c78998` | Claude MCP | `https://claude.ai/api/mcp/auth_callback` + `https://mcp.nuclide.systems/mcp/auth/callback` |
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
> **STATUS: STALE — many claims (PVE version, Pocket-ID port, Dockge as docker manager) no longer accurate. Source of truth is /CLAUDE.md and /docs/services/. This file kept for the original nexa-stack design notes only.**
|
||||
|
||||
---
|
||||
|
||||
# Instructions for Claude (and other agents)
|
||||
|
||||
This file tells future automated runs what they need to know about this repo.
|
||||
|
||||
+70
-383
@@ -1,409 +1,96 @@
|
||||
# Storage Layout — NUC 14 Docker Stacks
|
||||
# Storage layout
|
||||
|
||||
## Data Location Architecture
|
||||
> Verified state as of 2026-05-21. Source of truth for which service lives on which storage class.
|
||||
> Earlier revisions of this doc were stale on multiple items (Garage, n8n, Arcane, Karakeep, Pocket-ID, arr-stack, Nextcloud's storage protocol, missing Coder/Gitea). Reconciled via live audit.
|
||||
|
||||
All persistent service data is being consolidated to `/mnt/pve/unas/services/{service}/`
|
||||
on the UNAS NFS share, with media content in `/mnt/pve/unas/media/`.
|
||||
## Storage classes
|
||||
|
||||
**PostgreSQL databases stay on local NVMe** — NFS ownership checks (`stat(2)` UID match) prevent
|
||||
PostgreSQL from running on network filesystems. PostgreSQL is backed up via `pg_dump` to UNAS instead.
|
||||
|
||||
| Mount | Source | Content |
|
||||
|---|---|---|
|
||||
| `/mnt/pve/unas/services/` | UNAS NFS | SQLite DBs, configs, files, S3 data |
|
||||
| `/mnt/pve/unas/media/` | UNAS NFS | User media (photos, docs, books, audio) |
|
||||
| `/mnt/pve/unas/backup/` | UNAS NFS | Service backups |
|
||||
| `/var/lib/docker/volumes/` | Local NVMe | PostgreSQL data, Redis data |
|
||||
| `/opt/stacks/` | Local NVMe | Compose files, small configs |
|
||||
|
||||
## Migration Status
|
||||
|
||||
### ✅ Already on UNAS — `/mnt/pve/unas/services/`
|
||||
|
||||
| Service | UNAS Path(s) | Container Mount | Data Type |
|
||||
| Class | Substrate | Where | Typical use |
|
||||
|---|---|---|---|
|
||||
| gluetun | `gluetun/data/` | `/gluetun` | VPN config |
|
||||
| immich | `immich/{encoded-video,profile,thumbs}/` | `/usr/src/app/upload/{encoded-video,profile,thumbs}` | App derived data |
|
||||
| karakeep | `karakeep/data/`, `karakeep/meilisearch/` | `/data`, `/meili_data` | Bookmarks DB + Meilisearch index |
|
||||
| ntfy | `ntfy/{config,cache}/` | `/etc/ntfy`, `/var/lib/ntfy` | Notification config + cache |
|
||||
| paperless-ai | `paperless-ai/` | `/app/data` | AI processing data |
|
||||
| pocketid | `pocketid/` | `/app/data` | Auth data (SQLite) |
|
||||
| stremio | `stremio/` | `/root/.stremio-server` | Streaming server data |
|
||||
| traccar | `traccar/{data,logs,traccar.xml}` | `/opt/traccar/{data,logs,conf/traccar.xml}` | GPS tracking data |
|
||||
| vaultwarden | `vaultwarden/` | `/data` | Password vault (SQLite) |
|
||||
| garage | `shared-db/garage/{data,meta}/` | `/var/lib/garage/{data,meta}` | S3 object storage (Garage) |
|
||||
| Local zfs (rootfs) | NVMe in nuc | `/opt/stacks/<stack>/` on each LXC | Tier-1 hot state: databases, SQLite, secret stores, anything latency-sensitive |
|
||||
| Local zfs (subvol) | NVMe in nuc | `rpool/data/subvol-<NNN>-disk-0` per LXC | Container rootfs, image cache |
|
||||
| Docker named volume | NVMe in nuc | `/var/lib/docker/volumes/...` on each Docker CT | Per-container persistent state managed by Docker |
|
||||
| UNAS over **NFSv3** | `192.168.1.31` | `/mnt/pve/unas` on CTs 101, 103, 104, 111 | Bulk media, workspace home dirs, document storage |
|
||||
| UNAS over **CIFS/SMB 3.1.1** | `192.168.1.31` | `/mnt/pve/unas` on CT 105 (Nextcloud) | Nextcloud-only path — earlier docs mis-labelled this as NFS |
|
||||
|
||||
### 🆕 Shared Infrastructure — `/opt/stacks/shared-db/`
|
||||
## What lives where (verified 2026-05-21)
|
||||
|
||||
| Service | Hostname | Type | Data Location |
|
||||
|---|---|---|---|
|
||||
| shared-postgres | `shared-postgres` | PostgreSQL 16 (local volume) | `shared-db_shared-pgdata` volume |
|
||||
| garage | `garage` | S3-compatible object storage (on UNAS) | `/mnt/pve/unas/services/shared-db/garage/` |
|
||||
### UNAS NFS — `/mnt/pve/unas/services/...`
|
||||
|
||||
### ✅ Migrated (Phase 1) — SQLite to UNAS
|
||||
Bulk + media + non-latency-sensitive app data.
|
||||
|
||||
| Service | Old Path | New Path | Status |
|
||||
|---|---|---|---|
|
||||
| memos | `/opt/stacks/memos/data` | `/mnt/pve/unas/services/memos` | ✅ Healthy |
|
||||
| arcane | volume `arcane_arcane-data` | `/mnt/pve/unas/services/arcane` | ✅ Running |
|
||||
| n8n | volume `n8n_n8n_storage` | `/mnt/pve/unas/services/n8n` | ✅ Healthy |
|
||||
|
||||
### ⏳ Still Local — `/opt/stacks/` or Docker Volumes
|
||||
|
||||
| Service | Current Location | Data Type | Estimated Size | UNAS Target |
|
||||
| Service | CT | Path on UNAS → in container | Backed up? | Notes |
|
||||
|---|---|---|---|---|
|
||||
| arr-stack | `/opt/stacks/arr-stack/` | Configs + media | ~55 GB | `arr-stack/{config,audiobooks,ebooks,podcasts,torrents}` |
|
||||
| qdrant | `/opt/stacks/qdrant/qdrant_storage` | Vector DB | ~7 MB | `qdrant/` |
|
||||
| dozzle | `/opt/stacks/dozzle/dozzle_data` | Log viewer state | tiny | *(ephemeral, optional)* |
|
||||
| homepage | `/opt/stacks/homepage/{config,icons}` | Dashboard config | ~164 KB | *(keep in stack dir)* |
|
||||
| Immich (uploads, thumbs, derived) | 104 | `services/immich/{encoded-video,profile,thumbs}` + `media/images` + `backup/immich` | Immich own pg_dump → `media/images/db-dumps`; no off-host copy | |
|
||||
| Paperless-ngx (documents) | 104 | `media/documents/public/paperless-ngx/{consume,export,library}` | none | |
|
||||
| Paperless-AI | 104 | `services/paperless-ai` → `/app/data` | none | |
|
||||
| Vaultwarden (attachments + SQLite DB) | 104 | `services/vaultwarden` → `/data` | none | tier-1 DB on NFS — risky, plan to move local |
|
||||
| Traccar (logs, config) | 104 | `services/traccar/{logs,traccar.xml}` | none | data dir reverted to local `/opt/stacks/traccar/data` |
|
||||
| Memos | 104 | `services/memos` → `/var/opt/memos` | none | |
|
||||
| Arr-stack (Audiobookshelf, Prowlarr, RDTClient, Shelfarr) + media | 104 | `services/arr-stack/*` + `media/{audiobooks,ebooks,podcasts,Torrents}` | none | already migrated (older doc claimed "still local") |
|
||||
| Gluetun (VPN) | 104 | `services/gluetun/data` → `/gluetun` | none | |
|
||||
| Gitea | 111 | `services/gitea` → `/data` | none | new 2026-05-20 |
|
||||
| Coder workspace home dirs | 111 | `services/coder/<user>/<workspace>` → `/home/<user>` | none | new 2026-05-20 |
|
||||
| Backrest's jottacloud-mirrored data | 103 | `media/data-dir` only | **rclone → jottacloud (Backrest)** | only path with off-host backup |
|
||||
|
||||
### 🗄️ Databases Requiring Migration
|
||||
### Local zfs — `/opt/stacks/...`
|
||||
|
||||
| Service | Engine | Current Path | Target | Migration Method | Status |
|
||||
|---|---|---|---|---|---|---|
|
||||
| immich | PostgreSQL | `/opt/stacks/immich/postgres` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| litellm | PostgreSQL | `/opt/stacks/ai/postgres_data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| lobehub | PostgreSQL | `/opt/stacks/ai/lobehub/data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| daytona | PostgreSQL | volume `daytona-minimal_db_data` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| paperless-ngx | PostgreSQL | volume `paperless-ngx_pgdata` | → shared-postgres | `pg_dump` → `pg_restore` | ⏳ Pending |
|
||||
| memos | SQLite | `/opt/stacks/memos/data` | `/mnt/pve/unas/services/memos` | `cp` + `chown` | ✅ Done |
|
||||
| arcane | SQLite | volume `arcane_arcane-data` | `/mnt/pve/unas/services/arcane` | `cp` + `chown` | ✅ Done |
|
||||
| n8n | SQLite | volume `n8n_n8n_storage` | `/mnt/pve/unas/services/n8n` | `cp` + `chown` | ✅ Done |
|
||||
| pocketid | SQLite | `pocketid/` | *(already on UNAS)* | — | ✅ Done |
|
||||
| vaultwarden | SQLite | `vaultwarden/` | *(already on UNAS)* | — | ✅ Done |
|
||||
| karakeep | SQLite + Meilisearch | `karakeep/{data,meilisearch}` | *(already on UNAS)* | — | ✅ Done |
|
||||
Tier-1 state and anything that should NOT be NFS-backed.
|
||||
|
||||
### 🧠 ML Models & Caches (Regenerable — Can Stay Local)
|
||||
|
||||
| Resource | Path | Decision |
|
||||
|---|---|---|
|
||||
| Immich ML models | volume `immich_model-cache` | Cache — stay |
|
||||
| Paperless Redis | volume `paperless-ngx_redisdata` | Ephemeral — stay |
|
||||
| AI stack Redis | volume `nuc-ai-core_redis_data` | Ephemeral — stay |
|
||||
| SearXNG search cache | volume (hash) at `/var/lib/docker/volumes/bb5bb...` | Ephemeral — stay |
|
||||
| Daytona runner Docker | volume `daytona-minimal` runner data | Can stay |
|
||||
|
||||
## Media Content (Already on UNAS)
|
||||
|
||||
| Mount | Content |
|
||||
|---|---|
|
||||
| `/mnt/pve/unas/media/images/` | Immich photo library |
|
||||
| `/mnt/pve/unas/media/documents/public/paperless-ngx/` | Paperless documents (consume, export, library) |
|
||||
| `/mnt/pve/unas/media/audiobooks/` | Audiobookshelf media |
|
||||
| `/mnt/pve/unas/media/ebooks/` | Audiobookshelf ebooks |
|
||||
| `/mnt/pve/unas/media/podcasts/` | Audiobookshelf podcasts |
|
||||
| `/mnt/pve/unas/backup/immich/` | Immich DB backups |
|
||||
|
||||
## Prospective UNAS Layout
|
||||
|
||||
```
|
||||
/mnt/pve/unas/
|
||||
├── services/
|
||||
│ ├── gluetun/
|
||||
│ │ └── data/ ← VPN config
|
||||
│ ├── immich/
|
||||
│ │ ├── db/ ← PostgreSQL (to migrate)
|
||||
│ │ ├── encoded-video/ ← Already migrated
|
||||
│ │ ├── profile/ ← Already migrated
|
||||
│ │ └── thumbs/ ← Already migrated
|
||||
│ ├── karakeep/
|
||||
│ │ ├── data/ ← Already migrated
|
||||
│ │ └── meilisearch/ ← Already migrated
|
||||
│ ├── ntfy/
|
||||
│ │ ├── cache/ ← Already migrated
|
||||
│ │ └── config/ ← Already migrated
|
||||
│ ├── paperless-ai/ ← Already migrated
|
||||
│ ├── paperless-ngx/
|
||||
│ │ └── pgdata/ ← PostgreSQL (to migrate)
|
||||
│ ├── pocketid/ ← Already migrated
|
||||
│ ├── stremio/ ← Already migrated
|
||||
│ ├── traccar/
|
||||
│ │ ├── data/ ← Already migrated
|
||||
│ │ ├── logs/ ← Already migrated
|
||||
│ │ └── traccar.xml ← Already migrated
|
||||
│ ├── vaultwarden/ ← Already migrated
|
||||
│ ├── memos/ ← SQLite ✅ Done
|
||||
│ ├── qdrant/ ← Vector DB (to migrate)
|
||||
│ ├── arcane/ ← SQLite ✅ Done
|
||||
│ ├── n8n/ ← SQLite ✅ Done
|
||||
│ ├── shared-db/
|
||||
│ │ ├── postgres/ ← ⚠️ Local volume, NOT on UNAS
|
||||
│ │ └── garage/
|
||||
│ │ ├── data/ ← S3 data (on UNAS)
|
||||
│ │ └── meta/ ← S3 metadata (on UNAS)
|
||||
│ ├── arr-stack/
|
||||
│ │ ├── config/ ← Bind mounts (to migrate)
|
||||
│ │ ├── audiobooks/ ← Media (to migrate)
|
||||
│ │ ├── ebooks/ ← Media (to migrate)
|
||||
│ │ ├── podcasts/ ← Media (to migrate)
|
||||
│ │ └── torrents/ ← Downloads (to migrate)
|
||||
│
|
||||
├── media/
|
||||
│ ├── images/ ← Immich photo library
|
||||
│ └── documents/
|
||||
│ └── public/
|
||||
│ └── paperless-ngx/
|
||||
│ ├── consume/
|
||||
│ ├── export/
|
||||
│ └── library/
|
||||
│
|
||||
├── backups/
|
||||
│ └── immich/ ← Immich DB backups
|
||||
│
|
||||
└── (potential future: torrents/, cache/)
|
||||
```
|
||||
|
||||
## Garage S3 — `/opt/stacks/shared-db/`
|
||||
|
||||
S3-compatible object storage using [Garage](https://garagehq.deuxfleurs.fr/) v1.0.0.
|
||||
Replaces RustFS from the LobeHub AI stack. Single-node, replication_factor=1.
|
||||
|
||||
| Detail | Value |
|
||||
|---|---|
|
||||
| Container | `garage` (dxflrs/garage:v1.0.0) |
|
||||
| Data | `/mnt/pve/unas/services/shared-db/garage/{data,meta}` |
|
||||
| S3 API | `garage:3900` (internal, shared_backend) |
|
||||
| Admin API | `127.0.0.1:3901` (localhost only) |
|
||||
| Regional | No auth (internal network) |
|
||||
| Bucket | `lobe-files` |
|
||||
| Key ID | `GK55210bd6b56d92891db26fac` |
|
||||
| Secret Key | `d0af9e4eb69d7773ffa17950feee27925dbc70eb3efd4ee23a3e1324672c869a` |
|
||||
|
||||
## Shared PostgreSQL — `/opt/stacks/shared-db/`
|
||||
|
||||
Centralized PostgreSQL 16 instance for all non-immich services.
|
||||
|
||||
| Detail | Value |
|
||||
|---|---|
|
||||
| Container | `shared-postgres` (postgres:16) |
|
||||
| Data | `shared-db_shared-pgdata` (local Docker volume, NOT NFS) |
|
||||
| Networks | `shared_backend` |
|
||||
| Auth | `postgres` superuser with password from `.env` |
|
||||
|
||||
### Databases & Users (pre-created on init)
|
||||
|
||||
| Database | Owner | Password | Used By |
|
||||
|---|---|---|---|
|
||||
| `paperless` | `paperless` | `paperless` | paperless-ngx |
|
||||
| `daytona` | `user` | `pass` | daytona |
|
||||
| `litellm` | `litellm` | `litellm_password_here` | LiteLLM |
|
||||
| `lobechat` | `lobe` | `uWNZugjBqixf8dxC` | LobeHub |
|
||||
| `paradedb` | `lobe` | `uWNZugjBqixf8dxC` | LobeHub |
|
||||
|
||||
## Key Architectural Decision: PostgreSQL Stays Local
|
||||
|
||||
**Problem:** PostgreSQL checks data directory ownership via `stat(2)` and refuses to start
|
||||
if the process UID doesn't match the file owner UID. NFS with `all_squash` maps all UIDs
|
||||
to `65534` (nobody), breaking this check. `chown` also fails on NFS.
|
||||
|
||||
**Solution:** All PostgreSQL instances use local Docker volumes on NVMe:
|
||||
|
||||
```
|
||||
Local NVMe (/var/lib/docker/volumes/) UNAS NFS (/mnt/pve/unas/)
|
||||
├── shared-postgres data ├── pg_dump backups ← scheduled
|
||||
├── immich_postgres data ├── SQLite DBs (memos, arcane, n8n)
|
||||
├── paperless-ngx pgdata ├── Garage S3 data
|
||||
├── daytona-minimal db_data ├── Media files (immich, paperless)
|
||||
├── litellm-db / ai postgres_data ├── Configs / service data
|
||||
├── lobehub postgres data
|
||||
└── all Redis / cache volumes
|
||||
```
|
||||
|
||||
**Backup strategy:** `pg_dump` each database to UNAS on a schedule (e.g. daily cron).
|
||||
|
||||
## Migration Priority
|
||||
|
||||
| Priority | Service | Effort | Risk | Status | Reason |
|
||||
|---|---|---|---|---|---|
|
||||
| P0 | memos | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | n8n | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | arcane | Minimal (SQLite copy) | Low | ✅ Done | Simple file move |
|
||||
| P0 | qdrant | Minimal (file copy) | Low | ✅ Done | Fresh start on UNAS |
|
||||
| P1 | daytona → shared-postgres | Medium (pg_dump) | Low | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | litellm → shared-postgres | Medium (pg_dump) | Low | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | lobehub → shared-postgres | Medium (pg_dump) | Low | ⛔ Skipped | Uses ParadeDB (pg17+extensions), incompatible with shared-postgres (pg16) |
|
||||
| P1 | paperless-ngx → shared-postgres | Medium (pg_dump) | Medium | ✅ Done | pg_dump → restore → running |
|
||||
| P1 | immich → shared-postgres | Medium (pg_dump) | Medium | ⏳ | Custom pgvector image |
|
||||
| P2 | arr-stack | Large (55 GB copy) | Low | ⏳ | Simple copy, time-consuming |
|
||||
|
||||
## Mail Archive (Potential Savings)
|
||||
|
||||
Unattached anonymous volumes from old/stopped containers:
|
||||
|
||||
| Volume | Size | Likely From | Action |
|
||||
|---|---|---|---|
|
||||
| `0b6d07ff...` | ? | old daytona runner | prune |
|
||||
| (100+ hex-named volumes) | ? | ephemeral build cache | prune |
|
||||
|
||||
Recommend: `docker system df` then `docker volume prune` to clean.
|
||||
|
||||
---
|
||||
|
||||
## Database Consolidation Plan
|
||||
|
||||
### Current: 10 Database Instances
|
||||
|
||||
| # | Container | Type | Image | Used By | Networks |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | `immich_postgres` | PostgreSQL | `immich-app/postgres:14-vectorchord` (pgvector) | immich | `immich_default` |
|
||||
| 2 | `paperless-ngx-db-1` | PostgreSQL | `postgres:16` | paperless-ngx | `paperless-ngx_default` |
|
||||
| 3 | `daytona-minimal-db-1` | PostgreSQL | `postgres:16-alpine` | daytona | `daytona-minimal_daytona-network` |
|
||||
| 4 | `nuc-ai-core-litellm-db-1` | PostgreSQL | `postgres:16-alpine` | litellm | `ai-internal` |
|
||||
| 5 | `lobe-postgres` | PostgreSQL | `postgres:16-alpine` | lobehub (lobechat, paradedb) | `ai-internal` |
|
||||
| 6 | `paperless-ngx-broker-1` | Redis 7 | `redis:7` | paperless broker | `paperless-ngx_default` |
|
||||
| 7 | `immich_redis` | Valkey 9 | `valkey/valkey:9` | immich cache | `immich_default` |
|
||||
| 8 | `daytona-minimal-redis-1` | Redis Alpine | `redis:alpine` | daytona | `daytona-minimal_daytona-network` |
|
||||
| 9 | `lobe-redis` | Redis 7 | `redis:7-alpine` | lobehub | `ai-internal` |
|
||||
| 10 | `redis-searxng` | Redis 7 | `redis:7-alpine` | searxng | `ai-internal` |
|
||||
|
||||
### Target: 6 Database Instances (remove 4)
|
||||
|
||||
| # | Container | Type | DBs Hosted | Saved |
|
||||
| Service | CT | Local path → in container | Backed up? | Notes |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `immich_postgres` | PostgreSQL immich-app (pgvector) | immich | 🔒 Must stay separate |
|
||||
| 2 | **`shared-postgres`** 🆕 | **PostgreSQL 16** | **paperless, daytona, litellm, lobechat, paradedb** | **−3 containers** |
|
||||
| 3 | `paperless-ngx-broker-1` | Redis 7 | paperless broker | 🔒 Must stay separate |
|
||||
| 4 | `immich_redis` | Valkey 9 | immich cache | 🔒 Must stay separate |
|
||||
| 5 | `daytona-minimal-redis-1` | Redis Alpine | daytona | 🔒 Stay separate |
|
||||
| 6 | `lobe-redis` | Redis 7 | lobehub | 🔒 Stay separate |
|
||||
| 7 | `redis-searxng` | Redis 7 | searxng | 🔒 Stay separate |
|
||||
| 7 | ~~`nuc-ai-core-litellm-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 8 | ~~`lobe-postgres`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 9 | ~~`paperless-ngx-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| 10 | ~~`daytona-minimal-db-1`~~ | 🗑️ removed | → shared-postgres | ✅ |
|
||||
| Pocket-ID | **110** | `/opt/stacks/pocketid/data` → `/app/data` | none | CT 110 has no NFS mount at all. Earlier doc said UNAS — incorrect. |
|
||||
| Garage (S3 meta + data) | 104 | `/opt/stacks/shared-db/garage/{data,meta}` → `/var/lib/garage/*` | none | **moved off NFS** 2026-05-19 after WAL-G outage. Stale copy may still exist on UNAS. |
|
||||
| Shared Postgres (multi-stack) | 104 | docker named volume `shared-db_shared-pgdata` | **WAL-G → local Garage** (configured; outage 2026-05-19 prompted move) | Garage itself is single-disk local — no off-host copy. |
|
||||
| n8n | 104 | `/opt/stacks/n8n/data` → `/home/node/.n8n` | none | **back to local** after PG migration attempt failed. Stale 408 MB `database.sqlite` left on UNAS (May 15) — clean up. n8n now uses `shared-postgres` as its actual DB. |
|
||||
| Arcane | 104 | `/opt/stacks/arcane/data` → `/app/data` | none | reverted from UNAS to local 2026-05-19 (undocumented before now) |
|
||||
| Karakeep + Meilisearch | 104 | `/opt/stacks/karakeep/localdata/{data,meili}` | none | reverted from UNAS to local |
|
||||
| Immich Postgres (pgvector) | 104 | `/opt/stacks/immich/postgres` → `/var/lib/postgresql/data` | Immich pg_dump → UNAS | tier-1; PG demands local disk |
|
||||
| Homepage | 104 | `/opt/stacks/homepage/{config,icons}` | none | |
|
||||
| Dozzle | 104 | `/opt/stacks/dozzle/dozzle_data` | none | |
|
||||
| LiteLLM config | 104 | `/opt/stacks/ai/litellm-config` | none | |
|
||||
| SearXNG config | 104 | `/opt/stacks/ai/searxng` | none | |
|
||||
| Flaresolverr | 104 | `/var/lib/flaresolver` | none | |
|
||||
| AdGuard / Zoraxy / DNS / Shepard / Backrest binaries | 102/108/103/101 | local zfs only | none | Backrest itself has no self-backup |
|
||||
| Nextcloud config + sidecars | 105 | local zfs (CT rootfs) | none | data path is CIFS — see below |
|
||||
|
||||
**Result: 10 → 6 instances, removing 4 containers, saving ~160MB RAM.**
|
||||
### Docker named volumes
|
||||
|
||||
### Why Redis Stays Separate
|
||||
Local on `/var/lib/docker/volumes/`. Mostly databases and caches.
|
||||
|
||||
| Reason | Detail |
|
||||
|---|---|
|
||||
| **Valkey ≠ Redis** | Immich uses Valkey 9 which has diverged from Redis 7 |
|
||||
| **`SELECT` collisions** | Apps may use specific DB numbers that clash |
|
||||
| **Password mismatch** | Each app has its own `requirepass` / `REDIS_PASSWORD` |
|
||||
| **Ephemeral = low value** | Caches/brokers rebuild on restart, savings don't justify risk |
|
||||
| **`FLUSHALL` risk** | One misconfigured app could wipe another's cache |
|
||||
|
||||
### Why PG Consolidation Works
|
||||
|
||||
| Reason | Detail |
|
||||
|---|---|
|
||||
| **Same major version** | All target DBs use PostgreSQL 16 |
|
||||
| **Native compatibility** | No conflicting extensions (only immich has pgvector) |
|
||||
| **Isolation via databases** | Each service gets its own database + user within shared PG |
|
||||
| **Simple restore** | `pg_dump` + `pg_restore` into shared instance |
|
||||
| **Single backup** | One `pg_dumpall` covers all 5 databases |
|
||||
|
||||
### Implementation: Shared PostgreSQL
|
||||
|
||||
Create `/opt/stacks/shared-db/docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
container_name: shared-postgres
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/pve/unas/services/shared-db:/var/lib/postgresql/data
|
||||
environment:
|
||||
POSTGRES_PASSWORD: ${SHARED_DB_PASSWORD}
|
||||
ports:
|
||||
- "127.0.0.1:5432:5432"
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: "2.0"
|
||||
memory: 1G
|
||||
reservations:
|
||||
cpus: "1.0"
|
||||
memory: 512M
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
networks:
|
||||
default:
|
||||
external: true
|
||||
name: shared_backend
|
||||
```
|
||||
|
||||
### Migration Order (Phase 2)
|
||||
|
||||
```
|
||||
Step 0: Create shared-postgres container
|
||||
Step 1: Dump ALL 4 databases from their current instances
|
||||
Step 2: Create users + databases on shared-postgres
|
||||
Step 3: Restore each dump into shared-postgres
|
||||
Step 4: One by one, update each app's compose to point to shared-postgres
|
||||
```
|
||||
|
||||
#### Detailed: Paperless-ngx
|
||||
|
||||
| Change | Before | After |
|
||||
| Volume | Service / CT | Backed up? |
|
||||
|---|---|---|
|
||||
| DB service | `db` (in paperless compose) | remove entirely |
|
||||
| `PAPERLESS_DBHOST` | `db` | `shared-postgres` |
|
||||
| DB credentials | defaults (paperless/paperless) | keep same |
|
||||
| Network | `paperless-ngx_default` | `shared_backend` |
|
||||
| `shared-db_shared-pgdata` | shared-postgres / 104 | WAL-G → local Garage |
|
||||
| `paperless-ngx_pgdata`, `_redisdata`, `_data` | paperless-ngx / 104 | none |
|
||||
| `lobe-postgres` + `lobe-redis` | lobehub / 104 | none |
|
||||
| `nuc-ai-core_rustfs-data` | lobehub stack / 104 | none |
|
||||
| `pgadmin-data` | pgadmin / 104 | none (regenerable) |
|
||||
| `immich_model-cache` | immich-ml / 104 | regenerable |
|
||||
| `coder-db` + `gitea-db` (docker volume `coder-db`) | / 111 | none |
|
||||
| (orphaned) `daytona-minimal_db_data` + runner anon vol | / 104 | none — clean up post-decommission |
|
||||
|
||||
#### Detailed: Daytona
|
||||
### Nextcloud (CT 105) — the CIFS outlier
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `db` (in daytona compose) | remove entirely |
|
||||
| `DB_HOST` | `db` | `shared-postgres` |
|
||||
| `DB_USERNAME` | `user` | keep same |
|
||||
| `DB_DATABASE` | `daytona` | keep same |
|
||||
| Network | `daytona-minimal_daytona-network` | `shared_backend` |
|
||||
Nextcloud AIO mounts `//192.168.1.31/storage` over **CIFS/SMB 3.1.1**, not NFS like every other CT. Application data path is under that mount. The Postgres + Redis sidecars stay on local docker volumes.
|
||||
|
||||
#### Detailed: LiteLLM
|
||||
## Backup reality
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `litellm-db` (in ai compose) | remove entirely |
|
||||
| `DATABASE_URL` | `postgresql://litellm:...@litellm-db:5432/litellm` | `postgresql://litellm:...@shared-postgres:5432/litellm` |
|
||||
| Network | `ai-internal` | `shared_backend` |
|
||||
**There is effectively no off-host backup of service data.** The only configured Backrest plan covers `/mnt/pve/unas/media/data-dir → jottacloud` — i.e., Backrest backs up *one specific UNAS path*, not the services that write to UNAS.
|
||||
|
||||
#### Detailed: LobeHub
|
||||
Coverage:
|
||||
|
||||
| Change | Before | After |
|
||||
|---|---|---|
|
||||
| DB service | `postgresql` (in lobehub.yml) | remove entirely |
|
||||
| `DATABASE_URL` | `postgresql://postgres:...@postgresql:5432/lobechat` | `postgresql://postgres:...@shared-postgres:5432/lobechat` |
|
||||
| Lobe also needs `paradedb` DB | created manually on initial setup | created via SQL script |
|
||||
| Network | `ai-internal` | `shared_backend` |
|
||||
- Immich pg_dump → UNAS (same filer; survives container loss, not filer loss)
|
||||
- shared-postgres WAL-G → Garage (same host; survives container loss, not disk loss)
|
||||
- Everything else: zero
|
||||
|
||||
### Total Impact
|
||||
This is a known gap. Plans:
|
||||
1. Extend Backrest plans to snapshot tier-1 paths (Pocket-ID sqlite, Vaultwarden data, Gitea repos, Coder workspace homes) to jottacloud nightly.
|
||||
2. Once the second NVMe lands (see `infra/proxmox-state.md` §6), mirror `rpool` so a single disk death doesn't take everything.
|
||||
|
||||
| Metric | Before | After |
|
||||
|---|---|---|
|
||||
| PostgreSQL instances | 5 | 2 |
|
||||
| Redis instances | 5 | 5 (unchanged) |
|
||||
| Total container count | 10 DBs | 7 DBs (−3) |
|
||||
| RAM saved (estimated) | — | ~160 MB |
|
||||
| Complexity | Fragmented | Centralized |
|
||||
| Backup burden | 5 pg_dumps | 2 pg_dumps |
|
||||
## Drift cleanup TODO
|
||||
|
||||
### Timing Estimates
|
||||
|
||||
| Service | Downtime | Cumulative |
|
||||
|---|---|---|
|
||||
| Setup shared-postgres + dump all | 0 (live dump) | 0 |
|
||||
| daytona → shared | ~2 min | 2 min |
|
||||
| litellm → shared | ~2 min | 4 min |
|
||||
| lobehub → shared | ~3 min | 7 min |
|
||||
| paperless-ngx → shared | ~5 min | 12 min |
|
||||
| immich DB → UNAS bind mount | ~10 min | 22 min |
|
||||
| **Total** | **~22 min** | |
|
||||
|
||||
### Notes
|
||||
|
||||
- All compose files are version-controlled in `/opt/stacks/{service}/docker-compose.yml`
|
||||
- Always test with `docker compose config` before `up -d`
|
||||
- PostgreSQL stays on local NVMe — backed up via `pg_dump` to UNAS
|
||||
- Garage S3 replaces RustFS from LobeHub stack (RustFS will be removed when LobeHub is switched)
|
||||
- LobeHub's `paradedb` extension requires `CREATE EXTENSION IF NOT EXISTS paradedb;` on the shared instance
|
||||
- All PostgreSQL dbs consolidated into shared-postgres except immich (pgvector requirement)
|
||||
- [ ] Remove stale `services/n8n/database.sqlite` (408 MB, last write 2026-05-15) from UNAS.
|
||||
- [ ] Remove stale `services/shared-db/garage/` copy from UNAS if present.
|
||||
- [ ] Remove orphaned `daytona-minimal_db_data` Docker volume on CT 104.
|
||||
- [ ] Vaultwarden `data/` on UNAS — plan to migrate to local zfs (latency + tier-1).
|
||||
- [ ] Add a Backrest plan for Pocket-ID DB and Gitea data → jottacloud.
|
||||
|
||||
Reference in New Issue
Block a user