Flip Dockge → Arcane: Arcane is the active docker manager, Dockge is stale
The LXC was originally provisioned with the Dockge helper-script template, but the user moved on to Arcane. Dozzle stays as the log viewer (different role, not redundant). - docs/09 step 3: deployment goes via Arcane UI (not Dockge); reworded the deploy block accordingly. - docs/11 Q19: read the reference compose from Arcane, not Dockge. - docs/12 #33: was "stacks live in Dockge"; now "Arcane manages stacks, Dockge is stale, retire it" with the same tar-then-remove pattern as SiYuan and Open-WebUI. - docs/12 housekeeping campaign + #36: "walk every Arcane stack" rather than Dockge. - docs/13 Task 1: stack inventory comes from the Arcane UI (compose.yaml + .env screenshot/copy) rather than `ls /opt/stacks/` which was the Dockge default. The shell command for `docker ps -a` stays. - docs/02 Phase 6.1: Nexa polls Arcane (not Dockge) for inventory sync. - CLAUDE.md infra block: Arcane is the active manager, Dozzle is the log viewer, Dockge is stale; added services/dockge/ to the stale list alongside siyuan and open-webui.
This commit is contained in:
@@ -15,15 +15,15 @@ This file tells future automated runs what they need to know about this repo.
|
|||||||
- Steady state: ~32 GiB used (≈24 GiB of which is ZFS ARC, tunable via `zfs_arc_max`), CPU load <2.0, IO delay <0.05%.
|
- Steady state: ~32 GiB used (≈24 GiB of which is ZFS ARC, tunable via `zfs_arc_max`), CPU load <2.0, IO delay <0.05%.
|
||||||
- LXC 102 dns (AdGuard) — internal DNS, rewrites for `*.nuclide.systems`.
|
- LXC 102 dns (AdGuard) — internal DNS, rewrites for `*.nuclide.systems`.
|
||||||
- LXC 103 backrest — backup orchestration.
|
- LXC 103 backrest — backup orchestration.
|
||||||
- LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, provisioned from the **Dockge** flavor of `proxmox-helper-scripts` — stacks are managed in the Dockge web UI, not raw `docker compose` on disk. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways:
|
- LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, originally provisioned from the Dockge helper-script template, but **Arcane is the active docker manager** today (Dockge is stale, slated for retirement — see [docs/12 #33](./docs/12-optimization-opportunities.md#33)). **Dozzle** is the live log viewer. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways:
|
||||||
- **NFS** (Proxmox host-mounted at `/mnt/pve/unas`, 19.4 TiB / 16.3 TiB free) — fine for host-side admin tasks and read-mostly mounts.
|
- **NFS** (Proxmox host-mounted at `/mnt/pve/unas`, 19.4 TiB / 16.3 TiB free) — fine for host-side admin tasks and read-mostly mounts.
|
||||||
- **SMB** — **default for Nexa per-container volumes**. The user already hit NFS+unprivileged-LXC write-permission issues with Nextcloud; don't repeat that. Mount SMB shares directly as docker volumes inside the stack with explicit `username=`/`uid=`/`gid=` — sidesteps uid-mapping entirely.
|
- **SMB** — **default for Nexa per-container volumes**. The user already hit NFS+unprivileged-LXC write-permission issues with Nextcloud; don't repeat that. Mount SMB shares directly as docker volumes inside the stack with explicit `username=`/`uid=`/`gid=` — sidesteps uid-mapping entirely.
|
||||||
|
|
||||||
**UNAS layout conventions** (homelab-wide, all docker containers follow them):
|
**UNAS layout conventions** (homelab-wide, all docker containers follow them):
|
||||||
- **`services/<svc>/` is the general docker config store** — every container in LXC 104 binds its persistent data here. Existing tenants observed: immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun. **Stale (retire, do not consume):** `services/siyuan/` (migrated to Obsidian) and `services/open-webui/` (unused). **Nexa MUST follow the same pattern: `services/nexa/{qdrant,tei-cache,graphdb,...}`.** Don't invent a parallel layout.
|
- **`services/<svc>/` is the general docker config store** — every container in LXC 104 binds its persistent data here. Existing tenants observed: immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun. **Stale (retire, do not consume):** `services/siyuan/` (migrated to Obsidian), `services/open-webui/` (unused — LobeHub is the active LLM UI), and `services/dockge/` if it exists (retired in favor of Arcane). **Nexa MUST follow the same pattern: `services/nexa/{qdrant,tei-cache,graphdb,...}`.** Don't invent a parallel layout.
|
||||||
- `backup/<svc>/` — per-service backups (existing: home-assistant tars, immich pgdump, nextcloud borg). **Nexa snapshots → `backup/nexa/`.**
|
- `backup/<svc>/` — per-service backups (existing: home-assistant tars, immich pgdump, nextcloud borg). **Nexa snapshots → `backup/nexa/`.**
|
||||||
- `media/`, `code/`, `_sortMe/`, `dump/`, `test_perm` — user data, not Nexa's concern.
|
- `media/`, `code/`, `_sortMe/`, `dump/`, `test_perm` — user data, not Nexa's concern.
|
||||||
- **Before deploying any Nexa container, READ AN EXISTING STACK** (e.g. `services/karakeep/` or `services/immich/`'s compose under Dockge) to confirm the exact mount syntax in use — driver name, share path, credential injection pattern. Match it. The actual NFS export root is `/var/nfs/shared/storage`; the SMB share name is **still TBD — see Q19** in `docs/11`.
|
- **Before deploying any Nexa container, READ AN EXISTING STACK** in **Arcane** (e.g. `karakeep` or `immich`) to confirm the exact mount syntax in use — driver name, share path, credential injection pattern. Match it. The actual NFS export root is `/var/nfs/shared/storage`; the SMB share name is **still TBD — see Q19** in `docs/11`.
|
||||||
|
|
||||||
**Hard-blocklist for any Nexa indexer / agent** (never read these paths or matching glob):
|
**Hard-blocklist for any Nexa indexer / agent** (never read these paths or matching glob):
|
||||||
- `_sortMe/wallet/**` — contains PGP keys + bitcoin wallet files.
|
- `_sortMe/wallet/**` — contains PGP keys + bitcoin wallet files.
|
||||||
|
|||||||
+1
-1
@@ -52,7 +52,7 @@ Iterative build-out, value-first. Each phase is shippable on its own.
|
|||||||
**Focus:** Nexa actively maintains the homelab inventory instead of being told manually.
|
**Focus:** Nexa actively maintains the homelab inventory instead of being told manually.
|
||||||
**Milestone:** Nexa runs a daily *drift report* — what changed, what's stale, what's missing — and turns it into actionable comments under a pinned `[STEWARD]` memo.
|
**Milestone:** Nexa runs a daily *drift report* — what changed, what's stale, what's missing — and turns it into actionable comments under a pinned `[STEWARD]` memo.
|
||||||
|
|
||||||
- [ ] **6.1 Inventory sync** — Nexa polls Dockge / `docker ps` / Proxmox API on a schedule and stores the running-state snapshot in GraphDB (`nexa:Service`, `nexa:LastSeen`, `nexa:DataPath`, …).
|
- [ ] **6.1 Inventory sync** — Nexa polls Arcane / `docker ps` / Proxmox API on a schedule and stores the running-state snapshot in GraphDB (`nexa:Service`, `nexa:LastSeen`, `nexa:DataPath`, …).
|
||||||
- [ ] **6.2 Drift detection** — diff against the documented desired state in this repo (`docs/12`, `CLAUDE.md`, the housekeeping campaign at #35–37). Emit findings as structured Memos comments: *"`open-webui` last used 38 d ago — retire?"*, *"Container `foo` has no `services/foo/` bind — schedule UNAS migration?"*.
|
- [ ] **6.2 Drift detection** — diff against the documented desired state in this repo (`docs/12`, `CLAUDE.md`, the housekeeping campaign at #35–37). Emit findings as structured Memos comments: *"`open-webui` last used 38 d ago — retire?"*, *"Container `foo` has no `services/foo/` bind — schedule UNAS migration?"*.
|
||||||
- [ ] **6.3 Steward commands** — `#nexa:retire <svc>` (tar to `backup/`, stop, archive the stack), `#nexa:document <svc>` (template a CLAUDE.md entry), `#nexa:wishlist-status` (which entries in [docs/13](./13-information-wishlist.md) are still needed).
|
- [ ] **6.3 Steward commands** — `#nexa:retire <svc>` (tar to `backup/`, stop, archive the stack), `#nexa:document <svc>` (template a CLAUDE.md entry), `#nexa:wishlist-status` (which entries in [docs/13](./13-information-wishlist.md) are still needed).
|
||||||
- [ ] **6.4 Self-update of docs** — when the user resolves an open question via Memos, Nexa proposes the corresponding edit to `docs/11-open-questions.md` as a PR-ready diff (the doc itself becomes a Nexa-managed surface).
|
- [ ] **6.4 Self-update of docs** — when the user resolves an open question via Memos, Nexa proposes the corresponding edit to `docs/11-open-questions.md` as a PR-ready diff (the doc itself becomes a Nexa-managed surface).
|
||||||
|
|||||||
@@ -92,10 +92,10 @@ This means **no data is lost** between 3.1 and 3.2 — the queue is the GraphDB
|
|||||||
|
|
||||||
Use HuggingFace **text-embeddings-inference** — single Rust binary, ~500 MB image, OpenAI-compatible API, loads exactly one model. Lighter than Ollama because there's no LLM runtime, no GGUF loader, no model registry.
|
Use HuggingFace **text-embeddings-inference** — single Rust binary, ~500 MB image, OpenAI-compatible API, loads exactly one model. Lighter than Ollama because there's no LLM runtime, no GGUF loader, no model registry.
|
||||||
|
|
||||||
The docker LXC is the **Dockge** flavor of `proxmox-helper-scripts` — paste the stack into the Dockge UI (`http://192.168.1.40:5001` or whatever Zoraxy host is configured), name it `nexa`, save, start. Don't `docker compose up -d` over SSH; Dockge maintains the compose YAML at `/opt/stacks/nexa/compose.yaml`.
|
The active docker manager on this LXC is **Arcane** (visible from Homepage as the running container manager — the LXC was originally provisioned with the Dockge helper-script template, but Dockge is now stale; see [12/#33](./12-optimization-opportunities.md#33)). Paste the stack into Arcane → name it `nexa` → save → start. Don't `docker compose up -d` over SSH; Arcane manages the compose lifecycle.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# Dockge stack: nexa (paste in the UI)
|
# Nexa stack — paste into Arcane
|
||||||
services:
|
services:
|
||||||
nexa-embed:
|
nexa-embed:
|
||||||
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
|
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
|
||||||
@@ -135,7 +135,7 @@ Memory budget: ~1.1 GB resident. First start downloads `bge-m3` (~1 GB) into the
|
|||||||
|
|
||||||
> **Why SMB and not the Proxmox `/mnt/pve/unas` NFS mount?** The host-side NFS mount works fine for host admin, but **unprivileged LXC + NFS + per-container writes** historically breaks (user confirmed: "had it with Nextcloud"). Mounting SMB **directly as a docker volume** with explicit `username=`/`uid=`/`gid=` bypasses LXC uid-mapping entirely — the docker daemon is root inside the LXC so it can mount, and the container sees a regular volume with the right ownership.
|
> **Why SMB and not the Proxmox `/mnt/pve/unas` NFS mount?** The host-side NFS mount works fine for host admin, but **unprivileged LXC + NFS + per-container writes** historically breaks (user confirmed: "had it with Nextcloud"). Mounting SMB **directly as a docker volume** with explicit `username=`/`uid=`/`gid=` bypasses LXC uid-mapping entirely — the docker daemon is root inside the LXC so it can mount, and the container sees a regular volume with the right ownership.
|
||||||
>
|
>
|
||||||
> Add `UNAS_USER` / `UNAS_PASS` to the Dockge stack's `.env`, or — better — store them in Vaultwarden and inject via a secrets agent (see [12/#13](./12-optimization-opportunities.md)).
|
> Add `UNAS_USER` / `UNAS_PASS` to the stack's `.env` (Arcane provides an editor for this), or — better — store them in Vaultwarden and inject via a secrets agent (see [12/#11](./12-optimization-opportunities.md#11)).
|
||||||
|
|
||||||
Register it inside LiteLLM (admin UI → Models) with the OpenAI-compatible adapter:
|
Register it inside LiteLLM (admin UI → Models) with the OpenAI-compatible adapter:
|
||||||
|
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ Items that **block progress** and need a human decision before a workflow can be
|
|||||||
|
|
||||||
## Verification needed before Phase-3.1 deploy
|
## Verification needed before Phase-3.1 deploy
|
||||||
|
|
||||||
- [ ] **Q19 — Confirm the SMB share name + mount syntax** by reading an existing Dockge stack (recommended: `services/karakeep/` or `services/immich/`). The SMB share name on UNAS that maps to the NFS root `/var/nfs/shared/storage` is currently a placeholder `<unas-share>` in [docs/09](./09-deployment.md). Required action: open the running Karakeep stack in Dockge, copy its `volumes:` block verbatim, swap `karakeep` → `nexa` in the path, drop into the Nexa stack. This locks Nexa to the **same convention every other container in the homelab uses** — same driver, same credentials secret, same uid/gid — and avoids reinventing storage just for Nexa.
|
- [ ] **Q19 — Confirm the SMB share name + mount syntax** by reading an existing stack in **Arcane** (recommended: `karakeep` or `immich`). The SMB share name on UNAS that maps to the NFS root `/var/nfs/shared/storage` is currently a placeholder `<unas-share>` in [docs/09](./09-deployment.md). Required action: open the running Karakeep stack in Arcane, copy its `volumes:` block verbatim, swap `karakeep` → `nexa` in the path, drop into the Nexa stack. This locks Nexa to the **same convention every other container in the homelab uses** — same driver, same credentials secret, same uid/gid — and avoids reinventing storage just for Nexa.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -83,20 +83,20 @@ Confirmed allocation: 16 CPU, 31.25 GiB RAM (7.86 GiB used / 25%), 8 GiB swap (i
|
|||||||
|
|
||||||
Credentials (`UNAS_USER` / `UNAS_PASS`) go in Vaultwarden ([12/#11](#11)), injected at deploy. The host-side `/mnt/pve/unas` NFS mount stays useful for admin tasks (rsync, manual backups) — not for per-container write paths.
|
Credentials (`UNAS_USER` / `UNAS_PASS`) go in Vaultwarden ([12/#11](#11)), injected at deploy. The host-side `/mnt/pve/unas` NFS mount stays useful for admin tasks (rsync, manual backups) — not for per-container write paths.
|
||||||
32. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap.
|
32. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap.
|
||||||
33. **Stacks live in Dockge, not raw compose.** The LXC is the Dockge flavor of helper-scripts. Deployment writes a stack named `nexa` in the Dockge UI; the compose YAML is stored under `/opt/stacks/nexa/` (Dockge default) and edited from the web UI. This means [docs/09](./09-deployment.md) "deploy" snippets translate to "paste this into Dockge → save → start", not `docker compose up -d` over SSH.
|
33. **Stacks are managed in Arcane; Dockge is stale.** The LXC was originally provisioned with the Dockge helper-script template, but the user moved on to **Arcane** as the day-to-day docker manager. Deployment of the Nexa stack therefore goes through Arcane, not Dockge. Cleanup task: tar `services/dockge/` (if it exists) into `backup/dockge/`, retire the Dockge container, drop the stale data dir. **Dozzle** stays — it's the log viewer, not a manager, so it isn't redundant with Arcane.
|
||||||
34. **Untriaged local mail on the LXC.** Console shows `You have new mail.` at login — the system mail spool on `/var/mail/root` has unread messages, almost always cron job failures. `mailx` or `mutt` to inspect, then either fix the failing job or send the spool to `nexa.system` ntfy via a tiny aliases entry (`root: |/usr/local/bin/spool-to-ntfy.sh`).
|
34. **Untriaged local mail on the LXC.** Console shows `You have new mail.` at login — the system mail spool on `/var/mail/root` has unread messages, almost always cron job failures. `mailx` or `mutt` to inspect, then either fix the failing job or send the spool to `nexa.system` ntfy via a tiny aliases entry (`root: |/usr/local/bin/spool-to-ntfy.sh`).
|
||||||
|
|
||||||
## Housekeeping campaign (cross-cutting, do together)
|
## Housekeeping campaign (cross-cutting, do together)
|
||||||
|
|
||||||
These three are inter-related — picking them up as one campaign is cheaper than chasing each individually, because the audit step is the same. They also map cleanly onto **[Phase 6 (Nexa as homelab steward)](./02-roadmap.md#phase-6--nexa-as-homelab-steward)**: once Nexa can poll Dockge, diff against the documented state and emit actionable findings, items #35–37 become semi-automatic — Nexa proposes the migrations rather than us hunting them down.
|
These three are inter-related — picking them up as one campaign is cheaper than chasing each individually, because the audit step is the same. They also map cleanly onto **[Phase 6 (Nexa as homelab steward)](./02-roadmap.md#phase-6--nexa-as-homelab-steward)**: once Nexa can poll Arcane, diff against the documented state and emit actionable findings, items #35–37 become semi-automatic — Nexa proposes the migrations rather than us hunting them down.
|
||||||
|
|
||||||
35. **Consolidate Postgres instances.** Today there are at least four independent Postgres containers running — visible from Dockge: `immich_postgres`, `lobe-postgres`, `litellm_db`, `paperless-ngx-db-1` (Karakeep uses Meilisearch + maybe SQLite, separate). Each idles around 100–300 MB RAM and has its own backup story. Two paths:
|
35. **Consolidate Postgres instances.** Today there are at least four independent Postgres containers running — visible from Arcane: `immich_postgres`, `lobe-postgres`, `litellm_db`, `paperless-ngx-db-1` (Karakeep uses Meilisearch + maybe SQLite, separate). Each idles around 100–300 MB RAM and has its own backup story. Two paths:
|
||||||
- **Single shared cluster** (`pgsql` container with one role per app, one database per app). Most modern apps support `DATABASE_URL` → just point them at the shared instance. Saves ~600 MB–1 GB RAM and consolidates backups to one pg_dump cron.
|
- **Single shared cluster** (`pgsql` container with one role per app, one database per app). Most modern apps support `DATABASE_URL` → just point them at the shared instance. Saves ~600 MB–1 GB RAM and consolidates backups to one pg_dump cron.
|
||||||
- **Or** migrate the smallest ones to SQLite (LiteLLM has SQLite mode; Paperless does too) and keep Postgres only for Immich. Same outcome, less migration risk.
|
- **Or** migrate the smallest ones to SQLite (LiteLLM has SQLite mode; Paperless does too) and keep Postgres only for Immich. Same outcome, less migration risk.
|
||||||
|
|
||||||
Pre-step: list every running container with `docker ps --format '{{.Names}}\t{{.Image}}' | grep -i 'postgres\|mariadb\|mysql'` to inventory exactly what's running.
|
Pre-step: list every running container with `docker ps --format '{{.Names}}\t{{.Image}}' | grep -i 'postgres\|mariadb\|mysql'` to inventory exactly what's running.
|
||||||
|
|
||||||
36. **UNAS-integration audit.** `services/<svc>/` on UNAS is the homelab convention but not every container follows it yet. Walk every Dockge stack and check the `volumes:` block. Suspected non-compliant (need verification):
|
36. **UNAS-integration audit.** `services/<svc>/` on UNAS is the homelab convention but not every container follows it yet. Walk every Arcane stack and check the `volumes:` block. Suspected non-compliant (need verification):
|
||||||
- `qdrant_scientific` — vector data possibly on local boot disk; **critical to verify** before Phase-3.1 piles on a `nexa_knowledge_text` collection.
|
- `qdrant_scientific` — vector data possibly on local boot disk; **critical to verify** before Phase-3.1 piles on a `nexa_knowledge_text` collection.
|
||||||
- `litellm_db`, `lobe-postgres`, `lobe-redis` — DB containers, persistent state.
|
- `litellm_db`, `lobe-postgres`, `lobe-redis` — DB containers, persistent state.
|
||||||
- `memos` — the user-facing notes app; loss = data loss.
|
- `memos` — the user-facing notes app; loss = data loss.
|
||||||
|
|||||||
@@ -8,14 +8,16 @@ Each task header tells you **exactly where to run it** (which shell or which UI)
|
|||||||
|
|
||||||
## Task 1 — Stack inventory + reference compose **(highest leverage)**
|
## Task 1 — Stack inventory + reference compose **(highest leverage)**
|
||||||
|
|
||||||
📍 **Where:** Proxmox UI → Datacenter → `nuc` → LXC `104 (docker)` → **Console** (or `ssh root@192.168.1.40` from your PC).
|
📍 **Two halves — the canonical compose lives in the Arcane UI; the running-container view comes from a shell.**
|
||||||
|
|
||||||
Locks **Q19** (the SMB share name + driver options Nexa must mirror) and replaces screenshot-derived guesses about what's running.
|
Locks **Q19** (the SMB share name + driver options Nexa must mirror) and replaces screenshot-derived guesses about what's running.
|
||||||
|
|
||||||
|
**(a) Reference compose** — open **Arcane** in your browser → pick a clean small stack (Karakeep is a good choice) → screenshot or copy the **compose.yaml** *and* the **`.env`** (or whatever credential file is referenced). Just the `volumes:` block + the volume credential pattern is enough; everything else is bonus.
|
||||||
|
|
||||||
|
**(b) Running-container view** — Proxmox UI → Datacenter → `nuc` → LXC `104 (docker)` → **Console** (or `ssh root@192.168.1.40` from your PC):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker ps -a --format '{{.Names}}\t{{.Image}}\t{{.Status}}' | sort
|
docker ps -a --format '{{.Names}}\t{{.Image}}\t{{.Status}}' | sort
|
||||||
ls /opt/stacks/
|
|
||||||
cat /opt/stacks/karakeep/compose.yaml # or any clean small stack
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Unblocks:** Q19, the housekeeping migration template (#36), Phase-3.1 deployment YAML.
|
**Unblocks:** Q19, the housekeeping migration template (#36), Phase-3.1 deployment YAML.
|
||||||
|
|||||||
Reference in New Issue
Block a user