diff --git a/CLAUDE.md b/CLAUDE.md index bf4f73a..94d8b8f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,15 +15,15 @@ This file tells future automated runs what they need to know about this repo. - Steady state: ~32 GiB used (≈24 GiB of which is ZFS ARC, tunable via `zfs_arc_max`), CPU load <2.0, IO delay <0.05%. - LXC 102 dns (AdGuard) — internal DNS, rewrites for `*.nuclide.systems`. - LXC 103 backrest — backup orchestration. - - LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, provisioned from the **Dockge** flavor of `proxmox-helper-scripts` — stacks are managed in the Dockge web UI, not raw `docker compose` on disk. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways: + - LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, originally provisioned from the Dockge helper-script template, but **Arcane is the active docker manager** today (Dockge is stale, slated for retirement — see [docs/12 #33](./docs/12-optimization-opportunities.md#33)). **Dozzle** is the live log viewer. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways: - **NFS** (Proxmox host-mounted at `/mnt/pve/unas`, 19.4 TiB / 16.3 TiB free) — fine for host-side admin tasks and read-mostly mounts. - **SMB** — **default for Nexa per-container volumes**. The user already hit NFS+unprivileged-LXC write-permission issues with Nextcloud; don't repeat that. Mount SMB shares directly as docker volumes inside the stack with explicit `username=`/`uid=`/`gid=` — sidesteps uid-mapping entirely. **UNAS layout conventions** (homelab-wide, all docker containers follow them): - - **`services//` is the general docker config store** — every container in LXC 104 binds its persistent data here. Existing tenants observed: immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun. **Stale (retire, do not consume):** `services/siyuan/` (migrated to Obsidian) and `services/open-webui/` (unused). **Nexa MUST follow the same pattern: `services/nexa/{qdrant,tei-cache,graphdb,...}`.** Don't invent a parallel layout. + - **`services//` is the general docker config store** — every container in LXC 104 binds its persistent data here. Existing tenants observed: immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun. **Stale (retire, do not consume):** `services/siyuan/` (migrated to Obsidian), `services/open-webui/` (unused — LobeHub is the active LLM UI), and `services/dockge/` if it exists (retired in favor of Arcane). **Nexa MUST follow the same pattern: `services/nexa/{qdrant,tei-cache,graphdb,...}`.** Don't invent a parallel layout. - `backup//` — per-service backups (existing: home-assistant tars, immich pgdump, nextcloud borg). **Nexa snapshots → `backup/nexa/`.** - `media/`, `code/`, `_sortMe/`, `dump/`, `test_perm` — user data, not Nexa's concern. - - **Before deploying any Nexa container, READ AN EXISTING STACK** (e.g. `services/karakeep/` or `services/immich/`'s compose under Dockge) to confirm the exact mount syntax in use — driver name, share path, credential injection pattern. Match it. The actual NFS export root is `/var/nfs/shared/storage`; the SMB share name is **still TBD — see Q19** in `docs/11`. + - **Before deploying any Nexa container, READ AN EXISTING STACK** in **Arcane** (e.g. `karakeep` or `immich`) to confirm the exact mount syntax in use — driver name, share path, credential injection pattern. Match it. The actual NFS export root is `/var/nfs/shared/storage`; the SMB share name is **still TBD — see Q19** in `docs/11`. **Hard-blocklist for any Nexa indexer / agent** (never read these paths or matching glob): - `_sortMe/wallet/**` — contains PGP keys + bitcoin wallet files. diff --git a/docs/02-roadmap.md b/docs/02-roadmap.md index 7ebd58a..fff7ebf 100644 --- a/docs/02-roadmap.md +++ b/docs/02-roadmap.md @@ -52,7 +52,7 @@ Iterative build-out, value-first. Each phase is shippable on its own. **Focus:** Nexa actively maintains the homelab inventory instead of being told manually. **Milestone:** Nexa runs a daily *drift report* — what changed, what's stale, what's missing — and turns it into actionable comments under a pinned `[STEWARD]` memo. -- [ ] **6.1 Inventory sync** — Nexa polls Dockge / `docker ps` / Proxmox API on a schedule and stores the running-state snapshot in GraphDB (`nexa:Service`, `nexa:LastSeen`, `nexa:DataPath`, …). +- [ ] **6.1 Inventory sync** — Nexa polls Arcane / `docker ps` / Proxmox API on a schedule and stores the running-state snapshot in GraphDB (`nexa:Service`, `nexa:LastSeen`, `nexa:DataPath`, …). - [ ] **6.2 Drift detection** — diff against the documented desired state in this repo (`docs/12`, `CLAUDE.md`, the housekeeping campaign at #35–37). Emit findings as structured Memos comments: *"`open-webui` last used 38 d ago — retire?"*, *"Container `foo` has no `services/foo/` bind — schedule UNAS migration?"*. - [ ] **6.3 Steward commands** — `#nexa:retire ` (tar to `backup/`, stop, archive the stack), `#nexa:document ` (template a CLAUDE.md entry), `#nexa:wishlist-status` (which entries in [docs/13](./13-information-wishlist.md) are still needed). - [ ] **6.4 Self-update of docs** — when the user resolves an open question via Memos, Nexa proposes the corresponding edit to `docs/11-open-questions.md` as a PR-ready diff (the doc itself becomes a Nexa-managed surface). diff --git a/docs/09-deployment.md b/docs/09-deployment.md index 49d2eae..cbebddd 100644 --- a/docs/09-deployment.md +++ b/docs/09-deployment.md @@ -92,10 +92,10 @@ This means **no data is lost** between 3.1 and 3.2 — the queue is the GraphDB Use HuggingFace **text-embeddings-inference** — single Rust binary, ~500 MB image, OpenAI-compatible API, loads exactly one model. Lighter than Ollama because there's no LLM runtime, no GGUF loader, no model registry. -The docker LXC is the **Dockge** flavor of `proxmox-helper-scripts` — paste the stack into the Dockge UI (`http://192.168.1.40:5001` or whatever Zoraxy host is configured), name it `nexa`, save, start. Don't `docker compose up -d` over SSH; Dockge maintains the compose YAML at `/opt/stacks/nexa/compose.yaml`. +The active docker manager on this LXC is **Arcane** (visible from Homepage as the running container manager — the LXC was originally provisioned with the Dockge helper-script template, but Dockge is now stale; see [12/#33](./12-optimization-opportunities.md#33)). Paste the stack into Arcane → name it `nexa` → save → start. Don't `docker compose up -d` over SSH; Arcane manages the compose lifecycle. ```yaml -# Dockge stack: nexa (paste in the UI) +# Nexa stack — paste into Arcane services: nexa-embed: image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 @@ -135,7 +135,7 @@ Memory budget: ~1.1 GB resident. First start downloads `bge-m3` (~1 GB) into the > **Why SMB and not the Proxmox `/mnt/pve/unas` NFS mount?** The host-side NFS mount works fine for host admin, but **unprivileged LXC + NFS + per-container writes** historically breaks (user confirmed: "had it with Nextcloud"). Mounting SMB **directly as a docker volume** with explicit `username=`/`uid=`/`gid=` bypasses LXC uid-mapping entirely — the docker daemon is root inside the LXC so it can mount, and the container sees a regular volume with the right ownership. > -> Add `UNAS_USER` / `UNAS_PASS` to the Dockge stack's `.env`, or — better — store them in Vaultwarden and inject via a secrets agent (see [12/#13](./12-optimization-opportunities.md)). +> Add `UNAS_USER` / `UNAS_PASS` to the stack's `.env` (Arcane provides an editor for this), or — better — store them in Vaultwarden and inject via a secrets agent (see [12/#11](./12-optimization-opportunities.md#11)). Register it inside LiteLLM (admin UI → Models) with the OpenAI-compatible adapter: diff --git a/docs/11-open-questions.md b/docs/11-open-questions.md index 97b2083..b210c27 100644 --- a/docs/11-open-questions.md +++ b/docs/11-open-questions.md @@ -53,7 +53,7 @@ Items that **block progress** and need a human decision before a workflow can be ## Verification needed before Phase-3.1 deploy -- [ ] **Q19 — Confirm the SMB share name + mount syntax** by reading an existing Dockge stack (recommended: `services/karakeep/` or `services/immich/`). The SMB share name on UNAS that maps to the NFS root `/var/nfs/shared/storage` is currently a placeholder `` in [docs/09](./09-deployment.md). Required action: open the running Karakeep stack in Dockge, copy its `volumes:` block verbatim, swap `karakeep` → `nexa` in the path, drop into the Nexa stack. This locks Nexa to the **same convention every other container in the homelab uses** — same driver, same credentials secret, same uid/gid — and avoids reinventing storage just for Nexa. +- [ ] **Q19 — Confirm the SMB share name + mount syntax** by reading an existing stack in **Arcane** (recommended: `karakeep` or `immich`). The SMB share name on UNAS that maps to the NFS root `/var/nfs/shared/storage` is currently a placeholder `` in [docs/09](./09-deployment.md). Required action: open the running Karakeep stack in Arcane, copy its `volumes:` block verbatim, swap `karakeep` → `nexa` in the path, drop into the Nexa stack. This locks Nexa to the **same convention every other container in the homelab uses** — same driver, same credentials secret, same uid/gid — and avoids reinventing storage just for Nexa. --- diff --git a/docs/12-optimization-opportunities.md b/docs/12-optimization-opportunities.md index 4a91abb..e22ef3c 100644 --- a/docs/12-optimization-opportunities.md +++ b/docs/12-optimization-opportunities.md @@ -83,20 +83,20 @@ Confirmed allocation: 16 CPU, 31.25 GiB RAM (7.86 GiB used / 25%), 8 GiB swap (i Credentials (`UNAS_USER` / `UNAS_PASS`) go in Vaultwarden ([12/#11](#11)), injected at deploy. The host-side `/mnt/pve/unas` NFS mount stays useful for admin tasks (rsync, manual backups) — not for per-container write paths. 32. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap. -33. **Stacks live in Dockge, not raw compose.** The LXC is the Dockge flavor of helper-scripts. Deployment writes a stack named `nexa` in the Dockge UI; the compose YAML is stored under `/opt/stacks/nexa/` (Dockge default) and edited from the web UI. This means [docs/09](./09-deployment.md) "deploy" snippets translate to "paste this into Dockge → save → start", not `docker compose up -d` over SSH. +33. **Stacks are managed in Arcane; Dockge is stale.** The LXC was originally provisioned with the Dockge helper-script template, but the user moved on to **Arcane** as the day-to-day docker manager. Deployment of the Nexa stack therefore goes through Arcane, not Dockge. Cleanup task: tar `services/dockge/` (if it exists) into `backup/dockge/`, retire the Dockge container, drop the stale data dir. **Dozzle** stays — it's the log viewer, not a manager, so it isn't redundant with Arcane. 34. **Untriaged local mail on the LXC.** Console shows `You have new mail.` at login — the system mail spool on `/var/mail/root` has unread messages, almost always cron job failures. `mailx` or `mutt` to inspect, then either fix the failing job or send the spool to `nexa.system` ntfy via a tiny aliases entry (`root: |/usr/local/bin/spool-to-ntfy.sh`). ## Housekeeping campaign (cross-cutting, do together) -These three are inter-related — picking them up as one campaign is cheaper than chasing each individually, because the audit step is the same. They also map cleanly onto **[Phase 6 (Nexa as homelab steward)](./02-roadmap.md#phase-6--nexa-as-homelab-steward)**: once Nexa can poll Dockge, diff against the documented state and emit actionable findings, items #35–37 become semi-automatic — Nexa proposes the migrations rather than us hunting them down. +These three are inter-related — picking them up as one campaign is cheaper than chasing each individually, because the audit step is the same. They also map cleanly onto **[Phase 6 (Nexa as homelab steward)](./02-roadmap.md#phase-6--nexa-as-homelab-steward)**: once Nexa can poll Arcane, diff against the documented state and emit actionable findings, items #35–37 become semi-automatic — Nexa proposes the migrations rather than us hunting them down. -35. **Consolidate Postgres instances.** Today there are at least four independent Postgres containers running — visible from Dockge: `immich_postgres`, `lobe-postgres`, `litellm_db`, `paperless-ngx-db-1` (Karakeep uses Meilisearch + maybe SQLite, separate). Each idles around 100–300 MB RAM and has its own backup story. Two paths: +35. **Consolidate Postgres instances.** Today there are at least four independent Postgres containers running — visible from Arcane: `immich_postgres`, `lobe-postgres`, `litellm_db`, `paperless-ngx-db-1` (Karakeep uses Meilisearch + maybe SQLite, separate). Each idles around 100–300 MB RAM and has its own backup story. Two paths: - **Single shared cluster** (`pgsql` container with one role per app, one database per app). Most modern apps support `DATABASE_URL` → just point them at the shared instance. Saves ~600 MB–1 GB RAM and consolidates backups to one pg_dump cron. - **Or** migrate the smallest ones to SQLite (LiteLLM has SQLite mode; Paperless does too) and keep Postgres only for Immich. Same outcome, less migration risk. Pre-step: list every running container with `docker ps --format '{{.Names}}\t{{.Image}}' | grep -i 'postgres\|mariadb\|mysql'` to inventory exactly what's running. -36. **UNAS-integration audit.** `services//` on UNAS is the homelab convention but not every container follows it yet. Walk every Dockge stack and check the `volumes:` block. Suspected non-compliant (need verification): +36. **UNAS-integration audit.** `services//` on UNAS is the homelab convention but not every container follows it yet. Walk every Arcane stack and check the `volumes:` block. Suspected non-compliant (need verification): - `qdrant_scientific` — vector data possibly on local boot disk; **critical to verify** before Phase-3.1 piles on a `nexa_knowledge_text` collection. - `litellm_db`, `lobe-postgres`, `lobe-redis` — DB containers, persistent state. - `memos` — the user-facing notes app; loss = data loss. diff --git a/docs/13-information-wishlist.md b/docs/13-information-wishlist.md index 8787bab..dc34d92 100644 --- a/docs/13-information-wishlist.md +++ b/docs/13-information-wishlist.md @@ -8,14 +8,16 @@ Each task header tells you **exactly where to run it** (which shell or which UI) ## Task 1 — Stack inventory + reference compose **(highest leverage)** -📍 **Where:** Proxmox UI → Datacenter → `nuc` → LXC `104 (docker)` → **Console** (or `ssh root@192.168.1.40` from your PC). +📍 **Two halves — the canonical compose lives in the Arcane UI; the running-container view comes from a shell.** Locks **Q19** (the SMB share name + driver options Nexa must mirror) and replaces screenshot-derived guesses about what's running. +**(a) Reference compose** — open **Arcane** in your browser → pick a clean small stack (Karakeep is a good choice) → screenshot or copy the **compose.yaml** *and* the **`.env`** (or whatever credential file is referenced). Just the `volumes:` block + the volume credential pattern is enough; everything else is bonus. + +**(b) Running-container view** — Proxmox UI → Datacenter → `nuc` → LXC `104 (docker)` → **Console** (or `ssh root@192.168.1.40` from your PC): + ```bash docker ps -a --format '{{.Names}}\t{{.Image}}\t{{.Status}}' | sort -ls /opt/stacks/ -cat /opt/stacks/karakeep/compose.yaml # or any clean small stack ``` **Unblocks:** Q19, the housekeeping migration template (#36), Phase-3.1 deployment YAML.