Pin homelab-wide UNAS storage convention; add Phase 6 + info wishlist
- UNAS layout decoded from the user's tree: services/<svc>/ is the homelab-wide docker config store (every container follows the same pattern — immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun). Nexa MUST follow the same pattern at services/nexa/. backup/<svc>/ for snapshots. Pinned in CLAUDE.md and docs/12 #31. - Stale services flagged for retirement: services/siyuan/ (migrated to Obsidian) and services/open-webui/ (unused, only LobeHub is alive). Drops the "two LLM UIs" item (#15) — it's now "retire open-webui". - Hard-blocklist for any Nexa indexer pinned in CLAUDE.md: _sortMe/wallet/**, *.gpg/asc/key/pem/kdbx/credentials/secret, plus the Nextcloud appdata dir. - Three new "future user-facing wins" surfaced from the tree: Paperless-AI as a Phase-2.x triage helper for _sortMe/Downloads/, media/Recipes/ (~300 entries) as the showcase RAG corpus, and Paperless-AI's existing ChromaDB as a potential read-from source rather than re-embedding scanned docs. - Housekeeping campaign in docs/12 §35-37: consolidate postgres, audit UNAS-everywhere, S3 archive tier on s3.nuclide.systems. - Phase 6 added to the roadmap: "Nexa as homelab steward". Polls Dockge/docker/Proxmox, diffs vs documented state, emits a daily drift report. Most of the housekeeping campaign becomes semi-automatic once 6.1-6.3 ship. - New docs/13-information-wishlist.md packages the still-needed inventory as 5-6 paste-and-run tasks, each with explicit "📍 Where" markers (which shell or which UI) and what it unblocks. Highest leverage = Task 1 (read an existing compose stack to lock Q19). - docs/index.md TOC extended to 13 rows. Also fixed numbering drift in docs/12 (duplicate #28, missing #17/#18) and added Q19 to docs/11 covering the SMB-share verification step.
This commit is contained in:
@@ -17,7 +17,18 @@ This file tells future automated runs what they need to know about this repo.
|
|||||||
- LXC 103 backrest — backup orchestration.
|
- LXC 103 backrest — backup orchestration.
|
||||||
- LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, provisioned from the **Dockge** flavor of `proxmox-helper-scripts` — stacks are managed in the Dockge web UI, not raw `docker compose` on disk. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways:
|
- LXC 104 docker — main docker host at `192.168.1.40`, hostname `docker`, OS Debian 13. **Unprivileged**, provisioned from the **Dockge** flavor of `proxmox-helper-scripts` — stacks are managed in the Dockge web UI, not raw `docker compose` on disk. Allocated: 16 CPU, 31.25 GiB RAM (25% used), 8 GiB swap, 200 GiB boot disk (47.7% used). **Storage on the UNAS (`192.168.1.31`)** is reachable two ways:
|
||||||
- **NFS** (Proxmox host-mounted at `/mnt/pve/unas`, 19.4 TiB / 16.3 TiB free) — fine for host-side admin tasks and read-mostly mounts.
|
- **NFS** (Proxmox host-mounted at `/mnt/pve/unas`, 19.4 TiB / 16.3 TiB free) — fine for host-side admin tasks and read-mostly mounts.
|
||||||
- **SMB** — **default for Nexa per-container volumes**. The user already hit NFS+unprivileged-LXC write-permission issues with Nextcloud; don't repeat that. Mount SMB shares directly as docker volumes inside the stack with explicit `username=`/`uid=`/`gid=` — sidesteps uid-mapping entirely. Intel iGPU passthrough is *configured but currently broken* — see [docs/12 #26](./docs/12-optimization-opportunities.md#26).
|
- **SMB** — **default for Nexa per-container volumes**. The user already hit NFS+unprivileged-LXC write-permission issues with Nextcloud; don't repeat that. Mount SMB shares directly as docker volumes inside the stack with explicit `username=`/`uid=`/`gid=` — sidesteps uid-mapping entirely.
|
||||||
|
|
||||||
|
**UNAS layout conventions** (homelab-wide, all docker containers follow them):
|
||||||
|
- **`services/<svc>/` is the general docker config store** — every container in LXC 104 binds its persistent data here. Existing tenants observed: immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid, shelfmark, stremio, traccar, vaultwarden, gluetun. **Stale (retire, do not consume):** `services/siyuan/` (migrated to Obsidian) and `services/open-webui/` (unused). **Nexa MUST follow the same pattern: `services/nexa/{qdrant,tei-cache,graphdb,...}`.** Don't invent a parallel layout.
|
||||||
|
- `backup/<svc>/` — per-service backups (existing: home-assistant tars, immich pgdump, nextcloud borg). **Nexa snapshots → `backup/nexa/`.**
|
||||||
|
- `media/`, `code/`, `_sortMe/`, `dump/`, `test_perm` — user data, not Nexa's concern.
|
||||||
|
- **Before deploying any Nexa container, READ AN EXISTING STACK** (e.g. `services/karakeep/` or `services/immich/`'s compose under Dockge) to confirm the exact mount syntax in use — driver name, share path, credential injection pattern. Match it. The actual NFS export root is `/var/nfs/shared/storage`; the SMB share name is **still TBD — see Q19** in `docs/11`.
|
||||||
|
|
||||||
|
**Hard-blocklist for any Nexa indexer / agent** (never read these paths or matching glob):
|
||||||
|
- `_sortMe/wallet/**` — contains PGP keys + bitcoin wallet files.
|
||||||
|
- Any path matching `*.gpg`, `*.asc`, `*.key`, `*.pem`, `id_rsa*`, `*wallet*`, `*.kdbx`, `*credentials*`, `*secret*`.
|
||||||
|
- The Nextcloud appdata dir (`services/nextcloud/appdata_*`) — Nextcloud-internal, not user content. Intel iGPU passthrough is *configured but currently broken* — see [docs/12 #26](./docs/12-optimization-opportunities.md#26).
|
||||||
- LXC 105 nextcloud — Nextcloud at `nc.nuclide.systems`.
|
- LXC 105 nextcloud — Nextcloud at `nc.nuclide.systems`.
|
||||||
- LXC 106 octoprint — currently *Exited*; flagged in [docs/11](./docs/11-open-questions.md#q13).
|
- LXC 106 octoprint — currently *Exited*; flagged in [docs/11](./docs/11-open-questions.md#q13).
|
||||||
- LXC 108 zoraxy — reverse proxy at `192.168.1.4:8000`, TLS for `*.nuclide.systems`.
|
- LXC 108 zoraxy — reverse proxy at `192.168.1.4:8000`, TLS for `*.nuclide.systems`.
|
||||||
|
|||||||
@@ -46,3 +46,15 @@ Iterative build-out, value-first. Each phase is shippable on its own.
|
|||||||
|
|
||||||
- [ ] **5.1 HA Voice** — Wake-word integration, Wyoming protocol.
|
- [ ] **5.1 HA Voice** — Wake-word integration, Wyoming protocol.
|
||||||
- [ ] **5.2 System monitoring** — Proxmox / Backrest alerts into the Memos system feed via ntfy.
|
- [ ] **5.2 System monitoring** — Proxmox / Backrest alerts into the Memos system feed via ntfy.
|
||||||
|
|
||||||
|
## Phase 6 — Nexa as homelab steward
|
||||||
|
|
||||||
|
**Focus:** Nexa actively maintains the homelab inventory instead of being told manually.
|
||||||
|
**Milestone:** Nexa runs a daily *drift report* — what changed, what's stale, what's missing — and turns it into actionable comments under a pinned `[STEWARD]` memo.
|
||||||
|
|
||||||
|
- [ ] **6.1 Inventory sync** — Nexa polls Dockge / `docker ps` / Proxmox API on a schedule and stores the running-state snapshot in GraphDB (`nexa:Service`, `nexa:LastSeen`, `nexa:DataPath`, …).
|
||||||
|
- [ ] **6.2 Drift detection** — diff against the documented desired state in this repo (`docs/12`, `CLAUDE.md`, the housekeeping campaign at #35–37). Emit findings as structured Memos comments: *"`open-webui` last used 38 d ago — retire?"*, *"Container `foo` has no `services/foo/` bind — schedule UNAS migration?"*.
|
||||||
|
- [ ] **6.3 Steward commands** — `#nexa:retire <svc>` (tar to `backup/`, stop, archive the stack), `#nexa:document <svc>` (template a CLAUDE.md entry), `#nexa:wishlist-status` (which entries in [docs/13](./13-information-wishlist.md) are still needed).
|
||||||
|
- [ ] **6.4 Self-update of docs** — when the user resolves an open question via Memos, Nexa proposes the corresponding edit to `docs/11-open-questions.md` as a PR-ready diff (the doc itself becomes a Nexa-managed surface).
|
||||||
|
|
||||||
|
This phase is what turns Nexa from "an assistant that answers questions" into "an assistant that takes care of its own runtime", and it's the natural home for the cross-cutting [housekeeping campaign](./12-optimization-opportunities.md#housekeeping-campaign-cross-cutting-do-together) — most of those items become semi-automatic once 6.1–6.3 ship.
|
||||||
|
|||||||
@@ -111,11 +111,18 @@ volumes:
|
|||||||
# SMB volume on UNAS — sidesteps NFS+unprivileged-LXC permission issues
|
# SMB volume on UNAS — sidesteps NFS+unprivileged-LXC permission issues
|
||||||
# (the same kind that bit the Nextcloud rollout). Stores embedding model
|
# (the same kind that bit the Nextcloud rollout). Stores embedding model
|
||||||
# caches, Qdrant snapshots, future GraphDB exports.
|
# caches, Qdrant snapshots, future GraphDB exports.
|
||||||
|
#
|
||||||
|
# The exact mount block (driver, share name, credential injection) MUST
|
||||||
|
# be copied verbatim from an existing stack on the same docker host —
|
||||||
|
# services/<svc>/ on UNAS is the homelab-wide docker config store and
|
||||||
|
# every container already follows the same SMB pattern. See Q19 in
|
||||||
|
# docs/11. The block below is the *shape* but the share name "<unas-share>"
|
||||||
|
# is a placeholder until verified against e.g. services/karakeep/compose.
|
||||||
tei-cache:
|
tei-cache:
|
||||||
driver: local
|
driver: local
|
||||||
driver_opts:
|
driver_opts:
|
||||||
type: cifs
|
type: cifs
|
||||||
device: //192.168.1.31/nexa/tei-cache
|
device: //192.168.1.31/<unas-share>/services/nexa/tei-cache
|
||||||
o: "username=${UNAS_USER},password=${UNAS_PASS},uid=1000,gid=1000,vers=3.0,iocharset=utf8"
|
o: "username=${UNAS_USER},password=${UNAS_PASS},uid=1000,gid=1000,vers=3.0,iocharset=utf8"
|
||||||
|
|
||||||
networks:
|
networks:
|
||||||
|
|||||||
@@ -51,6 +51,10 @@ Items that **block progress** and need a human decision before a workflow can be
|
|||||||
- **Verdict: no blockers for Phase-3.1.** Revisit once 3.4 GraphDB lands.
|
- **Verdict: no blockers for Phase-3.1.** Revisit once 3.4 GraphDB lands.
|
||||||
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
|
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
|
||||||
|
|
||||||
|
## Verification needed before Phase-3.1 deploy
|
||||||
|
|
||||||
|
- [ ] **Q19 — Confirm the SMB share name + mount syntax** by reading an existing Dockge stack (recommended: `services/karakeep/` or `services/immich/`). The SMB share name on UNAS that maps to the NFS root `/var/nfs/shared/storage` is currently a placeholder `<unas-share>` in [docs/09](./09-deployment.md). Required action: open the running Karakeep stack in Dockge, copy its `volumes:` block verbatim, swap `karakeep` → `nexa` in the path, drop into the Nexa stack. This locks Nexa to the **same convention every other container in the homelab uses** — same driver, same credentials secret, same uid/gid — and avoids reinventing storage just for Nexa.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Speed budget (Q3 follow-up)
|
## Speed budget (Q3 follow-up)
|
||||||
|
|||||||
@@ -20,34 +20,45 @@ Observations from the running infrastructure. Each item is independent — accep
|
|||||||
11. **Vaultwarden as the secret store** for Nexa secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, …) — read at bootstrap via the Bitwarden CLI from inside the docker host. Removes the need for a `.env` on disk.
|
11. **Vaultwarden as the secret store** for Nexa secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, …) — read at bootstrap via the Bitwarden CLI from inside the docker host. Removes the need for a `.env` on disk.
|
||||||
12. **AdGuard as DNS-based control plane.** Since AdGuard is the resolver for the LAN, you can rewrite `*.nuclide.systems` to `192.168.1.4` (Zoraxy) internally and avoid a hairpin via the WAN — already the case if AdGuard rewrite rules are set, worth verifying.
|
12. **AdGuard as DNS-based control plane.** Since AdGuard is the resolver for the LAN, you can rewrite `*.nuclide.systems` to `192.168.1.4` (Zoraxy) internally and avoid a hairpin via the WAN — already the case if AdGuard rewrite rules are set, worth verifying.
|
||||||
13. **Disk usage on LXC 104** is 47.7 % (Proxmox). Monitor; n8n execution logs and Dozzle history are the usual culprits. Setting `EXECUTIONS_DATA_PRUNE=true` and `EXECUTIONS_DATA_MAX_AGE=168` (7 days) on n8n keeps it bounded.
|
13. **Disk usage on LXC 104** is 47.7 % (Proxmox). Monitor; n8n execution logs and Dozzle history are the usual culprits. Setting `EXECUTIONS_DATA_PRUNE=true` and `EXECUTIONS_DATA_MAX_AGE=168` (7 days) on n8n keeps it bounded.
|
||||||
14. **Obsidian plugin embeddings collide with Nexa's.** The vault already runs Obsidian Copilot (`.copilot`, `.copilot-index`) and Smart Connections / Smart Composer (`.smart-env`, ~13 MB). They each embed the same notes into their own vector stores — three indexes for the same content. Nexa's value is the **cross-source** index (memos + mail + obsidian + RDF graph), so it has to embed independently, but the plugins could be retired once Nexa's RAG is satisfying. Track separately, decide later.
|
14. **Vector-store sprawl in the homelab — three competing indexes today.** Nexa is about to be the fourth. Track for eventual consolidation:
|
||||||
15. **Real-time Obsidian sync via `notify_push`.** Phase 3.1 polls WebDAV every 15 min (Q4 resolution). Once that works, swap to Nextcloud's `notify_push` app for sub-second propagation. One-line workflow change in n8n.
|
- **Obsidian Copilot** (`.copilot`, `.copilot-index`) and **Smart Connections / Smart Composer** (`.smart-env`, ~13 MB) — embed the Obsidian vault into two separate vector stores inside the vault.
|
||||||
16. **`assets/` is 186 MB of binaries** in the Obsidian vault — worth a glance to confirm it's mostly images (Phase-3.2 visual queue) rather than something that should live in Nextcloud Files proper.
|
- **Paperless-AI ChromaDB** (`services/paperless-ai/chromadb/`) — embeds scanned documents from Paperless-ngx for Q&A.
|
||||||
|
- **Nexa Qdrant** (`nexa_knowledge_text`, soon `_visual`) — embeds the cross-source corpus.
|
||||||
|
|
||||||
|
Long-term, Nexa is the natural single source of truth (it sees memos + mail + obsidian + RDF graph). Once its RAG is satisfying, retire the Obsidian-plugin indexes and consider letting Nexa **read from** Paperless-AI's ChromaDB rather than re-embed PDFs (one-line ChromaDB query, much cheaper than redoing OCR-to-vector). Track but don't act yet.
|
||||||
|
|
||||||
|
15. **Retire `open-webui`.** Confirmed stale by the user — only LobeHub is in active use as the LiteLLM chat front-end (`ai.nuclide.systems`). Stop the container, tar `services/open-webui/` into `backup/open-webui/`, then remove the stack. Frees ~500 MB RAM + a couple of GB of model cache.
|
||||||
|
|
||||||
|
16. **`services/siyuan/workspace/` is dead data.** SiYuan retired, content migrated to Obsidian (Nextcloud `Notizen/`). Keep a final tar in `backup/siyuan/`, then `rm -rf services/siyuan/`. Frees disk + removes a "is this still authoritative?" question for future agents (and for Nexa's classifier if it ever sees the path).
|
||||||
|
17. **Real-time Obsidian sync via `notify_push`.** Phase 3.1 polls WebDAV every 15 min (Q4 resolution). Once that works, swap to Nextcloud's `notify_push` app for sub-second propagation. One-line workflow change in n8n.
|
||||||
|
18. **`assets/` is 186 MB of binaries** in the Obsidian vault — worth a glance to confirm it's mostly images (Phase-3.2 visual queue) rather than something that should live in Nextcloud Files proper.
|
||||||
|
19. **Paperless-AI as a Phase-2.x triage helper for `_sortMe/Downloads/`.** UNAS shows ~230 PDFs/docs in `_sortMe/Downloads/` plus another batch under `_sortMe/Anne/`. Paperless-AI (already running) can ingest, OCR, classify and route them; Nexa's role is to *delegate* — wire a workflow that posts a batch to Paperless-AI and reports the result back via Memos.
|
||||||
|
20. **`media/Recipes/`** has ~300 individually-named recipe folders. Once Phase-3.1 text indexing works, this becomes a high-quality test corpus for `#nexa:ask` (e.g. "what was that gochujang noodle recipe with no anchovies?"). Out-of-scope for the deployment plan, but a satisfying first user-facing win.
|
||||||
|
|
||||||
## Proxmox host (NUC 14 Pro) tuning
|
## Proxmox host (NUC 14 Pro) tuning
|
||||||
|
|
||||||
Observed from the node summary: 22 threads, 62 GiB RAM (32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35% used), load avg <2.0, IO delay 0.04%, kernel 6.17.13-4-pve, PVE 9.1.9, EFI. Suggestions in priority order:
|
Observed from the node summary: 22 threads, 62 GiB RAM (32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35% used), load avg <2.0, IO delay 0.04%, kernel 6.17.13-4-pve, PVE 9.1.9, EFI. Suggestions in priority order:
|
||||||
|
|
||||||
17. **Cap ZFS ARC.** Default is 50% of RAM (~31 GiB); current actual ~24 GiB. For a node that runs services rather than a pure storage box, capping at 8–12 GiB frees ~12–16 GiB for guests without measurable IO impact (disk is 1.64 TiB and 0.35% used — there's nothing hot to cache):
|
21. **Cap ZFS ARC.** Default is 50% of RAM (~31 GiB); current actual ~24 GiB. For a node that runs services rather than a pure storage box, capping at 8–12 GiB frees ~12–16 GiB for guests without measurable IO impact (disk is 1.64 TiB and 0.35% used — there's nothing hot to cache):
|
||||||
```
|
```
|
||||||
echo 'options zfs zfs_arc_max=8589934592' > /etc/modprobe.d/zfs.conf # 8 GiB
|
echo 'options zfs zfs_arc_max=8589934592' > /etc/modprobe.d/zfs.conf # 8 GiB
|
||||||
update-initramfs -u
|
update-initramfs -u
|
||||||
```
|
```
|
||||||
Reboot or `echo 8589934592 > /sys/module/zfs/parameters/zfs_arc_max` to apply live.
|
Reboot or `echo 8589934592 > /sys/module/zfs/parameters/zfs_arc_max` to apply live.
|
||||||
18. **Enable KSM (Kernel Same-page Merging).** With ~40 docker containers + several LXCs, KSM typically frees 1–3 GiB by deduplicating identical memory pages. Currently `KSM sharing: 0 B` in the summary. PVE has `ksmtuned` available — `systemctl enable --now ksmtuned`.
|
22. **Enable KSM (Kernel Same-page Merging).** With ~40 docker containers + several LXCs, KSM typically frees 1–3 GiB by deduplicating identical memory pages. Currently `KSM sharing: 0 B` in the summary. PVE has `ksmtuned` available — `systemctl enable --now ksmtuned`.
|
||||||
19. **Suppress the `pve-no-subscription` repository warning** — either accept it (it's a homelab) and apply the `pve-no-subscription-warning` polyfill, or move to the enterprise repo. Pure cosmetic, but the orange banner in the UI is noise.
|
23. **Suppress the `pve-no-subscription` repository warning** — either accept it (it's a homelab) and apply the `pve-no-subscription-warning` polyfill, or move to the enterprise repo. Pure cosmetic, but the orange banner in the UI is noise.
|
||||||
20. **Swap is 31 GiB on a 62 GiB box with ZFS root** — almost certainly oversized. Drop `vm.swappiness` to 10 (`sysctl -w vm.swappiness=10` + persist) so swap is only used under genuine pressure, and consider shrinking the swap volume if disk-layout permits.
|
24. **Swap is 31 GiB on a 62 GiB box with ZFS root** — almost certainly oversized. Drop `vm.swappiness` to 10 (`sysctl -w vm.swappiness=10` + persist) so swap is only used under genuine pressure, and consider shrinking the swap volume if disk-layout permits.
|
||||||
21. **Verify scheduled ZFS scrub is enabled.** PVE ships `zfs-scrub-monthly@.timer` — `systemctl list-timers | grep zfs` to confirm. Cheap insurance on a 1.6 TiB pool.
|
25. **Verify scheduled ZFS scrub is enabled.** PVE ships `zfs-scrub-monthly@.timer` — `systemctl list-timers | grep zfs` to confirm. Cheap insurance on a 1.6 TiB pool.
|
||||||
22. **SMART monitoring on the NVMe.** `smartctl -a /dev/nvme0` should be regularly polled; PVE's notification target can ntfy on degradation. Combine with the existing `nexa.system` ntfy topic (optimization #4) so disk-health alerts land in the same Memos system feed as everything else.
|
26. **SMART monitoring on the NVMe.** `smartctl -a /dev/nvme0` should be regularly polled; PVE's notification target can ntfy on degradation. Combine with the existing `nexa.system` ntfy topic (optimization #4) so disk-health alerts land in the same Memos system feed as everything else.
|
||||||
23. **NTP source via AdGuard.** AdGuard already resolves DNS for the LAN; pointing the host's `systemd-timesyncd` at `pool.ntp.org` resolved through AdGuard avoids any external dependency for time. One-line change in `/etc/systemd/timesyncd.conf`.
|
27. **NTP source via AdGuard.** AdGuard already resolves DNS for the LAN; pointing the host's `systemd-timesyncd` at `pool.ntp.org` resolved through AdGuard avoids any external dependency for time. One-line change in `/etc/systemd/timesyncd.conf`.
|
||||||
24. **`fstrim.timer` enabled** for the SSD pool — verify with `systemctl status fstrim.timer`. Default-on in modern PVE, but quick to confirm.
|
28. **`fstrim.timer` enabled** for the SSD pool — verify with `systemctl status fstrim.timer`. Default-on in modern PVE, but quick to confirm.
|
||||||
25. **Watchdog config** is irrelevant for a single-node setup (HA is the use case), so leave the default. Mentioned only so future agents don't add it speculatively.
|
29. **Watchdog config** is irrelevant for a single-node setup (HA is the use case), so leave the default. Mentioned only so future agents don't add it speculatively.
|
||||||
|
|
||||||
## Docker LXC (104) — observations & wins
|
## Docker LXC (104) — observations & wins
|
||||||
|
|
||||||
Confirmed allocation: 16 CPU, 31.25 GiB RAM (7.86 GiB used / 25%), 8 GiB swap (idle), **200 GiB boot disk at 47.7% used** — disk pressure outranks RAM pressure.
|
Confirmed allocation: 16 CPU, 31.25 GiB RAM (7.86 GiB used / 25%), 8 GiB swap (idle), **200 GiB boot disk at 47.7% used** — disk pressure outranks RAM pressure.
|
||||||
|
|
||||||
26. **Fix the Intel iGPU passthrough.** The container's own notes flag the binding as "likely failing". Once `/dev/dri/{card0,renderD128}` is visible inside the LXC, both TEI and `infinity` can run embeddings on the Arc iGPU via OpenVINO / IPEX-LLM — typically **5–10× faster** than CPU. Equally, Immich's CLIP can be GPU-accelerated. The required config in `/etc/pve/lxc/104.conf`:
|
30. **Fix the Intel iGPU passthrough.** The container's own notes flag the binding as "likely failing". Once `/dev/dri/{card0,renderD128}` is visible inside the LXC, both TEI and `infinity` can run embeddings on the Arc iGPU via OpenVINO / IPEX-LLM — typically **5–10× faster** than CPU. Equally, Immich's CLIP can be GPU-accelerated. The required config in `/etc/pve/lxc/104.conf`:
|
||||||
```
|
```
|
||||||
lxc.cgroup2.devices.allow: c 226:0 rwm
|
lxc.cgroup2.devices.allow: c 226:0 rwm
|
||||||
lxc.cgroup2.devices.allow: c 226:128 rwm
|
lxc.cgroup2.devices.allow: c 226:128 rwm
|
||||||
@@ -60,16 +71,45 @@ Confirmed allocation: 16 CPU, 31.25 GiB RAM (7.86 GiB used / 25%), 8 GiB swap (i
|
|||||||
lxc.idmap: g 104 104 1 # render group on host
|
lxc.idmap: g 104 104 1 # render group on host
|
||||||
```
|
```
|
||||||
Then inside the container: `usermod -aG video,render <docker-user>` and run TEI with `--device cuda` replaced by the OpenVINO build (`text-embeddings-inference:cpu-1.5-openvino`). Not needed for Phase-3.1's volume but a clean upgrade path.
|
Then inside the container: `usermod -aG video,render <docker-user>` and run TEI with `--device cuda` replaced by the OpenVINO build (`text-embeddings-inference:cpu-1.5-openvino`). Not needed for Phase-3.1's volume but a clean upgrade path.
|
||||||
27. **Push persistent data to UNAS via SMB-backed docker volumes.** The LXC also has a Proxmox-level NFS mount at `/mnt/pve/unas` (19.4 TiB / 16.3 TiB free), but the user has already lived through the unprivileged-LXC + NFS + per-container-write trap with Nextcloud. **For Nexa volumes the default is SMB**, mounted directly as a docker volume (driver `local`, type `cifs`) with explicit `username=`/`uid=`/`gid=` — that bypasses LXC uid-mapping entirely. Targets:
|
31. **Match the homelab-wide UNAS storage convention.** Every existing docker stack on LXC 104 already binds its persistent data into `services/<svc>/` on UNAS via SMB (immich, karakeep, nextcloud, ntfy, open-webui, paperless-ai, pocketid, shelfmark, siyuan, stremio, traccar, vaultwarden, gluetun all follow this). Nexa **must** follow the same pattern at `services/nexa/`. Do not invent a parallel layout. Targets:
|
||||||
- **Qdrant data dir** (`qdrant_scientific`) → SMB share `//192.168.1.31/nexa/qdrant`. Move it before the visual collection lands and starts adding GBs.
|
- `services/nexa/qdrant/` — Qdrant data dir (move before the visual collection lands).
|
||||||
- **GraphDB repo** (Phase 3.4) → `//192.168.1.31/nexa/graphdb`. The 4-GB heap is in RAM, but the on-disk RDF store will grow.
|
- `services/nexa/tei-cache/` — embedding model cache (keeps multi-GB models off the boot disk).
|
||||||
- **Embedding model caches** (TEI/infinity) → `//192.168.1.31/nexa/tei-cache` — keeps the boot disk free of multi-GB model files.
|
- `services/nexa/graphdb/` — Phase-3.4 RDF store.
|
||||||
- **Daily Qdrant snapshots** → `//192.168.1.31/nexa/snapshots/qdrant/<date>/` before rsync to S3 (Phase 3.3 — Q12 still open). With UNAS in the path, S3 becomes a *colder* tier rather than the only copy.
|
- `backup/nexa/snapshots/qdrant/<date>/` — daily Qdrant snapshots before rsync to S3 (Phase 3.3 — Q12 open). Mirrors the `backup/<svc>/` pattern used by Home Assistant / Immich.
|
||||||
- **n8n executions retention** still useful: `EXECUTIONS_DATA_PRUNE=true`, `EXECUTIONS_DATA_MAX_AGE=168`.
|
- **n8n executions retention**: `EXECUTIONS_DATA_PRUNE=true`, `EXECUTIONS_DATA_MAX_AGE=168`.
|
||||||
- **Monthly `docker image prune --all --filter "until=720h"`** to clear dangling layers.
|
- **Monthly `docker image prune --all --filter "until=720h"`** to clear dangling layers.
|
||||||
|
|
||||||
Credentials: store `UNAS_USER` / `UNAS_PASS` in Vaultwarden ([12/#13](#13)) rather than a `.env` on disk; inject via a secrets agent or per-stack `.env` that is itself stored on the SMB share with restricted ACLs. The host-side `/mnt/pve/unas` NFS mount stays useful for admin tasks (rsync to/from snapshots, manual backups) — just not for per-container write paths.
|
Why SMB and not the host's NFS mount: the user already lived through the unprivileged-LXC + NFS + per-container-write trap with Nextcloud. SMB mounted directly as a docker volume (driver `local`, type `cifs`) with explicit `username=`/`uid=`/`gid=` bypasses LXC uid-mapping entirely. **Pre-deploy step:** read the volume block from an existing stack (Karakeep is small and clean) and copy it verbatim — this guarantees Nexa uses the same SMB share name, credential pattern, uid/gid as the rest of the homelab. Tracked as [Q19](./11-open-questions.md#q19).
|
||||||
28. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap.
|
|
||||||
29. **Stacks live in Dockge, not raw compose.** The LXC is the Dockge flavor of helper-scripts. Deployment writes a stack named `nexa` in the Dockge UI; the compose YAML is stored under `/opt/stacks/nexa/` (Dockge default) and edited from the web UI. This means [docs/09](./09-deployment.md) "deploy" snippets translate to "paste this into Dockge → save → start", not `docker compose up -d` over SSH.
|
Credentials (`UNAS_USER` / `UNAS_PASS`) go in Vaultwarden ([12/#11](#11)), injected at deploy. The host-side `/mnt/pve/unas` NFS mount stays useful for admin tasks (rsync, manual backups) — not for per-container write paths.
|
||||||
30. **Untriaged local mail on the LXC.** Console shows `You have new mail.` at login — the system mail spool on `/var/mail/root` has unread messages, almost always cron job failures. `mailx` or `mutt` to inspect, then either fix the failing job or send the spool to `nexa.system` ntfy via a tiny aliases entry (`root: |/usr/local/bin/spool-to-ntfy.sh`).
|
32. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap.
|
||||||
28. **The LXC has its own 8 GiB swap.** Combined with the host's 31 GiB, that's a lot of swap for guests that should never page. Drop the LXC swap allocation to 1–2 GiB (`pct set 104 -swap 2048`) — frees disk on the LVM-thin pool and forces issues to surface earlier rather than silently swap.
|
33. **Stacks live in Dockge, not raw compose.** The LXC is the Dockge flavor of helper-scripts. Deployment writes a stack named `nexa` in the Dockge UI; the compose YAML is stored under `/opt/stacks/nexa/` (Dockge default) and edited from the web UI. This means [docs/09](./09-deployment.md) "deploy" snippets translate to "paste this into Dockge → save → start", not `docker compose up -d` over SSH.
|
||||||
|
34. **Untriaged local mail on the LXC.** Console shows `You have new mail.` at login — the system mail spool on `/var/mail/root` has unread messages, almost always cron job failures. `mailx` or `mutt` to inspect, then either fix the failing job or send the spool to `nexa.system` ntfy via a tiny aliases entry (`root: |/usr/local/bin/spool-to-ntfy.sh`).
|
||||||
|
|
||||||
|
## Housekeeping campaign (cross-cutting, do together)
|
||||||
|
|
||||||
|
These three are inter-related — picking them up as one campaign is cheaper than chasing each individually, because the audit step is the same. They also map cleanly onto **[Phase 6 (Nexa as homelab steward)](./02-roadmap.md#phase-6--nexa-as-homelab-steward)**: once Nexa can poll Dockge, diff against the documented state and emit actionable findings, items #35–37 become semi-automatic — Nexa proposes the migrations rather than us hunting them down.
|
||||||
|
|
||||||
|
35. **Consolidate Postgres instances.** Today there are at least four independent Postgres containers running — visible from Dockge: `immich_postgres`, `lobe-postgres`, `litellm_db`, `paperless-ngx-db-1` (Karakeep uses Meilisearch + maybe SQLite, separate). Each idles around 100–300 MB RAM and has its own backup story. Two paths:
|
||||||
|
- **Single shared cluster** (`pgsql` container with one role per app, one database per app). Most modern apps support `DATABASE_URL` → just point them at the shared instance. Saves ~600 MB–1 GB RAM and consolidates backups to one pg_dump cron.
|
||||||
|
- **Or** migrate the smallest ones to SQLite (LiteLLM has SQLite mode; Paperless does too) and keep Postgres only for Immich. Same outcome, less migration risk.
|
||||||
|
|
||||||
|
Pre-step: list every running container with `docker ps --format '{{.Names}}\t{{.Image}}' | grep -i 'postgres\|mariadb\|mysql'` to inventory exactly what's running.
|
||||||
|
|
||||||
|
36. **UNAS-integration audit.** `services/<svc>/` on UNAS is the homelab convention but not every container follows it yet. Walk every Dockge stack and check the `volumes:` block. Suspected non-compliant (need verification):
|
||||||
|
- `qdrant_scientific` — vector data possibly on local boot disk; **critical to verify** before Phase-3.1 piles on a `nexa_knowledge_text` collection.
|
||||||
|
- `litellm_db`, `lobe-postgres`, `lobe-redis` — DB containers, persistent state.
|
||||||
|
- `memos` — the user-facing notes app; loss = data loss.
|
||||||
|
- `n8n` — workflows + executions DB.
|
||||||
|
- `audiobookshelf` — listening progress + library metadata.
|
||||||
|
- the `*-mcp` containers — likely stateless (cache only), low priority.
|
||||||
|
|
||||||
|
Output: a one-page table `service | persistent? | currently bound to | should be bound to`. Then migrate the non-compliant ones one-by-one (stop → rsync data to `services/<svc>/` on UNAS → re-create stack with the new bind → verify → keep the old volume for 7 days as a rollback). Lock the convention for any new stack going forward.
|
||||||
|
|
||||||
|
37. **S3 archive tier (`s3.nuclide.systems`).** Currently UNAS holds backups in `backup/<svc>/` and there's no cold offsite. Make `s3.nuclide.systems` the canonical *cold* tier:
|
||||||
|
- **Restic / borg** repos targeting S3 for: `backup/home-assistant/`, `backup/immich/`, `backup/nextcloud/`, future `backup/nexa/`. Backrest already orchestrates Borg — just add an S3 destination.
|
||||||
|
- **Qdrant + GraphDB snapshots** flow UNAS → S3 weekly (daily on UNAS, weekly to S3 = cheap, keeps the most recent locally for fast restore).
|
||||||
|
- **Immich photo originals** to S3 if the photo library grows past UNAS comfort. Immich has native S3 support via its `IMMICH_*` env vars.
|
||||||
|
- Resolves [Q12](./11-open-questions.md#q12) — once the bucket name + access key are set, every other archive just adds a target.
|
||||||
|
|
||||||
|
This is the right time to do it because we're about to add Nexa snapshots; designing the path for *all* services rather than just Nexa avoids one-offs.
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
# 13 — Information Wishlist
|
||||||
|
|
||||||
|
What additional system inventory would sharpen future decisions, packaged as the **smallest set of paste-and-run tasks** that still answers everything material. Each task is independent — run any subset, in any order.
|
||||||
|
|
||||||
|
Each task header tells you **exactly where to run it** (which shell or which UI). Output goes into a Memos draft, a comment in this conversation, or `docs/inventory/<NN>-<slug>.txt` — whichever is easiest.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 1 — Stack inventory + reference compose **(highest leverage)**
|
||||||
|
|
||||||
|
📍 **Where:** Proxmox UI → Datacenter → `nuc` → LXC `104 (docker)` → **Console** (or `ssh root@192.168.1.40` from your PC).
|
||||||
|
|
||||||
|
Locks **Q19** (the SMB share name + driver options Nexa must mirror) and replaces screenshot-derived guesses about what's running.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker ps -a --format '{{.Names}}\t{{.Image}}\t{{.Status}}' | sort
|
||||||
|
ls /opt/stacks/
|
||||||
|
cat /opt/stacks/karakeep/compose.yaml # or any clean small stack
|
||||||
|
```
|
||||||
|
|
||||||
|
**Unblocks:** Q19, the housekeeping migration template (#36), Phase-3.1 deployment YAML.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 2 — Storage map (Proxmox + docker)
|
||||||
|
|
||||||
|
📍 **Where:** *Two shells* — first the **Proxmox host** (Datacenter → `nuc` → **Shell**, or `ssh root@192.168.1.20`), then back into the **LXC 104 console**.
|
||||||
|
|
||||||
|
Tells us which Proxmox storage backs what, and which docker volumes are local vs. SMB.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# (a) on the Proxmox host (192.168.1.20):
|
||||||
|
cat /etc/pve/storage.cfg
|
||||||
|
cat /etc/pve/lxc/104.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# (b) on LXC 104 (192.168.1.40):
|
||||||
|
docker volume ls
|
||||||
|
ls -l /dev/dri/ # confirms whether iGPU passthrough actually works
|
||||||
|
```
|
||||||
|
|
||||||
|
**Unblocks:** optimization #30 (iGPU), #36 (UNAS audit), B6/B7/C17.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 3 — Postgres / DB workload inventory
|
||||||
|
|
||||||
|
📍 **Where:** **LXC 104 console** (same shell as Task 1).
|
||||||
|
|
||||||
|
Direct input to housekeeping #35 (consolidate postgres instances).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
for c in $(docker ps --format '{{.Names}}' | grep -iE 'postgres|mariadb|mysql|_db$'); do
|
||||||
|
echo "=== $c ==="
|
||||||
|
docker exec "$c" sh -c 'psql -U postgres -l 2>/dev/null || mysql -e "show databases" 2>/dev/null'
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**Unblocks:** #35 (consolidate vs. migrate-to-SQLite decision).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 4 — LiteLLM model list for the Nexa key
|
||||||
|
|
||||||
|
📍 **Where:** *Either* the **LiteLLM admin UI** (one screenshot) *or* any shell with `$SAIA_API_KEY` exported.
|
||||||
|
|
||||||
|
- **Screenshot path:** open `https://ai.nuclide.systems` (Lobehub) → Settings → Model List, filtered to the Nexa virtual key. Screenshot the model rows.
|
||||||
|
- **CLI path:** any shell on your PC or the LXC, with `SAIA_API_KEY` set:
|
||||||
|
```bash
|
||||||
|
curl -sH "Authorization: Bearer $SAIA_API_KEY" \
|
||||||
|
https://ai.nuclide.systems/v1/models | jq '.data[].id'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Unblocks:** Q18 follow-up — which embedding model SAIA proxies and at what dim, so we can decide if any flow can short-circuit TEI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 5 — S3 archive credentials
|
||||||
|
|
||||||
|
📍 **Where:** **`s3.nuclide.systems` admin UI** in your browser (it's already proxied via Zoraxy — same login as the rest of the homelab).
|
||||||
|
|
||||||
|
Walk to: **Buckets** → either pick an existing Nexa-suitable bucket or create one called `nexa` → note the bucket name. Then **Access Keys** → create a key named `nexa-snapshots` with read/write on that bucket → drop the access-key + secret into Vaultwarden under "Nexa S3", and reply here with **just the bucket name** (the secret stays in Vaultwarden).
|
||||||
|
|
||||||
|
**Unblocks:** Q12, housekeeping #37 (S3 archive tier), Phase-3.3.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 6 — Reverse-proxy + DNS authority *(only if needed)*
|
||||||
|
|
||||||
|
📍 **Where:** two browser UIs.
|
||||||
|
|
||||||
|
- **Zoraxy** (`http://192.168.1.4:8000`) → HTTP Proxy → either click "Export" if available, or take a full screenshot of the table.
|
||||||
|
- **AdGuard** (`http://192.168.1.20` LXC 102 web UI) → Filters → DNS Rewrites → screenshot.
|
||||||
|
|
||||||
|
Skip unless we hit a routing surprise during Phase 1.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## When something else is needed
|
||||||
|
|
||||||
|
The smaller items (n8n credentials list, Memos webhook config, smartctl, sample of `_sortMe/`, what cron is failing) only matter when we touch that specific area. The agent will ask for them at the moment they're needed, with the same "📍 Where" framing.
|
||||||
|
|
||||||
|
## Self-serve once Phase 6 ships
|
||||||
|
|
||||||
|
Once [Phase 6 — Nexa as homelab steward](./02-roadmap.md#phase-6--nexa-as-homelab-steward) lands, Nexa runs Tasks 1–3 itself on a schedule and folds the results into a daily drift report. This wishlist becomes a Nexa-managed surface (`#nexa:wishlist-status`) rather than something the user has to remember.
|
||||||
@@ -22,6 +22,7 @@ This is the documentation entry point. Read top-to-bottom for first-time setup,
|
|||||||
| 10 | [Operations](./10-operations.md) | You need backup, monitoring or troubleshooting. |
|
| 10 | [Operations](./10-operations.md) | You need backup, monitoring or troubleshooting. |
|
||||||
| 11 | [Open Questions (user-info-required)](./11-open-questions.md) | Items the user still has to answer before progress. |
|
| 11 | [Open Questions (user-info-required)](./11-open-questions.md) | Items the user still has to answer before progress. |
|
||||||
| 12 | [Optimization Opportunities](./12-optimization-opportunities.md) | Ideas worth considering for the wider homelab. |
|
| 12 | [Optimization Opportunities](./12-optimization-opportunities.md) | Ideas worth considering for the wider homelab. |
|
||||||
|
| 13 | [Information Wishlist](./13-information-wishlist.md) | What additional system inventory would sharpen future decisions. |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user