Commit Graph

14 Commits

Author SHA1 Message Date
Claude e0061ee173 Defer Q12 and Q20: backup-tier decisions parked until Phase 3.3 is up next 2026-05-05 05:18:58 +00:00
Claude a8446f8371 Q19 resolved: storage convention is plain host bind-mount, not SMB volume
The Karakeep compose user shared confirms every container in LXC 104
uses host bind-mounts of /mnt/pve/unas/services/<svc>/<vol> directly.
No driver_opts, no CIFS, no per-volume credentials. The earlier
SMB-as-docker-volume proposal in this repo was over-fitting to the
Nextcloud-specific NFS issue and is withdrawn.

The Nextcloud failure was Nextcloud-specific: its setup tooling chowns
the data dir to www-data, which fails against root_squash exports.
Normal containers don't trigger that path, so plain bind-mounts work
fine.

Changes:

- docs/09 §Step 3: compose block dropped from ~30 lines (volume,
  driver_opts, CIFS device, credentials env) to a one-line bind-mount
  + env_file. Added a pre-deploy `mkdir -p` for the UNAS service +
  backup paths. Explanatory note retained for future agents.
- docs/12 #27 marked withdrawn pointing to #31. #31 rewritten with the
  bind-mount pattern (was SMB-default). #36 (UNAS audit) now looks for
  `- /var/lib/docker/...` or anonymous volumes as the non-compliant
  signal.
- CLAUDE.md storage block: bind-mount is the standard, SMB is
  documented escape-hatch only.
- docs/11: Q19 moved into Resolved with the new framing. Q12 + Q20
  grouped under a new "## Backups" section header (the now-empty
  "Verification needed" section removed).
- docs/13 Task 1 struck through with done-marker pointing to docs/09.

Net: Phase-3.1 deployment YAML is now actually deployable as written.
2026-05-05 05:01:48 +00:00
Claude dfd9aa7d2a 3-2-1 backup tiering with off-site cold (Jottacloud candidate)
- s3.nuclide.systems is on-site only — recasting it as the WARM tier,
  not the disaster-recovery copy. Off-site cold becomes a separate
  decision tracked as Q20.
- docs/12 #37 expanded with full provider comparison: Jottacloud
  Unlimited (~€9.50/mo, EU/Norway, soft-cap ~5 TB) is the user's
  stated candidate and a reasonable fit for the current 2 TB scale
  with ~10 years of headroom. Alternatives priced: Hetzner Storage
  Box, Backblaze B2, rsync.net, Storj. Always restic/rclone-crypt
  before upload.
- Off-site priority list pinned: Immich originals + documents +
  Nextcloud user data + Vaultwarden DB go off-site; movies / music /
  ROMs / derived caches don't.
- docs/10 backup section rewritten as the 3-2-1 pipeline (source →
  UNAS native snapshot → warm → encrypted cold). Nexa-specific
  snapshot rows updated for the new collection names.
- docs/11 Q20 added with the decision criteria.
- CLAUDE.md: backup model pinned so future agents don't propose
  alternative paths.
2026-05-05 04:50:43 +00:00
Claude e8f0d9a2c2 Memory retention design + open-question re-asker + UNAS Pro specifics
Two design questions answered:

1. Web/news/Karakeep INTO long-term memory? Yes, but with per-source TTL.
   docs/08 gains a "Memory sources & retention" section that pins TTLs:
   memo/obsidian/karakeep = permanent, mail = 365d, mail_digest = 90d,
   rss = 30d, web_search = 90d, system = 30d. Every Qdrant point carries
   payload.expires_at; a daily prune workflow honours it. New commands
   in docs/05: #nexa:learn --permanent, #nexa:forget, #nexa:retain,
   #nexa:ask --web (SearXNG → crawl4ai-mcp → markitdown-mcp → embed).
   Phase 2.4 added to roadmap.

2. Re-ask unanswered open questions. Backoff schedule (3d → 7d → 21d →
   60d) tracked in GraphDB per question. Surface ONE question per day
   in the morning digest, but only when the digest is otherwise short
   (capacity guard < 800 chars). User reply parsed → question auto-
   resolved → docs/11 diff proposed (Phase 6.4 hook). New commands in
   docs/05: #nexa:digest, #nexa:remind, #nexa:answered. Phase 5.3 added.

UNAS Pro details from the UniFi Drive dashboard:
- It's a Ubiquiti UNAS Pro (UniFi Drive 4.1.16 on UniFi OS 5.0.17),
  SFP+, RAID 6, 19.96 TiB raw, 2.05 TiB used. Recorded in CLAUDE.md.
- SMB native paths: smb://192.168.1.31/<share> (mac) / \\192.168.1.31\
  <share> (win). UniFi recommends SMB as the modern path — matches our
  decision to default Nexa volumes to SMB.
- ⚠️ Storage-pool snapshots NOT configured ("Click to Setup"). Added as
  optimization #38: highest-leverage data-protection change in the
  homelab right now. Daily + weekly UniFi Drive snapshot, native, no
  agent. RAID 6 doesn't protect against rm -rf or accidental mass-
  delete; snapshots do.
- #39: Nexa workflows that mutate large state can use the same native
  snapshots for fast rollback (pre-snapshot → operate → verify).
2026-05-05 04:45:29 +00:00
Claude 74368100a3 Flip Dockge → Arcane: Arcane is the active docker manager, Dockge is stale
The LXC was originally provisioned with the Dockge helper-script template,
but the user moved on to Arcane. Dozzle stays as the log viewer (different
role, not redundant).

- docs/09 step 3: deployment goes via Arcane UI (not Dockge); reworded
  the deploy block accordingly.
- docs/11 Q19: read the reference compose from Arcane, not Dockge.
- docs/12 #33: was "stacks live in Dockge"; now "Arcane manages stacks,
  Dockge is stale, retire it" with the same tar-then-remove pattern as
  SiYuan and Open-WebUI.
- docs/12 housekeeping campaign + #36: "walk every Arcane stack" rather
  than Dockge.
- docs/13 Task 1: stack inventory comes from the Arcane UI (compose.yaml +
  .env screenshot/copy) rather than `ls /opt/stacks/` which was the Dockge
  default. The shell command for `docker ps -a` stays.
- docs/02 Phase 6.1: Nexa polls Arcane (not Dockge) for inventory sync.
- CLAUDE.md infra block: Arcane is the active manager, Dozzle is the log
  viewer, Dockge is stale; added services/dockge/ to the stale list
  alongside siyuan and open-webui.
2026-05-04 22:31:10 +00:00
Claude 250a5cb2a6 Pin homelab-wide UNAS storage convention; add Phase 6 + info wishlist
- UNAS layout decoded from the user's tree: services/<svc>/ is the
  homelab-wide docker config store (every container follows the same
  pattern — immich, karakeep, nextcloud, ntfy, paperless-ai, pocketid,
  shelfmark, stremio, traccar, vaultwarden, gluetun). Nexa MUST follow
  the same pattern at services/nexa/. backup/<svc>/ for snapshots.
  Pinned in CLAUDE.md and docs/12 #31.
- Stale services flagged for retirement: services/siyuan/ (migrated to
  Obsidian) and services/open-webui/ (unused, only LobeHub is alive).
  Drops the "two LLM UIs" item (#15) — it's now "retire open-webui".
- Hard-blocklist for any Nexa indexer pinned in CLAUDE.md:
  _sortMe/wallet/**, *.gpg/asc/key/pem/kdbx/credentials/secret, plus
  the Nextcloud appdata dir.
- Three new "future user-facing wins" surfaced from the tree:
  Paperless-AI as a Phase-2.x triage helper for _sortMe/Downloads/,
  media/Recipes/ (~300 entries) as the showcase RAG corpus, and
  Paperless-AI's existing ChromaDB as a potential read-from source
  rather than re-embedding scanned docs.
- Housekeeping campaign in docs/12 §35-37: consolidate postgres,
  audit UNAS-everywhere, S3 archive tier on s3.nuclide.systems.
- Phase 6 added to the roadmap: "Nexa as homelab steward". Polls
  Dockge/docker/Proxmox, diffs vs documented state, emits a daily
  drift report. Most of the housekeeping campaign becomes
  semi-automatic once 6.1-6.3 ship.
- New docs/13-information-wishlist.md packages the still-needed
  inventory as 5-6 paste-and-run tasks, each with explicit "📍 Where"
  markers (which shell or which UI) and what it unblocks. Highest
  leverage = Task 1 (read an existing compose stack to lock Q19).
- docs/index.md TOC extended to 13 rows.

Also fixed numbering drift in docs/12 (duplicate #28, missing #17/#18)
and added Q19 to docs/11 covering the SMB-share verification step.
2026-05-04 22:24:24 +00:00
Claude a0a67eefab SMB-default for Nexa volumes (user confirmed NFS+unpriv-LXC writes break)
User's Nextcloud rollout already hit the classic trap of unprivileged-LXC
write attempts against an NFS share, so don't repeat that pattern for
Nexa. Updated docs to:

- Use SMB volumes mounted directly as docker volumes (driver: local,
  type: cifs) with explicit username/uid/gid — bypasses LXC uid-mapping
  entirely. Concrete YAML in docs/09 step 3 against
  //192.168.1.31/nexa/<volume>.
- Keep the host-side /mnt/pve/unas NFS mount for admin-only use (rsync,
  manual snapshot copies). Don't bind-mount it into containers.
- docs/12 #27 reordered to lead with SMB targets (qdrant, graphdb,
  tei-cache, snapshots) and only mention NFS for host admin.
- CLAUDE.md storage block flipped to "SMB = default per-container
  storage; NFS = host-side admin only".
- Also captured: deployments go through the Dockge UI (LXC is
  the Dockge flavor of helper-scripts), not raw `docker compose up`.
- New optimization #30 about the local mail spool nag on the docker LXC
  ("You have new mail." at login).
2026-05-04 22:14:12 +00:00
Claude 12ed4426e0 Refine capacity numbers from LXC 104 summary; flag iGPU passthrough fix
- LXC 104 (docker) confirmed: 16 CPU, 31.25 GiB RAM (7.86 used / 25%),
  200 GiB boot disk at 47.7% — disk is the real near-term constraint,
  not RAM. Numbers propagated into Q11 (docs/11), docs/03 architecture
  table, and CLAUDE.md infra block.
- New optimization #26 in docs/12: fix the Intel Arc iGPU passthrough
  flagged as broken in the LXC notes. Concrete /etc/pve/lxc/104.conf
  snippet provided. Once working, TEI / infinity / Immich CLIP can run
  GPU-accelerated for ~5–10x throughput.
- New optimization #27: disk-pressure mitigation playbook (move Qdrant
  to unas if it grows, prune docker layers monthly, cap n8n executions).
- New optimization #28: shrink the LXC's 8 GiB swap to 2 GiB.
2026-05-04 22:10:51 +00:00
Claude 0b470b3e12 Resolve Q6/Q7/Q8/Q9/Q11; pin actual list names; add host tuning suggestions
Verified from screenshots:

- Q6/Q7 → Lists & calendars discovered by NAME, not by ID. Names confirmed:
  "Persönlich" (personal), "DLR" (work), "Einkaufsliste" (shopping),
  "Wunschliste" (wishes). Self-healing: #nexa:config runs daily and on
  cache-miss, so user-added lists become routable without code changes.
  Routing logic in docs/07 expanded from 2-way to 4-way switch with a
  Resolver step that consults the Qdrant _config cache before fanning out;
  default route is "Persönlich" when SAIA confidence < 0.7.
- Q8 → Single Nextcloud Mail account fkrebs@nucli.de. No separate IMAP.
  The "Waiting" folder is honored as a manual user signal — items there are
  excluded from digest.
- Q9 → Pocket-ID SSO is global at the Zoraxy layer; Nexa surfaces inherit
  it. No app-level basic-auth. Q10 folded in.
- Q11 → Capacity confirmed: 22 threads (Core Ultra 7 155H), 62 GiB RAM
  (~32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35%).
  Phase-3 budget (~7 GB) is comfortable; two levers documented (raise
  LXC cap, or cap zfs_arc_max). Earlier "16 CPU" references corrected.

Propagation:
- system_prime.txt rewritten with the four context labels and the new
  list names; legacy NC_LIST_ID_* placeholders removed.
- docs/04 integration matrix uses the German names directly.
- docs/05 documents that #nexa:config also runs scheduled and on
  cache-miss, with the example output reflecting real list IDs.
- docs/09 step 6 mirrors the same flow + lists the Waiting-folder
  semantics.
- CLAUDE.md gets an authoritative block of homelab decisions so future
  agents don't re-litigate any of these.

New section in docs/12 (#17–25): Proxmox host tuning suggestions
based on the live node summary — cap ZFS ARC (frees ~16 GiB),
enable KSM (~1-3 GiB savings across 40 containers), suppress the
no-subscription banner, drop swappiness, verify scrub/SMART/fstrim
timers, NTP via AdGuard.
2026-05-04 22:08:10 +00:00
Claude e1c2ba96e2 Resolve Q4/Q5/Q13/Q14/Q18: Obsidian via Nextcloud WebDAV, Karakeep, Octoprint suppress
- Q4 → Obsidian vault is the Notizen/ folder in Nextcloud (multi-device sync).
  Nexa reads via WebDAV using the existing NC_APP_PASSWORD — no filesystem
  mount. Ignore list pinned: .copilot/, .copilot-index/, .smart-env/,
  .caldav-sync/, assets/ (routed to Phase-3.2 visual queue), Templates/,
  BMO/, Excalidraw/. Index target: Notizen/**/*.md.
- Q5 → canonical name is Karakeep; legacy Zoraxy alias hoarder.nuclide.systems
  kept for compatibility.
- Q13 → Octoprint container is intentionally powered down most of the time.
  Phase-5 monitoring must skip names matching octoprint*. Reflected in
  docs/10 monitoring table and CLAUDE.md.
- Q14 → Homepage Zoraxy widget config error; cosmetic, dropped from open
  questions and from docs/12 optimization list.
- Q18 → SAIA does proxy embeddings but rate limit is 10 msg/min, unusable
  for ingest. TEI stays in Phase 3.1.
- docs/12: stale items removed (Octoprint, Zoraxy widget, generic Immich
  vector idea); added new ones derived from the Obsidian discovery (plugin
  embedding collision, notify_push upgrade path, assets/ size note).
- docs/04 integration matrix updated to describe Obsidian-via-Nextcloud
  read path explicitly.
- docs/09 step-4 credentials list collapses NC Tasks / Calendar / WebDAV
  onto a single app password.
2026-05-04 21:55:47 +00:00
Claude 5c9034c861 Resolve Q16/Q17: media_uri at source, Immich out-of-band
- Q16 → image bytes stay in their source (Memos / Nextcloud / Obsidian);
  Phase-3.2 backfill fetches by media_uri. Zero copy.
- Q17 → Immich stays out-of-band; Nexa does not call Immich's smart-search.
- New Q18 carries forward the unresolved sub-question of whether SAIA's
  LiteLLM gateway already proxies an embedding model — a 1-line check that
  could let us skip TEI in 3.1.
2026-05-04 21:49:08 +00:00
Claude 17c7f5033f Stage embeddings: ship Path A (text) now, prepare Path C (text+visual) additively
Decision (Q15 resolved): start with text-only via TEI + bge-m3 in Phase 3.1,
prepare data shapes so Phase 3.2 (visual collection via infinity + jina-clip-v2)
is a pure additive operation — no rename, no schema migration, no n8n rewiring.

Concretely:
- Qdrant collection renamed nexa_knowledge → nexa_knowledge_text (1024-dim
  for bge-m3) with modality-aware payload (modality, source_type, media_uri,
  graph_iri, content_hash, context). Visual placeholder schema committed
  alongside (qdrant_schema_visual.json, 768-dim, jina-clip-v2).
- Image attachments captured in 3.1 are recorded in GraphDB as nexa:Note with
  nexa:modality "image" + nexa:pendingVisualIndex true; the 3.2 backfill
  workflow picks them up and embeds. No data lost between phases — the queue
  is the GraphDB itself.
- RDF schema (docs/08) gains nexa:modality, nexa:mediaUri,
  nexa:vectorCollection, nexa:pendingVisualIndex from day one.
- docs/02 roadmap split: 3.1 = text RAG (Path A), 3.2 = visual collection
  (Path C), 3.4 = Ontotext GraphDB.
- docs/09 grows a "Phase add-on: visual collection (Phase 3.2)" section with
  the TEI→infinity swap, second collection create, LiteLLM second model
  registration, and the SPARQL-driven backfill query.
- New open questions: Q16 (queue ergonomics + does SAIA already proxy an
  embed model?), Q17 (reuse Immich's CLIP for photo-library queries?).
- docs/03 + CLAUDE.md updated so future runs use the new collection names
  and don't re-decide the staging.
2026-05-04 21:33:59 +00:00
Claude a1e14c64c3 Apply Q1–Q3 decisions: GraphDB/SPARQL, reuse Qdrant, self-host embeddings via TEI
- Q1 → Ontotext GraphDB (SPARQL). docs/08 fully rewritten with RDF schema,
  example SPARQL queries (transitive deps via property paths, time-windowed
  topic counts, cross-pillar joins via nexa:vectorId).
- Q2 → reuse qdrant_scientific with nexa_* collection prefix; docs/09 step 2
  now points there explicitly.
- Q3 → no OpenAI embeddings. Self-host on the docker host. Use TEI
  (HuggingFace text-embeddings-inference) — single Rust binary, ~500 MB image,
  OpenAI-compatible — instead of Ollama, since we only need embeddings.
- docs/09 Phase-3.4 add-on simplified to a single Ontotext compose snippet
  (Neo4j option dropped) plus repo creation curl.
- docs/11 Q3 marked resolved; new Q15 picks the model (bge-m3 vs nomic-embed)
  and adds the open question of whether SAIA already proxies an embedding
  model that would let us skip TEI entirely.
- docs/03 + CLAUDE.md updated with the new decisions so future runs don't
  re-litigate.
2026-05-04 21:26:08 +00:00
Claude 9b18e2710d Reorganize /docs with numbered TOC; deployment rewrite for actual homelab
- /docs/ now contains 12 numbered guides + index.md (navigator).
- Old duplicates removed: nexa-core/doc/, DEPLOYMENT.md, PLAN.md,
  config/{classification_logic,workflows_spec}.md.
- 09-deployment.md targets the running infra (LXC 104 docker host,
  Zoraxy at 192.168.1.4, existing Memos/n8n/LiteLLM/Nextcloud/Qdrant)
  rather than spinning a parallel stack; minimal-config approach.
- 11-open-questions.md tracks user-info-required blockers.
- 12-optimization-opportunities.md captures homelab tweaks.
- CLAUDE.md added so future agent runs know repo conventions and infra.
2026-05-04 21:18:28 +00:00