Claude
0b470b3e12
Verified from screenshots: - Q6/Q7 → Lists & calendars discovered by NAME, not by ID. Names confirmed: "Persönlich" (personal), "DLR" (work), "Einkaufsliste" (shopping), "Wunschliste" (wishes). Self-healing: #nexa:config runs daily and on cache-miss, so user-added lists become routable without code changes. Routing logic in docs/07 expanded from 2-way to 4-way switch with a Resolver step that consults the Qdrant _config cache before fanning out; default route is "Persönlich" when SAIA confidence < 0.7. - Q8 → Single Nextcloud Mail account fkrebs@nucli.de. No separate IMAP. The "Waiting" folder is honored as a manual user signal — items there are excluded from digest. - Q9 → Pocket-ID SSO is global at the Zoraxy layer; Nexa surfaces inherit it. No app-level basic-auth. Q10 folded in. - Q11 → Capacity confirmed: 22 threads (Core Ultra 7 155H), 62 GiB RAM (~32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35%). Phase-3 budget (~7 GB) is comfortable; two levers documented (raise LXC cap, or cap zfs_arc_max). Earlier "16 CPU" references corrected. Propagation: - system_prime.txt rewritten with the four context labels and the new list names; legacy NC_LIST_ID_* placeholders removed. - docs/04 integration matrix uses the German names directly. - docs/05 documents that #nexa:config also runs scheduled and on cache-miss, with the example output reflecting real list IDs. - docs/09 step 6 mirrors the same flow + lists the Waiting-folder semantics. - CLAUDE.md gets an authoritative block of homelab decisions so future agents don't re-litigate any of these. New section in docs/12 (#17–25): Proxmox host tuning suggestions based on the live node summary — cap ZFS ARC (frees ~16 GiB), enable KSM (~1-3 GiB savings across 40 containers), suppress the no-subscription banner, drop swappiness, verify scrub/SMART/fstrim timers, NTP via AdGuard.
67 lines
7.8 KiB
Markdown
67 lines
7.8 KiB
Markdown
# 11 — Open Questions (user-info-required)
|
||
|
||
Items that **block progress** and need a human decision before a workflow can be implemented or a service deployed. Tick them off as you decide.
|
||
|
||
## Resolved
|
||
|
||
- [x] **Q1 — Graph DB choice → Ontotext GraphDB (SPARQL).** Rationale: explore Nexa's memory through SPARQL is a stated goal. [08-graphrag-architecture](./08-graphrag-architecture.md) is rewritten accordingly.
|
||
- [x] **Q2 — Vector store → reuse `qdrant_scientific`** with a `nexa_*` collection prefix. No dedicated container.
|
||
- [x] **Q3 — Embeddings model → not OpenAI.** Self-host on the docker host via **TEI** (HF `text-embeddings-inference`) — Rust single-binary, OpenAI-compatible, ~500 MB image, no LLM runtime overhead. Speed analysis in §"Speed budget" below.
|
||
- [x] **Q5 — Display name → Karakeep.** The Zoraxy host alias `hoarder.nuclide.systems` is legacy — keep it for compatibility, but all docs, prompts and new workflow nodes use "Karakeep".
|
||
- [x] **Q13 — Octoprint container is *intentionally* temporary.** Suppress from Phase-5 monitoring: container-up/down alerts must skip names matching `octoprint*` (or any container tagged `proxmox-he 3d-printing`).
|
||
- [x] **Q14 — Homepage Zoraxy widget → no such widget.** Config error in `homepage/services.yaml`; cosmetic, not Nexa-related.
|
||
- [x] **Q15 — Embeddings staging plan → A now, C prepared.**
|
||
- **Phase 3.1 (now)**: TEI + `BAAI/bge-m3`, single collection `nexa_knowledge_text` (1024-dim). DE/EN multilingual, fits the corpus.
|
||
- **Phase 3.2 (later)**: swap TEI → `infinity`, add `jinaai/jina-clip-v2` (768-dim), second collection `nexa_knowledge_visual`. Backfill from the queue (see Q16).
|
||
- All schema fields needed for 3.2 (`modality`, `media_uri`, `graph_iri`, `nexa:pendingVisualIndex`) are introduced **now** so 3.2 is purely additive — no rename, no migration. See [`qdrant_schema.json`](../nexa-core/config/qdrant_schema.json) and [`qdrant_schema_visual.json`](../nexa-core/config/qdrant_schema_visual.json).
|
||
- [x] **Q16 — Image-attachment queue ergonomics → leave bytes at source, reference by `media_uri`.** Zero copy. Memos attachments stay in Memos's data dir, Nextcloud images stay in Nextcloud, Obsidian images stay in the `Notizen` folder; the Phase-3.2 backfill workflow fetches them on demand via the URI scheme.
|
||
- [x] **Q17 — Immich out-of-band.** Nexa does not call the Immich smart-search API. Photo-library queries stay inside Immich.
|
||
- [x] **Q18 — SAIA proxies an embedding model, but 10 msg/min rate limit makes it unusable for ingest.** A 2 k-note Obsidian backfill would take ~3.3 h; real-time `#nexa:ask` would queue for tens of seconds during a writing burst. **Decision: deploy TEI as planned.** SAIA embeddings remain available as a manual fallback (e.g. for one-off `#nexa:learn` calls where rate is irrelevant).
|
||
- [x] **Q4 — Obsidian sync via Nextcloud WebDAV.** Confirmed the vault is `nc.nuclide.systems/Notizen/` (multi-device sync via Nextcloud client). Nexa accesses it through WebDAV (`/remote.php/dav/files/<user>/Notizen/`) reusing the existing `NC_APP_PASSWORD` — no filesystem mount, no LXC-to-LXC privilege escalation. Phase 3.1 polls every 15 min; an upgrade to Nextcloud's `notify_push` for sub-second updates is captured as optimization #15. Ignore list (don't index):
|
||
- `.copilot/`, `.copilot-index/` — Obsidian Copilot's own embeddings cache.
|
||
- `.smart-env/` — Smart Connections / Smart Composer plugin data (~13 MB).
|
||
- `.caldav-sync/` — calendar sync, not notes.
|
||
- `assets/` — 186 MB of binaries; routed through the Phase-3.2 visual queue (`nexa:pendingVisualIndex`), not the text path.
|
||
- `Templates/` — empty templates, low semantic value.
|
||
- `BMO/`, `Excalidraw/` — plugin folders.
|
||
Anything else under `Notizen/**/*.md` is fair game.
|
||
- [x] **Q6 + Q7 — Lists & calendars discovered by name, not ID. Self-healing.**
|
||
Confirmed names from screenshots:
|
||
- **Tasks**: `Persönlich` (14), `DLR` (13), `Einkaufsliste` (13), `Wunschliste` (9).
|
||
- **Calendars**: `Persönlich`, `DLR`, `Einkaufsliste`, `Wunschliste` (Nextcloud Tasks is calendar-backed, so the names are shared; Tasks lives on the calendar of the same name).
|
||
- Mapping in classification: `DLR` = Work-Kontext, `Persönlich` = Personal-Kontext, `Einkaufsliste` = Shopping, `Wunschliste` = Wishes.
|
||
|
||
**Self-healing requirement (the user explicitly noted lists may change/grow):** Nexa must not cache IDs forever. The `#nexa:config` workflow runs (a) on demand, (b) once daily as a scheduled refresh, and (c) automatically as a retry whenever a list/calendar lookup returns 404 or "not found". The runtime-config record in Qdrant's `_config` namespace stores `{name → id, discovered_at}` and gets invalidated on cache-miss. New lists added in Nextcloud surface in the next scheduled refresh and Nexa starts honouring `#einkaufsliste` / `#dlr` etc. without code changes.
|
||
|
||
- [x] **Q8 — Mail via Nextcloud, single account `fkrebs@nucli.de`.** No extra IMAP entry. n8n's IMAP node uses the same server credentials Nextcloud Mail already holds for that account; Nexa never sees a second password. Folders confirmed: `Posteingang` (default), `Archiv`, `Junk` (61 — auto-filtered, ignored by Nexa), `Papierkorb`, `Waiting`. The `Waiting` folder is a useful manual signal — items moved there by the user are skipped from digest (treat as "in flight").
|
||
|
||
- [x] **Q9 — Pocket-ID SSO is configured everywhere.** No separate auth for Nexa: Memos / n8n / future Nexa dashboard authenticate humans via Pocket-ID at the Zoraxy layer. Machine-to-machine calls (n8n → Memos webhook, n8n → Nextcloud, n8n → Qdrant, n8n → LiteLLM) keep using API keys / app passwords — SSO is for human UIs only. Don't add basic-auth or per-app login screens.
|
||
|
||
This **resolves Q10 too** (the proposed "Pocket-ID SSO in front of n8n / Memos" is already done; Nexa just inherits it).
|
||
|
||
- [x] **Q10 — folded into Q9.**
|
||
|
||
## Hardware / capacity
|
||
|
||
- [x] **Q11 — Capacity confirmed from Proxmox node summary.**
|
||
- Host (NUC 14 Pro): **22 threads** (Intel Core Ultra 7 155H, 1 socket), **62.12 GiB RAM**, **1.64 TiB disk** (0.35% used).
|
||
- Current: 32.4 GiB used (~52%), of which **ZFS ARC accounts for ~24 GiB**. Load avg 1.99 / 1.39 / 1.15 — idle. IO delay 0.04% — disk is not a bottleneck.
|
||
- LXC 104 (docker) is currently capped at ~31 GiB; if Phase-3 squeezes (~7 GB additional: Qdrant ~1.5 GB + TEI/bge-m3 ~1.1 GB + GraphDB 4 GB heap + later +infinity/jina-clip-v2 ~1 GB) we have two levers, in order of preference: (1) raise the LXC cap (host has plenty), (2) cap `zfs_arc_max` lower (ARC currently uses 24 GB). Either is one line of config.
|
||
- **Verdict: comfortable. No action needed at Phase-3.1 start.** Revisit if/when 3.4 GraphDB lands and actual heap usage exceeds the budget.
|
||
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
|
||
|
||
---
|
||
|
||
## Speed budget (Q3 follow-up)
|
||
|
||
Workload on the docker host (host: 22-thread Intel Core Ultra 7 155H, 62 GiB RAM, ~30 GiB free; LXC 104 currently capped at 31 GiB):
|
||
|
||
| Task | Volume | Latency target | Achievable on CPU with `bge-m3` | Achievable with `nomic-embed-text` |
|
||
|------|--------|----------------|---------------------------------|------------------------------------|
|
||
| Real-time memo embed | 1 doc | <500 ms incl. n8n round-trip | ✅ ~50–100 ms | ✅ ~20 ms |
|
||
| Daily ingest | ~70 docs | <60 s | ✅ ~5–10 s | ✅ ~2 s |
|
||
| Obsidian backfill (one-shot) | ~2 000 docs | <15 min | ✅ ~2–4 min | ✅ <1 min |
|
||
| RAG query embed (`#nexa:ask`) | 1 doc | <300 ms | ✅ ~50 ms | ✅ ~20 ms |
|
||
|
||
Conclusion: CPU-only TEI is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings. SAIA's own embedding endpoint is rate-limited to **10 req/min** which would block real-time embed; self-hosted TEI side-steps that completely.
|