Resolve Q4/Q5/Q13/Q14/Q18: Obsidian via Nextcloud WebDAV, Karakeep, Octoprint suppress

- Q4 → Obsidian vault is the Notizen/ folder in Nextcloud (multi-device sync).
  Nexa reads via WebDAV using the existing NC_APP_PASSWORD — no filesystem
  mount. Ignore list pinned: .copilot/, .copilot-index/, .smart-env/,
  .caldav-sync/, assets/ (routed to Phase-3.2 visual queue), Templates/,
  BMO/, Excalidraw/. Index target: Notizen/**/*.md.
- Q5 → canonical name is Karakeep; legacy Zoraxy alias hoarder.nuclide.systems
  kept for compatibility.
- Q13 → Octoprint container is intentionally powered down most of the time.
  Phase-5 monitoring must skip names matching octoprint*. Reflected in
  docs/10 monitoring table and CLAUDE.md.
- Q14 → Homepage Zoraxy widget config error; cosmetic, dropped from open
  questions and from docs/12 optimization list.
- Q18 → SAIA does proxy embeddings but rate limit is 10 msg/min, unusable
  for ingest. TEI stays in Phase 3.1.
- docs/12: stale items removed (Octoprint, Zoraxy widget, generic Immich
  vector idea); added new ones derived from the Obsidian discovery (plugin
  embedding collision, notify_push upgrade path, assets/ size note).
- docs/04 integration matrix updated to describe Obsidian-via-Nextcloud
  read path explicitly.
- docs/09 step-4 credentials list collapses NC Tasks / Calendar / WebDAV
  onto a single app password.
This commit is contained in:
Claude
2026-05-04 21:55:47 +00:00
parent 5c9034c861
commit e1c2ba96e2
6 changed files with 37 additions and 32 deletions
+4 -2
View File
@@ -59,8 +59,10 @@ Nexa muss strikt zwischen zwei Kontexten unterscheiden, da die Datenquellen vari
* Wichtige private Termine werden in den privaten Kalender extrahiert.
* Newsletter landen als Zusammenfassung in Memos.
### Wissens-Management (Obsidian & Qdrant)
* Obsidian-Notizen (Arbeit & Privat) werden in Qdrant indiziert.
### Wissens-Management (Obsidian via Nextcloud & Qdrant)
* Der Obsidian-Vault liegt im Nextcloud-Ordner `Notizen/` (Multi-Device-Sync via Nextcloud-Client). Nexa liest **read-only über WebDAV** (`/remote.php/dav/files/<user>/Notizen/`) — kein Mount, kein zweiter Sync-Mechanismus.
* Markdown-Dateien (`Notizen/**/*.md`) werden in Qdrant (`nexa_knowledge_text`) indiziert; Bilder unter `assets/` landen im Phase-3.2-Visual-Queue (siehe [docs/11/Q4](./11-open-questions.md)).
* Plugin-Internals (`.copilot/`, `.smart-env/`, `.caldav-sync/`, `Templates/`, `BMO/`, `Excalidraw/`) sind explizit von der Indizierung ausgeschlossen.
* Nexa nutzt dieses Wissen, um Arbeits-Tasks besser zu beschreiben, auch wenn sie keinen Zugriff auf die Arbeits-E-Mails hat.
---
+3 -3
View File
@@ -13,7 +13,7 @@ Surveyed from Homepage / Dozzle / Proxmox / Zoraxy:
| LiteLLM (SAIA gateway) | docker LXC 104 → `:4000` | `https://ai.nuclide.systems` (proxies LobeHub UI :3210; API on :4000) |
| Nextcloud | LXC 105 | `https://nc.nuclide.systems` |
| ntfy | docker LXC 104 → `:7998` | `https://ntfy.nuclide.systems` |
| Karakeep (Hoarder) | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` |
| Karakeep | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` (legacy host alias kept for compatibility) |
| Home Assistant | VM 100 (HAOS) | `https://ha.nuclide.systems` |
| Pocket-ID (OAuth/SSO) | docker LXC 104 → `:1411` | `https://id.nuclide.systems` |
| Vaultwarden | docker LXC 104 → `:11001` | `https://vault.nuclide.systems` |
@@ -145,8 +145,8 @@ done
Inside n8n, attach credentials to the imported nodes:
- **Memos** → HTTP header `Authorization: Bearer $MEMOS_API_KEY`
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY`
- **Nextcloud** → app password
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY` (chat + `nexa-embed`)
- **Nextcloud Tasks / Calendar / WebDAV (Obsidian vault under `Notizen/`)** → one app password (`$NC_APP_PASSWORD`), reused across all three node types
- **Qdrant** → header `api-key: $QDRANT_API_KEY`
Activate each workflow individually after smoke-test.
+1 -1
View File
@@ -18,7 +18,7 @@ Backrest (LXC 103) already handles file-level snapshots. Add Nexa-specific items
| Signal | Source | Sink |
|--------|--------|------|
| Container up/down | Dozzle (`192.168.1.40:3553`) + docker healthchecks | Memos system feed via ntfy |
| Container up/down | Dozzle (`192.168.1.40:3553`) + docker healthchecks. **Skip names matching `octoprint*`** — that container is intentionally powered down most of the time. | Memos system feed via ntfy |
| n8n workflow failures | n8n built-in failure-webhook | ntfy → `nexa.system` topic |
| Proxmox / disk / memory alerts | Proxmox notification target → ntfy | ntfy → Memos digest |
| LiteLLM rate-limit | LiteLLM logs / cost-tracking | Memos morning digest |
+16 -15
View File
@@ -7,18 +7,24 @@ Items that **block progress** and need a human decision before a workflow can be
- [x] **Q1 — Graph DB choice → Ontotext GraphDB (SPARQL).** Rationale: explore Nexa's memory through SPARQL is a stated goal. [08-graphrag-architecture](./08-graphrag-architecture.md) is rewritten accordingly.
- [x] **Q2 — Vector store → reuse `qdrant_scientific`** with a `nexa_*` collection prefix. No dedicated container.
- [x] **Q3 — Embeddings model → not OpenAI.** Self-host on the docker host via **TEI** (HF `text-embeddings-inference`) — Rust single-binary, OpenAI-compatible, ~500 MB image, no LLM runtime overhead. Speed analysis in §"Speed budget" below.
- [x] **Q5 — Display name → Karakeep.** The Zoraxy host alias `hoarder.nuclide.systems` is legacy — keep it for compatibility, but all docs, prompts and new workflow nodes use "Karakeep".
- [x] **Q13 — Octoprint container is *intentionally* temporary.** Suppress from Phase-5 monitoring: container-up/down alerts must skip names matching `octoprint*` (or any container tagged `proxmox-he 3d-printing`).
- [x] **Q14 — Homepage Zoraxy widget → no such widget.** Config error in `homepage/services.yaml`; cosmetic, not Nexa-related.
- [x] **Q15 — Embeddings staging plan → A now, C prepared.**
- **Phase 3.1 (now)**: TEI + `BAAI/bge-m3`, single collection `nexa_knowledge_text` (1024-dim). DE/EN multilingual, fits the corpus.
- **Phase 3.2 (later)**: swap TEI → `infinity`, add `jinaai/jina-clip-v2` (768-dim), second collection `nexa_knowledge_visual`. Backfill from the queue (see Q16).
- All schema fields needed for 3.2 (`modality`, `media_uri`, `graph_iri`, `nexa:pendingVisualIndex`) are introduced **now** so 3.2 is purely additive — no rename, no migration. See [`qdrant_schema.json`](../nexa-core/config/qdrant_schema.json) and [`qdrant_schema_visual.json`](../nexa-core/config/qdrant_schema_visual.json).
## Architectural decisions
- [ ] **Q4 — Obsidian sync mechanism.** `system_prime.txt` references Obsidian Context, but the current setup syncs via Nextcloud (`nc.nuclide.systems``Notizen` folder, ~200 MB). Should Nexa watch the **filesystem on LXC 105** (NC data dir) or **the Nextcloud WebDAV API**? FS is cheaper, WebDAV is portable.
- [ ] **Q5 — Karakeep vs. Hoarder naming.** Zoraxy host is `hoarder.nuclide.systems` but containers are `karakeep-*` and Homepage labels it Karakeep. Same product (rename 2024). Pick one display name for docs and prompts.
- [x] **Q16 — Image-attachment queue ergonomics → leave bytes at source, reference by `media_uri`.** Zero copy. Memos attachments stay in Memos's data dir, Nextcloud images stay in Nextcloud, Obsidian images stay in the `Notizen` folder; the Phase-3.2 backfill workflow fetches them on demand via the URI scheme. Sub-question on whether SAIA already proxies an embedding model is still worth a 1-line check in the LiteLLM admin UI before deploying TEI — moved to Q18.
- [x] **Q17 — Immich out-of-band.** Nexa does not call the Immich smart-search API. Photo-library queries stay inside Immich; if a Nexa workflow ever needs photo context it will go through a Memos-mediated handoff rather than a direct API.
- [ ] **Q18 — Does SAIA already proxy an embedding model?** 1-line check in the LiteLLM admin UI: list models on the Nexa virtual key. If the SAIA backend offers e.g. `mistral-embed`, we can skip TEI in Phase 3.1 entirely. If not, deploy TEI as planned.
- [x] **Q16 — Image-attachment queue ergonomics → leave bytes at source, reference by `media_uri`.** Zero copy. Memos attachments stay in Memos's data dir, Nextcloud images stay in Nextcloud, Obsidian images stay in the `Notizen` folder; the Phase-3.2 backfill workflow fetches them on demand via the URI scheme.
- [x] **Q17 — Immich out-of-band.** Nexa does not call the Immich smart-search API. Photo-library queries stay inside Immich.
- [x] **Q18 — SAIA proxies an embedding model, but 10 msg/min rate limit makes it unusable for ingest.** A 2 k-note Obsidian backfill would take ~3.3 h; real-time `#nexa:ask` would queue for tens of seconds during a writing burst. **Decision: deploy TEI as planned.** SAIA embeddings remain available as a manual fallback (e.g. for one-off `#nexa:learn` calls where rate is irrelevant).
- [x] **Q4 — Obsidian sync via Nextcloud WebDAV.** Confirmed the vault is `nc.nuclide.systems/Notizen/` (multi-device sync via Nextcloud client). Nexa accesses it through WebDAV (`/remote.php/dav/files/<user>/Notizen/`) reusing the existing `NC_APP_PASSWORD` — no filesystem mount, no LXC-to-LXC privilege escalation. Phase 3.1 polls every 15 min; an upgrade to Nextcloud's `notify_push` for sub-second updates is captured as optimization #15. Ignore list (don't index):
- `.copilot/`, `.copilot-index/` — Obsidian Copilot's own embeddings cache.
- `.smart-env/` — Smart Connections / Smart Composer plugin data (~13 MB).
- `.caldav-sync/` — calendar sync, not notes.
- `assets/` — 186 MB of binaries; routed through the Phase-3.2 visual queue (`nexa:pendingVisualIndex`), not the text path.
- `Templates/` — empty templates, low semantic value.
- `BMO/`, `Excalidraw/` — plugin folders.
Anything else under `Notizen/**/*.md` is fair game.
## Identifiers needed (auto-discoverable, but list now if known)
@@ -30,14 +36,9 @@ Items that **block progress** and need a human decision before a workflow can be
## Hardware / capacity
- [ ] **Q11 — RAM headroom on docker host.** 29.5 GiB free / ~31 GiB total, ~8.5 GB used. Phase-3 Qdrant indexing + TEI/`bge-m3` (~1.1 GB) + Ontotext GraphDB (~4 GB heap) ⇒ ~14 GB used worst case, still ample. Confirm acceptable.
- [ ] **Q11 — RAM headroom on docker host.** Sized against 31 GiB allocated to the LXC; the NUC 14 Pro itself has 64 GB so the LXC's memory cap can be lifted if needed. Phase-3 stack (Qdrant + TEI/bge-m3 ~1.1 GB + Ontotext GraphDB ~4 GB heap, later +infinity/jina-clip-v2 ~1 GB) tops out around ~14 GB. Comfortable. Confirm acceptable.
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
## Process
- [ ] **Q13 — Octoprint container is Exited** (Homepage). Out of scope for Nexa, but Phase-5 monitoring would alert on it. Suppress or is it intentional?
- [ ] **Q14 — `Missing Widget Type: zoraxy`** on Homepage. Cosmetic, unrelated to Nexa.
---
## Speed budget (Q3 follow-up)
@@ -51,4 +52,4 @@ Workload on the docker host (16 CPU, ~30 GB free RAM):
| Obsidian backfill (one-shot) | ~2 000 docs | <15 min | ✅ ~24 min | ✅ <1 min |
| RAG query embed (`#nexa:ask`) | 1 doc | <300 ms | ✅ ~50 ms | ✅ ~20 ms |
Conclusion: CPU-only Ollama is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings.
Conclusion: CPU-only TEI is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings. SAIA's own embedding endpoint is rate-limited to **10 req/min** which would block real-time embed; self-hosted TEI side-steps that completely.
+10 -10
View File
@@ -13,13 +13,13 @@ Observations from the running infrastructure. Each item is independent — accep
## For the wider homelab (out of scope but worth noting)
7. **Octoprint `EXITED`** — either remove the Homepage entry or fix the container; right now it permanently shows red.
8. **Zoraxy widget missing** on Homepage. Homepage v1.x has no built-in Zoraxy widget, but you can use the generic `customapi` widget against `/api/stats/summary` to surface request counts.
9. **AI gateway naming.** `ai.nuclide.systems` currently proxies LobeHub (a chat UI on `:3210`), while the LiteLLM API lives on `:4000`. For Nexa, point n8n directly at LiteLLM (`http://192.168.1.40:4000` over the docker net — no public TLS hop needed) to save latency and isolate from UI restarts.
10. **MCP servers consolidation.** Dozzle shows `crawl4ai-mcp`, `markitdown-mcp`, `papersearch-mcp` running individually. They're all MCP servers — Nexa Phase-3 could pull from these via LiteLLM's MCP support to enrich the embedding pipeline (e.g. fetch + markitdown a Karakeep link before embedding).
11. **Backup the n8n SQLite file** — Backrest covers `/home/node/.n8n` if added; today the only "backup" is the workflow JSON which omits credentials and execution history.
12. **Pocket-ID SSO in front of n8n** would let you remove n8n basic-auth and unify session management across the whole stack. One-time setup, large UX win.
13. **Vaultwarden as the secret store** for Nexa secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, …) — read at bootstrap via the Bitwarden CLI from inside the docker host. Removes the need for a `.env` on disk.
14. **AdGuard as DNS-based control plane.** Since AdGuard is the resolver for the LAN, you can rewrite `*.nuclide.systems` to `192.168.1.4` (Zoraxy) internally and avoid a hairpin via the WAN — already the case if AdGuard rewrite rules are set, worth verifying.
15. **Immich + Qdrant.** Immich is already running and embeds photos with CLIP. If you ever want photo-aware Nexa answers, you can mount Immich's existing search via its API — no second embedding pass.
16. **Disk usage on LXC 104** is 47.7 % (Proxmox). Monitor; n8n execution logs and Dozzle history are the usual culprits. Setting `EXECUTIONS_DATA_PRUNE=true` and `EXECUTIONS_DATA_MAX_AGE=168` (7 days) on n8n keeps it bounded.
7. **AI gateway naming.** `ai.nuclide.systems` currently proxies LobeHub (a chat UI on `:3210`), while the LiteLLM API lives on `:4000`. For Nexa, point n8n directly at LiteLLM (`http://192.168.1.40:4000` over the docker net — no public TLS hop needed) to save latency and isolate from UI restarts.
8. **MCP servers consolidation.** Dozzle shows `crawl4ai-mcp`, `markitdown-mcp`, `papersearch-mcp` running individually. They're all MCP servers — Nexa Phase-3 could pull from these via LiteLLM's MCP support to enrich the embedding pipeline (e.g. fetch + markitdown a Karakeep link before embedding).
9. **Backup the n8n SQLite file** — Backrest covers `/home/node/.n8n` if added; today the only "backup" is the workflow JSON which omits credentials and execution history.
10. **Pocket-ID SSO in front of n8n** would let you remove n8n basic-auth and unify session management across the whole stack. One-time setup, large UX win.
11. **Vaultwarden as the secret store** for Nexa secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, …) — read at bootstrap via the Bitwarden CLI from inside the docker host. Removes the need for a `.env` on disk.
12. **AdGuard as DNS-based control plane.** Since AdGuard is the resolver for the LAN, you can rewrite `*.nuclide.systems` to `192.168.1.4` (Zoraxy) internally and avoid a hairpin via the WAN — already the case if AdGuard rewrite rules are set, worth verifying.
13. **Disk usage on LXC 104** is 47.7 % (Proxmox). Monitor; n8n execution logs and Dozzle history are the usual culprits. Setting `EXECUTIONS_DATA_PRUNE=true` and `EXECUTIONS_DATA_MAX_AGE=168` (7 days) on n8n keeps it bounded.
14. **Obsidian plugin embeddings collide with Nexa's.** The vault already runs Obsidian Copilot (`.copilot`, `.copilot-index`) and Smart Connections / Smart Composer (`.smart-env`, ~13 MB). They each embed the same notes into their own vector stores — three indexes for the same content. Nexa's value is the **cross-source** index (memos + mail + obsidian + RDF graph), so it has to embed independently, but the plugins could be retired once Nexa's RAG is satisfying. Track separately, decide later.
15. **Real-time Obsidian sync via `notify_push`.** Phase 3.1 polls WebDAV every 15 min (Q4 resolution). Once that works, swap to Nextcloud's `notify_push` app for sub-second propagation. One-line workflow change in n8n.
16. **`assets/` is 186 MB of binaries** in the Obsidian vault — worth a glance to confirm it's mostly images (Phase-3.2 visual queue) rather than something that should live in Nextcloud Files proper.