fkrebs/nexa

Files

T

Claude a1e14c64c3 Apply Q1–Q3 decisions: GraphDB/SPARQL, reuse Qdrant, self-host embeddings via TEI

- Q1 → Ontotext GraphDB (SPARQL). docs/08 fully rewritten with RDF schema,
  example SPARQL queries (transitive deps via property paths, time-windowed
  topic counts, cross-pillar joins via nexa:vectorId).
- Q2 → reuse qdrant_scientific with nexa_* collection prefix; docs/09 step 2
  now points there explicitly.
- Q3 → no OpenAI embeddings. Self-host on the docker host. Use TEI
  (HuggingFace text-embeddings-inference) — single Rust binary, ~500 MB image,
  OpenAI-compatible — instead of Ollama, since we only need embeddings.
- docs/09 Phase-3.4 add-on simplified to a single Ontotext compose snippet
  (Neo4j option dropped) plus repo creation curl.
- docs/11 Q3 marked resolved; new Q15 picks the model (bge-m3 vs nomic-embed)
  and adds the open question of whether SAIA already proxies an embedding
  model that would let us skip TEI entirely.
- docs/03 + CLAUDE.md updated with the new decisions so future runs don't
  re-litigate.

2026-05-04 21:26:08 +00:00

4.2 KiB

Raw Blame History

11 — Open Questions (user-info-required)

Items that block progress and need a human decision before a workflow can be implemented or a service deployed. Tick them off as you decide.

Resolved

Q1 — Graph DB choice → Ontotext GraphDB (SPARQL). Rationale: explore Nexa's memory through SPARQL is a stated goal. 08-graphrag-architecture is rewritten accordingly.
Q2 — Vector store → reuse qdrant_scientific with a nexa_* collection prefix. No dedicated container.
Q3 — Embeddings model → not OpenAI. Self-host on the docker host via TEI (HF text-embeddings-inference) — Rust single-binary, OpenAI-compatible, ~500 MB image, no LLM runtime overhead. Speed analysis in §"Speed budget" below; concrete model still open as Q15.

Architectural decisions

Q4 — Obsidian sync mechanism. system_prime.txt references Obsidian Context, but the current setup syncs via Nextcloud (nc.nuclide.systems → Notizen folder, ~200 MB). Should Nexa watch the filesystem on LXC 105 (NC data dir) or the Nextcloud WebDAV API? FS is cheaper, WebDAV is portable.
Q5 — Karakeep vs. Hoarder naming. Zoraxy host is hoarder.nuclide.systems but containers are karakeep-* and Homepage labels it Karakeep. Same product (rename 2024). Pick one display name for docs and prompts.
Q15 — Embedding model (served via TEI). Two viable options on the docker host (CPU only — see speed budget):
- BAAI/bge-m3 (568 M params, ~1 GB RAM, multilingual incl. German, 1024-dim, ~10–20 docs/s on CPU). Recommended because the corpus is DE/EN-mixed. Requires qdrant_schema.json vectors.size = 1024.
- nomic-ai/nomic-embed-text-v1.5 (137 M, ~250 MB RAM, EN-leaning, 768-dim, ~50 docs/s CPU). Lighter / faster but weaker on German.
- (maybe) does SAIA already proxy any embedding model? If the SAIA backend offers e.g. mistral-embed for free, we can skip TEI entirely. Worth a 1-line check in the LiteLLM admin UI before deploying TEI.
Decide before Phase 3.1.

Identifiers needed (auto-discoverable, but list now if known)

Q6 — Nextcloud Tasks list IDs for: Work_Tasks, Personal_Tasks, Shopping, Wishes. Discovery via #nexa:config will fill these — confirm names match.
Q7 — Nextcloud Calendar IDs for: Work_Calendar, primary personal calendar.
Q8 — IMAP credentials for the personal mail account. Can n8n reuse a Nextcloud Mail account (preferred — no extra password) or must we add a dedicated IMAP entry?
Q9 — ntfy topic name for nexa.system. Is the topic public on ntfy.nuclide.systems or should it be authenticated?
Q10 — Pocket-ID role. id.nuclide.systems is running. Do we want SSO in front of the n8n / Memos UIs, or skip for now?

Hardware / capacity

Q11 — RAM headroom on docker host. 29.5 GiB free / ~31 GiB total, ~8.5 GB used. Phase-3 Qdrant indexing + TEI/bge-m3 (~1.1 GB) + Ontotext GraphDB (~4 GB heap) ⇒ ~14 GB used worst case, still ample. Confirm acceptable.
Q12 — S3 archive bucket. s3.nuclide.systems is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?

Process

Q13 — Octoprint container is Exited (Homepage). Out of scope for Nexa, but Phase-5 monitoring would alert on it. Suppress or is it intentional?
Q14 — Missing Widget Type: zoraxy on Homepage. Cosmetic, unrelated to Nexa.

Speed budget (Q3 follow-up)

Workload on the docker host (16 CPU, ~30 GB free RAM):

Task	Volume	Latency target	Achievable on CPU with `bge-m3`	Achievable with `nomic-embed-text`
Real-time memo embed	1 doc	<500 ms incl. n8n round-trip	✅ ~50–100 ms	✅ ~20 ms
Daily ingest	~70 docs	<60 s	✅ ~5–10 s	✅ ~2 s
Obsidian backfill (one-shot)	~2 000 docs	<15 min	✅ ~2–4 min	✅ <1 min
RAG query embed (`#nexa:ask`)	1 doc	<300 ms	✅ ~50 ms	✅ ~20 ms

Conclusion: CPU-only Ollama is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings.