fkrebs/nexa

Files

T

Claude 17c7f5033f Stage embeddings: ship Path A (text) now, prepare Path C (text+visual) additively

Decision (Q15 resolved): start with text-only via TEI + bge-m3 in Phase 3.1,
prepare data shapes so Phase 3.2 (visual collection via infinity + jina-clip-v2)
is a pure additive operation — no rename, no schema migration, no n8n rewiring.

Concretely:
- Qdrant collection renamed nexa_knowledge → nexa_knowledge_text (1024-dim
  for bge-m3) with modality-aware payload (modality, source_type, media_uri,
  graph_iri, content_hash, context). Visual placeholder schema committed
  alongside (qdrant_schema_visual.json, 768-dim, jina-clip-v2).
- Image attachments captured in 3.1 are recorded in GraphDB as nexa:Note with
  nexa:modality "image" + nexa:pendingVisualIndex true; the 3.2 backfill
  workflow picks them up and embeds. No data lost between phases — the queue
  is the GraphDB itself.
- RDF schema (docs/08) gains nexa:modality, nexa:mediaUri,
  nexa:vectorCollection, nexa:pendingVisualIndex from day one.
- docs/02 roadmap split: 3.1 = text RAG (Path A), 3.2 = visual collection
  (Path C), 3.4 = Ontotext GraphDB.
- docs/09 grows a "Phase add-on: visual collection (Phase 3.2)" section with
  the TEI→infinity swap, second collection create, LiteLLM second model
  registration, and the SPARQL-driven backfill query.
- New open questions: Q16 (queue ergonomics + does SAIA already proxy an
  embed model?), Q17 (reuse Immich's CLIP for photo-library queries?).
- docs/03 + CLAUDE.md updated so future runs use the new collection names
  and don't re-decide the staging.

2026-05-04 21:33:59 +00:00

5.2 KiB

Raw Blame History

11 — Open Questions (user-info-required)

Items that block progress and need a human decision before a workflow can be implemented or a service deployed. Tick them off as you decide.

Resolved

Q1 — Graph DB choice → Ontotext GraphDB (SPARQL). Rationale: explore Nexa's memory through SPARQL is a stated goal. 08-graphrag-architecture is rewritten accordingly.
Q2 — Vector store → reuse qdrant_scientific with a nexa_* collection prefix. No dedicated container.
Q3 — Embeddings model → not OpenAI. Self-host on the docker host via TEI (HF text-embeddings-inference) — Rust single-binary, OpenAI-compatible, ~500 MB image, no LLM runtime overhead. Speed analysis in §"Speed budget" below.
Q15 — Embeddings staging plan → A now, C prepared.
- Phase 3.1 (now): TEI + BAAI/bge-m3, single collection nexa_knowledge_text (1024-dim). DE/EN multilingual, fits the corpus.
- Phase 3.2 (later): swap TEI → infinity, add jinaai/jina-clip-v2 (768-dim), second collection nexa_knowledge_visual. Backfill from the queue (see Q16).
- All schema fields needed for 3.2 (modality, media_uri, graph_iri, nexa:pendingVisualIndex) are introduced now so 3.2 is purely additive — no rename, no migration. See qdrant_schema.json and qdrant_schema_visual.json.

Architectural decisions

Q4 — Obsidian sync mechanism. system_prime.txt references Obsidian Context, but the current setup syncs via Nextcloud (nc.nuclide.systems → Notizen folder, ~200 MB). Should Nexa watch the filesystem on LXC 105 (NC data dir) or the Nextcloud WebDAV API? FS is cheaper, WebDAV is portable.
Q5 — Karakeep vs. Hoarder naming. Zoraxy host is hoarder.nuclide.systems but containers are karakeep-* and Homepage labels it Karakeep. Same product (rename 2024). Pick one display name for docs and prompts.
Q16 — Image-attachment queue ergonomics (Phase 3.2). The agreed plan is: in 3.1 we record images as nexa:Note with nexa:pendingVisualIndex true but don't embed them. Open sub-questions for when 3.2 lands:
- Where to store image bytes between capture and indexing? Three options: (a) leave them in their source (Memos attachments dir / Nextcloud / Obsidian), reference by media_uri; (b) copy to a staging area on the docker host; (c) push to s3.nuclide.systems immediately. Recommendation: (a) — zero copy, smallest blast radius.
- Does SAIA already proxy any embedding model? If the SAIA backend offers e.g. mistral-embed we could simplify 3.1 by skipping TEI. Worth a 1-line check in the LiteLLM admin UI.
Q17 — Reuse Immich's CLIP for photo-library queries? Immich already runs CLIP server-side on the photo library. For images that live in Immich, querying its smart-search API is cheaper than re-embedding. Is the Immich API key OK to add to the n8n workflow, or do we treat Immich as out-of-band?

Identifiers needed (auto-discoverable, but list now if known)

Q6 — Nextcloud Tasks list IDs for: Work_Tasks, Personal_Tasks, Shopping, Wishes. Discovery via #nexa:config will fill these — confirm names match.
Q7 — Nextcloud Calendar IDs for: Work_Calendar, primary personal calendar.
Q8 — IMAP credentials for the personal mail account. Can n8n reuse a Nextcloud Mail account (preferred — no extra password) or must we add a dedicated IMAP entry?
Q9 — ntfy topic name for nexa.system. Is the topic public on ntfy.nuclide.systems or should it be authenticated?
Q10 — Pocket-ID role. id.nuclide.systems is running. Do we want SSO in front of the n8n / Memos UIs, or skip for now?

Hardware / capacity

Q11 — RAM headroom on docker host. 29.5 GiB free / ~31 GiB total, ~8.5 GB used. Phase-3 Qdrant indexing + TEI/bge-m3 (~1.1 GB) + Ontotext GraphDB (~4 GB heap) ⇒ ~14 GB used worst case, still ample. Confirm acceptable.
Q12 — S3 archive bucket. s3.nuclide.systems is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?

Process

Q13 — Octoprint container is Exited (Homepage). Out of scope for Nexa, but Phase-5 monitoring would alert on it. Suppress or is it intentional?
Q14 — Missing Widget Type: zoraxy on Homepage. Cosmetic, unrelated to Nexa.

Speed budget (Q3 follow-up)

Workload on the docker host (16 CPU, ~30 GB free RAM):

Task	Volume	Latency target	Achievable on CPU with `bge-m3`	Achievable with `nomic-embed-text`
Real-time memo embed	1 doc	<500 ms incl. n8n round-trip	✅ ~50–100 ms	✅ ~20 ms
Daily ingest	~70 docs	<60 s	✅ ~5–10 s	✅ ~2 s
Obsidian backfill (one-shot)	~2 000 docs	<15 min	✅ ~2–4 min	✅ <1 min
RAG query embed (`#nexa:ask`)	1 doc	<300 ms	✅ ~50 ms	✅ ~20 ms

Conclusion: CPU-only Ollama is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings.