Claude
a1e14c64c3
- Q1 → Ontotext GraphDB (SPARQL). docs/08 fully rewritten with RDF schema, example SPARQL queries (transitive deps via property paths, time-windowed topic counts, cross-pillar joins via nexa:vectorId). - Q2 → reuse qdrant_scientific with nexa_* collection prefix; docs/09 step 2 now points there explicitly. - Q3 → no OpenAI embeddings. Self-host on the docker host. Use TEI (HuggingFace text-embeddings-inference) — single Rust binary, ~500 MB image, OpenAI-compatible — instead of Ollama, since we only need embeddings. - docs/09 Phase-3.4 add-on simplified to a single Ontotext compose snippet (Neo4j option dropped) plus repo creation curl. - docs/11 Q3 marked resolved; new Q15 picks the model (bge-m3 vs nomic-embed) and adds the open question of whether SAIA already proxies an embedding model that would let us skip TEI entirely. - docs/03 + CLAUDE.md updated with the new decisions so future runs don't re-litigate.
54 lines
4.2 KiB
Markdown
54 lines
4.2 KiB
Markdown
# 11 — Open Questions (user-info-required)
|
||
|
||
Items that **block progress** and need a human decision before a workflow can be implemented or a service deployed. Tick them off as you decide.
|
||
|
||
## Resolved
|
||
|
||
- [x] **Q1 — Graph DB choice → Ontotext GraphDB (SPARQL).** Rationale: explore Nexa's memory through SPARQL is a stated goal. [08-graphrag-architecture](./08-graphrag-architecture.md) is rewritten accordingly.
|
||
- [x] **Q2 — Vector store → reuse `qdrant_scientific`** with a `nexa_*` collection prefix. No dedicated container.
|
||
- [x] **Q3 — Embeddings model → not OpenAI.** Self-host on the docker host via **TEI** (HF `text-embeddings-inference`) — Rust single-binary, OpenAI-compatible, ~500 MB image, no LLM runtime overhead. Speed analysis in §"Speed budget" below; concrete model still open as **Q15**.
|
||
|
||
## Architectural decisions
|
||
|
||
- [ ] **Q4 — Obsidian sync mechanism.** `system_prime.txt` references Obsidian Context, but the current setup syncs via Nextcloud (`nc.nuclide.systems` → `Notizen` folder, ~200 MB). Should Nexa watch the **filesystem on LXC 105** (NC data dir) or **the Nextcloud WebDAV API**? FS is cheaper, WebDAV is portable.
|
||
- [ ] **Q5 — Karakeep vs. Hoarder naming.** Zoraxy host is `hoarder.nuclide.systems` but containers are `karakeep-*` and Homepage labels it Karakeep. Same product (rename 2024). Pick one display name for docs and prompts.
|
||
- [ ] **Q15 — Embedding model (served via TEI).** Two viable options on the docker host (CPU only — see speed budget):
|
||
- **`BAAI/bge-m3`** (568 M params, ~1 GB RAM, multilingual incl. German, 1024-dim, ~10–20 docs/s on CPU). **Recommended** because the corpus is DE/EN-mixed. Requires `qdrant_schema.json` `vectors.size = 1024`.
|
||
- **`nomic-ai/nomic-embed-text-v1.5`** (137 M, ~250 MB RAM, EN-leaning, 768-dim, ~50 docs/s CPU). Lighter / faster but weaker on German.
|
||
- **(maybe) does SAIA already proxy any embedding model?** If the SAIA backend offers e.g. `mistral-embed` for free, we can skip TEI entirely. Worth a 1-line check in the LiteLLM admin UI before deploying TEI.
|
||
|
||
Decide before Phase 3.1.
|
||
|
||
## Identifiers needed (auto-discoverable, but list now if known)
|
||
|
||
- [ ] **Q6 — Nextcloud Tasks list IDs** for: `Work_Tasks`, `Personal_Tasks`, `Shopping`, `Wishes`. Discovery via `#nexa:config` will fill these — confirm names match.
|
||
- [ ] **Q7 — Nextcloud Calendar IDs** for: `Work_Calendar`, primary personal calendar.
|
||
- [ ] **Q8 — IMAP credentials for the personal mail account.** Can n8n reuse a Nextcloud Mail account (preferred — no extra password) or must we add a dedicated IMAP entry?
|
||
- [ ] **Q9 — ntfy topic name** for `nexa.system`. Is the topic public on `ntfy.nuclide.systems` or should it be authenticated?
|
||
- [ ] **Q10 — Pocket-ID role.** `id.nuclide.systems` is running. Do we want SSO in front of the n8n / Memos UIs, or skip for now?
|
||
|
||
## Hardware / capacity
|
||
|
||
- [ ] **Q11 — RAM headroom on docker host.** 29.5 GiB free / ~31 GiB total, ~8.5 GB used. Phase-3 Qdrant indexing + TEI/`bge-m3` (~1.1 GB) + Ontotext GraphDB (~4 GB heap) ⇒ ~14 GB used worst case, still ample. Confirm acceptable.
|
||
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
|
||
|
||
## Process
|
||
|
||
- [ ] **Q13 — Octoprint container is Exited** (Homepage). Out of scope for Nexa, but Phase-5 monitoring would alert on it. Suppress or is it intentional?
|
||
- [ ] **Q14 — `Missing Widget Type: zoraxy`** on Homepage. Cosmetic, unrelated to Nexa.
|
||
|
||
---
|
||
|
||
## Speed budget (Q3 follow-up)
|
||
|
||
Workload on the docker host (16 CPU, ~30 GB free RAM):
|
||
|
||
| Task | Volume | Latency target | Achievable on CPU with `bge-m3` | Achievable with `nomic-embed-text` |
|
||
|------|--------|----------------|---------------------------------|------------------------------------|
|
||
| Real-time memo embed | 1 doc | <500 ms incl. n8n round-trip | ✅ ~50–100 ms | ✅ ~20 ms |
|
||
| Daily ingest | ~70 docs | <60 s | ✅ ~5–10 s | ✅ ~2 s |
|
||
| Obsidian backfill (one-shot) | ~2 000 docs | <15 min | ✅ ~2–4 min | ✅ <1 min |
|
||
| RAG query embed (`#nexa:ask`) | 1 doc | <300 ms | ✅ ~50 ms | ✅ ~20 ms |
|
||
|
||
Conclusion: CPU-only Ollama is sufficient — no GPU needed for current scope. Bottleneck is SAIA chat (already remote), not embeddings.
|