fkrebs/nexa
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Stage embeddings: ship Path A (text) now, prepare Path C (text+visual) additively

Browse Source 
Decision (Q15 resolved): start with text-only via TEI + bge-m3 in Phase 3.1,
prepare data shapes so Phase 3.2 (visual collection via infinity + jina-clip-v2)
is a pure additive operation — no rename, no schema migration, no n8n rewiring.

Concretely:
- Qdrant collection renamed nexa_knowledge → nexa_knowledge_text (1024-dim
  for bge-m3) with modality-aware payload (modality, source_type, media_uri,
  graph_iri, content_hash, context). Visual placeholder schema committed
  alongside (qdrant_schema_visual.json, 768-dim, jina-clip-v2).
- Image attachments captured in 3.1 are recorded in GraphDB as nexa:Note with
  nexa:modality "image" + nexa:pendingVisualIndex true; the 3.2 backfill
  workflow picks them up and embeds. No data lost between phases — the queue
  is the GraphDB itself.
- RDF schema (docs/08) gains nexa:modality, nexa:mediaUri,
  nexa:vectorCollection, nexa:pendingVisualIndex from day one.
- docs/02 roadmap split: 3.1 = text RAG (Path A), 3.2 = visual collection
  (Path C), 3.4 = Ontotext GraphDB.
- docs/09 grows a "Phase add-on: visual collection (Phase 3.2)" section with
  the TEI→infinity swap, second collection create, LiteLLM second model
  registration, and the SPARQL-driven backfill query.
- New open questions: Q16 (queue ergonomics + does SAIA already proxy an
  embed model?), Q17 (reuse Immich's CLIP for photo-library queries?).
- docs/03 + CLAUDE.md updated so future runs use the new collection names
  and don't re-decide the staging.
This commit is contained in:
Claude
2026-05-04 21:33:59 +00:00
parent a1e14c64c3
commit 17c7f5033f
8 changed files with 142 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+3 -3
View File
@@ -21,9 +21,9 @@ This file tells future automated runs what they need to know about this repo.
- **Already-running services on docker host** (don't redeploy):
- Memos `:5230`, n8n `:5678`, LiteLLM `:4000` (UI LobeHub `:3210`), Qdrant (`qdrant_scientific`), ntfy `:7998`, Karakeep/Hoarder, Vaultwarden `:11001`, Pocket-ID `:1411`, Immich, Audiobookshelf, Paperless-ngx, Traccar, Prowlarr, plus MCP containers (`crawl4ai-mcp`, `markitdown-mcp`, `papersearch-mcp`).
- **Decided for Nexa** (don't re-litigate without user input):
- Vector store: **reuse `qdrant_scientific`** with `nexa_*` collection prefix.
- Embeddings: **TEI** (HF text-embeddings-inference), self-hosted, `bge-m3` candidate (Q15 still pending the model name).
- Graph store: **Ontotext GraphDB** (SPARQL/RDF), Phase 3.4.
- Vector store: **reuse `qdrant_scientific`** with collections suffixed by modality (`nexa_knowledge_text`, `nexa_knowledge_visual`).
- Embeddings staged: **Phase 3.1** TEI + `BAAI/bge-m3` (text-only, 1024-dim). **Phase 3.2** swap to `infinity` and add `jinaai/jina-clip-v2` (768-dim, joint text+image space). All forward-compat fields (`modality`, `media_uri`, `graph_iri`, `nexa:pendingVisualIndex`) exist from 3.1 — adding the visual collection is additive.
- Graph store: **Ontotext GraphDB** (SPARQL/RDF), Phase 3.4. RDF schema in docs/08 already includes `nexa:modality` / `nexa:mediaUri` / `nexa:vectorCollection` / `nexa:pendingVisualIndex`.
- Chat model: SAIA via LiteLLM virtual key.
## When working on Nexa