Stage embeddings: ship Path A (text) now, prepare Path C (text+visual) additively
Decision (Q15 resolved): start with text-only via TEI + bge-m3 in Phase 3.1, prepare data shapes so Phase 3.2 (visual collection via infinity + jina-clip-v2) is a pure additive operation — no rename, no schema migration, no n8n rewiring. Concretely: - Qdrant collection renamed nexa_knowledge → nexa_knowledge_text (1024-dim for bge-m3) with modality-aware payload (modality, source_type, media_uri, graph_iri, content_hash, context). Visual placeholder schema committed alongside (qdrant_schema_visual.json, 768-dim, jina-clip-v2). - Image attachments captured in 3.1 are recorded in GraphDB as nexa:Note with nexa:modality "image" + nexa:pendingVisualIndex true; the 3.2 backfill workflow picks them up and embeds. No data lost between phases — the queue is the GraphDB itself. - RDF schema (docs/08) gains nexa:modality, nexa:mediaUri, nexa:vectorCollection, nexa:pendingVisualIndex from day one. - docs/02 roadmap split: 3.1 = text RAG (Path A), 3.2 = visual collection (Path C), 3.4 = Ontotext GraphDB. - docs/09 grows a "Phase add-on: visual collection (Phase 3.2)" section with the TEI→infinity swap, second collection create, LiteLLM second model registration, and the SPARQL-driven backfill query. - New open questions: Q16 (queue ergonomics + does SAIA already proxy an embed model?), Q17 (reuse Immich's CLIP for photo-library queries?). - docs/03 + CLAUDE.md updated so future runs use the new collection names and don't re-decide the staging.
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"collection_name": "nexa_knowledge_visual",
|
||||
"vector_config": {
|
||||
"size": 768,
|
||||
"distance": "Cosine"
|
||||
},
|
||||
"payload_schema": {
|
||||
"source": "keyword",
|
||||
"source_type": "keyword",
|
||||
"modality": "keyword",
|
||||
"mime_type": "keyword",
|
||||
"media_uri": "keyword",
|
||||
"context": "keyword",
|
||||
"tags": "keyword",
|
||||
"graph_iri": "keyword",
|
||||
"content_hash":"keyword",
|
||||
"created_at": "datetime"
|
||||
},
|
||||
"_notes": {
|
||||
"status": "PLACEHOLDER — not deployed in Phase 3.1. Created in Phase 3.2.",
|
||||
"vector_size_rationale": "768 = jina-clip-v2 joint text/image space. The same model embeds both text queries and images, so semantic queries against this collection are simple kNN.",
|
||||
"served_by": "Will be served by `infinity` (https://github.com/michaelfeil/infinity) — drop-in replacement for TEI that supports CLIP-family models with the same OpenAI-compatible API.",
|
||||
"modality_field": "'image' for indexed images, 'text' if a text query has been embedded into the visual space for retrieval (rare — usually we embed the text query at search time and don't store it).",
|
||||
"media_uri_field": "Required for image rows. Examples: 'memos://memos/123/attachment/4', 'nextcloud://files/photos/whiteboard-2025-09.jpg', 'obsidian://attachments/diagram.png'."
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user