doc-ingestion: nomic embedding stack, Bifrost semantic cache, GPU passthrough status

services/doc-ingestion.md

@@ -22,17 +22,22 @@ directly from Paperless's REST API without re-running Docling for already-OCR'd
 
 | Component | Location | Endpoint |
 |-----------|----------|---------|
-| **Qdrant** | CT 104, `ai-internal` net | `http://qdrant:6333` |
+| **Qdrant** | CT 104, `ai-internal` net | `http://qdrant:6333` (REST), `qdrant:6334` (gRPC) |
-| **TEI** (Text Embeddings Inference) | CT 104, `ai-internal` net | `http://tei:80` |
+| **nomic** | CT 104, `ai-internal` net | `http://nomic:80` — text+vision 768d |
-| **Open WebUI** | CT 104, port 14002 | Uses Qdrant + TEI natively |
+| **TEI** (Text Embeddings Inference) | CT 104, `ai-internal` net | `http://tei:80` — text-only 768d (standby) |
+| **Open WebUI** | CT 104, port 14002 | Uses Qdrant + nomic natively |
+| **Bifrost** | CT 104, port 14003 | Semantic cache → Qdrant gRPC, mistral-embed 1024d |
 | **Docling MCP** | CT 104, port 18005 | MCP server in gateway |
 
 ### Embedding stack
 
-- **Model**: `intfloat/multilingual-e5-base` (768d, multilingual DE+EN, ~278 MB ONNX)
+| Service | Model | Dimensions | Use |
-- **Server**: HuggingFace TEI `cpu-1.6` — OpenAI-compatible at `http://tei:80/v1/embeddings`
+|---------|-------|-----------|-----|
-- **Auth**: none (internal network only)
+| **nomic** | `nomic-ai/nomic-embed-text-v1.5` + `nomic-embed-vision-v1.5` | 768d | OWUI RAG, ingest pipeline |
-- **Upgrade path**: switch to `mistral/mistral-embed` via Bifrost (1024d, better quality) — requires re-index (drop + recreate Qdrant collections)
+| **TEI** | `intfloat/multilingual-e5-base` | 768d | Standby; same vector space as nomic text |
+| **Bifrost cache** | `mistral/mistral-embed` via Bifrost | 1024d | Semantic cache only (separate Qdrant collection) |
+
+Both nomic models share a 768d embedding space — text and image queries work on the same `documents` Qdrant collection.
 
 ### OWUI RAG config (env-driven)
 
@@ -40,15 +45,21 @@ directly from Paperless's REST API without re-running Docling for already-OCR'd
 VECTOR_DB=qdrant
 QDRANT_URI=http://qdrant:6333
 RAG_EMBEDDING_ENGINE=openai
-RAG_OPENAI_API_BASE_URL=http://tei:80
+RAG_OPENAI_API_BASE_URL=http://nomic:80
 RAG_OPENAI_API_KEY=none
-RAG_EMBEDDING_MODEL=intfloat/multilingual-e5-base
+RAG_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5
 CONTENT_EXTRACTION_ENGINE=docling
 CHUNK_SIZE=1200
 CHUNK_OVERLAP=150
 ENABLE_RAG_HYBRID_SEARCH=true
 ```
 
+To switch to Bifrost embeddings (1024d, better quality — requires re-indexing `documents` collection):
+```
+RAG_OPENAI_API_BASE_URL=http://bifrost:8080/v1
+RAG_EMBEDDING_MODEL=mistral/mistral-embed
+```
+
 ### Bifrost embedding models (available for external services / upgrade)
 
 Three embedding models tested and working via `http://bifrost:8080/v1/embeddings`:
@@ -75,19 +86,37 @@ from qdrant_client import QdrantClient
 client = QdrantClient(url="http://qdrant:6333")
 ```
 
-n8n, MCP tools, and custom pipelines should use `http://tei:80/v1/embeddings` for
+n8n, MCP tools, and custom pipelines should use `http://nomic:80/v1/embeddings` for
 consistent 768d vectors. Mixing models/dimensions in the same collection will fail.
 
-## Intel Arc GPU passthrough (planned)
+## Bifrost semantic caching
 
-CT 104 has Intel Core Ultra 7 155H iGPU but no `/dev/dri/render*` device is
+Bifrost uses Qdrant (gRPC port 6334) as a semantic cache backend. Config lives in
-passed through. When passthrough is enabled, switch TEI to the Intel image:
+`/opt/stacks/ai/bifrost/data/config.json`:
+
+```json
+{
+  "$schema": "https://www.getbifrost.ai/schema",
+  "vector_store": {"enabled": true, "type": "qdrant", "config": {"host": "qdrant", "port": 6334}},
+  "plugins": [{
+    "enabled": true, "name": "semantic_cache",
+    "config": {
+      "provider": "mistral", "embedding_model": "mistral-embed", "dimension": 1024,
+      "ttl": "10m", "threshold": 0.85, "conversation_history_threshold": 3, "exclude_system_prompt": true
+    }
+  }]
+}
 ```
-image: ghcr.io/huggingface/text-embeddings-inference:intel-1.6
+
-```
+The semantic cache uses a separate Qdrant collection (auto-created) at 1024d — no collision with the `documents` collection at 768d. TTL: 10 min, similarity threshold: 0.85.
-This uses IPEX and runs ~5–10× faster for embedding batches.
+
+## Intel Arc GPU passthrough (enabled)
+
+CT 104 Intel Core Ultra 7 155H iGPU is passed through via PVE `dev3`/`dev4` entries
+(`/dev/dri/renderD128` and `/dev/dri/card1`). No TEI Intel image exists currently — passthrough is available for future inference acceleration.
 
 ## Pending
 
-- n8n workflow: Nextcloud/Paperless → Docling → TEI → Qdrant
+- n8n workflow: Nextcloud/Paperless → Docling → nomic → Qdrant
 - MCP knowledge-search tool pointing at Qdrant (replaces LobeChat search intent)
+- `ingest.py` filesystem scan script (see [ingest-pipeline.md](ingest-pipeline.md))