327 lines
15 KiB
Markdown
327 lines
15 KiB
Markdown
# 09 — Deployment
|
||
|
||
Pragmatic deployment guide that **assumes the existing homelab** and adds only what's missing.
|
||
|
||
## What's already running (no action required)
|
||
|
||
Surveyed from Homepage / Dozzle / Proxmox / Zoraxy:
|
||
|
||
| Service | Host / port | URL |
|
||
|---------|-------------|-----|
|
||
| Memos | docker LXC 104 → `:5230` | `https://memos.nuclide.systems` |
|
||
| n8n | docker LXC 104 → `:5678` | `https://n8n.nuclide.systems` |
|
||
| LiteLLM (SAIA gateway) | docker LXC 104 → `:4000` | `https://ai.nuclide.systems` (proxies LobeHub UI :3210; API on :4000) |
|
||
| Nextcloud | LXC 105 | `https://nc.nuclide.systems` |
|
||
| ntfy | docker LXC 104 → `:7998` | `https://ntfy.nuclide.systems` |
|
||
| Karakeep | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` (legacy host alias kept for compatibility) |
|
||
| Home Assistant | VM 100 (HAOS) | `https://ha.nuclide.systems` |
|
||
| Pocket-ID (OAuth/SSO) | docker LXC 104 → `:1411` | `https://id.nuclide.systems` |
|
||
| Vaultwarden | docker LXC 104 → `:11001` | `https://vault.nuclide.systems` |
|
||
| Backrest | LXC 103 | (internal) |
|
||
| AdGuard DNS | LXC 102 | (internal) |
|
||
| Zoraxy reverse proxy | LXC 108 → `192.168.1.4:8000` | TLS for *.nuclide.systems |
|
||
| `qdrant_scientific` (existing) | docker LXC 104 | **reused** — Nexa uses `nexa_*` collections in this instance |
|
||
|
||
The deployment task is **not** "spin up the stack" — most of the stack is already up. It is **wire Nexa across these services + add the small bits that are missing**.
|
||
|
||
## What's missing for Nexa
|
||
|
||
1. **Qdrant collection** for Nexa (`nexa_knowledge`) inside the existing `qdrant_scientific` instance — vector dim follows Q15 (1024 for `bge-m3`, 768 for `nomic-embed-text`).
|
||
2. **TEI** (HF text-embeddings-inference) on the docker host for self-hosted embeddings (LiteLLM key is **not** authorised for OpenAI embeddings — see [11/Q3+Q15](./11-open-questions.md)). Lighter than Ollama: single Rust binary, ~500 MB image, no LLM runtime.
|
||
3. **n8n workflows** (`./nexa-core/n8n-workflows/`) imported into the running n8n.
|
||
4. **Nextcloud lists & calendars** for Work / Personal / Shopping / Wishes (auto-discovered via `#nexa:config`).
|
||
5. **Memos webhook → n8n** wired through the Memos config.
|
||
6. **LiteLLM virtual key** for the `nexa` user with chat-only access (no embeddings — handled by Ollama).
|
||
7. **A Zoraxy host entry** is *not* needed — Memos / n8n / LiteLLM are already proxied.
|
||
8. **(Phase 3.4) Ontotext GraphDB** for the SPARQL pillar — see add-on at the bottom of this doc.
|
||
|
||
---
|
||
|
||
## Step 1 — Secrets
|
||
|
||
Copy `nexa-core/.env.example` → `nexa-core/.env` and fill **only** the secrets:
|
||
|
||
```bash
|
||
cd nexa-core
|
||
cp .env.example .env
|
||
$EDITOR .env # MEMOS_API_KEY, SAIA_API_KEY, NC_APP_PASSWORD, QDRANT_API_KEY
|
||
```
|
||
|
||
The `.env` is **only used at bootstrap time**. Everything else (list IDs, calendar IDs, collection sizes) is discovered at runtime via `#nexa:config` (see [05](./05-command-system.md)). No secrets should ever live in n8n workflow JSON — use n8n credentials instead.
|
||
|
||
## Step 2 — Qdrant collection (`nexa_knowledge_text`)
|
||
|
||
Phase 3.1 ships **Path A** (text-only) but the schema and naming already make room for **Path C** (text + visual) so adding a `nexa_knowledge_visual` collection later is a pure additive operation — no rename, no migration, no n8n rewiring.
|
||
|
||
```bash
|
||
# adjust QDRANT_HOST in .env first
|
||
source nexa-core/.env
|
||
|
||
# create the text collection from the schema file
|
||
curl -X PUT "$QDRANT_HOST/collections/nexa_knowledge_text" \
|
||
-H "Content-Type: application/json" \
|
||
-H "api-key: $QDRANT_API_KEY" \
|
||
-d @nexa-core/config/qdrant_schema.json
|
||
```
|
||
|
||
The collection name is **always suffixed with the modality** (`_text`, `_visual`) so logic in n8n and SPARQL stays modality-aware from day one. Indexed rows carry these payload fields ([source](../nexa-core/config/qdrant_schema.json)):
|
||
|
||
| Field | Why it's there now |
|
||
|-------|--------------------|
|
||
| `modality` | Always `"text"` in `_text`, `"image"` in `_visual`. Future-proofs cross-modality filters. |
|
||
| `source_type` | `memo` / `mail` / `obsidian` / `screenshot` / `image` — used by classification and digest workflows. |
|
||
| `media_uri` | `memos://…`, `nextcloud://…`, `obsidian://…`. Empty for text-only rows; populated when Path C ships. |
|
||
| `graph_iri` | IRI of the corresponding `nexa:Note` in GraphDB. The same value is stored on the GraphDB side as `nexa:vectorId` — this is the cross-pillar bridge. |
|
||
| `content_hash` | de-dup. |
|
||
| `context` | `work` / `personal`. |
|
||
|
||
> Targets the existing `qdrant_scientific` instance — just an extra collection, no new container.
|
||
> The `vectors.size` field follows [Q15](./11-open-questions.md): **1024** for `bge-m3`, **768** for `nomic-embed-text-v1.5`.
|
||
|
||
### Image attachments today (queue them)
|
||
|
||
Memos can already attach images. Until Phase 3.2 the indexer **does not** embed them, but it **does** record them so they can be replayed later:
|
||
|
||
- Memo with an image → text body still goes into `nexa_knowledge_text`.
|
||
- The image attachment(s) are written as `nexa:Note` triples in GraphDB with `nexa:modality "image"` and `nexa:vectorId` left empty (`nexa:pendingVisualIndex true`).
|
||
- A Phase-3.2 backfill workflow will pick up everything where `?n nexa:pendingVisualIndex true` and embed it through the visual collection.
|
||
|
||
This means **no data is lost** between 3.1 and 3.2 — the queue is the GraphDB itself.
|
||
|
||
## Step 3 — Self-hosted embeddings (TEI)
|
||
|
||
Use HuggingFace **text-embeddings-inference** — single Rust binary, ~500 MB image, OpenAI-compatible API, loads exactly one model. Lighter than Ollama because there's no LLM runtime, no GGUF loader, no model registry.
|
||
|
||
The active docker manager on this LXC is **Arcane** (visible from Homepage as the running container manager — the LXC was originally provisioned with the Dockge helper-script template, but Dockge is now stale; see [12/#33](./12-optimization-opportunities.md#33)). Paste the stack into Arcane → name it `nexa` → save → start. Don't `docker compose up -d` over SSH; Arcane manages the compose lifecycle.
|
||
|
||
```yaml
|
||
# Nexa stack — paste into Arcane.
|
||
# Storage convention matches the rest of the homelab (verified against
|
||
# the running Karakeep stack, Q19): host bind-mount of
|
||
# /mnt/pve/unas/services/<svc>/<vol>. No volume-driver, no CIFS, no
|
||
# credentials in the compose — the LXC's NFS mount is already there.
|
||
services:
|
||
nexa-embed:
|
||
image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
|
||
container_name: nexa-embed
|
||
restart: unless-stopped
|
||
command: ["--model-id", "BAAI/bge-m3"]
|
||
ports:
|
||
- "127.0.0.1:8080:80"
|
||
volumes:
|
||
- /mnt/pve/unas/services/nexa/tei-cache:/data
|
||
env_file:
|
||
- .env
|
||
|
||
networks: {}
|
||
```
|
||
|
||
Pre-deploy step on the docker LXC (one-time):
|
||
|
||
```bash
|
||
mkdir -p /mnt/pve/unas/services/nexa/{tei-cache,qdrant,graphdb}
|
||
mkdir -p /mnt/pve/unas/backup/nexa/snapshots/{qdrant,graphdb}
|
||
```
|
||
|
||
Memory budget: ~1.1 GB resident. First start downloads `bge-m3` (~1 GB) into `/mnt/pve/unas/services/nexa/tei-cache/`; subsequent restarts are instant.
|
||
|
||
Secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, `QDRANT_API_KEY`, `NC_APP_PASSWORD`) go in the stack's `.env` next to the compose — same pattern Karakeep uses (`env_file: .env`). Arcane has an editor for it. Vaultwarden becomes the source-of-truth long-term ([12/#11](./12-optimization-opportunities.md#11)) but isn't required for the first cut.
|
||
|
||
> **Why bind-mount and not SMB?** Earlier drafts of this doc proposed an SMB-via-docker-volume pattern because of the user's "had it with Nextcloud" experience. The Karakeep stack confirms the actual convention is the simpler one: **host bind-mount of the LXC's existing `/mnt/pve/unas` NFS mount**. The Nextcloud failure was Nextcloud-specific (its setup tooling chowns the data dir to `www-data`, which fails against `root_squash` exports) and doesn't apply to normal containers. SMB-as-docker-volume stays documented in [12/#27](./12-optimization-opportunities.md#27) only as an escape hatch *if* a future service hits Nextcloud-style issues — Nexa doesn't, so we don't use it.
|
||
|
||
Register it inside LiteLLM (admin UI → Models) with the OpenAI-compatible adapter:
|
||
|
||
- model name: `nexa-embed`
|
||
- provider: `openai`
|
||
- model: `bge-m3`
|
||
- api_base: `http://nexa-embed:80/v1`
|
||
- api_key: any non-empty string (TEI ignores it)
|
||
|
||
Now n8n only ever talks to LiteLLM and the model is swappable without touching workflows.
|
||
|
||
## Step 4 — LiteLLM virtual key
|
||
|
||
In the LiteLLM admin UI (`ai.nuclide.systems`):
|
||
|
||
1. Create user `nexa`.
|
||
2. Issue a virtual key with access to:
|
||
- one chat model (already-available model from your SAIA gateway).
|
||
- the `nexa-embed` model from Step 3.
|
||
3. Paste the key into `SAIA_API_KEY` in `.env`.
|
||
|
||
## Step 4 — n8n workflows
|
||
|
||
Import the JSON exports — credentials are filled inside n8n, not in the JSON:
|
||
|
||
```bash
|
||
# n8n personal access token from the n8n UI: Settings → API
|
||
N8N_URL=https://n8n.nuclide.systems
|
||
N8N_TOKEN=... # from the n8n UI
|
||
|
||
for f in nexa-core/n8n-workflows/phase-1/*.json \
|
||
nexa-core/n8n-workflows/phase-2/*.json; do
|
||
curl -X POST "$N8N_URL/api/v1/workflows" \
|
||
-H "X-N8N-API-KEY: $N8N_TOKEN" \
|
||
-H "Content-Type: application/json" \
|
||
--data-binary "@$f"
|
||
done
|
||
```
|
||
|
||
Inside n8n, attach credentials to the imported nodes:
|
||
|
||
- **Memos** → HTTP header `Authorization: Bearer $MEMOS_API_KEY`
|
||
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY` (chat + `nexa-embed`)
|
||
- **Nextcloud Tasks / Calendar / WebDAV (Obsidian vault under `Notizen/`)** → one app password (`$NC_APP_PASSWORD`), reused across all three node types
|
||
- **Qdrant** → header `api-key: $QDRANT_API_KEY`
|
||
|
||
Activate each workflow individually after smoke-test.
|
||
|
||
## Step 5 — Memos webhook
|
||
|
||
In the Memos admin UI, set the webhook URL to the production address of the discovery workflow:
|
||
|
||
```
|
||
https://n8n.nuclide.systems/webhook/memos
|
||
```
|
||
|
||
The same URL is the one the workflow exposes; verify with:
|
||
|
||
```bash
|
||
curl -i https://n8n.nuclide.systems/webhook/memos
|
||
# expect 200 / 405, never 404
|
||
```
|
||
|
||
## Step 6 — Bootstrap commands via Memos
|
||
|
||
Create a memo with body `#nexa:config` — the discovery workflow:
|
||
|
||
1. Lists Nextcloud Tasks lists, looking for `Persönlich`, `DLR`, `Einkaufsliste`, `Wunschliste` (and any new lists added later — see [05](./05-command-system.md)). Caches `name → id`.
|
||
2. Lists Nextcloud Calendars by the same names; caches IDs.
|
||
3. Verifies the Nextcloud Mail account `fkrebs@nucli.de` and the folder list (`Posteingang`, `Archiv`, `Junk`, `Waiting`).
|
||
4. Counts existing Qdrant points in `nexa_knowledge_text`.
|
||
5. Verifies LiteLLM reachability + lists available models for the Nexa virtual key.
|
||
6. Replies as a comment with a **runtime-config snapshot** that's stored in the Qdrant `_config` namespace (and mirrored as `nexa-core/config/runtime_config.json`, gitignored).
|
||
|
||
The same workflow is also triggered by:
|
||
- A daily cron inside n8n (so newly added Nextcloud lists become routable without intervention).
|
||
- Cache-miss in the task-router — if a list ID 404s, the router fires `#nexa:config` once and retries.
|
||
|
||
After this point, `.env` is read-once. Subsequent runs read config from Qdrant.
|
||
|
||
## Step 7 — Smoke tests
|
||
|
||
```bash
|
||
# (1) Memos round-trip — should produce a comment within ~5 s
|
||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"content":"- [ ] testing the router #nexa"}'
|
||
|
||
# (2) Classification dry-run
|
||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||
-d '{"content":"#nexa:route-test buy milk"}'
|
||
|
||
# (3) RAG test (requires at least one indexed memo/note)
|
||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||
-d '{"content":"#nexa:ask what is the goal of nexa?"}'
|
||
```
|
||
|
||
## Step 8 — Reverse proxy
|
||
|
||
Already done — Zoraxy at `192.168.1.4:8000` terminates TLS for `*.nuclide.systems` and forwards to docker LXC 104 (`192.168.1.40`). **No new entry is required for Nexa**: every service Nexa talks to already has a host entry.
|
||
|
||
## Step 9 — Backups
|
||
|
||
Already covered by Backrest (LXC 103). Add:
|
||
|
||
- **n8n workflows** → `nexa-core/scripts/backup_workflows.sh` (already present) into a Backrest schedule.
|
||
- **Qdrant snapshots** → schedule a daily `POST /collections/nexa_knowledge/snapshots` and rsync to S3 (`s3.nuclide.systems`). Add as a Backrest pre-hook on the docker host.
|
||
|
||
For deeper detail: [10 — Operations](./10-operations.md).
|
||
|
||
---
|
||
|
||
## Phase add-on: Ontotext GraphDB (Phase 3.4)
|
||
|
||
Defer until 3.1–3.3 ship.
|
||
|
||
```yaml
|
||
# nexa-core/docker-compose.graph.yml
|
||
services:
|
||
graphdb:
|
||
image: ontotext/graphdb:10.7.0
|
||
container_name: nexa-graphdb
|
||
ports: ["127.0.0.1:7200:7200"]
|
||
environment:
|
||
GDB_JAVA_OPTS: "-Xmx4g -Xms1g"
|
||
volumes:
|
||
- ./data/graphdb:/opt/graphdb/home
|
||
restart: unless-stopped
|
||
```
|
||
|
||
After first start, create the repository (one-time):
|
||
|
||
```bash
|
||
curl -X POST http://localhost:7200/rest/repositories \
|
||
-H 'Content-Type: application/json' \
|
||
-d '{
|
||
"id": "nexa_knowledge",
|
||
"title": "Nexa Knowledge Graph",
|
||
"type": "graphdb",
|
||
"params": {
|
||
"ruleset": {"value": "rdfsplus-optimized"},
|
||
"baseURL": {"value": "https://nuclide.systems/nexa/"}
|
||
}
|
||
}'
|
||
```
|
||
|
||
Optional Zoraxy entry `graph.nuclide.systems` → `192.168.1.40:7200` if you want the SPARQL Workbench in a browser; otherwise n8n talks to it on the docker network at `http://nexa-graphdb:7200`.
|
||
|
||
For schema and example queries: [08-graphrag-architecture](./08-graphrag-architecture.md).
|
||
|
||
---
|
||
|
||
## Phase add-on: visual collection (Phase 3.2)
|
||
|
||
Adds Path C — image embeddings without disturbing the text path. Schema is already in `nexa-core/config/qdrant_schema_visual.json`.
|
||
|
||
```bash
|
||
# (1) replace TEI with infinity (or run alongside) for CLIP-family support
|
||
docker rm -f nexa-embed
|
||
docker run -d --name nexa-embed \
|
||
--restart unless-stopped \
|
||
-p 127.0.0.1:8080:80 \
|
||
-v infinity-data:/app/.cache \
|
||
michaelf34/infinity:latest \
|
||
v2 \
|
||
--model-id BAAI/bge-m3 \
|
||
--model-id jinaai/jina-clip-v2 \
|
||
--port 80
|
||
|
||
# (2) create the visual collection
|
||
curl -X PUT "$QDRANT_HOST/collections/nexa_knowledge_visual" \
|
||
-H "Content-Type: application/json" \
|
||
-H "api-key: $QDRANT_API_KEY" \
|
||
-d @nexa-core/config/qdrant_schema_visual.json
|
||
|
||
# (3) register the second model in LiteLLM as `nexa-embed-visual`
|
||
# (same OpenAI-compatible route, different model id)
|
||
|
||
# (4) backfill queued images:
|
||
# SPARQL: SELECT ?note ?uri WHERE { ?note nexa:pendingVisualIndex true ; nexa:mediaUri ?uri }
|
||
# For each row: fetch the bytes, embed via nexa-embed-visual, upsert into the visual collection,
|
||
# UPDATE GraphDB to set nexa:vectorId and DELETE nexa:pendingVisualIndex.
|
||
```
|
||
|
||
n8n RAG workflow gains a parallel branch: text-query → both `nexa-embed-text` and `nexa-embed-visual` text encoders → kNN against both collections → merge by score before SAIA prompt.
|
||
|
||
---
|
||
|
||
## Step-back / rollback
|
||
|
||
- **Disable any Nexa workflow** in n8n — deactivates the side effect immediately, Memos webhooks become no-ops.
|
||
- **Drop a Qdrant collection** — `curl -X DELETE $QDRANT_HOST/collections/nexa_knowledge_text -H "api-key: $QDRANT_API_KEY"`.
|
||
- **Re-discover** — `#nexa:reset-config` then `#nexa:config`.
|