Files
nexa/docs/09-deployment.md
T
Claude 9b18e2710d Reorganize /docs with numbered TOC; deployment rewrite for actual homelab
- /docs/ now contains 12 numbered guides + index.md (navigator).
- Old duplicates removed: nexa-core/doc/, DEPLOYMENT.md, PLAN.md,
  config/{classification_logic,workflows_spec}.md.
- 09-deployment.md targets the running infra (LXC 104 docker host,
  Zoraxy at 192.168.1.4, existing Memos/n8n/LiteLLM/Nextcloud/Qdrant)
  rather than spinning a parallel stack; minimal-config approach.
- 11-open-questions.md tracks user-info-required blockers.
- 12-optimization-opportunities.md captures homelab tweaks.
- CLAUDE.md added so future agent runs know repo conventions and infra.
2026-05-04 21:18:28 +00:00

194 lines
7.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 09 — Deployment
Pragmatic deployment guide that **assumes the existing homelab** and adds only what's missing.
## What's already running (no action required)
Surveyed from Homepage / Dozzle / Proxmox / Zoraxy:
| Service | Host / port | URL |
|---------|-------------|-----|
| Memos | docker LXC 104 → `:5230` | `https://memos.nuclide.systems` |
| n8n | docker LXC 104 → `:5678` | `https://n8n.nuclide.systems` |
| LiteLLM (SAIA gateway) | docker LXC 104 → `:4000` | `https://ai.nuclide.systems` (proxies LobeHub UI :3210; API on :4000) |
| Nextcloud | LXC 105 | `https://nc.nuclide.systems` |
| ntfy | docker LXC 104 → `:7998` | `https://ntfy.nuclide.systems` |
| Karakeep (Hoarder) | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` |
| Home Assistant | VM 100 (HAOS) | `https://ha.nuclide.systems` |
| Pocket-ID (OAuth/SSO) | docker LXC 104 → `:1411` | `https://id.nuclide.systems` |
| Vaultwarden | docker LXC 104 → `:11001` | `https://vault.nuclide.systems` |
| Backrest | LXC 103 | (internal) |
| AdGuard DNS | LXC 102 | (internal) |
| Zoraxy reverse proxy | LXC 108 → `192.168.1.4:8000` | TLS for *.nuclide.systems |
| `qdrant_scientific` (existing) | docker LXC 104 | reuse — see [11](./11-open-questions.md) |
The deployment task is **not** "spin up the stack" — most of the stack is already up. It is **wire Nexa across these services + add the small bits that are missing**.
## What's missing for Nexa
1. **Qdrant collection** for Nexa (`nexa_knowledge`) — either on the existing `qdrant_scientific` or a dedicated container.
2. **n8n workflows** (`./nexa-core/n8n-workflows/`) imported into the running n8n.
3. **Nextcloud lists & calendars** for Work / Personal / Shopping / Wishes (auto-discovered via `#nexa:config`).
4. **Memos webhook → n8n** wired through the Memos config.
5. **LiteLLM virtual key** for the `nexa` user with embedding + chat models.
6. **A Zoraxy host entry** is *not* needed — Memos / n8n / LiteLLM are already proxied.
7. **(Phase 3.4)** decision on graph DB (Neo4j vs. Ontotext GraphDB) — see [11](./11-open-questions.md).
---
## Step 1 — Secrets
Copy `nexa-core/.env.example``nexa-core/.env` and fill **only** the secrets:
```bash
cd nexa-core
cp .env.example .env
$EDITOR .env # MEMOS_API_KEY, SAIA_API_KEY, NC_APP_PASSWORD, QDRANT_API_KEY
```
The `.env` is **only used at bootstrap time**. Everything else (list IDs, calendar IDs, collection sizes) is discovered at runtime via `#nexa:config` (see [05](./05-command-system.md)). No secrets should ever live in n8n workflow JSON — use n8n credentials instead.
## Step 2 — Qdrant collection
```bash
# adjust QDRANT_HOST in .env first
source nexa-core/.env
curl -X PUT "$QDRANT_HOST/collections/nexa_knowledge" \
-H "Content-Type: application/json" \
-H "api-key: $QDRANT_API_KEY" \
-d @nexa-core/config/qdrant_schema.json
```
> If reusing `qdrant_scientific`, this just adds another collection; no new container needed.
## Step 3 — LiteLLM virtual key
In the LiteLLM admin UI (`ai.nuclide.systems`):
1. Create user `nexa`.
2. Issue a virtual key with access to one chat model (e.g. `gpt-4o-mini` / a local Sonnet equivalent) and one embedding model (`text-embedding-3-small`).
3. Paste the key into `SAIA_API_KEY` in `.env`.
## Step 4 — n8n workflows
Import the JSON exports — credentials are filled inside n8n, not in the JSON:
```bash
# n8n personal access token from the n8n UI: Settings → API
N8N_URL=https://n8n.nuclide.systems
N8N_TOKEN=... # from the n8n UI
for f in nexa-core/n8n-workflows/phase-1/*.json \
nexa-core/n8n-workflows/phase-2/*.json; do
curl -X POST "$N8N_URL/api/v1/workflows" \
-H "X-N8N-API-KEY: $N8N_TOKEN" \
-H "Content-Type: application/json" \
--data-binary "@$f"
done
```
Inside n8n, attach credentials to the imported nodes:
- **Memos** → HTTP header `Authorization: Bearer $MEMOS_API_KEY`
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY`
- **Nextcloud** → app password
- **Qdrant** → header `api-key: $QDRANT_API_KEY`
Activate each workflow individually after smoke-test.
## Step 5 — Memos webhook
In the Memos admin UI, set the webhook URL to the production address of the discovery workflow:
```
https://n8n.nuclide.systems/webhook/memos
```
The same URL is the one the workflow exposes; verify with:
```bash
curl -i https://n8n.nuclide.systems/webhook/memos
# expect 200 / 405, never 404
```
## Step 6 — Bootstrap commands via Memos
Create a memo with body `#nexa:config` — the discovery workflow:
1. Lists Nextcloud Tasks lists, picks Work / Personal / Shopping / Wishes by name.
2. Counts existing Qdrant points in `nexa_knowledge`.
3. Verifies LiteLLM reachability + lists available models.
4. Replies as a comment with a **runtime-config snapshot** that's stored as Qdrant metadata (`_config` namespace) and as `nexa-core/config/runtime_config.json` (gitignored).
After this point, `.env` is read-once. Subsequent runs read config from Qdrant.
## Step 7 — Smoke tests
```bash
# (1) Memos round-trip — should produce a comment within ~5 s
curl -X POST https://memos.nuclide.systems/api/v1/memos \
-H "Authorization: Bearer $MEMOS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"content":"- [ ] testing the router #nexa"}'
# (2) Classification dry-run
curl -X POST https://memos.nuclide.systems/api/v1/memos \
-H "Authorization: Bearer $MEMOS_API_KEY" \
-d '{"content":"#nexa:route-test buy milk"}'
# (3) RAG test (requires at least one indexed memo/note)
curl -X POST https://memos.nuclide.systems/api/v1/memos \
-H "Authorization: Bearer $MEMOS_API_KEY" \
-d '{"content":"#nexa:ask what is the goal of nexa?"}'
```
## Step 8 — Reverse proxy
Already done — Zoraxy at `192.168.1.4:8000` terminates TLS for `*.nuclide.systems` and forwards to docker LXC 104 (`192.168.1.40`). **No new entry is required for Nexa**: every service Nexa talks to already has a host entry.
## Step 9 — Backups
Already covered by Backrest (LXC 103). Add:
- **n8n workflows** → `nexa-core/scripts/backup_workflows.sh` (already present) into a Backrest schedule.
- **Qdrant snapshots** → schedule a daily `POST /collections/nexa_knowledge/snapshots` and rsync to S3 (`s3.nuclide.systems`). Add as a Backrest pre-hook on the docker host.
For deeper detail: [10 — Operations](./10-operations.md).
---
## Phase add-on: graph DB (Phase 3.4)
Defer until 3.13.3 ship. When needed, two minimal options — pick one:
### Option A — Neo4j (lighter, popular for graph-RAG)
```yaml
# nexa-core/docker-compose.graph.yml
services:
neo4j:
image: neo4j:5-community
container_name: nexa-neo4j
ports: ["7687:7687", "7474:7474"]
environment:
NEO4J_AUTH: "neo4j/${NEO4J_PASSWORD}"
NEO4J_server_memory_heap_max__size: 2G
volumes: ["./data/neo4j:/data"]
restart: unless-stopped
```
### Option B — Ontotext GraphDB (SPARQL, RDF native)
Keep the snippet from the original `DEPLOYMENT.md` Schritt 1.3 for `graphdb`. Heavier (~4 GB heap), only justified if you want SHACL / OWL reasoning.
Add a Zoraxy host entry `graph.nuclide.systems``192.168.1.40:7474` (Neo4j) or `:7200` (GraphDB) **only if** browser access is desired; otherwise n8n talks to it on the docker network.
---
## Step-back / rollback
- **Disable any Nexa workflow** in n8n — deactivates the side effect immediately, Memos webhooks become no-ops.
- **Drop a Qdrant collection** — `curl -X DELETE $QDRANT_HOST/collections/nexa_knowledge -H "api-key: $QDRANT_API_KEY"`.
- **Re-discover** — `#nexa:reset-config` then `#nexa:config`.