Reorganize /docs with numbered TOC; deployment rewrite for actual homelab
- /docs/ now contains 12 numbered guides + index.md (navigator).
- Old duplicates removed: nexa-core/doc/, DEPLOYMENT.md, PLAN.md,
config/{classification_logic,workflows_spec}.md.
- 09-deployment.md targets the running infra (LXC 104 docker host,
Zoraxy at 192.168.1.4, existing Memos/n8n/LiteLLM/Nextcloud/Qdrant)
rather than spinning a parallel stack; minimal-config approach.
- 11-open-questions.md tracks user-info-required blockers.
- 12-optimization-opportunities.md captures homelab tweaks.
- CLAUDE.md added so future agent runs know repo conventions and infra.
This commit is contained in:
Claude
@@ -0,0 +1,193 @@
|
||||
# 09 — Deployment
|
||||
|
||||
Pragmatic deployment guide that **assumes the existing homelab** and adds only what's missing.
|
||||
|
||||
## What's already running (no action required)
|
||||
|
||||
Surveyed from Homepage / Dozzle / Proxmox / Zoraxy:
|
||||
|
||||
| Service | Host / port | URL |
|
||||
|---------|-------------|-----|
|
||||
| Memos | docker LXC 104 → `:5230` | `https://memos.nuclide.systems` |
|
||||
| n8n | docker LXC 104 → `:5678` | `https://n8n.nuclide.systems` |
|
||||
| LiteLLM (SAIA gateway) | docker LXC 104 → `:4000` | `https://ai.nuclide.systems` (proxies LobeHub UI :3210; API on :4000) |
|
||||
| Nextcloud | LXC 105 | `https://nc.nuclide.systems` |
|
||||
| ntfy | docker LXC 104 → `:7998` | `https://ntfy.nuclide.systems` |
|
||||
| Karakeep (Hoarder) | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` |
|
||||
| Home Assistant | VM 100 (HAOS) | `https://ha.nuclide.systems` |
|
||||
| Pocket-ID (OAuth/SSO) | docker LXC 104 → `:1411` | `https://id.nuclide.systems` |
|
||||
| Vaultwarden | docker LXC 104 → `:11001` | `https://vault.nuclide.systems` |
|
||||
| Backrest | LXC 103 | (internal) |
|
||||
| AdGuard DNS | LXC 102 | (internal) |
|
||||
| Zoraxy reverse proxy | LXC 108 → `192.168.1.4:8000` | TLS for *.nuclide.systems |
|
||||
| `qdrant_scientific` (existing) | docker LXC 104 | reuse — see [11](./11-open-questions.md) |
|
||||
|
||||
The deployment task is **not** "spin up the stack" — most of the stack is already up. It is **wire Nexa across these services + add the small bits that are missing**.
|
||||
|
||||
## What's missing for Nexa
|
||||
|
||||
1. **Qdrant collection** for Nexa (`nexa_knowledge`) — either on the existing `qdrant_scientific` or a dedicated container.
|
||||
2. **n8n workflows** (`./nexa-core/n8n-workflows/`) imported into the running n8n.
|
||||
3. **Nextcloud lists & calendars** for Work / Personal / Shopping / Wishes (auto-discovered via `#nexa:config`).
|
||||
4. **Memos webhook → n8n** wired through the Memos config.
|
||||
5. **LiteLLM virtual key** for the `nexa` user with embedding + chat models.
|
||||
6. **A Zoraxy host entry** is *not* needed — Memos / n8n / LiteLLM are already proxied.
|
||||
7. **(Phase 3.4)** decision on graph DB (Neo4j vs. Ontotext GraphDB) — see [11](./11-open-questions.md).
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Secrets
|
||||
|
||||
Copy `nexa-core/.env.example` → `nexa-core/.env` and fill **only** the secrets:
|
||||
|
||||
```bash
|
||||
cd nexa-core
|
||||
cp .env.example .env
|
||||
$EDITOR .env # MEMOS_API_KEY, SAIA_API_KEY, NC_APP_PASSWORD, QDRANT_API_KEY
|
||||
```
|
||||
|
||||
The `.env` is **only used at bootstrap time**. Everything else (list IDs, calendar IDs, collection sizes) is discovered at runtime via `#nexa:config` (see [05](./05-command-system.md)). No secrets should ever live in n8n workflow JSON — use n8n credentials instead.
|
||||
|
||||
## Step 2 — Qdrant collection
|
||||
|
||||
```bash
|
||||
# adjust QDRANT_HOST in .env first
|
||||
source nexa-core/.env
|
||||
|
||||
curl -X PUT "$QDRANT_HOST/collections/nexa_knowledge" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "api-key: $QDRANT_API_KEY" \
|
||||
-d @nexa-core/config/qdrant_schema.json
|
||||
```
|
||||
|
||||
> If reusing `qdrant_scientific`, this just adds another collection; no new container needed.
|
||||
|
||||
## Step 3 — LiteLLM virtual key
|
||||
|
||||
In the LiteLLM admin UI (`ai.nuclide.systems`):
|
||||
|
||||
1. Create user `nexa`.
|
||||
2. Issue a virtual key with access to one chat model (e.g. `gpt-4o-mini` / a local Sonnet equivalent) and one embedding model (`text-embedding-3-small`).
|
||||
3. Paste the key into `SAIA_API_KEY` in `.env`.
|
||||
|
||||
## Step 4 — n8n workflows
|
||||
|
||||
Import the JSON exports — credentials are filled inside n8n, not in the JSON:
|
||||
|
||||
```bash
|
||||
# n8n personal access token from the n8n UI: Settings → API
|
||||
N8N_URL=https://n8n.nuclide.systems
|
||||
N8N_TOKEN=... # from the n8n UI
|
||||
|
||||
for f in nexa-core/n8n-workflows/phase-1/*.json \
|
||||
nexa-core/n8n-workflows/phase-2/*.json; do
|
||||
curl -X POST "$N8N_URL/api/v1/workflows" \
|
||||
-H "X-N8N-API-KEY: $N8N_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data-binary "@$f"
|
||||
done
|
||||
```
|
||||
|
||||
Inside n8n, attach credentials to the imported nodes:
|
||||
|
||||
- **Memos** → HTTP header `Authorization: Bearer $MEMOS_API_KEY`
|
||||
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY`
|
||||
- **Nextcloud** → app password
|
||||
- **Qdrant** → header `api-key: $QDRANT_API_KEY`
|
||||
|
||||
Activate each workflow individually after smoke-test.
|
||||
|
||||
## Step 5 — Memos webhook
|
||||
|
||||
In the Memos admin UI, set the webhook URL to the production address of the discovery workflow:
|
||||
|
||||
```
|
||||
https://n8n.nuclide.systems/webhook/memos
|
||||
```
|
||||
|
||||
The same URL is the one the workflow exposes; verify with:
|
||||
|
||||
```bash
|
||||
curl -i https://n8n.nuclide.systems/webhook/memos
|
||||
# expect 200 / 405, never 404
|
||||
```
|
||||
|
||||
## Step 6 — Bootstrap commands via Memos
|
||||
|
||||
Create a memo with body `#nexa:config` — the discovery workflow:
|
||||
|
||||
1. Lists Nextcloud Tasks lists, picks Work / Personal / Shopping / Wishes by name.
|
||||
2. Counts existing Qdrant points in `nexa_knowledge`.
|
||||
3. Verifies LiteLLM reachability + lists available models.
|
||||
4. Replies as a comment with a **runtime-config snapshot** that's stored as Qdrant metadata (`_config` namespace) and as `nexa-core/config/runtime_config.json` (gitignored).
|
||||
|
||||
After this point, `.env` is read-once. Subsequent runs read config from Qdrant.
|
||||
|
||||
## Step 7 — Smoke tests
|
||||
|
||||
```bash
|
||||
# (1) Memos round-trip — should produce a comment within ~5 s
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"content":"- [ ] testing the router #nexa"}'
|
||||
|
||||
# (2) Classification dry-run
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-d '{"content":"#nexa:route-test buy milk"}'
|
||||
|
||||
# (3) RAG test (requires at least one indexed memo/note)
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-d '{"content":"#nexa:ask what is the goal of nexa?"}'
|
||||
```
|
||||
|
||||
## Step 8 — Reverse proxy
|
||||
|
||||
Already done — Zoraxy at `192.168.1.4:8000` terminates TLS for `*.nuclide.systems` and forwards to docker LXC 104 (`192.168.1.40`). **No new entry is required for Nexa**: every service Nexa talks to already has a host entry.
|
||||
|
||||
## Step 9 — Backups
|
||||
|
||||
Already covered by Backrest (LXC 103). Add:
|
||||
|
||||
- **n8n workflows** → `nexa-core/scripts/backup_workflows.sh` (already present) into a Backrest schedule.
|
||||
- **Qdrant snapshots** → schedule a daily `POST /collections/nexa_knowledge/snapshots` and rsync to S3 (`s3.nuclide.systems`). Add as a Backrest pre-hook on the docker host.
|
||||
|
||||
For deeper detail: [10 — Operations](./10-operations.md).
|
||||
|
||||
---
|
||||
|
||||
## Phase add-on: graph DB (Phase 3.4)
|
||||
|
||||
Defer until 3.1–3.3 ship. When needed, two minimal options — pick one:
|
||||
|
||||
### Option A — Neo4j (lighter, popular for graph-RAG)
|
||||
|
||||
```yaml
|
||||
# nexa-core/docker-compose.graph.yml
|
||||
services:
|
||||
neo4j:
|
||||
image: neo4j:5-community
|
||||
container_name: nexa-neo4j
|
||||
ports: ["7687:7687", "7474:7474"]
|
||||
environment:
|
||||
NEO4J_AUTH: "neo4j/${NEO4J_PASSWORD}"
|
||||
NEO4J_server_memory_heap_max__size: 2G
|
||||
volumes: ["./data/neo4j:/data"]
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
### Option B — Ontotext GraphDB (SPARQL, RDF native)
|
||||
|
||||
Keep the snippet from the original `DEPLOYMENT.md` Schritt 1.3 for `graphdb`. Heavier (~4 GB heap), only justified if you want SHACL / OWL reasoning.
|
||||
|
||||
Add a Zoraxy host entry `graph.nuclide.systems` → `192.168.1.40:7474` (Neo4j) or `:7200` (GraphDB) **only if** browser access is desired; otherwise n8n talks to it on the docker network.
|
||||
|
||||
---
|
||||
|
||||
## Step-back / rollback
|
||||
|
||||
- **Disable any Nexa workflow** in n8n — deactivates the side effect immediately, Memos webhooks become no-ops.
|
||||
- **Drop a Qdrant collection** — `curl -X DELETE $QDRANT_HOST/collections/nexa_knowledge -H "api-key: $QDRANT_API_KEY"`.
|
||||
- **Re-discover** — `#nexa:reset-config` then `#nexa:config`.
|
||||
Reference in New Issue
Block a user