Files

T

Claude 9b18e2710d Reorganize /docs with numbered TOC; deployment rewrite for actual homelab

- /docs/ now contains 12 numbered guides + index.md (navigator).
- Old duplicates removed: nexa-core/doc/, DEPLOYMENT.md, PLAN.md,
  config/{classification_logic,workflows_spec}.md.
- 09-deployment.md targets the running infra (LXC 104 docker host,
  Zoraxy at 192.168.1.4, existing Memos/n8n/LiteLLM/Nextcloud/Qdrant)
  rather than spinning a parallel stack; minimal-config approach.
- 11-open-questions.md tracks user-info-required blockers.
- 12-optimization-opportunities.md captures homelab tweaks.
- CLAUDE.md added so future agent runs know repo conventions and infra.

2026-05-04 21:18:28 +00:00

3.3 KiB

Raw Blame History

10 — Operations

Day-2 concerns. Backups, monitoring, troubleshooting.

Backups

Backrest (LXC 103) already handles file-level snapshots. Add Nexa-specific items:

What	How	Frequency
n8n workflow JSON	`nexa-core/scripts/backup_workflows.sh` → git push	hourly cron in n8n container
Qdrant `nexa_knowledge`	`POST /collections/nexa_knowledge/snapshots` → rsync to `s3.nuclide.systems`	daily (Backrest pre-hook)
Memos DB	Backrest snapshot of Memos data dir	already covered
Nextcloud	Nextcloud's own backup app + Backrest of `/var/www/nextcloud/data`	already covered
`runtime_config.json` (Qdrant `_config` namespace)	Qdrant snapshot covers it	n/a

Monitoring

Signal	Source	Sink
Container up/down	Dozzle (`192.168.1.40:3553`) + docker healthchecks	Memos system feed via ntfy
n8n workflow failures	n8n built-in failure-webhook	ntfy → `nexa.system` topic
Proxmox / disk / memory alerts	Proxmox notification target → ntfy	ntfy → Memos digest
LiteLLM rate-limit	LiteLLM logs / cost-tracking	Memos morning digest
Backrest run status	Backrest webhook	ntfy

A single ntfy topic nexa.system is the convention; n8n has one workflow that re-broadcasts it as a Memos comment under a pinned [SYSTEM] memo.

Troubleshooting

Memos webhook not firing

# is the Memos webhook config still pointing at n8n?
curl -s -H "Authorization: Bearer $MEMOS_API_KEY" \
  https://memos.nuclide.systems/api/v1/workspace/setting | jq '.webhooks'

# does n8n still expose the path?
curl -i https://n8n.nuclide.systems/webhook/memos     # expect 200/405, never 404

If 404 → workflow is inactive in n8n. Activate.

LiteLLM 401 / 429

401 → the Nexa virtual key was rotated. Reissue, update .env, re-create the n8n credential.
429 → set per-key TPM/RPM limits in LiteLLM admin; classification calls are tiny (<300 tok), embeddings are the bulk.

Qdrant `nexa_knowledge` empty

# is the indexing workflow active?
docker logs nexa-n8n 2>&1 | grep memos_bridge | tail

# manual index test
curl -X POST https://memos.nuclide.systems/api/v1/memos \
  -H "Authorization: Bearer $MEMOS_API_KEY" \
  -d '{"content":"manual probe #nexa"}'

# point count
curl -s -H "api-key: $QDRANT_API_KEY" \
  $QDRANT_HOST/collections/nexa_knowledge | jq '.result.points_count'

Wrong list routing (Work vs Personal)

Run #nexa:route-test <text> and check the confidence value.
If <0.7 the router defaults to Personal (by design — see 06).
Tune the system prompt in nexa-core/ai-prompts/system_prime.txt; commit the change.

"Where did Nexa store this?"

# state dump
curl -X POST https://memos.nuclide.systems/api/v1/memos \
  -d '{"content":"#nexa:export-state"}'
# returns runtime_config + counters as a JSON memo

Log locations

Component	Where
Memos	`docker logs nexa-memos` (LXC 104)
n8n	`docker logs nexa-n8n`
LiteLLM	`docker logs litellm`
Qdrant	`docker logs qdrant_scientific`
Zoraxy	LXC 108 web UI → Statistical Analysis
Backrest	LXC 103 web UI

For a one-shot dump:

ssh nuc 'docker compose -p nexa logs --tail 500' > /tmp/nexa.log

3.3 KiB Raw Blame History