Reorganize /docs with numbered TOC; deployment rewrite for actual homelab
- /docs/ now contains 12 numbered guides + index.md (navigator).
- Old duplicates removed: nexa-core/doc/, DEPLOYMENT.md, PLAN.md,
config/{classification_logic,workflows_spec}.md.
- 09-deployment.md targets the running infra (LXC 104 docker host,
Zoraxy at 192.168.1.4, existing Memos/n8n/LiteLLM/Nextcloud/Qdrant)
rather than spinning a parallel stack; minimal-config approach.
- 11-open-questions.md tracks user-info-required blockers.
- 12-optimization-opportunities.md captures homelab tweaks.
- CLAUDE.md added so future agent runs know repo conventions and infra.
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
# 01 — Vision & Scope
|
||||
|
||||
## Vision
|
||||
|
||||
Nexa is the **central nervous system** of a personal IT setup: an intelligent middleware sitting between input sources (Memos, e-mail, RSS, Karakeep, Bluesky), knowledge stores (Obsidian, Qdrant, Nextcloud) and organization tools (Nextcloud Calendar/Tasks).
|
||||
|
||||
### Core goals
|
||||
|
||||
- **Cognitive offload** — Nexa sorts, filters and proposes; it doesn't just store.
|
||||
- **Context separation** — clean AI-driven split between *Work* and *Personal*.
|
||||
- **Single point of interaction** — Memos is the primary surface ("voice & ear").
|
||||
- **Knowledge synergy** — link ephemeral memos to deep notes via semantic search.
|
||||
|
||||
## Scope
|
||||
|
||||
### In-scope
|
||||
|
||||
- Triaging ~20 personal e-mails per day.
|
||||
- Routing tasks to the right Nextcloud list (Work vs. Personal).
|
||||
- Suggesting time-boxes in the work calendar.
|
||||
- Long-term memory in Qdrant (and optionally a graph DB).
|
||||
- Aggregating critical IT alerts (Proxmox, Backrest) into a Memos system feed.
|
||||
|
||||
### Out-of-scope
|
||||
|
||||
- Direct access to work IT (work mail, work servers).
|
||||
- Active document editing (metadata extraction only).
|
||||
- Replacing specialist UIs (Arcane / Dozzle / Portainer for containers).
|
||||
|
||||
## Success metrics
|
||||
|
||||
- **Reduced mail interaction** — less time in the mail app, more on the Nexa digest.
|
||||
- **Calendar fill-rate** — focus slots auto-blocked on the work calendar.
|
||||
- **Search latency** — old memos / Obsidian notes resurface instantly via `#nexa:ask`.
|
||||
@@ -0,0 +1,48 @@
|
||||
# 02 — Roadmap & Phases
|
||||
|
||||
Iterative build-out, value-first. Each phase is shippable on its own.
|
||||
|
||||
## Phase 1 — The Spine (connectivity)
|
||||
|
||||
**Focus:** datapath between Memos, n8n and SAIA.
|
||||
**Milestone:** Nexa replies on Memos and answers simple questions.
|
||||
|
||||
- [ ] **1.1 Memos ↔ n8n bridge** — webhook ingress, content filter, comment-back.
|
||||
- [ ] **1.2 SAIA integration** — call LiteLLM proxy from n8n; chat via Memos comments.
|
||||
- [ ] **1.3 Git-sync** — periodic export of n8n workflows into this repo.
|
||||
|
||||
## Phase 2 — Senses (input channels)
|
||||
|
||||
**Focus:** e-mail filter and the Work-vs-Personal router.
|
||||
**Milestone:** Nexa distinguishes work tasks from personal tasks without manual tags.
|
||||
|
||||
- [ ] **2.1 Email butler** — filter & summarize the ~20 daily personal mails.
|
||||
- [ ] **2.2 RSS morning digest** — curated daily summary in Memos.
|
||||
- [ ] **2.3 Bluesky antenna** — store liked posts as knowledge snippets.
|
||||
|
||||
## Phase 3 — Memory (Qdrant & RAG)
|
||||
|
||||
**Focus:** Qdrant + (optional) graph DB for retrieval-augmented answers.
|
||||
**Milestone:** RAG works — Nexa answers from archived notes.
|
||||
|
||||
- [ ] **3.1 Embedding pipeline** — automatic vectorization of Memos & Obsidian.
|
||||
- [ ] **3.2 Context retrieval** — `#nexa:ask` reads Qdrant before answering.
|
||||
- [ ] **3.3 S3 long-term archive** — Qdrant snapshots to the S3 server.
|
||||
- [ ] **3.4 Graph layer (deferred decision — see [11-open-questions](./11-open-questions.md))** — Neo4j *or* Ontotext GraphDB for structural queries.
|
||||
|
||||
## Phase 4 — Motor (organization & action)
|
||||
|
||||
**Focus:** calendar integration and time-boxing.
|
||||
**Milestone:** Nexa proactively proposes morning focus slots in the work calendar.
|
||||
|
||||
- [ ] **4.1 Nextcloud Tasks sync** — bidirectional status sync (Memos ↔ NC Tasks).
|
||||
- [ ] **4.2 Time-boxing agent** — calendar-aware slot suggestions.
|
||||
- [ ] **4.3 Karakeep curation** — automated review cycle for saved links.
|
||||
|
||||
## Phase 5 — Daily integration (polish)
|
||||
|
||||
**Focus:** Home Assistant Voice and monitoring.
|
||||
**Milestone:** Nexa speaks via HA-Voice and surfaces critical system states proactively.
|
||||
|
||||
- [ ] **5.1 HA Voice** — Wake-word integration, Wyoming protocol.
|
||||
- [ ] **5.2 System monitoring** — Proxmox / Backrest alerts into the Memos system feed via ntfy.
|
||||
@@ -0,0 +1,63 @@
|
||||
# 03 — Architecture Overview
|
||||
|
||||
A one-page mental model. For details follow the cross-links.
|
||||
|
||||
## Topology
|
||||
|
||||
```
|
||||
┌────────────────┐
|
||||
voice / typing ──────▶│ Memos │◀──── Nexa replies as comments
|
||||
│ (interface) │
|
||||
└───────┬────────┘
|
||||
│ webhook (- [ ] / #nexa:*)
|
||||
▼
|
||||
┌────────────────┐
|
||||
IMAP / RSS / NC ────▶│ n8n │◀──── workflows live in
|
||||
Karakeep / Bluesky │ (logic) │ ./nexa-core/n8n-workflows
|
||||
└───┬────────┬───┘
|
||||
classify ▲ │ │ ▲ retrieve
|
||||
│ ▼ ▼ │
|
||||
┌──────────┐ ┌──────────┐
|
||||
│ SAIA │ │ Qdrant │
|
||||
│ LiteLLM │ │ vector │
|
||||
└──────────┘ └────┬─────┘
|
||||
│ optional
|
||||
▼
|
||||
┌──────────┐
|
||||
│ Graph │ (Neo4j or Ontotext —
|
||||
│ DB │ see open questions)
|
||||
└──────────┘
|
||||
│
|
||||
▼ writes
|
||||
┌──────────────────────────────┐
|
||||
│ Nextcloud (Tasks, Calendar, │
|
||||
│ Mail, Files / Obsidian) │
|
||||
└──────────────────────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Role | Where it runs (today) |
|
||||
|-----------|------|-----------------------|
|
||||
| **Memos** | Interface, voice input, webhook source | docker host LXC 104 → `memos.nuclide.systems` |
|
||||
| **n8n** | Workflow / logic engine | docker host LXC 104 → `n8n.nuclide.systems` |
|
||||
| **SAIA / LiteLLM** | Model gateway, embeddings, classification | docker host LXC 104 → `ai.nuclide.systems` (LiteLLM internal :4000) |
|
||||
| **Qdrant** | Vector memory (semantic recall) | docker host LXC 104 (existing `qdrant_scientific` may be reused — see [11](./11-open-questions.md)) |
|
||||
| **Graph DB** | Structural memory (deferred) | not yet deployed |
|
||||
| **Nextcloud** | Tasks, calendar, mail, files | dedicated LXC 105 → `nc.nuclide.systems` |
|
||||
| **ntfy** | Push channel for system alerts | docker host → `ntfy.nuclide.systems` |
|
||||
| **Backrest** | Backup orchestration | LXC 103 |
|
||||
| **Zoraxy** | Reverse proxy + TLS | LXC 108 (`192.168.1.4:8000`) |
|
||||
| **AdGuard DNS** | Internal name resolution | LXC 102 |
|
||||
| **Home Assistant** | Voice + house automation | VM 100 (HAOS) |
|
||||
|
||||
## Two-pillar memory
|
||||
|
||||
- **Qdrant** answers *"what is similar / relevant?"* (cosine search over embeddings).
|
||||
- **Graph DB** answers *"who, what depends on what, how is it structured?"* (Cypher / SPARQL).
|
||||
|
||||
Both pillars are queried in parallel for `#nexa:ask` and merged before SAIA generates the final answer. See [08 — GraphRAG architecture](./08-graphrag-architecture.md).
|
||||
|
||||
## Dual-context routing
|
||||
|
||||
Every input is classified `work` or `personal` before any side effect (task creation, calendar write). See [04 — Integration matrix](./04-integration-matrix.md) and [06 — Classification logic](./06-classification-logic.md).
|
||||
@@ -0,0 +1,79 @@
|
||||
# 04 — Integrations-Matrix
|
||||
|
||||
Diese Matrix definiert die logische Trennung zwischen privaten und beruflichen Datenströmen sowie die Anbindung der Infrastruktur.
|
||||
|
||||
---
|
||||
|
||||
## 1. Die Dualität: Arbeit vs. Privat
|
||||
Nexa muss strikt zwischen zwei Kontexten unterscheiden, da die Datenquellen variieren:
|
||||
|
||||
### A. Bereich: ARBEIT (Work)
|
||||
* **Sichtbarkeit:** Eingeschränkt (kein Zugriff auf Arbeits-E-Mails oder Firmen-Server).
|
||||
* **Datenquellen:**
|
||||
* Separate Nextcloud Task-Liste (`Work_Tasks`).
|
||||
* Separater Nextcloud Kalender (`Work_Calendar`).
|
||||
* Memos mit Tag `#work` oder semantischer Erkennung.
|
||||
* **Nexa-Fokus:** Timeblocking, Fokus-Zeiten, Vorbereitung von Meetings basierend auf Obsidian-Notizen.
|
||||
|
||||
### B. Bereich: PRIVAT (Personal)
|
||||
* **Sichtbarkeit:** Vollständig.
|
||||
* **Datenquellen:**
|
||||
* Nextcloud Task-Liste (`Personal_Tasks`).
|
||||
* Haupt-Kalender.
|
||||
* E-Mails (IMAP - 20 Mails/Tag).
|
||||
* Bluesky, RSS, Karakeep.
|
||||
* **Nexa-Fokus:** Automatisierung des Alltags, Kuratierung von Wissen, E-Mail-Management.
|
||||
|
||||
### Karakeep & Shopping-Logik
|
||||
* **Schnittstelle:** Karakeep API / RSS.
|
||||
* **Listen-Routing:**
|
||||
* **Einkaufsliste:** Automatischer Sync für Items mit hoher Frequenz (Lebensmittel, Drogerie).
|
||||
* **Wunschliste:** Speicherort für "Entdeckungen". Nexa fügt bei Wunschlisten-Items automatisch den aktuellen Preis und eine kurze SAIA-Zusammenfassung hinzu ("Warum du das speichern wolltest").
|
||||
* **Review-Cycle:** Einmal im Monat fragt Nexa bei Wunschlisten-Items nach: "Immer noch interessiert oder kann das weg?"
|
||||
|
||||
---
|
||||
|
||||
## 2. Kern-Infrastruktur (The Brain & Spine)
|
||||
|
||||
| Komponente | Rolle im System |
|
||||
| :--- | :--- |
|
||||
| **Memos** | Zentraler Input (Drafts, Ideen, schnelle Tasks). Schnittstelle für Nexa-Antworten. |
|
||||
| **n8n** | Logik-Engine. Führt die Klassifizierung Arbeit vs. Privat durch. |
|
||||
| **SAIA (LiteLLM)** | Entscheidet anhand des Inhalts, in welchen Kalender/Liste ein Eintrag gehört. |
|
||||
| **Qdrant** | Langzeitgedächtnis. Speichert Projektwissen (Work) und privates Wissen getrennt. |
|
||||
| **Arcane** | Verwaltung der Docker-Container (n8n, Qdrant, Memos). |
|
||||
|
||||
---
|
||||
|
||||
## 3. Spezifische Datenflüsse & Logik
|
||||
|
||||
### Task-Routing (Das Gehirn-Filter)
|
||||
1. **Input:** Neues Memo oder Spracheingabe.
|
||||
2. **Analyse:** SAIA prüft: "Ist das Business oder Privat?"
|
||||
3. **Routing:** * `Arbeits-Kontext` -> Eintrag in `Work_Tasks`.
|
||||
* `Privat-Kontext` -> Eintrag in `Personal_Tasks`.
|
||||
4. **Timeboxing:** Nexa scannt den `Work_Calendar` auf Lücken und schlägt Slots für `Work_Tasks` vor.
|
||||
|
||||
### E-Mail Management (Nur Privat)
|
||||
* Nexa filtert die 20 täglichen Privat-Mails.
|
||||
* Wichtige private Termine werden in den privaten Kalender extrahiert.
|
||||
* Newsletter landen als Zusammenfassung in Memos.
|
||||
|
||||
### Wissens-Management (Obsidian & Qdrant)
|
||||
* Obsidian-Notizen (Arbeit & Privat) werden in Qdrant indiziert.
|
||||
* Nexa nutzt dieses Wissen, um Arbeits-Tasks besser zu beschreiben, auch wenn sie keinen Zugriff auf die Arbeits-E-Mails hat.
|
||||
|
||||
---
|
||||
|
||||
## 4. Monitoring & Hardware (Home Assistant)
|
||||
|
||||
| Dienst | Reiz-Typ | Nexa-Reaktion |
|
||||
| :--- | :--- | :--- |
|
||||
| **Home Assistant** | Voice / Sensorik | Nexa nimmt Sprachbefehle entgegen und meldet Alarme (Haus). |
|
||||
| **Proxmox** | Stabilität | Meldung an Nexa bei Hardware-Problemen. |
|
||||
| **S3 Server** | Archiv | Langzeit-Backups der Wissensdatenbank. |
|
||||
|
||||
---
|
||||
|
||||
## 5. Ergänzungen für das Setup (Vorschlag)
|
||||
* **Zwei-Faktor-Klassifizierung:** Einführung von expliziten Tags (`#w` für Work, `#p` für Privat) für Grenzfälle, in denen SAIA den Kontext nicht eindeutig bestimmen kann.
|
||||
@@ -0,0 +1,131 @@
|
||||
# 05 — Command System (`#nexa:*`)
|
||||
|
||||
Nexa "hört" auf folgende Kommandos in Memos-Kommentaren oder als Memo-Inhalt mit Hashtag:
|
||||
|
||||
## 🔧 Konfigurations-Kommandos
|
||||
|
||||
### #nexa:config
|
||||
- **Beschreibung**: Triggert Autodiscovery aller Services (Nextcloud, Qdrant, etc.)
|
||||
- **Antwort**: Postet Report mit gefundenen Konfigurationen
|
||||
- **Speichert**: Settings in DynoDB / Qdrant Metadata (nicht in .env)
|
||||
|
||||
```
|
||||
#nexa:config
|
||||
```
|
||||
|
||||
**Beispiel-Response:**
|
||||
```
|
||||
✅ Autodiscovery abgeschlossen:
|
||||
- Nextcloud Listen: Work (id=123), Personal (id=456), Shopping (id=789), Wishes (id=1011)
|
||||
- Qdrant Collection: nexa_knowledge (1536 dims, Cosine)
|
||||
- Nextcloud Mail: 3 Accounts erkannt (Office 365, Gmail, Privat)
|
||||
```
|
||||
|
||||
### #nexa:status
|
||||
- **Beschreibung**: Zeigt den aktuellen System-Status
|
||||
- **Antwort**: Uptime, verbundene Systeme, Memory-Stats
|
||||
|
||||
```
|
||||
#nexa:status
|
||||
```
|
||||
|
||||
### #nexa:sync-obsidian
|
||||
- **Beschreibung**: Triggert sofortige Synchronisierung mit Obsidian Vault
|
||||
- **Optional Parameter**: `--vault=/path/to/vault` oder `--force` für Neuindexierung
|
||||
|
||||
```
|
||||
#nexa:sync-obsidian --force
|
||||
```
|
||||
|
||||
## 📊 Memory & RAG Kommandos
|
||||
|
||||
### #nexa:ask [question]
|
||||
- **Beschreibung**: Semantische Suche in Qdrant (RAG) basierend auf Obsidian + Memos
|
||||
- **Antwort**: Top-3 Treffer mit Quellen
|
||||
|
||||
```
|
||||
#nexa:ask Wie implementierten wir das JWT-Middleware-Pattern?
|
||||
```
|
||||
|
||||
### #nexa:learn [topic]
|
||||
- **Beschreibung**: Explizit neue Information in Qdrant speichern (mit Tags)
|
||||
- **Optional**: `--tag=architecture` `--source=obsidian/notes/arch.md`
|
||||
|
||||
```
|
||||
#nexa:learn Das Routing-Schema unterscheidet Work vs. Personal via SAIA-Kontext-Analyse --tag=architecture
|
||||
```
|
||||
|
||||
## 🎯 Workflow-Kommandos
|
||||
|
||||
### #nexa:route-test [text]
|
||||
- **Beschreibung**: Testet die Klassifizierung (Work/Personal) ohne Task zu erstellen
|
||||
|
||||
```
|
||||
#nexa:route-test Muss morgen die Präsentation für den Client fertigstellen
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```
|
||||
💬 Klassifizierung (Test-Mode):
|
||||
- Kontext: WORK
|
||||
- Vertrauen: 0.95
|
||||
- Begründung: "Client-Präsentation → professioneller Kontext"
|
||||
```
|
||||
|
||||
### #nexa:email-digest
|
||||
- **Beschreibung**: Triggert sofortige Email-Zusammenfassung (sonst tägl. 7 Uhr)
|
||||
|
||||
```
|
||||
#nexa:email-digest
|
||||
```
|
||||
|
||||
## 🔐 Admin-Kommandos
|
||||
|
||||
### #nexa:reset-config
|
||||
- **Beschreibung**: Löscht gespeicherte Konfiguration, triggert neuen Autodiscovery
|
||||
- **Warnung**: Alle benutzerdefinierten Settings werden vergessen
|
||||
|
||||
```
|
||||
#nexa:reset-config
|
||||
```
|
||||
|
||||
### #nexa:export-state
|
||||
- **Beschreibung**: Exportiert aktuellen State als JSON (für Backups)
|
||||
|
||||
```
|
||||
#nexa:export-state
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Implementierung in n8n
|
||||
|
||||
Ein **Command Parser** läuft immer mit:
|
||||
|
||||
1. **Memos Webhook** empfängt alle Memos
|
||||
2. **Regex Check**: Sucht nach `#nexa:command`
|
||||
3. **Router**: Versendet an entsprechenden n8n-Workflow
|
||||
4. **Antwort**: Postet Reply als Kommentar/Edit
|
||||
|
||||
**Command Parser Regex:**
|
||||
```
|
||||
^#nexa:(\w+)(?:\s+([^\n]*?))?(?:$|\s*--)
|
||||
```
|
||||
|
||||
Extrahiert: `[command, parameters]`
|
||||
|
||||
---
|
||||
|
||||
## 🎛️ Dynamische Konfigurationsspeicherung
|
||||
|
||||
Statt .env zu editieren:
|
||||
|
||||
1. **First Run**: `#nexa:config` speichert zu lokalen Metadata
|
||||
2. **Speich-Ziel**:
|
||||
- **Primary**: Qdrant Metadata (als `_config` Namespace)
|
||||
- **Fallback**: `nexa-core/config/runtime_config.json` (gitignored)
|
||||
- **Notfall**: .env (nur initial)
|
||||
|
||||
3. **Reload-Logik**: Bei jedem Workflow-Start werden Settings aus Qdrant geladen
|
||||
|
||||
Das macht Nexa vollständig selbstständig nach dem initialem Setup!
|
||||
@@ -0,0 +1,19 @@
|
||||
# 06 — Klassifizierungs-Logik
|
||||
|
||||
Dieser Fragebogen dient der Feinabstimmung von SAIA, um Tasks korrekt zu routen.
|
||||
|
||||
### A. Schlüsselwörter & Projekte (Arbeit)
|
||||
*Welche Begriffe triggern zwingend die Work-Liste?*
|
||||
- [ ] Projekt-Namen (z.B. "Nexa-Core", "Infrastruktur-Audit")
|
||||
- [ ] Rollenspezifische Begriffe ("Meeting", "Report", "Deadline")
|
||||
- [ ] Tools, die nur im Job vorkommen.
|
||||
|
||||
### B. Ausschlusskriterien (Privat)
|
||||
*Was darf niemals in die Work-Liste?*
|
||||
- [ ] Lebensmittel, Rezepte, Haushalt.
|
||||
- [ ] Bluesky-Input (sofern nicht explizit als Recherche markiert).
|
||||
- [ ] Finanz-Mails (Privat-Bank).
|
||||
|
||||
### C. Umgang mit Unschärfe
|
||||
- **Default:** Wenn < 70% Sicherheit -> `Personal`.
|
||||
- **Interaktion:** Nexa erstellt einen Kommentar: "In welche Liste soll das? [W/P]".
|
||||
@@ -0,0 +1,16 @@
|
||||
# 07 — Workflow Spec: Phase-2 Task-Router
|
||||
|
||||
## 1. Trigger
|
||||
* **Typ:** Webhook (Memos)
|
||||
* **Filter:** `payload.content` enthält `- [ ]` ODER `#todo`.
|
||||
|
||||
## 2. Intelligence Node (SAIA)
|
||||
* **Prompt:** "Analysiere folgende Notiz. Gib als JSON zurück: { "context": "work" | "personal", "summary": "string", "urgency": 1-5 }."
|
||||
* **Kontext-Zufuhr:** Lade die letzten 5 Obsidian-Projekttitel aus Qdrant als Referenz hoch.
|
||||
|
||||
## 3. Switch Node
|
||||
* **Route 1:** Context == "work" -> Nextcloud Create Task (List ID: Work).
|
||||
* **Route 2:** Context == "personal" -> Nextcloud Create Task (List ID: Personal).
|
||||
|
||||
## 4. Feedback Loop
|
||||
* Poste die Task-ID und den gewählten Kontext als Kommentar unter das Original-Memo.
|
||||
@@ -0,0 +1,277 @@
|
||||
# 08 — GraphRAG: Structural Knowledge & Relations
|
||||
|
||||
> ⚠️ **Open decision (see [11-open-questions](./11-open-questions.md))**: this document describes the design with **Neo4j**. The earlier deployment draft referenced **Ontotext GraphDB** (SPARQL). Pick one before Phase 3.4.
|
||||
|
||||
## GraphRAG für Nexa — Strukturelles Wissen & Beziehungen
|
||||
|
||||
## 🧠 Zwei-Säulen-Memory-Architektur
|
||||
|
||||
Nexa nutzt zwei komplementäre Speicher für vollständiges Verständnis:
|
||||
|
||||
### Säule 1: Qdrant (Vector DB) - Semantische Ähnlichkeit
|
||||
- **Frage**: "Was ist ähnlich, was ist relevant?"
|
||||
- **Speichert**: Embeddings von Memos, Obsidian-Notes, E-Mail-Digests
|
||||
- **Suche**: Cosine Similarity über 1536-dimensionale Vektoren
|
||||
- **Antwort**: Top-k semantisch ähnliche Dokumente (RAG)
|
||||
|
||||
**Beispiel:**
|
||||
```
|
||||
Frage: "Wie war die JWT-Implementation in unserem letzten Projekt?"
|
||||
→ Qdrant findet: 10 ähnliche Snippets → Prompt-Context
|
||||
```
|
||||
|
||||
### Säule 2: Neo4j (Graph DB) - Strukturelle Beziehungen
|
||||
- **Frage**: "Wer ist beteiligt? Was hängt davon ab? Wie ist es organisiert?"
|
||||
- **Speichert**:
|
||||
- **Knoten (Nodes)**: Projects, Tasks, People, Technologies, Dates
|
||||
- **Kanten (Relationships)**: `USES`, `OWNS`, `DEPENDS_ON`, `MENTIONS`, `CREATED_BY`
|
||||
- **Suche**: Cypher-Queries über Netzwerk-Topologie
|
||||
- **Antwort**: Pfade, zentrale Knoten, Abhängigkeitsanalyse
|
||||
|
||||
**Beispiel:**
|
||||
```cypher
|
||||
MATCH (tech:Technology)-[USES*]->*(service:Service),
|
||||
(tech)-[:MENTIONED_IN]->(task:Task)
|
||||
WHERE tech.name = "JWT" AND task.status = "DONE"
|
||||
RETURN tech, service, task
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Neo4j Schema für Nexa
|
||||
|
||||
### Node-Typen
|
||||
|
||||
```cypher
|
||||
(Project {name, startDate, status, priority})
|
||||
(Task {title, description, status, urgency: 1-5, context: "work"|"personal"})
|
||||
(Person {name, role, email})
|
||||
(Technology {name, category: "framework"|"language"|"tool"})
|
||||
(Topic {name, domain})
|
||||
(File {path, type, lastModified})
|
||||
(Note {id, content_hash, created_at})
|
||||
```
|
||||
|
||||
### Relationship-Typen
|
||||
|
||||
| Beziehung | Von | Zu | Beschreibung |
|
||||
|-----------|-----|-----------|-------------|
|
||||
| `OWNS` | Person | Task/Project | Wer ist verantwortlich |
|
||||
| `USES` | Task/Project | Technology | Welche Tech wird verwendet |
|
||||
| `DEPENDS_ON` | Task/Project | Task/Project | Hängt davon ab |
|
||||
| `CREATED_BY` | Note | Person | Autor einer Notiz |
|
||||
| `MENTIONS` | Note/Task | Topic/Technology | Verweist auf |
|
||||
| `RELATED_TO` | Task | Task | Zusammenhängend |
|
||||
| `CHILD_OF` | Task | Project | Task gehört zu Project |
|
||||
| `SCHEDULED_FOR` | Task | Date | Zeitliche Zuordnung |
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Synchronisierungs-Flows
|
||||
|
||||
### 1. **Memos → Neo4j** (automatisch beim Erstellen)
|
||||
|
||||
Wenn ein Memo mit `- [ ]` erstellt wird:
|
||||
|
||||
```
|
||||
Memo-Content: "Muss JWT-Middleware für Auth-Service refaktorieren"
|
||||
↓
|
||||
SAIA extrahiert Entitäten:
|
||||
- Technologie: "JWT-Middleware", "Auth-Service"
|
||||
- Kontext: "work"
|
||||
- Urgenz: 3
|
||||
↓
|
||||
Erstelle in Neo4j:
|
||||
- Task node (title="JWT-Middleware refaktorieren")
|
||||
- Technology nodes: JWT, Auth-Service
|
||||
- Relationships: Task -[USES]-> JWT, Task -[USES]-> Auth-Service
|
||||
- Task -[CREATED_BY]-> Me (Person node)
|
||||
```
|
||||
|
||||
### 2. **Obsidian → Neo4j** (#nexa:sync-obsidian)
|
||||
|
||||
Beim Sync von Obsidian-Vault:
|
||||
|
||||
```
|
||||
File: obsidian/projects/nexa-deployment.md
|
||||
↓
|
||||
Header Struktur wird zu Graph:
|
||||
- Project "Nexa Deployment"
|
||||
- Technologies: Docker, n8n, Nextcloud
|
||||
- Dependencies: "requires infrastructure setup"
|
||||
↓
|
||||
Erstelle:
|
||||
- Project nodes mit Metadata
|
||||
- Hierarchie: Project -[CONTAINS]-> Task -[USES]-> Technology
|
||||
```
|
||||
|
||||
### 3. **Nextcloud Tasks → Neo4j** (bidirektional)
|
||||
|
||||
Wenn Task in Nextcloud erstellt/gelöcht:
|
||||
|
||||
```
|
||||
Nextcloud Task: "Review JWT implementation" (in Work-List)
|
||||
↓
|
||||
n8n Trigger erkennt Änderung
|
||||
↓
|
||||
Update Neo4j:
|
||||
- Task node status = "active"
|
||||
- Task -[DEPENDS_ON]-> related Task (falls Beziehung existiert)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🤖 GraphRAG Queries - Praktische Anwendungen
|
||||
|
||||
### Query 1: "Zeige alle offenen Tasks, die JWT betreffen"
|
||||
|
||||
```cypher
|
||||
MATCH (tech:Technology {name:"JWT"}),
|
||||
(tech)-[:MENTIONED_IN|USES]-(task:Task),
|
||||
(task)-[:CHILD_OF]->(project:Project)
|
||||
WHERE task.status IN ["NEEDS-ACTION", "IN-PROGRESS"]
|
||||
RETURN project.name, task.title, task.urgency
|
||||
ORDER BY task.urgency DESC
|
||||
```
|
||||
|
||||
### Query 2: "Von welchen Tasks/Projects hängt Auth-Service ab?"
|
||||
|
||||
```cypher
|
||||
MATCH path = (dependent)-[:DEPENDS_ON*]->(service:Technology {name:"Auth-Service"})
|
||||
RETURN [x IN nodes(path) | x.name], length(path)
|
||||
ORDER BY length(path) ASC
|
||||
```
|
||||
|
||||
### Query 3: "Wer arbeitet an diesem Projekt und auf welchen Technologien?"
|
||||
|
||||
```cypher
|
||||
MATCH (person:Person)-[:OWNS]->(task:Task)-[:CHILD_OF]->(project:Project {name:"Nexa"}),
|
||||
(task)-[:USES]->(tech:Technology)
|
||||
RETURN person.name, collect(task.title), collect(tech.name)
|
||||
```
|
||||
|
||||
### Query 4: "Was ist seit gestern neu an Technologien/Themen gelernt?"
|
||||
|
||||
```cypher
|
||||
MATCH (note:Note)-[:MENTIONS]->(topic:Topic)
|
||||
WHERE note.created_at > datetime.now() - duration('P1D')
|
||||
RETURN DISTINCT topic.name, count(note) as mentions
|
||||
ORDER BY mentions DESC
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔗 GraphRAG-Enhanced Answer Generation
|
||||
|
||||
Wenn Nutzer `#nexa:ask` fragt:
|
||||
|
||||
```
|
||||
#nexa:ask Wie stehen die JWT-Tasks zur Deadline?
|
||||
↓
|
||||
┌─────────────┴──────────────┐
|
||||
↓ ↓
|
||||
[Qdrant] [Neo4j]
|
||||
Semantic Structural
|
||||
Search Search
|
||||
↓ ↓
|
||||
Top-3 Notes (JWT)-[DEPENDS_ON]
|
||||
mit ähnlichem → Connected Tasks
|
||||
Kontext → Critical Path
|
||||
↓ ↓
|
||||
Combine: Merge + Rank
|
||||
↓
|
||||
System Prompt (SAIA):
|
||||
"Basierend auf diesem Wissen und diesen Abhängigkeiten, beantworte:"
|
||||
↓
|
||||
Response mit:
|
||||
- Semantisch relevanter Information (Qdrant)
|
||||
- Strukturellem Kontext (Neo4j)
|
||||
- Abhängigkeitsanalyse
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ n8n Integration (Phase 3)
|
||||
|
||||
### Workflow: Graph-Sync Trigger
|
||||
|
||||
```
|
||||
[Memos Webhook]
|
||||
↓
|
||||
[Parse Content]
|
||||
↓
|
||||
[SAIA: Extract Entities]
|
||||
├→ Tasks, Projects, Technologies, People
|
||||
└→ Relationships (uses, depends_on, mentions)
|
||||
↓
|
||||
[Neo4j: Create/Update Nodes & Relationships]
|
||||
↓
|
||||
[Index in Qdrant Metadata: Add graph_node_id]
|
||||
```
|
||||
|
||||
### Workflow: Question Router
|
||||
|
||||
```
|
||||
[#nexa:ask Query]
|
||||
↓
|
||||
[Parallel Search]
|
||||
├→ [Qdrant: Vector Similarity]
|
||||
├→ [Neo4j: Graph Query (auto-generated Cypher)]
|
||||
└→ [Rank & Merge Results]
|
||||
↓
|
||||
[SAIA: Generate Answer with Context]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Commands für Graph-Management
|
||||
|
||||
### #nexa:graph-status
|
||||
Zeigt Statistiken der Graph-DB
|
||||
```
|
||||
#nexa:graph-status
|
||||
```
|
||||
|
||||
Response:
|
||||
```
|
||||
📊 Neo4j Graph Status:
|
||||
- Nodes: 342 (Tasks: 145, Projects: 12, Technologies: 52, Notes: 133)
|
||||
- Relationships: 1,247
|
||||
- Density: 0.34
|
||||
- Most Connected: "JWT-Middleware" (27 connections)
|
||||
```
|
||||
|
||||
### #nexa:graph-trace [entity]
|
||||
Zeigt alle Verbindungen für eine Entität
|
||||
```
|
||||
#nexa:graph-trace JWT-Middleware
|
||||
```
|
||||
|
||||
### #nexa:graph-rebuild
|
||||
Rekonstruiert Graph aus Obsidian + Memos (einmalig, danach inkrementell)
|
||||
```
|
||||
#nexa:graph-rebuild --force
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Phase 3 Roadmap (Graph Integration)
|
||||
|
||||
- [ ] **3.1 Neo4j Setup**: Collection, Constraints, Indexes
|
||||
- [ ] **3.2 Graph-Sync Workflow**: Automatische Entity Extraction
|
||||
- [ ] **3.3 GraphRAG Pipeline**: Kombinierte Qdrant + Neo4j Queries
|
||||
- [ ] **3.4 Dependency Tracker**: Warnt vor zirkulären Dependencies
|
||||
- [ ] **3.5 Critical Path Analysis**: Berechnet längsten Weg im Graph
|
||||
|
||||
---
|
||||
|
||||
## 💡 Warum zwei DBs?
|
||||
|
||||
| Szenario | Qdrant | Neo4j | Kombi |
|
||||
|----------|--------|-------|-------|
|
||||
| "Welche Note war ähnlich?" | ✅ Best | ❌ | Qdrant |
|
||||
| "Welche Tasks blockieren diesen Task?" | ❌ | ✅ Best | Neo4j |
|
||||
| "Erkläre mir dieses Projekt umfassend" | ✅ Context | ✅ Structure | **Beide!** |
|
||||
| "Finde alle JWT-bezogenen offenen Work" | ✅ Semantic | ✅ Crisp | **Beide!** |
|
||||
|
||||
Kombiniert: **Vollständiges Verständnis** statt nur Suchindex oder nur Struktur.
|
||||
@@ -0,0 +1,193 @@
|
||||
# 09 — Deployment
|
||||
|
||||
Pragmatic deployment guide that **assumes the existing homelab** and adds only what's missing.
|
||||
|
||||
## What's already running (no action required)
|
||||
|
||||
Surveyed from Homepage / Dozzle / Proxmox / Zoraxy:
|
||||
|
||||
| Service | Host / port | URL |
|
||||
|---------|-------------|-----|
|
||||
| Memos | docker LXC 104 → `:5230` | `https://memos.nuclide.systems` |
|
||||
| n8n | docker LXC 104 → `:5678` | `https://n8n.nuclide.systems` |
|
||||
| LiteLLM (SAIA gateway) | docker LXC 104 → `:4000` | `https://ai.nuclide.systems` (proxies LobeHub UI :3210; API on :4000) |
|
||||
| Nextcloud | LXC 105 | `https://nc.nuclide.systems` |
|
||||
| ntfy | docker LXC 104 → `:7998` | `https://ntfy.nuclide.systems` |
|
||||
| Karakeep (Hoarder) | docker LXC 104 → `:3090` | `https://hoarder.nuclide.systems` |
|
||||
| Home Assistant | VM 100 (HAOS) | `https://ha.nuclide.systems` |
|
||||
| Pocket-ID (OAuth/SSO) | docker LXC 104 → `:1411` | `https://id.nuclide.systems` |
|
||||
| Vaultwarden | docker LXC 104 → `:11001` | `https://vault.nuclide.systems` |
|
||||
| Backrest | LXC 103 | (internal) |
|
||||
| AdGuard DNS | LXC 102 | (internal) |
|
||||
| Zoraxy reverse proxy | LXC 108 → `192.168.1.4:8000` | TLS for *.nuclide.systems |
|
||||
| `qdrant_scientific` (existing) | docker LXC 104 | reuse — see [11](./11-open-questions.md) |
|
||||
|
||||
The deployment task is **not** "spin up the stack" — most of the stack is already up. It is **wire Nexa across these services + add the small bits that are missing**.
|
||||
|
||||
## What's missing for Nexa
|
||||
|
||||
1. **Qdrant collection** for Nexa (`nexa_knowledge`) — either on the existing `qdrant_scientific` or a dedicated container.
|
||||
2. **n8n workflows** (`./nexa-core/n8n-workflows/`) imported into the running n8n.
|
||||
3. **Nextcloud lists & calendars** for Work / Personal / Shopping / Wishes (auto-discovered via `#nexa:config`).
|
||||
4. **Memos webhook → n8n** wired through the Memos config.
|
||||
5. **LiteLLM virtual key** for the `nexa` user with embedding + chat models.
|
||||
6. **A Zoraxy host entry** is *not* needed — Memos / n8n / LiteLLM are already proxied.
|
||||
7. **(Phase 3.4)** decision on graph DB (Neo4j vs. Ontotext GraphDB) — see [11](./11-open-questions.md).
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Secrets
|
||||
|
||||
Copy `nexa-core/.env.example` → `nexa-core/.env` and fill **only** the secrets:
|
||||
|
||||
```bash
|
||||
cd nexa-core
|
||||
cp .env.example .env
|
||||
$EDITOR .env # MEMOS_API_KEY, SAIA_API_KEY, NC_APP_PASSWORD, QDRANT_API_KEY
|
||||
```
|
||||
|
||||
The `.env` is **only used at bootstrap time**. Everything else (list IDs, calendar IDs, collection sizes) is discovered at runtime via `#nexa:config` (see [05](./05-command-system.md)). No secrets should ever live in n8n workflow JSON — use n8n credentials instead.
|
||||
|
||||
## Step 2 — Qdrant collection
|
||||
|
||||
```bash
|
||||
# adjust QDRANT_HOST in .env first
|
||||
source nexa-core/.env
|
||||
|
||||
curl -X PUT "$QDRANT_HOST/collections/nexa_knowledge" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "api-key: $QDRANT_API_KEY" \
|
||||
-d @nexa-core/config/qdrant_schema.json
|
||||
```
|
||||
|
||||
> If reusing `qdrant_scientific`, this just adds another collection; no new container needed.
|
||||
|
||||
## Step 3 — LiteLLM virtual key
|
||||
|
||||
In the LiteLLM admin UI (`ai.nuclide.systems`):
|
||||
|
||||
1. Create user `nexa`.
|
||||
2. Issue a virtual key with access to one chat model (e.g. `gpt-4o-mini` / a local Sonnet equivalent) and one embedding model (`text-embedding-3-small`).
|
||||
3. Paste the key into `SAIA_API_KEY` in `.env`.
|
||||
|
||||
## Step 4 — n8n workflows
|
||||
|
||||
Import the JSON exports — credentials are filled inside n8n, not in the JSON:
|
||||
|
||||
```bash
|
||||
# n8n personal access token from the n8n UI: Settings → API
|
||||
N8N_URL=https://n8n.nuclide.systems
|
||||
N8N_TOKEN=... # from the n8n UI
|
||||
|
||||
for f in nexa-core/n8n-workflows/phase-1/*.json \
|
||||
nexa-core/n8n-workflows/phase-2/*.json; do
|
||||
curl -X POST "$N8N_URL/api/v1/workflows" \
|
||||
-H "X-N8N-API-KEY: $N8N_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data-binary "@$f"
|
||||
done
|
||||
```
|
||||
|
||||
Inside n8n, attach credentials to the imported nodes:
|
||||
|
||||
- **Memos** → HTTP header `Authorization: Bearer $MEMOS_API_KEY`
|
||||
- **LiteLLM** → header `Authorization: Bearer $SAIA_API_KEY`
|
||||
- **Nextcloud** → app password
|
||||
- **Qdrant** → header `api-key: $QDRANT_API_KEY`
|
||||
|
||||
Activate each workflow individually after smoke-test.
|
||||
|
||||
## Step 5 — Memos webhook
|
||||
|
||||
In the Memos admin UI, set the webhook URL to the production address of the discovery workflow:
|
||||
|
||||
```
|
||||
https://n8n.nuclide.systems/webhook/memos
|
||||
```
|
||||
|
||||
The same URL is the one the workflow exposes; verify with:
|
||||
|
||||
```bash
|
||||
curl -i https://n8n.nuclide.systems/webhook/memos
|
||||
# expect 200 / 405, never 404
|
||||
```
|
||||
|
||||
## Step 6 — Bootstrap commands via Memos
|
||||
|
||||
Create a memo with body `#nexa:config` — the discovery workflow:
|
||||
|
||||
1. Lists Nextcloud Tasks lists, picks Work / Personal / Shopping / Wishes by name.
|
||||
2. Counts existing Qdrant points in `nexa_knowledge`.
|
||||
3. Verifies LiteLLM reachability + lists available models.
|
||||
4. Replies as a comment with a **runtime-config snapshot** that's stored as Qdrant metadata (`_config` namespace) and as `nexa-core/config/runtime_config.json` (gitignored).
|
||||
|
||||
After this point, `.env` is read-once. Subsequent runs read config from Qdrant.
|
||||
|
||||
## Step 7 — Smoke tests
|
||||
|
||||
```bash
|
||||
# (1) Memos round-trip — should produce a comment within ~5 s
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"content":"- [ ] testing the router #nexa"}'
|
||||
|
||||
# (2) Classification dry-run
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-d '{"content":"#nexa:route-test buy milk"}'
|
||||
|
||||
# (3) RAG test (requires at least one indexed memo/note)
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-d '{"content":"#nexa:ask what is the goal of nexa?"}'
|
||||
```
|
||||
|
||||
## Step 8 — Reverse proxy
|
||||
|
||||
Already done — Zoraxy at `192.168.1.4:8000` terminates TLS for `*.nuclide.systems` and forwards to docker LXC 104 (`192.168.1.40`). **No new entry is required for Nexa**: every service Nexa talks to already has a host entry.
|
||||
|
||||
## Step 9 — Backups
|
||||
|
||||
Already covered by Backrest (LXC 103). Add:
|
||||
|
||||
- **n8n workflows** → `nexa-core/scripts/backup_workflows.sh` (already present) into a Backrest schedule.
|
||||
- **Qdrant snapshots** → schedule a daily `POST /collections/nexa_knowledge/snapshots` and rsync to S3 (`s3.nuclide.systems`). Add as a Backrest pre-hook on the docker host.
|
||||
|
||||
For deeper detail: [10 — Operations](./10-operations.md).
|
||||
|
||||
---
|
||||
|
||||
## Phase add-on: graph DB (Phase 3.4)
|
||||
|
||||
Defer until 3.1–3.3 ship. When needed, two minimal options — pick one:
|
||||
|
||||
### Option A — Neo4j (lighter, popular for graph-RAG)
|
||||
|
||||
```yaml
|
||||
# nexa-core/docker-compose.graph.yml
|
||||
services:
|
||||
neo4j:
|
||||
image: neo4j:5-community
|
||||
container_name: nexa-neo4j
|
||||
ports: ["7687:7687", "7474:7474"]
|
||||
environment:
|
||||
NEO4J_AUTH: "neo4j/${NEO4J_PASSWORD}"
|
||||
NEO4J_server_memory_heap_max__size: 2G
|
||||
volumes: ["./data/neo4j:/data"]
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
### Option B — Ontotext GraphDB (SPARQL, RDF native)
|
||||
|
||||
Keep the snippet from the original `DEPLOYMENT.md` Schritt 1.3 for `graphdb`. Heavier (~4 GB heap), only justified if you want SHACL / OWL reasoning.
|
||||
|
||||
Add a Zoraxy host entry `graph.nuclide.systems` → `192.168.1.40:7474` (Neo4j) or `:7200` (GraphDB) **only if** browser access is desired; otherwise n8n talks to it on the docker network.
|
||||
|
||||
---
|
||||
|
||||
## Step-back / rollback
|
||||
|
||||
- **Disable any Nexa workflow** in n8n — deactivates the side effect immediately, Memos webhooks become no-ops.
|
||||
- **Drop a Qdrant collection** — `curl -X DELETE $QDRANT_HOST/collections/nexa_knowledge -H "api-key: $QDRANT_API_KEY"`.
|
||||
- **Re-discover** — `#nexa:reset-config` then `#nexa:config`.
|
||||
@@ -0,0 +1,95 @@
|
||||
# 10 — Operations
|
||||
|
||||
Day-2 concerns. Backups, monitoring, troubleshooting.
|
||||
|
||||
## Backups
|
||||
|
||||
Backrest (LXC 103) already handles file-level snapshots. Add Nexa-specific items:
|
||||
|
||||
| What | How | Frequency |
|
||||
|------|-----|-----------|
|
||||
| n8n workflow JSON | `nexa-core/scripts/backup_workflows.sh` → git push | hourly cron in n8n container |
|
||||
| Qdrant `nexa_knowledge` | `POST /collections/nexa_knowledge/snapshots` → rsync to `s3.nuclide.systems` | daily (Backrest pre-hook) |
|
||||
| Memos DB | Backrest snapshot of Memos data dir | already covered |
|
||||
| Nextcloud | Nextcloud's own backup app + Backrest of `/var/www/nextcloud/data` | already covered |
|
||||
| `runtime_config.json` (Qdrant `_config` namespace) | Qdrant snapshot covers it | n/a |
|
||||
|
||||
## Monitoring
|
||||
|
||||
| Signal | Source | Sink |
|
||||
|--------|--------|------|
|
||||
| Container up/down | Dozzle (`192.168.1.40:3553`) + docker healthchecks | Memos system feed via ntfy |
|
||||
| n8n workflow failures | n8n built-in failure-webhook | ntfy → `nexa.system` topic |
|
||||
| Proxmox / disk / memory alerts | Proxmox notification target → ntfy | ntfy → Memos digest |
|
||||
| LiteLLM rate-limit | LiteLLM logs / cost-tracking | Memos morning digest |
|
||||
| Backrest run status | Backrest webhook | ntfy |
|
||||
|
||||
A **single ntfy topic `nexa.system`** is the convention; n8n has one workflow that re-broadcasts it as a Memos comment under a pinned `[SYSTEM]` memo.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Memos webhook not firing
|
||||
|
||||
```bash
|
||||
# is the Memos webhook config still pointing at n8n?
|
||||
curl -s -H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
https://memos.nuclide.systems/api/v1/workspace/setting | jq '.webhooks'
|
||||
|
||||
# does n8n still expose the path?
|
||||
curl -i https://n8n.nuclide.systems/webhook/memos # expect 200/405, never 404
|
||||
```
|
||||
|
||||
If 404 → workflow is inactive in n8n. Activate.
|
||||
|
||||
### LiteLLM 401 / 429
|
||||
|
||||
- 401 → the Nexa virtual key was rotated. Reissue, update `.env`, re-create the n8n credential.
|
||||
- 429 → set per-key TPM/RPM limits in LiteLLM admin; classification calls are tiny (<300 tok), embeddings are the bulk.
|
||||
|
||||
### Qdrant `nexa_knowledge` empty
|
||||
|
||||
```bash
|
||||
# is the indexing workflow active?
|
||||
docker logs nexa-n8n 2>&1 | grep memos_bridge | tail
|
||||
|
||||
# manual index test
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-H "Authorization: Bearer $MEMOS_API_KEY" \
|
||||
-d '{"content":"manual probe #nexa"}'
|
||||
|
||||
# point count
|
||||
curl -s -H "api-key: $QDRANT_API_KEY" \
|
||||
$QDRANT_HOST/collections/nexa_knowledge | jq '.result.points_count'
|
||||
```
|
||||
|
||||
### Wrong list routing (Work vs Personal)
|
||||
|
||||
1. Run `#nexa:route-test <text>` and check the confidence value.
|
||||
2. If <0.7 the router defaults to Personal (by design — see [06](./06-classification-logic.md)).
|
||||
3. Tune the system prompt in `nexa-core/ai-prompts/system_prime.txt`; commit the change.
|
||||
|
||||
### "Where did Nexa store this?"
|
||||
|
||||
```bash
|
||||
# state dump
|
||||
curl -X POST https://memos.nuclide.systems/api/v1/memos \
|
||||
-d '{"content":"#nexa:export-state"}'
|
||||
# returns runtime_config + counters as a JSON memo
|
||||
```
|
||||
|
||||
## Log locations
|
||||
|
||||
| Component | Where |
|
||||
|-----------|-------|
|
||||
| Memos | `docker logs nexa-memos` (LXC 104) |
|
||||
| n8n | `docker logs nexa-n8n` |
|
||||
| LiteLLM | `docker logs litellm` |
|
||||
| Qdrant | `docker logs qdrant_scientific` |
|
||||
| Zoraxy | LXC 108 web UI → Statistical Analysis |
|
||||
| Backrest | LXC 103 web UI |
|
||||
|
||||
For a one-shot dump:
|
||||
|
||||
```bash
|
||||
ssh nuc 'docker compose -p nexa logs --tail 500' > /tmp/nexa.log
|
||||
```
|
||||
@@ -0,0 +1,29 @@
|
||||
# 11 — Open Questions (user-info-required)
|
||||
|
||||
Items that **block progress** and need a human decision before a workflow can be implemented or a service deployed. Tick them off as you decide.
|
||||
|
||||
## Architectural decisions
|
||||
|
||||
- [ ] **Q1 — Graph DB choice.** Phase 3.4 says *Neo4j* in [08](./08-graphrag-architecture.md) but the original deployment draft used *Ontotext GraphDB*. **Recommendation:** Neo4j (smaller footprint, Cypher is friendlier for n8n HTTP nodes, no SHACL/OWL needs identified). Confirm or override.
|
||||
- [ ] **Q2 — Reuse `qdrant_scientific` or run a dedicated `nexa-qdrant`?** Reuse keeps memory usage flat (~670 MB seen on Homepage); dedicated gives clean isolation and independent snapshot lifecycle. **Recommendation:** reuse with a `nexa_*` collection prefix.
|
||||
- [ ] **Q3 — Embeddings model.** Default in `infra_manifest.yaml` is `text-embedding-3-small` (OpenAI). Is the LiteLLM virtual key authorised to call it, or should we route to a self-hosted alternative (e.g. `bge-m3` via Ollama on the same host)?
|
||||
- [ ] **Q4 — Obsidian sync mechanism.** `system_prime.txt` references Obsidian Context, but the current setup syncs via Nextcloud (`nc.nuclide.systems` → `Notizen` folder, ~200 MB). Should Nexa watch the **filesystem on LXC 105** (NC data dir) or **the Nextcloud WebDAV API**? FS is cheaper, WebDAV is portable.
|
||||
- [ ] **Q5 — Karakeep vs. Hoarder naming.** The Zoraxy host is `hoarder.nuclide.systems` but the container is `karakeep-*` and Homepage labels it Karakeep. They're the same product (rename in 2024). Pick one display name to use in docs and prompts.
|
||||
|
||||
## Identifiers needed (auto-discoverable, but list now if known)
|
||||
|
||||
- [ ] **Q6 — Nextcloud Tasks list IDs** for: `Work_Tasks`, `Personal_Tasks`, `Shopping`, `Wishes`. Discovery via `#nexa:config` will fill these — confirm names match.
|
||||
- [ ] **Q7 — Nextcloud Calendar IDs** for: `Work_Calendar`, primary personal calendar.
|
||||
- [ ] **Q8 — IMAP credentials for the personal mail account.** Can n8n reuse a Nextcloud Mail account (preferred — no extra password) or must we add a dedicated IMAP entry?
|
||||
- [ ] **Q9 — ntfy topic name** for `nexa.system`. Is the topic public on `ntfy.nuclide.systems` or should it be authenticated?
|
||||
- [ ] **Q10 — Pocket-ID role.** `id.nuclide.systems` is running. Do we want SSO in front of the n8n / Memos UIs, or skip for now?
|
||||
|
||||
## Hardware / capacity
|
||||
|
||||
- [ ] **Q11 — RAM headroom on docker host.** Homepage shows 29.5 GiB free out of ~31 GiB total, ~8.5 GB used. Phase-3 Qdrant indexing of Obsidian + Memos history is fine here, but a Phase-3.4 GraphDB choice changes this (Neo4j ~2 GB heap, Ontotext ~4 GB). Confirm acceptable.
|
||||
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots?
|
||||
|
||||
## Process
|
||||
|
||||
- [ ] **Q13 — Octoprint container is Exited** (Homepage). Out of scope for Nexa, but flagging it because Phase 5 monitoring would alert on it. Suppress, or is it intentional?
|
||||
- [ ] **Q14 — `Missing Widget Type: zoraxy`** on Homepage — Homepage doesn't have a Zoraxy widget yet. Cosmetic, unrelated to Nexa.
|
||||
@@ -0,0 +1,25 @@
|
||||
# 12 — Optimization Opportunities
|
||||
|
||||
Observations from the running infrastructure. Each item is independent — accept, defer, or reject.
|
||||
|
||||
## For Nexa directly
|
||||
|
||||
1. **Reuse, don't redeploy.** The earlier `DEPLOYMENT.md` would have spun a second Memos / n8n / Qdrant. The current homelab already runs all three. The new [09-deployment](./09-deployment.md) treats these as pre-existing — keeps the config minimal and avoids port collisions.
|
||||
2. **Use LiteLLM virtual keys per logical caller.** Today there's one SAIA key. Issuing one key per workflow (`nexa-router`, `nexa-embed`, `nexa-digest`) lets you set different per-key rate/cost limits and disable a single workflow without rotating everything.
|
||||
3. **Use n8n's *credential* objects, never inline secrets.** The current workflows under `nexa-core/n8n-workflows/phase-1/*.json` should be reviewed — if any header `Authorization` is hardcoded, replace with credential references before importing.
|
||||
4. **Centralise system alerts on a single ntfy topic** (`nexa.system`). Backrest, Proxmox notifications, n8n failure-webhook and the Octoprint Exited state all go to that topic; one Memos system memo aggregates them.
|
||||
5. **Defer graph DB until Phase 3.4.** Qdrant alone covers ~80% of the assistant's daily value. The graph DB is justified once you actually need dependency analysis or critical-path queries.
|
||||
6. **Auto-export n8n workflows.** `nexa-core/scripts/backup_workflows.sh` already exists. Schedule it inside the n8n container (cron) and let it `git commit && git push` — this is the cheapest disaster recovery.
|
||||
|
||||
## For the wider homelab (out of scope but worth noting)
|
||||
|
||||
7. **Octoprint `EXITED`** — either remove the Homepage entry or fix the container; right now it permanently shows red.
|
||||
8. **Zoraxy widget missing** on Homepage. Homepage v1.x has no built-in Zoraxy widget, but you can use the generic `customapi` widget against `/api/stats/summary` to surface request counts.
|
||||
9. **AI gateway naming.** `ai.nuclide.systems` currently proxies LobeHub (a chat UI on `:3210`), while the LiteLLM API lives on `:4000`. For Nexa, point n8n directly at LiteLLM (`http://192.168.1.40:4000` over the docker net — no public TLS hop needed) to save latency and isolate from UI restarts.
|
||||
10. **MCP servers consolidation.** Dozzle shows `crawl4ai-mcp`, `markitdown-mcp`, `papersearch-mcp` running individually. They're all MCP servers — Nexa Phase-3 could pull from these via LiteLLM's MCP support to enrich the embedding pipeline (e.g. fetch + markitdown a Karakeep link before embedding).
|
||||
11. **Backup the n8n SQLite file** — Backrest covers `/home/node/.n8n` if added; today the only "backup" is the workflow JSON which omits credentials and execution history.
|
||||
12. **Pocket-ID SSO in front of n8n** would let you remove n8n basic-auth and unify session management across the whole stack. One-time setup, large UX win.
|
||||
13. **Vaultwarden as the secret store** for Nexa secrets (`SAIA_API_KEY`, `MEMOS_API_KEY`, …) — read at bootstrap via the Bitwarden CLI from inside the docker host. Removes the need for a `.env` on disk.
|
||||
14. **AdGuard as DNS-based control plane.** Since AdGuard is the resolver for the LAN, you can rewrite `*.nuclide.systems` to `192.168.1.4` (Zoraxy) internally and avoid a hairpin via the WAN — already the case if AdGuard rewrite rules are set, worth verifying.
|
||||
15. **Immich + Qdrant.** Immich is already running and embeds photos with CLIP. If you ever want photo-aware Nexa answers, you can mount Immich's existing search via its API — no second embedding pass.
|
||||
16. **Disk usage on LXC 104** is 47.7 % (Proxmox). Monitor; n8n execution logs and Dozzle history are the usual culprits. Setting `EXECUTIONS_DATA_PRUNE=true` and `EXECUTIONS_DATA_MAX_AGE=168` (7 days) on n8n keeps it bounded.
|
||||
@@ -0,0 +1,50 @@
|
||||
# NEXA Documentation
|
||||
|
||||
**Neural Nexus for Information & Automation** — the central nervous system that ties Memos, n8n, SAIA (LiteLLM), Nextcloud and a vector store into one assistant.
|
||||
|
||||
This is the documentation entry point. Read top-to-bottom for first-time setup, or jump to the section you need.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
| # | Document | Read when… |
|
||||
|---|----------|------------|
|
||||
| 01 | [Vision & Scope](./01-vision-and-scope.md) | You want to understand *what* Nexa is and isn't. |
|
||||
| 02 | [Roadmap & Phases](./02-roadmap.md) | You want to know the implementation order. |
|
||||
| 03 | [Architecture Overview](./03-architecture.md) | You need a one-page mental model. |
|
||||
| 04 | [Integration Matrix](./04-integration-matrix.md) | You're wiring up a new data source or mapping work-vs-personal flows. |
|
||||
| 05 | [Command System](./05-command-system.md) | You want to know what `#nexa:*` commands do. |
|
||||
| 06 | [Classification Logic](./06-classification-logic.md) | You're tuning the work/personal router. |
|
||||
| 07 | [Workflow Spec — Task Router](./07-workflow-spec.md) | You're building the Phase-2 router workflow. |
|
||||
| 08 | [GraphRAG Architecture](./08-graphrag-architecture.md) | You're working on Phase 3 (Qdrant + Graph). |
|
||||
| 09 | [Deployment](./09-deployment.md) | You're bringing Nexa up on the real infrastructure. |
|
||||
| 10 | [Operations](./10-operations.md) | You need backup, monitoring or troubleshooting. |
|
||||
| 11 | [Open Questions (user-info-required)](./11-open-questions.md) | Items the user still has to answer before progress. |
|
||||
| 12 | [Optimization Opportunities](./12-optimization-opportunities.md) | Ideas worth considering for the wider homelab. |
|
||||
|
||||
---
|
||||
|
||||
## Reading paths
|
||||
|
||||
- **First-time installer:** 01 → 03 → 09 → 10 → 11
|
||||
- **Workflow author:** 04 → 05 → 07 → 06
|
||||
- **Phase-3 work (memory):** 08 → 09 (§ Phase 3 add-on)
|
||||
- **Stakeholder / quick read:** 01 → 02 → 12
|
||||
|
||||
---
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
nexa/
|
||||
├── README.md → points here
|
||||
├── docs/ → you are here
|
||||
└── nexa-core/ → the actual project
|
||||
├── ai-prompts/ → SAIA system prompts
|
||||
├── config/ → runtime config (YAML, JSON schema)
|
||||
├── n8n-workflows/ → exported workflows, version-controlled
|
||||
└── scripts/ → automation helpers
|
||||
```
|
||||
|
||||
Source-of-truth for *runnable* config stays in `nexa-core/`. Documentation lives here in `docs/`.
|
||||
Reference in New Issue
Block a user