fkrebs/nexa
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Resolve Q6/Q7/Q8/Q9/Q11; pin actual list names; add host tuning suggestions

Browse Source 
Verified from screenshots:

- Q6/Q7 → Lists & calendars discovered by NAME, not by ID. Names confirmed:
  "Persönlich" (personal), "DLR" (work), "Einkaufsliste" (shopping),
  "Wunschliste" (wishes). Self-healing: #nexa:config runs daily and on
  cache-miss, so user-added lists become routable without code changes.
  Routing logic in docs/07 expanded from 2-way to 4-way switch with a
  Resolver step that consults the Qdrant _config cache before fanning out;
  default route is "Persönlich" when SAIA confidence < 0.7.
- Q8 → Single Nextcloud Mail account fkrebs@nucli.de. No separate IMAP.
  The "Waiting" folder is honored as a manual user signal — items there are
  excluded from digest.
- Q9 → Pocket-ID SSO is global at the Zoraxy layer; Nexa surfaces inherit
  it. No app-level basic-auth. Q10 folded in.
- Q11 → Capacity confirmed: 22 threads (Core Ultra 7 155H), 62 GiB RAM
  (~32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35%).
  Phase-3 budget (~7 GB) is comfortable; two levers documented (raise
  LXC cap, or cap zfs_arc_max). Earlier "16 CPU" references corrected.

Propagation:
- system_prime.txt rewritten with the four context labels and the new
  list names; legacy NC_LIST_ID_* placeholders removed.
- docs/04 integration matrix uses the German names directly.
- docs/05 documents that #nexa:config also runs scheduled and on
  cache-miss, with the example output reflecting real list IDs.
- docs/09 step 6 mirrors the same flow + lists the Waiting-folder
  semantics.
- CLAUDE.md gets an authoritative block of homelab decisions so future
  agents don't re-litigate any of these.

New section in docs/12 (#17–25): Proxmox host tuning suggestions
based on the live node summary — cap ZFS ARC (frees ~16 GiB),
enable KSM (~1-3 GiB savings across 40 containers), suppress the
no-subscription banner, drop swappiness, verify scrub/SMART/fstrim
timers, NTP via AdGuard.
This commit is contained in:
Claude
2026-05-04 22:08:10 +00:00
parent e1c2ba96e2
commit 0b470b3e12
8 changed files with 100 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/04-integration-matrix.md
+3 -5
View File
@@ -10,17 +10,15 @@ Nexa muss strikt zwischen zwei Kontexten unterscheiden, da die Datenquellen vari
### A. Bereich: ARBEIT (Work)
* **Sichtbarkeit:** Eingeschränkt (kein Zugriff auf Arbeits-E-Mails oder Firmen-Server).
* **Datenquellen:**
* Separate Nextcloud Task-Liste (`Work_Tasks`).
* Separater Nextcloud Kalender (`Work_Calendar`).
* Nextcloud Task-Liste **`DLR`** (sowie der gleichnamige Kalender).
* Memos mit Tag `#work` oder semantischer Erkennung.
* **Nexa-Fokus:** Timeblocking, Fokus-Zeiten, Vorbereitung von Meetings basierend auf Obsidian-Notizen.
### B. Bereich: PRIVAT (Personal)
* **Sichtbarkeit:** Vollständig.
* **Datenquellen:**
* Nextcloud Task-Liste (`Personal_Tasks`).
* Haupt-Kalender.
* E-Mails (IMAP - 20 Mails/Tag).
* Nextcloud Task-Liste **`Persönlich`** und gleichnamiger Kalender.
* Nextcloud Mail (`fkrebs@nucli.de`, einziges Konto). Ordner `Waiting` ist ein manuelles Signal — Mails dort werden vom Digest ausgeschlossen.
* Bluesky, RSS, Karakeep.
* **Nexa-Fokus:** Automatisierung des Alltags, Kuratierung von Wissen, E-Mail-Management.
docs/05-command-system.md
+12 -7
View File
@@ -5,9 +5,12 @@ Nexa "hört" auf folgende Kommandos in Memos-Kommentaren oder als Memo-Inhalt mi
## 🔧 Konfigurations-Kommandos
### #nexa:config
- **Beschreibung**: Triggert Autodiscovery aller Services (Nextcloud, Qdrant, etc.)
- **Antwort**: Postet Report mit gefundenen Konfigurationen
- **Speichert**: Settings in DynoDB / Qdrant Metadata (nicht in .env)
- **Beschreibung**: Triggert Autodiscovery aller Services (Nextcloud Listen/Kalender, Qdrant Collections, LiteLLM-Modelle, Mail-Konto).
- **Läuft auch automatisch**:
- Täglich (cron in n8n) — neue Listen / Kalender werden ohne manuelles Eingreifen aufgenommen.
- Bei Cache-Miss — wenn ein Workflow eine Liste/ID nicht mehr findet, läuft Discovery einmal, dann Retry.
- **Antwort**: Postet Report mit den aktuellen Konfigurationen als Memos-Kommentar.
- **Speichert**: Konfiguration in der Qdrant `_config` Namespace (nicht in `.env`).
```
#nexa:config
@@ -15,10 +18,12 @@ Nexa "hört" auf folgende Kommandos in Memos-Kommentaren oder als Memo-Inhalt mi
**Beispiel-Response:**
```
✅ Autodiscovery abgeschlossen:
- Nextcloud Listen: Work (id=123), Personal (id=456), Shopping (id=789), Wishes (id=1011)
- Qdrant Collection: nexa_knowledge (1536 dims, Cosine)
- Nextcloud Mail: 3 Accounts erkannt (Office 365, Gmail, Privat)
✅ Autodiscovery abgeschlossen (2026-05-04T23:59):
- Nextcloud Listen: Persönlich (id=14), DLR (id=dlr-1), Einkaufsliste (id=…), Wunschliste (id=…)
- Nextcloud Kalender: Persönlich, DLR, Einkaufsliste, Wunschliste
- Mail-Konto: fkrebs@nucli.de (Posteingang, Archiv, Junk, Waiting)
- Qdrant Collection: nexa_knowledge_text (1024 dims, Cosine, 0 Punkte)
- LiteLLM-Modelle: nexa-chat, nexa-embed
```
### #nexa:status
docs/07-workflow-spec.md
+16 -6
View File
@@ -5,12 +5,22 @@
* **Filter:** `payload.content` enthält `- [ ]` ODER `#todo`.
## 2. Intelligence Node (SAIA)
* **Prompt:** "Analysiere folgende Notiz. Gib als JSON zurück: { "context": "work" | "personal", "summary": "string", "urgency": 1-5 }."
* **Prompt:** "Analysiere folgende Notiz. Gib als JSON zurück: `{ "context": "work" | "personal" | "shopping" | "wishes", "summary": "string", "urgency": 1-5 }`."
* **Kontext-Zufuhr:** Lade die letzten 5 Obsidian-Projekttitel aus Qdrant als Referenz hoch.
## 3. Switch Node
* **Route 1:** Context == "work" -> Nextcloud Create Task (List ID: Work).
* **Route 2:** Context == "personal" -> Nextcloud Create Task (List ID: Personal).
## 3. List Resolver (vor dem Switch)
* Lese den Cache `nexa._config.lists` aus Qdrant: `{ "Persönlich": <id>, "DLR": <id>, "Einkaufsliste": <id>, "Wunschliste": <id>, discovered_at: <ts> }`.
* Bei Cache-Miss oder 404 vom nachfolgenden Create-Task → triggere `#nexa:config` (Discovery), update den Cache, retry **einmal**.
## 4. Feedback Loop
* Poste die Task-ID und den gewählten Kontext als Kommentar unter das Original-Memo.
## 4. Switch Node (4-Wege)
* **Route 1:** `context == "work"` → Nextcloud Create Task auf Liste `DLR`.
* **Route 2:** `context == "personal"` → Nextcloud Create Task auf Liste `Persönlich`.
* **Route 3:** `context == "shopping"` → Nextcloud Create Task auf Liste `Einkaufsliste`.
* **Route 4:** `context == "wishes"` → Nextcloud Create Task auf Liste `Wunschliste`.
* **Default (Konfidenz < 0.7):** `Persönlich` (siehe [06](./06-classification-logic.md)).
## 5. Feedback Loop
* Poste die Task-ID und den gewählten Kontext als Kommentar unter das Original-Memo.
## 6. Listen-Diskoverability
Listennamen sind nicht hardgecodet — die Resolver-Step liest sie aus dem Runtime-Config-Cache. Wenn der User in Nextcloud eine neue Liste anlegt (z. B. `Reisen`), erscheint sie nach dem nächsten geplanten `#nexa:config`-Lauf (täglich) automatisch als Routing-Ziel — die System-Prompt der Intelligence Node wird zusammen mit den verfügbaren Listen versorgt, sodass SAIA neue Kontexte vorschlagen kann.
docs/09-deployment.md
+10 -4
View File
@@ -170,10 +170,16 @@ curl -i https://n8n.nuclide.systems/webhook/memos
Create a memo with body `#nexa:config` — the discovery workflow:
1. Lists Nextcloud Tasks lists, picks Work / Personal / Shopping / Wishes by name.
2. Counts existing Qdrant points in `nexa_knowledge`.
3. Verifies LiteLLM reachability + lists available models.
4. Replies as a comment with a **runtime-config snapshot** that's stored as Qdrant metadata (`_config` namespace) and as `nexa-core/config/runtime_config.json` (gitignored).
1. Lists Nextcloud Tasks lists, looking for `Persönlich`, `DLR`, `Einkaufsliste`, `Wunschliste` (and any new lists added later — see [05](./05-command-system.md)). Caches `name → id`.
2. Lists Nextcloud Calendars by the same names; caches IDs.
3. Verifies the Nextcloud Mail account `fkrebs@nucli.de` and the folder list (`Posteingang`, `Archiv`, `Junk`, `Waiting`).
4. Counts existing Qdrant points in `nexa_knowledge_text`.
5. Verifies LiteLLM reachability + lists available models for the Nexa virtual key.
6. Replies as a comment with a **runtime-config snapshot** that's stored in the Qdrant `_config` namespace (and mirrored as `nexa-core/config/runtime_config.json`, gitignored).
The same workflow is also triggered by:
- A daily cron inside n8n (so newly added Nextcloud lists become routable without intervention).
- Cache-miss in the task-router — if a list ID 404s, the router fires `#nexa:config` once and retries.
After this point, `.env` is read-once. Subsequent runs read config from Qdrant.
docs/11-open-questions.md
+19 -8
View File
@@ -25,25 +25,36 @@ Items that **block progress** and need a human decision before a workflow can be
- `Templates/` — empty templates, low semantic value.
- `BMO/`, `Excalidraw/` — plugin folders.
Anything else under `Notizen/**/*.md` is fair game.
- [x] **Q6 + Q7 — Lists & calendars discovered by name, not ID. Self-healing.**
Confirmed names from screenshots:
- **Tasks**: `Persönlich` (14), `DLR` (13), `Einkaufsliste` (13), `Wunschliste` (9).
- **Calendars**: `Persönlich`, `DLR`, `Einkaufsliste`, `Wunschliste` (Nextcloud Tasks is calendar-backed, so the names are shared; Tasks lives on the calendar of the same name).
- Mapping in classification: `DLR` = Work-Kontext, `Persönlich` = Personal-Kontext, `Einkaufsliste` = Shopping, `Wunschliste` = Wishes.
## Identifiers needed (auto-discoverable, but list now if known)
**Self-healing requirement (the user explicitly noted lists may change/grow):** Nexa must not cache IDs forever. The `#nexa:config` workflow runs (a) on demand, (b) once daily as a scheduled refresh, and (c) automatically as a retry whenever a list/calendar lookup returns 404 or "not found". The runtime-config record in Qdrant's `_config` namespace stores `{name → id, discovered_at}` and gets invalidated on cache-miss. New lists added in Nextcloud surface in the next scheduled refresh and Nexa starts honouring `#einkaufsliste` / `#dlr` etc. without code changes.
- [ ] **Q6 — Nextcloud Tasks list IDs** for: `Work_Tasks`, `Personal_Tasks`, `Shopping`, `Wishes`. Discovery via `#nexa:config` will fill these — confirm names match.
- [ ] **Q7 — Nextcloud Calendar IDs** for: `Work_Calendar`, primary personal calendar.
- [ ] **Q8 — IMAP credentials for the personal mail account.** Can n8n reuse a Nextcloud Mail account (preferred — no extra password) or must we add a dedicated IMAP entry?
- [ ] **Q9 — ntfy topic name** for `nexa.system`. Is the topic public on `ntfy.nuclide.systems` or should it be authenticated?
- [ ] **Q10 — Pocket-ID role.** `id.nuclide.systems` is running. Do we want SSO in front of the n8n / Memos UIs, or skip for now?
- [x] **Q8 — Mail via Nextcloud, single account `fkrebs@nucli.de`.** No extra IMAP entry. n8n's IMAP node uses the same server credentials Nextcloud Mail already holds for that account; Nexa never sees a second password. Folders confirmed: `Posteingang` (default), `Archiv`, `Junk` (61 — auto-filtered, ignored by Nexa), `Papierkorb`, `Waiting`. The `Waiting` folder is a useful manual signal — items moved there by the user are skipped from digest (treat as "in flight").
- [x] **Q9 — Pocket-ID SSO is configured everywhere.** No separate auth for Nexa: Memos / n8n / future Nexa dashboard authenticate humans via Pocket-ID at the Zoraxy layer. Machine-to-machine calls (n8n → Memos webhook, n8n → Nextcloud, n8n → Qdrant, n8n → LiteLLM) keep using API keys / app passwords — SSO is for human UIs only. Don't add basic-auth or per-app login screens.
This **resolves Q10 too** (the proposed "Pocket-ID SSO in front of n8n / Memos" is already done; Nexa just inherits it).
- [x] **Q10 — folded into Q9.**
## Hardware / capacity
- [ ] **Q11 — RAM headroom on docker host.** Sized against 31 GiB allocated to the LXC; the NUC 14 Pro itself has 64 GB so the LXC's memory cap can be lifted if needed. Phase-3 stack (Qdrant + TEI/bge-m3 ~1.1 GB + Ontotext GraphDB ~4 GB heap, later +infinity/jina-clip-v2 ~1 GB) tops out around ~14 GB. Comfortable. Confirm acceptable.
- [x] **Q11 — Capacity confirmed from Proxmox node summary.**
- Host (NUC 14 Pro): **22 threads** (Intel Core Ultra 7 155H, 1 socket), **62.12 GiB RAM**, **1.64 TiB disk** (0.35% used).
- Current: 32.4 GiB used (~52%), of which **ZFS ARC accounts for ~24 GiB**. Load avg 1.99 / 1.39 / 1.15 — idle. IO delay 0.04% — disk is not a bottleneck.
- LXC 104 (docker) is currently capped at ~31 GiB; if Phase-3 squeezes (~7 GB additional: Qdrant ~1.5 GB + TEI/bge-m3 ~1.1 GB + GraphDB 4 GB heap + later +infinity/jina-clip-v2 ~1 GB) we have two levers, in order of preference: (1) raise the LXC cap (host has plenty), (2) cap `zfs_arc_max` lower (ARC currently uses 24 GB). Either is one line of config.
- **Verdict: comfortable. No action needed at Phase-3.1 start.** Revisit if/when 3.4 GraphDB lands and actual heap usage exceeds the budget.
- [ ] **Q12 — S3 archive bucket.** `s3.nuclide.systems` is up. Bucket name + access key for Qdrant snapshots and GraphDB exports?
---
## Speed budget (Q3 follow-up)
Workload on the docker host (16 CPU, ~30 GB free RAM):
Workload on the docker host (host: 22-thread Intel Core Ultra 7 155H, 62 GiB RAM, ~30 GiB free; LXC 104 currently capped at 31 GiB):
| Task | Volume | Latency target | Achievable on CPU with `bge-m3` | Achievable with `nomic-embed-text` |
|------|--------|----------------|---------------------------------|------------------------------------|
docs/12-optimization-opportunities.md
+19
View File
@@ -23,3 +23,22 @@ Observations from the running infrastructure. Each item is independent — accep
14. **Obsidian plugin embeddings collide with Nexa's.** The vault already runs Obsidian Copilot (`.copilot`, `.copilot-index`) and Smart Connections / Smart Composer (`.smart-env`, ~13 MB). They each embed the same notes into their own vector stores — three indexes for the same content. Nexa's value is the **cross-source** index (memos + mail + obsidian + RDF graph), so it has to embed independently, but the plugins could be retired once Nexa's RAG is satisfying. Track separately, decide later.
15. **Real-time Obsidian sync via `notify_push`.** Phase 3.1 polls WebDAV every 15 min (Q4 resolution). Once that works, swap to Nextcloud's `notify_push` app for sub-second propagation. One-line workflow change in n8n.
16. **`assets/` is 186 MB of binaries** in the Obsidian vault — worth a glance to confirm it's mostly images (Phase-3.2 visual queue) rather than something that should live in Nextcloud Files proper.
## Proxmox host (NUC 14 Pro) tuning
Observed from the node summary: 22 threads, 62 GiB RAM (32 GiB used, ~24 GiB of which is ZFS ARC), 1.64 TiB disk (0.35% used), load avg <2.0, IO delay 0.04%, kernel 6.17.13-4-pve, PVE 9.1.9, EFI. Suggestions in priority order:
17. **Cap ZFS ARC.** Default is 50% of RAM (~31 GiB); current actual ~24 GiB. For a node that runs services rather than a pure storage box, capping at 8–12 GiB frees ~12–16 GiB for guests without measurable IO impact (disk is 1.64 TiB and 0.35% used — there's nothing hot to cache):
```
echo 'options zfs zfs_arc_max=8589934592' > /etc/modprobe.d/zfs.conf # 8 GiB
update-initramfs -u
```
Reboot or `echo 8589934592 > /sys/module/zfs/parameters/zfs_arc_max` to apply live.
18. **Enable KSM (Kernel Same-page Merging).** With ~40 docker containers + several LXCs, KSM typically frees 1–3 GiB by deduplicating identical memory pages. Currently `KSM sharing: 0 B` in the summary. PVE has `ksmtuned` available — `systemctl enable --now ksmtuned`.
19. **Suppress the `pve-no-subscription` repository warning** — either accept it (it's a homelab) and apply the `pve-no-subscription-warning` polyfill, or move to the enterprise repo. Pure cosmetic, but the orange banner in the UI is noise.
20. **Swap is 31 GiB on a 62 GiB box with ZFS root** — almost certainly oversized. Drop `vm.swappiness` to 10 (`sysctl -w vm.swappiness=10` + persist) so swap is only used under genuine pressure, and consider shrinking the swap volume if disk-layout permits.
21. **Verify scheduled ZFS scrub is enabled.** PVE ships `zfs-scrub-monthly@.timer` — `systemctl list-timers | grep zfs` to confirm. Cheap insurance on a 1.6 TiB pool.
22. **SMART monitoring on the NVMe.** `smartctl -a /dev/nvme0` should be regularly polled; PVE's notification target can ntfy on degradation. Combine with the existing `nexa.system` ntfy topic (optimization #4) so disk-health alerts land in the same Memos system feed as everything else.
23. **NTP source via AdGuard.** AdGuard already resolves DNS for the LAN; pointing the host's `systemd-timesyncd` at `pool.ntp.org` resolved through AdGuard avoids any external dependency for time. One-line change in `/etc/systemd/timesyncd.conf`.
24. **`fstrim.timer` enabled** for the SSD pool — verify with `systemctl status fstrim.timer`. Default-on in modern PVE, but quick to confirm.
25. **Watchdog config** is irrelevant for a single-node setup (HA is the use case), so leave the default. Mentioned only so future agents don't add it speculatively.