Files
nexa/docs/08-graphrag-architecture.md
T
Claude 17c7f5033f Stage embeddings: ship Path A (text) now, prepare Path C (text+visual) additively
Decision (Q15 resolved): start with text-only via TEI + bge-m3 in Phase 3.1,
prepare data shapes so Phase 3.2 (visual collection via infinity + jina-clip-v2)
is a pure additive operation — no rename, no schema migration, no n8n rewiring.

Concretely:
- Qdrant collection renamed nexa_knowledge → nexa_knowledge_text (1024-dim
  for bge-m3) with modality-aware payload (modality, source_type, media_uri,
  graph_iri, content_hash, context). Visual placeholder schema committed
  alongside (qdrant_schema_visual.json, 768-dim, jina-clip-v2).
- Image attachments captured in 3.1 are recorded in GraphDB as nexa:Note with
  nexa:modality "image" + nexa:pendingVisualIndex true; the 3.2 backfill
  workflow picks them up and embeds. No data lost between phases — the queue
  is the GraphDB itself.
- RDF schema (docs/08) gains nexa:modality, nexa:mediaUri,
  nexa:vectorCollection, nexa:pendingVisualIndex from day one.
- docs/02 roadmap split: 3.1 = text RAG (Path A), 3.2 = visual collection
  (Path C), 3.4 = Ontotext GraphDB.
- docs/09 grows a "Phase add-on: visual collection (Phase 3.2)" section with
  the TEI→infinity swap, second collection create, LiteLLM second model
  registration, and the SPARQL-driven backfill query.
- New open questions: Q16 (queue ergonomics + does SAIA already proxy an
  embed model?), Q17 (reuse Immich's CLIP for photo-library queries?).
- docs/03 + CLAUDE.md updated so future runs use the new collection names
  and don't re-decide the staging.
2026-05-04 21:33:59 +00:00

8.4 KiB
Raw Blame History

08 — GraphRAG: Structural Knowledge & Relations

Decision: graph layer = Ontotext GraphDB with SPARQL (resolved in 11/Q1). Rationale: SPARQL + RDF lets Nexa's memory be browsed and queried with the same standard tooling that's used for any open-data corpus, and it leaves the door open for SHACL / OWL reasoning later.


Two-pillar memory

Pillar Question it answers Backed by
Qdrant (vectors) "What is similar / relevant?" Cosine search over embeddings
GraphDB (RDF) "What is connected? What depends on what? Who is involved?" SPARQL over a typed graph

Both pillars are queried in parallel for #nexa:ask and merged before SAIA generates the final answer.


RDF schema

Compact, opinionated. One namespace, one ontology file, no v2/v3 inheritance pain.

@prefix nexa: <https://nuclide.systems/nexa/ontology#> .
@prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
@prefix prov: <http://www.w3.org/ns/prov#> .

# Classes
nexa:Project       a rdfs:Class .
nexa:Task          a rdfs:Class .
nexa:Person        a rdfs:Class .
nexa:Technology    a rdfs:Class .
nexa:Topic         a rdfs:Class .
nexa:Note          a rdfs:Class .          # Memos / Obsidian / mail digests
nexa:File          a rdfs:Class .

# Properties
nexa:owns          a rdf:Property ;  rdfs:domain nexa:Person     ;  rdfs:range nexa:Task .
nexa:uses          a rdf:Property ;  rdfs:domain nexa:Task       ;  rdfs:range nexa:Technology .
nexa:dependsOn     a rdf:Property ;  rdfs:domain nexa:Task       ;  rdfs:range nexa:Task .
nexa:childOf       a rdf:Property ;  rdfs:domain nexa:Task       ;  rdfs:range nexa:Project .
nexa:mentions      a rdf:Property ;  rdfs:domain nexa:Note       ;  rdfs:range nexa:Topic .
nexa:scheduledFor  a rdf:Property ;  rdfs:domain nexa:Task       ;  rdfs:range xsd:dateTime .

# Datatype properties
nexa:status        a rdf:Property ; rdfs:range xsd:string .   # "needs-action" | "in-progress" | "done"
nexa:context       a rdf:Property ; rdfs:range xsd:string .   # "work" | "personal"
nexa:urgency       a rdf:Property ; rdfs:range xsd:integer .  # 15
nexa:contentHash   a rdf:Property ; rdfs:range xsd:string .   # for de-dup

# Cross-pillar / multimodality
nexa:modality              a rdf:Property ; rdfs:range xsd:string .   # "text" | "image"
nexa:mediaUri              a rdf:Property ; rdfs:range xsd:anyURI .   # memos://… , nextcloud://… , obsidian://…
nexa:vectorCollection      a rdf:Property ; rdfs:range xsd:string .   # "nexa_knowledge_text" | "nexa_knowledge_visual"
nexa:vectorId              a rdf:Property ; rdfs:range xsd:string .   # Qdrant point ID
nexa:pendingVisualIndex    a rdf:Property ; rdfs:range xsd:boolean .  # set true on image notes until Phase 3.2 backfills them

nexa:vectorId + nexa:vectorCollection together are the bridge between graph and vector store. A SPARQL hit can trigger a vector lookup, and a Qdrant payload's graph_iri field walks back the other way.

nexa:modality, nexa:mediaUri and nexa:pendingVisualIndex exist from Phase 3.1 even though only the text path is wired up. Image attachments captured in 3.1 are recorded as nexa:Note with modality "image" and pendingVisualIndex true, then picked up by the Phase-3.2 backfill workflow — no data loss across phases.


Sync flows

1. Memos → GraphDB (real-time)

Memo content: "Muss JWT-Middleware für Auth-Service refaktorieren"
       │
       ▼ SAIA extracts entities + relations as JSON
       │   { tasks: [{title, urgency}], technologies: [...],
       │     relations: [{type:"uses", from:..., to:...}] }
       │
       ▼ n8n turns JSON into a SPARQL UPDATE
       │
       └──▶ INSERT DATA { ... } against GraphDB repo "nexa_knowledge"

2. Obsidian → GraphDB (#nexa:sync-obsidian)

For each Obsidian note: parse front-matter + headings → emit nexa:Project, nexa:Task, nexa:Note triples; nexa:mentions for [[wikilinks]].

3. Nextcloud Tasks ↔ GraphDB (bidirectional)

n8n trigger on Nextcloud CalDAV/Tasks change → INSERT/DELETE DATA to keep nexa:status and nexa:scheduledFor in sync.


Example SPARQL queries

Q1 — All open tasks involving JWT, by urgency

PREFIX nexa: <https://nuclide.systems/nexa/ontology#>

SELECT ?taskTitle ?urgency ?projectName
WHERE {
  ?tech rdfs:label "JWT" .
  ?task nexa:uses ?tech ;
        rdfs:label ?taskTitle ;
        nexa:status ?status ;
        nexa:urgency ?urgency .
  FILTER (?status IN ("needs-action", "in-progress"))
  OPTIONAL { ?task nexa:childOf ?project . ?project rdfs:label ?projectName . }
}
ORDER BY DESC(?urgency)

Q2 — What does Auth-Service transitively depend on?

PREFIX nexa: <https://nuclide.systems/nexa/ontology#>

SELECT DISTINCT ?dep ?label
WHERE {
  ?root rdfs:label "Auth-Service" .
  ?root nexa:dependsOn+ ?dep .
  ?dep  rdfs:label ?label .
}

(+ is SPARQL property-paths — transitive closure, free.)

Q3 — Topics with the most note-mentions in the last day

PREFIX nexa: <https://nuclide.systems/nexa/ontology#>
PREFIX xsd:  <http://www.w3.org/2001/XMLSchema#>

SELECT ?topic (COUNT(?note) AS ?n)
WHERE {
  ?note a nexa:Note ;
        prov:generatedAtTime ?ts ;
        nexa:mentions ?topic .
  FILTER (?ts > NOW() - "P1D"^^xsd:duration)
}
GROUP BY ?topic
ORDER BY DESC(?n)
LIMIT 10

Q4 — Cross-pillar: "find vectors for tasks blocking project X"

PREFIX nexa: <https://nuclide.systems/nexa/ontology#>

SELECT ?taskTitle ?vectorId
WHERE {
  ?proj rdfs:label "Nexa" .
  ?task nexa:childOf ?proj ;
        nexa:status "in-progress" ;
        nexa:vectorId ?vectorId ;
        rdfs:label ?taskTitle .
}

n8n then takes each ?vectorId, fetches the embedding from Qdrant, and runs a "more like this" search for richer context.


GraphRAG answer pipeline (#nexa:ask)

       #nexa:ask <question>
              │
       ┌──────┴──────┐
       ▼             ▼
   [Qdrant]      [GraphDB]
   semantic       structural
   top-k          SPARQL — auto-generated
   notes          paths / dependencies
       │             │
       └──────┬──────┘
              ▼
       merge + rank
              │
              ▼
       SAIA prompt:
        "Given these passages and these relations, answer …"
              │
              ▼
        comment under the original memo

Auto-generation of SPARQL: SAIA is given the ontology (above) as a system prompt and asked to emit a SELECT/CONSTRUCT query for the user's natural-language question. n8n executes it, falls back to a templated query on parse failure.


n8n integration sketch

Workflow: Graph-Sync Trigger

[Memos Webhook]
     │
[Parse Content]
     │
[SAIA: Extract entities + relations as JSON]
     │
[Build SPARQL UPDATE INSERT DATA { ... }]
     │
[HTTP POST → /repositories/nexa_knowledge/statements]
     │
[Index in Qdrant; write Qdrant point id back via second SPARQL UPDATE]

Workflow: Question Router

[#nexa:ask Query]
     │
   ┌─┴────────────────┐
   ▼                  ▼
[SAIA: NL → SPARQL]  [Qdrant: kNN]
   │                  │
[POST → SPARQL endpoint]
   │                  │
   └──────┬───────────┘
          ▼
    rank + merge → SAIA answer

Graph-management commands

#nexa:graph-status

Returns triple count, class histogram, most-connected entity. Implemented as one SPARQL SELECT (COUNT).

#nexa:graph-trace [entity]

Returns the 1-hop (and optionally 2-hop) neighbourhood — a DESCRIBE <iri> plus a templated outgoing/incoming query.

#nexa:graph-rebuild

Clears the named graph and replays Obsidian + Memos. SPARQL: CLEAR GRAPH <https://nuclide.systems/nexa/runtime> followed by the import workflow.


Why two stores

Scenario Qdrant GraphDB Best
"Which note was similar to this one?" Qdrant
"What blocks this task?" GraphDB
"Explain this project" context structure both
"All JWT-related open work" semantic crisp both

Combined: complete understanding rather than a search index or a structure index.