security: add data-leak audits (2 sessions + comparison + meta + transcript) and SAIA reclassification
This commit is contained in:
@@ -0,0 +1,144 @@
|
||||
# Data-leak audit — 2026-05-21 · `Analyzing LUMEN TR004 Test Data`
|
||||
|
||||
**Conversation**: `ec5fba44-Analyzing_LUMEN_TR004_Test_Data.json`
|
||||
**LobeHub session model**: `qwen3-coder-30b-a3b-instruct` (19 turns) + `llama-3.3-70b-instruct` (2 turns)
|
||||
**Total messages**: 41 (3 user · 21 assistant · 17 tool)
|
||||
**Auditor**: agent doctrine-driven scan, 2026-05-21
|
||||
|
||||
## TL;DR
|
||||
|
||||
**Verdict: NO unapproved data egress. All conversation data stayed within the homelab + approved-partner perimeter.**
|
||||
|
||||
- **Zero** `lobe-cloud-sandbox` calls — the breach channel from the prior day is absent. Model used LobeHub's **builtin `Artifacts`** tool instead (SVG + interactive HTML generators) — those run in-process.
|
||||
- LLM inference went to **SAIA / GWDG** via LiteLLM (21 turns) — **approved partner**, [DLR-vetted](https://docs.hpc.gwdg.de/services/ai-services/saia/index.html), integrated with the DLR IdP federation. Not a leak in this context.
|
||||
|
||||
Two off-host channels (both approved or low-risk):
|
||||
|
||||
1. **SAIA (GWDG academic)** — 21 assistant turns sent prompt + Shepard data + tool messages. ✅ approved partner.
|
||||
2. **`cdn.jsdelivr.net`** — Chart.js library reference in one generated HTML artifact. The HTML never rendered (see "Why pictures didn't render"), so the fetch never happened. If/when it does, it's a public-CDN library fetch, no payload data. Low risk; informational.
|
||||
|
||||
Shepard MCP fetches and the Artifacts tool stayed on-LAN.
|
||||
|
||||
## Data flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
user(["You · browser"]) -->|HTTPS via Zoraxy| lobe["LobeHub · CT 104"]
|
||||
lobe -->|MCP, internal| shep[Shepard API · CT 101]
|
||||
shep -->|test data, lab notes| lobe
|
||||
lobe -->|prompt + Shepard results +<br/>tool messages| litellm[LiteLLM proxy · CT 104]
|
||||
litellm -.->|qwen3-coder-30b-a3b-instruct<br/>llama-3.3-70b-instruct<br/>19+2 calls| saia[(SAIA / GWDG<br/>academic provider)]
|
||||
litellm -. fallback only .- cerebras[(Cerebras)]
|
||||
litellm -. fallback only .- gemini[(Gemini)]
|
||||
litellm -. fallback only .- mistral[(Mistral)]
|
||||
lobe -->|builtin Artifacts<br/>generateSVG + generateInteractiveHTML| af[Artifacts plugin<br/>in-container]
|
||||
af -.failed render.-> user
|
||||
af -. would have fetched if rendered .-> cdn[(cdn.jsdelivr.net)]
|
||||
|
||||
classDef leak fill:#5a2a2a,stroke:#c44,color:#fcc
|
||||
classDef ok fill:#234c2a,stroke:#4c8,color:#cfc
|
||||
classDef stale stroke-dasharray:4 4,color:#888
|
||||
class saia leak
|
||||
class shep,lobe,litellm,user,af ok
|
||||
class cerebras,gemini,mistral,cdn stale
|
||||
```
|
||||
|
||||
## Tool / channel inventory
|
||||
|
||||
| Channel | Calls | Destination | Trust |
|
||||
|---|---|---|---|
|
||||
| `shepard` MCP | 9 | `shepard-api.nuclide.systems` (CT 101) | ✅ LAN |
|
||||
| `Artifacts` builtin (`generateSVG` + `generateInteractiveHTML`) | 5 | In-container (broken — empty result) | ✅ LAN |
|
||||
| `lobe-agent-documents` (`createDocument`, `readDocument`, `replaceDocumentContent`) | 3 | In-container | ✅ LAN |
|
||||
| LLM inference (`qwen3-coder-30b-a3b-instruct` + `llama-3.3-70b-instruct`) | 21 | **SAIA (GWDG)** via LiteLLM | ✅ approved partner (DLR-vetted, IdP-federated) |
|
||||
|
||||
## Why picture rendering didn't work
|
||||
|
||||
Three layered failures.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant M as Model (qwen3-coder)
|
||||
participant T as Artifacts tool
|
||||
participant U as LobeHub UI
|
||||
participant B as Your browser
|
||||
|
||||
M->>T: generateSVG(content="<svg>…</svg>")
|
||||
T-->>M: "" (empty response)
|
||||
Note over M,T: Tool result is length 0 — the SVG was<br/>accepted but no URL / handle came back.
|
||||
M->>U: markdown with relative path:<br/>
|
||||
U->>B: render markdown as-is
|
||||
B->>U: GET /timeline_view.svg
|
||||
U-->>B: 200 SPA index.html (catch-all route)
|
||||
Note over B: "links take me to chat.nuclide.systems/"
|
||||
```
|
||||
|
||||
1. **`Artifacts` tool returns empty.** Every `generateSVG` / `generateInteractiveHTML` result had `content` length 0. The plugin is supposed to register the SVG/HTML as a side-panel "artifact" that the UI surfaces inline, but it returns nothing useful to the model — so the model has no handle/URL to reference.
|
||||
2. **Model invents relative paths.** Without a real URL, the model writes markdown like `` — relative paths against the SPA route, which returns `index.html` for any unknown path. That's why every "link takes you to `chat.nuclide.systems/`".
|
||||
3. **No content store on the homelab.** Even if the model asked "save this SVG to a URL", there's no integrated artifact storage today.
|
||||
|
||||
LobeHub's `Artifacts` works in Anthropic's hosted claude.ai because of client-side inline rendering. The self-hosted version's behavior here is broken / incomplete — either a config gap or the build is newer than the artifact-render code.
|
||||
|
||||
## Fix: S3 as a data-exchange hub
|
||||
|
||||
You already have **Garage S3** on CT 104 (`/opt/stacks/shared-db/garage/`, moved to local NVMe 2026-05-19). Repurpose it as the artifact store for every AI surface.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph LH[CT 104 LobeHub]
|
||||
model[Model + Artifacts tool]
|
||||
interceptor["upload sidecar / fork:<br/>capture generateSVG / HTML output"]
|
||||
end
|
||||
subgraph S3[CT 104 Garage S3]
|
||||
bucket[(chat-artifacts bucket<br/>public-read on /pub/* prefix)]
|
||||
end
|
||||
cs[("Coder workspaces<br/>CT 111<br/>S3 SDK"
|
||||
)]
|
||||
user(["Browser"])
|
||||
zx[Zoraxy<br/>s3.nuclide.systems]
|
||||
|
||||
model -->|content| interceptor
|
||||
interceptor -->|PUT /chat-artifacts/<chatId>/<n>.svg| bucket
|
||||
interceptor -->|public URL| model
|
||||
model -->|markdown with absolute URL| user
|
||||
user -->|GET| zx -->|TLS+ACME| bucket
|
||||
cs <-->|S3 SDK| bucket
|
||||
```
|
||||
|
||||
### Build order (parallel-able after the first two)
|
||||
|
||||
```
|
||||
[no prereqs]
|
||||
└── B. Fix s3.nuclide.systems Zoraxy route + Garage external endpoint (~30 min)
|
||||
└── C. chat-artifacts bucket + ACL + lifecycle (~20 min)
|
||||
├── D. upload_artifact MCP server (CT 104 gateway, ~1 h)
|
||||
├── E. LobeHub Artifacts patch → S3 upload (TypeScript, ~3-4 h)
|
||||
└── F. Coder savefig helper into dotfiles (~30 min)
|
||||
```
|
||||
|
||||
D, E, F can run concurrently once C is up. Total elapsed if D+F land in parallel and E is deferred: ~2 hours wall time.
|
||||
|
||||
## Criticality matrix
|
||||
|
||||
| Channel | Sensitivity | Likelihood | Trust | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| SAIA inference (via LiteLLM) | High | 100% | ✅ approved partner | **OK** |
|
||||
| `cdn.jsdelivr.net` (CDN libs) | Low | ~Low (only when artifact renders) | external CDN, no payload data | LOW |
|
||||
| Shepard MCP fetches | Internal | Every related chat | ✅ LAN | LOW |
|
||||
| `Artifacts` plugin | In-container | 100% in this chat | ✅ LAN (just broken) | n/a |
|
||||
|
||||
## Recommended actions (ordered, with parallelism)
|
||||
|
||||
```
|
||||
[no prereqs — start any]
|
||||
├── A. Pin sensitive chats to a local LLM (LobeHub + LiteLLM)
|
||||
├── B. Fix s3.nuclide.systems Zoraxy route (Zoraxy — needs operator OK)
|
||||
│ └── C. chat-artifacts bucket + ACL + lifecycle (Garage)
|
||||
│ ├── D. upload_artifact MCP server (CT 104 gateway)
|
||||
│ ├── E. Fix LobeHub Artifacts to push S3 (TypeScript patch)
|
||||
│ └── F. Coder savefig helper (dotfiles)
|
||||
├── G. Clean stale DAYTONA_API_KEY from LobeHub (CT 104 env)
|
||||
└── H. Egress firewall block (Cerebras / lobehub / codesandbox) (UniFi UDM)
|
||||
```
|
||||
|
||||
See also: [`data-leak-audit-2026-05-20-tr004-cloud-sandbox.md`](data-leak-audit-2026-05-20-tr004-cloud-sandbox.md) for the prior session, and [`data-leak-audit-comparison.md`](data-leak-audit-comparison.md) for the side-by-side.
|
||||
Reference in New Issue
Block a user