145 lines
7.3 KiB
Markdown
145 lines
7.3 KiB
Markdown
# Data-leak audit — 2026-05-21 · `Analyzing LUMEN TR004 Test Data`
|
|
|
|
**Conversation**: `ec5fba44-Analyzing_LUMEN_TR004_Test_Data.json`
|
|
**LobeHub session model**: `qwen3-coder-30b-a3b-instruct` (19 turns) + `llama-3.3-70b-instruct` (2 turns)
|
|
**Total messages**: 41 (3 user · 21 assistant · 17 tool)
|
|
**Auditor**: agent doctrine-driven scan, 2026-05-21
|
|
|
|
## TL;DR
|
|
|
|
**Verdict: NO unapproved data egress. All conversation data stayed within the homelab + approved-partner perimeter.**
|
|
|
|
- **Zero** `lobe-cloud-sandbox` calls — the breach channel from the prior day is absent. Model used LobeHub's **builtin `Artifacts`** tool instead (SVG + interactive HTML generators) — those run in-process.
|
|
- LLM inference went to **SAIA / GWDG** via LiteLLM (21 turns) — **approved partner**, [DLR-vetted](https://docs.hpc.gwdg.de/services/ai-services/saia/index.html), integrated with the DLR IdP federation. Not a leak in this context.
|
|
|
|
Two off-host channels (both approved or low-risk):
|
|
|
|
1. **SAIA (GWDG academic)** — 21 assistant turns sent prompt + Shepard data + tool messages. ✅ approved partner.
|
|
2. **`cdn.jsdelivr.net`** — Chart.js library reference in one generated HTML artifact. The HTML never rendered (see "Why pictures didn't render"), so the fetch never happened. If/when it does, it's a public-CDN library fetch, no payload data. Low risk; informational.
|
|
|
|
Shepard MCP fetches and the Artifacts tool stayed on-LAN.
|
|
|
|
## Data flow
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
user(["You · browser"]) -->|HTTPS via Zoraxy| lobe["LobeHub · CT 104"]
|
|
lobe -->|MCP, internal| shep[Shepard API · CT 101]
|
|
shep -->|test data, lab notes| lobe
|
|
lobe -->|prompt + Shepard results +<br/>tool messages| litellm[LiteLLM proxy · CT 104]
|
|
litellm -.->|qwen3-coder-30b-a3b-instruct<br/>llama-3.3-70b-instruct<br/>19+2 calls| saia[(SAIA / GWDG<br/>academic provider)]
|
|
litellm -. fallback only .- cerebras[(Cerebras)]
|
|
litellm -. fallback only .- gemini[(Gemini)]
|
|
litellm -. fallback only .- mistral[(Mistral)]
|
|
lobe -->|builtin Artifacts<br/>generateSVG + generateInteractiveHTML| af[Artifacts plugin<br/>in-container]
|
|
af -.failed render.-> user
|
|
af -. would have fetched if rendered .-> cdn[(cdn.jsdelivr.net)]
|
|
|
|
classDef leak fill:#5a2a2a,stroke:#c44,color:#fcc
|
|
classDef ok fill:#234c2a,stroke:#4c8,color:#cfc
|
|
classDef stale stroke-dasharray:4 4,color:#888
|
|
class saia leak
|
|
class shep,lobe,litellm,user,af ok
|
|
class cerebras,gemini,mistral,cdn stale
|
|
```
|
|
|
|
## Tool / channel inventory
|
|
|
|
| Channel | Calls | Destination | Trust |
|
|
|---|---|---|---|
|
|
| `shepard` MCP | 9 | `shepard-api.nuclide.systems` (CT 101) | ✅ LAN |
|
|
| `Artifacts` builtin (`generateSVG` + `generateInteractiveHTML`) | 5 | In-container (broken — empty result) | ✅ LAN |
|
|
| `lobe-agent-documents` (`createDocument`, `readDocument`, `replaceDocumentContent`) | 3 | In-container | ✅ LAN |
|
|
| LLM inference (`qwen3-coder-30b-a3b-instruct` + `llama-3.3-70b-instruct`) | 21 | **SAIA (GWDG)** via LiteLLM | ✅ approved partner (DLR-vetted, IdP-federated) |
|
|
|
|
## Why picture rendering didn't work
|
|
|
|
Three layered failures.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant M as Model (qwen3-coder)
|
|
participant T as Artifacts tool
|
|
participant U as LobeHub UI
|
|
participant B as Your browser
|
|
|
|
M->>T: generateSVG(content="<svg>…</svg>")
|
|
T-->>M: "" (empty response)
|
|
Note over M,T: Tool result is length 0 — the SVG was<br/>accepted but no URL / handle came back.
|
|
M->>U: markdown with relative path:<br/>
|
|
U->>B: render markdown as-is
|
|
B->>U: GET /timeline_view.svg
|
|
U-->>B: 200 SPA index.html (catch-all route)
|
|
Note over B: "links take me to chat.nuclide.systems/"
|
|
```
|
|
|
|
1. **`Artifacts` tool returns empty.** Every `generateSVG` / `generateInteractiveHTML` result had `content` length 0. The plugin is supposed to register the SVG/HTML as a side-panel "artifact" that the UI surfaces inline, but it returns nothing useful to the model — so the model has no handle/URL to reference.
|
|
2. **Model invents relative paths.** Without a real URL, the model writes markdown like `` — relative paths against the SPA route, which returns `index.html` for any unknown path. That's why every "link takes you to `chat.nuclide.systems/`".
|
|
3. **No content store on the homelab.** Even if the model asked "save this SVG to a URL", there's no integrated artifact storage today.
|
|
|
|
LobeHub's `Artifacts` works in Anthropic's hosted claude.ai because of client-side inline rendering. The self-hosted version's behavior here is broken / incomplete — either a config gap or the build is newer than the artifact-render code.
|
|
|
|
## Fix: S3 as a data-exchange hub
|
|
|
|
You already have **Garage S3** on CT 104 (`/opt/stacks/shared-db/garage/`, moved to local NVMe 2026-05-19). Repurpose it as the artifact store for every AI surface.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph LH[CT 104 LobeHub]
|
|
model[Model + Artifacts tool]
|
|
interceptor["upload sidecar / fork:<br/>capture generateSVG / HTML output"]
|
|
end
|
|
subgraph S3[CT 104 Garage S3]
|
|
bucket[(chat-artifacts bucket<br/>public-read on /pub/* prefix)]
|
|
end
|
|
cs[("Coder workspaces<br/>CT 111<br/>S3 SDK"
|
|
)]
|
|
user(["Browser"])
|
|
zx[Zoraxy<br/>s3.nuclide.systems]
|
|
|
|
model -->|content| interceptor
|
|
interceptor -->|PUT /chat-artifacts/<chatId>/<n>.svg| bucket
|
|
interceptor -->|public URL| model
|
|
model -->|markdown with absolute URL| user
|
|
user -->|GET| zx -->|TLS+ACME| bucket
|
|
cs <-->|S3 SDK| bucket
|
|
```
|
|
|
|
### Build order (parallel-able after the first two)
|
|
|
|
```
|
|
[no prereqs]
|
|
└── B. Fix s3.nuclide.systems Zoraxy route + Garage external endpoint (~30 min)
|
|
└── C. chat-artifacts bucket + ACL + lifecycle (~20 min)
|
|
├── D. upload_artifact MCP server (CT 104 gateway, ~1 h)
|
|
├── E. LobeHub Artifacts patch → S3 upload (TypeScript, ~3-4 h)
|
|
└── F. Coder savefig helper into dotfiles (~30 min)
|
|
```
|
|
|
|
D, E, F can run concurrently once C is up. Total elapsed if D+F land in parallel and E is deferred: ~2 hours wall time.
|
|
|
|
## Criticality matrix
|
|
|
|
| Channel | Sensitivity | Likelihood | Trust | Verdict |
|
|
|---|---|---|---|---|
|
|
| SAIA inference (via LiteLLM) | High | 100% | ✅ approved partner | **OK** |
|
|
| `cdn.jsdelivr.net` (CDN libs) | Low | ~Low (only when artifact renders) | external CDN, no payload data | LOW |
|
|
| Shepard MCP fetches | Internal | Every related chat | ✅ LAN | LOW |
|
|
| `Artifacts` plugin | In-container | 100% in this chat | ✅ LAN (just broken) | n/a |
|
|
|
|
## Recommended actions (ordered, with parallelism)
|
|
|
|
```
|
|
[no prereqs — start any]
|
|
├── A. Pin sensitive chats to a local LLM (LobeHub + LiteLLM)
|
|
├── B. Fix s3.nuclide.systems Zoraxy route (Zoraxy — needs operator OK)
|
|
│ └── C. chat-artifacts bucket + ACL + lifecycle (Garage)
|
|
│ ├── D. upload_artifact MCP server (CT 104 gateway)
|
|
│ ├── E. Fix LobeHub Artifacts to push S3 (TypeScript patch)
|
|
│ └── F. Coder savefig helper (dotfiles)
|
|
├── G. Clean stale DAYTONA_API_KEY from LobeHub (CT 104 env)
|
|
└── H. Egress firewall block (Cerebras / lobehub / codesandbox) (UniFi UDM)
|
|
```
|
|
|
|
See also: [`data-leak-audit-2026-05-20-tr004-cloud-sandbox.md`](data-leak-audit-2026-05-20-tr004-cloud-sandbox.md) for the prior session, and [`data-leak-audit-comparison.md`](data-leak-audit-comparison.md) for the side-by-side.
|