92 lines
4.6 KiB
Markdown
92 lines
4.6 KiB
Markdown
# MCP Gateway v2 — Operational
|
|
|
|
Finished implementation of the DinD MCP gateway (Option A). Supersedes the
|
|
"Phase 1" stub described in `mcp-gateway-requirements.md`.
|
|
|
|
## What it is
|
|
|
|
`ai/mcp-gateway/` — FastAPI app, container `mcp-gateway`, public via Zoraxy at
|
|
`https://mcp.nuclide.systems` (→ `192.168.1.40:8080`). Single OAuth-gated
|
|
entrypoint fronting several MCP servers.
|
|
|
|
## Architecture (as built)
|
|
|
|
- **Spawn model**: Docker-in-Docker. Each spawnable server runs as
|
|
`mcp-<name>` on the **`ai-internal`** network with **no host ports**
|
|
(fixes the old `8000:8000` collision — servers talk over internal DNS).
|
|
- **Transport bridge**: generic streaming reverse proxy
|
|
`https://mcp.nuclide.systems/<server>/<path>` → `http://<upstream>/<path>`.
|
|
Transport-agnostic — works for streamable-HTTP (`/mcp`) and legacy SSE
|
|
(`/sse`+`/messages`). Registered LAST so specific routes win.
|
|
- **Auth**: Pocket ID. Two paths off the same OIDC app:
|
|
- **MCP clients / API / proxy** — `Bearer` token, validated via OIDC
|
|
userinfo (300s TTL cache). `401` carries `WWW-Authenticate` + RFC 9728
|
|
`/.well-known/oauth-protected-resource` for discovery.
|
|
- **Browser Web UI** — full Authorization-Code flow with an httpOnly
|
|
session cookie: unauthenticated `/ui` → `302 /login` → Pocket ID
|
|
`authorize` (CSRF `state` cookie) → `/sso/callback` (state-checked
|
|
code→token exchange) sets the `mcp_session` cookie and shows the page +
|
|
a copyable Bearer token for MCP clients. `/logout` clears it. The UI and
|
|
`/api/*` accept **either** the cookie or a Bearer header; the `/{srv}/`
|
|
proxy stays Bearer-only. `/health` + the well-known metadata are open.
|
|
- **Secrets**: all per-server OAuth client secrets come from `ai/.env`
|
|
(`env_file: ../.env`), read via `os.environ` — none hardcoded in source.
|
|
|
|
## Server registry (`SERVERS` in server.py)
|
|
|
|
| name | kind | upstream | notes |
|
|
|---|---|---|---|
|
|
| comfyui | static | `http://comfyui-mcp:8000` | runs as its own stack (`ai/comfyui-mcp.yml`); gateway only proxies |
|
|
| nextcloud | spawn | `mcp-nextcloud:8000` | `ghcr.io/cbcoutinho/nextcloud-mcp-server` streamable-http |
|
|
| crawl4ai | spawn | `mcp-crawl4ai:11235` | `unclecode/crawl4ai` |
|
|
| mermaid | spawn | `mcp-mermaid:8000` | `node:20-slim` + npx mcp-mermaid (validated working) |
|
|
| markitdown | spawn | `mcp-markitdown:8000` | `ghcr.io/astral-sh/uv` + uvx markitdown-mcp |
|
|
| papersearch | spawn | `mcp-papersearch:8000` | python + uv + mcp-proxy → paper_search_mcp |
|
|
|
|
## Verified
|
|
|
|
- `/health` 200 (no auth); `/.well-known/oauth-protected-resource` JSON (no auth).
|
|
- `/mcp.json`, `/<server>/mcp` → 401 without token; 401 with bad token
|
|
(userinfo path); correct `WWW-Authenticate`.
|
|
- Upstream path proven: gateway → `comfyui-mcp` `/mcp` initialize → 200;
|
|
gateway → spawned `mcp-mermaid` `/mcp` → 200 (spawn pattern + proxy).
|
|
- Route precedence correct (specific routes beat the catch-all proxy).
|
|
|
|
## Not yet exercised (needs a real token / operational bring-up)
|
|
|
|
- **Token-authenticated end-to-end** requires an interactive Pocket ID login.
|
|
Get a token: open `https://mcp.nuclide.systems/sso/callback` via the OAuth
|
|
authorize flow (or any Pocket ID token with scopes `openid,mcp`); the
|
|
callback page prints the access token.
|
|
- **Bring up the 4 remaining spawnable servers** (nextcloud/crawl4ai/
|
|
markitdown/papersearch): `POST /api/servers/<name>/start` (bearer) or
|
|
`POST /api/servers/bulk/start`. Each may need a transport/flag tweak on
|
|
first run — mermaid is the proven reference.
|
|
|
|
## Client registration
|
|
|
|
- **LobeChat**: Settings → Skills → Skill Store → Custom → Import JSON, e.g.
|
|
```json
|
|
{ "mcpServers": { "comfyui": { "type": "http",
|
|
"url": "https://mcp.nuclide.systems/comfyui/mcp",
|
|
"auth": { "type": "bearer", "accessToken": "<pocket-id token>" } } } }
|
|
```
|
|
(Or, for ComfyUI specifically, connect LobeChat straight to
|
|
`http://comfyui-mcp:8000/mcp` internally — see `comfyui-mcp.md` — no token.)
|
|
- **Claude.ai**: add `https://mcp.nuclide.systems/<server>/mcp` as a custom
|
|
connector; it will discover auth via the protected-resource metadata.
|
|
|
|
## Ops
|
|
|
|
- `cd /opt/stacks/ai/mcp-gateway && docker compose up -d --build`
|
|
- Logs: `docker logs mcp-gateway`; UI: `https://mcp.nuclide.systems/ui`
|
|
(Pocket-ID-gated; `/login` to sign in).
|
|
- Config overlay persisted in `ai/mcp-gateway/config.json`.
|
|
- **UI check (Playwright)** — the live UI is auth-gated, so the check
|
|
serves the local template + mocks `/api/servers`, screenshots, and
|
|
asserts cards render:
|
|
```
|
|
uv run --with playwright -- python -m playwright install chromium # once
|
|
uv run --with playwright -- python ai/mcp-gateway/ui_check.py # -> /tmp/mcp_ui.png
|
|
```
|