From 0600032b75331ffdbaf3238f1109072e9e2ce586 Mon Sep 17 00:00:00 2001 From: "fkrebs (via Claude)" Date: Tue, 26 May 2026 12:55:33 +0200 Subject: [PATCH] docs: migrate MCP gateway docs from mcp-gateway to Bifrost - mcp-gateway.md: full rewrite for Bifrost at ai.nuclide.systems/mcp; 29 clients / ~760 tools; VK auth; n8n+shepard blocked on streamable-HTTP; legacy gateway decommission checklist added - connection-hosts.md: Bifrost at port 14003 is now ai.nuclide.systems; LiteLLM demoted to internal-only; mcp.nuclide.systems marked pending decommission; OIDC client 78c78998 flagged for removal Co-Authored-By: Claude Sonnet 4.6 --- infra/connection-hosts.md | 19 +-- services/mcp-gateway.md | 235 +++++++++++++++++++------------------- 2 files changed, 127 insertions(+), 127 deletions(-) diff --git a/infra/connection-hosts.md b/infra/connection-hosts.md index 4fab2e2..943b2c0 100644 --- a/infra/connection-hosts.md +++ b/infra/connection-hosts.md @@ -204,16 +204,17 @@ All behind `vpn_gluetun` container network. | Port | Container | Public URL | Notes | |---|---|---|---| -| 14000 | `litellm` | `https://ai.nuclide.systems` | LLM proxy; internal: `http://litellm:4000`; master key: `sk-tapirnase` | +| 14000 | `litellm` | — | LLM proxy (internal only: `http://litellm:4000`); master key: `sk-tapirnase` | | 14001 | `lobehub` | `https://chat.nuclide.systems` | Chat UI (OIDC `26f3c26b`) | | 14002 | `qdrant_scientific` | — | Vector DB (LAN/internal only) | -| 8080 | `mcp-gateway` | `https://mcp.nuclide.systems` | MCP gateway (OIDC `e73bb7b9`); API key: `JxxCqKw32XoN4LOHunDikS6u1RpS7R5ythzaqADPuIA` | +| 14003 | `bifrost` | `https://ai.nuclide.systems` | LLM+MCP gateway; `/mcp` = MCP endpoint (29 clients, ~760 tools); master key: `sk-tapirnase` | +| 8080 | `mcp-gateway` | `https://mcp.nuclide.systems` | **PENDING DECOMMISSION** — legacy FastAPI DinD gateway; see [mcp-gateway.md](../services/mcp-gateway.md) | | 18002 | `comfyui` | — | Image gen (Intel Arc); LAN only | -| 18003 | `comfyui-mcp` | via gateway `/comfyui/mcp` | FastMCP | -| 18005 | `docling-mcp` | via gateway `/docling/mcp` | PDF→Markdown | -| 18007 | `kroki-mcp` | via gateway `/kroki/mcp` | Diagram rendering | +| 18003 | `comfyui-mcp` | via Bifrost `comfyui` client | FastMCP | +| 18005 | `docling-mcp` | via Bifrost `docling` client | PDF→Markdown | +| 18007 | `kroki-mcp` | via Bifrost `kroki` client | Diagram rendering | | 18009 | `speaches` | — | TTS/STT; LAN only | -| 18011 | `upload-artifact-mcp` | via gateway `/upload-artifact/mcp` | S3 artifact upload | +| 18011 | `upload-artifact-mcp` | via Bifrost `upload_artifact` client | S3 artifact upload | | internal | `searxng` | — | `shared_backend`; used by LobeChat | ### Documents (15000–15999) @@ -285,7 +286,7 @@ All behind `vpn_gluetun` container network. | `38469e7e` | Proxmox VE | `https://192.168.1.20:8006` | | `cbbf20d5` | Audiobookshelf | `https://abs.nuclide.systems/…` + `*` | | `d8733fcc` | shelfarr | `*` | -| `78c78998` | Claude MCP | `https://claude.ai/api/mcp/auth_callback` + `https://mcp.nuclide.systems/mcp/auth/callback` | +| `78c78998` | Claude MCP (legacy) | `https://claude.ai/api/mcp/auth_callback` + `https://mcp.nuclide.systems/mcp/auth/callback` — **remove when mcp-gateway decommissioned** | | `82ca2d53` | nuc-ai (MCP spawned servers) | (empty) | | `7fe1a14b` | zoraxy | (empty) | | (new) | vaultwarden | `https://vault.nuclide.systems/auth/callback` — SSO not yet wired | @@ -297,9 +298,9 @@ All behind `vpn_gluetun` container network. | URL | Backend | Service | |---|---|---| -| `https://ai.nuclide.systems` | `192.168.1.40:14000` | LiteLLM (LLM proxy) | +| `https://ai.nuclide.systems` | `192.168.1.40:14003` | Bifrost (LLM+MCP gateway) — `/mcp` for MCP tools | | `https://chat.nuclide.systems` | `192.168.1.40:14001` | LobeChat | -| `https://mcp.nuclide.systems` | `192.168.1.40:8080` | MCP gateway | +| `https://mcp.nuclide.systems` | `192.168.1.40:8080` | MCP gateway (**pending decommission**) | | `https://id.nuclide.systems` | `192.168.1.5:11000` | Pocket-ID (OIDC) | | `https://nc.nuclide.systems` | `192.168.1.41:11000` | Nextcloud | | `https://ha.nuclide.systems` | `192.168.1.60:8123` | Home Assistant | diff --git a/services/mcp-gateway.md b/services/mcp-gateway.md index 1008124..371bf26 100644 --- a/services/mcp-gateway.md +++ b/services/mcp-gateway.md @@ -1,155 +1,154 @@ -# MCP Gateway v2 — Operational +# MCP Gateway — Bifrost (migrated 2026-05-26) -DinD MCP gateway on CT 104. Single OAuth-gated entry point fronting 24 MCP servers (mix of static, spawned, and Docker-catalog-bridged). Public via Zoraxy at `https://mcp.nuclide.systems` (→ `192.168.1.40:8080`). +MCP servers are now aggregated by **Bifrost** at `https://ai.nuclide.systems/mcp`. +The legacy FastAPI DinD mcp-gateway (`mcp.nuclide.systems`) is pending decommission. ## Architecture -- **Stack**: `ai/mcp-gateway/` — FastAPI app, container `mcp-gateway`. -- **Spawn model**: Docker-in-Docker. Each spawnable server runs as `mcp-` on the `ai-internal` network with no host ports — servers talk over internal DNS. -- **Transport bridge**: generic streaming reverse proxy `https://mcp.nuclide.systems//` → `http:///`. Transport-agnostic (streamable-HTTP `/mcp` and legacy SSE `/sse` + `/messages`). Registered last so specific routes win. -- **Auth**: Pocket ID. Two paths off the same OIDC app: - - **MCP clients / API / proxy** — `Bearer` token, validated via OIDC userinfo (300 s TTL cache). `401` carries `WWW-Authenticate` + RFC 9728 `/.well-known/oauth-protected-resource` for discovery. - - **Browser Web UI** — Authorization-Code flow with httpOnly session cookie. `/ui` and `/api/*` accept cookie or Bearer; `/{srv}/` stays Bearer-only. -- **Secrets**: per-server OAuth client secrets live in `ai/.env` (read via `os.environ`), never hardcoded. +- **Stack**: `ai/bifrost/` on CT 104 — Bifrost LLM+MCP gateway, SQLite state at `data/config.db`. +- **Endpoint**: `https://ai.nuclide.systems/mcp` (Zoraxy → `192.168.1.40:14003`) +- **Auth**: Virtual Key (`sk-bf-` prefix) via `Authorization: Bearer `. +- **Client model**: Each upstream MCP server is a registered client in Bifrost (`auth_type=none`, `allow_on_all_virtual_keys=true`). Tools are auto-discovered and enabled via `tools_to_execute_json`. +- **Internal upstreams**: All child MCP containers run on the `ai-internal` Docker network at `http://-mcp:8000/mcp` (streamable-HTTP) or as dedicated stacks. -## Server inventory (as of 2026-05-23) +## Virtual keys -29 servers configured in `mcp-gateway/config.json`. **Status: 30 healthy** (includes gitea, paperless, proxmox added 2026-05-23). +| Name | Key prefix | Use | +|---|---|---| +| `claude-code` | `sk-bf-bfc19117-4c46-4d48-9b10-85d78b1ae2b3` | Claude Code + Claude.ai | +| `mcp-dev` | `sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52` | Testing / dev | -| Group | Server | Kind | Notes | -|---|---|---|---| -| **dev** | context7 | catalog | `mcp/context7` | -| | fetch | spawn | `mcp-server-fetch` via uvx | -| | git | catalog | `mcp/git` — Docker MCP Catalog; `mcp-git` bridge + DinD child | -| | gitlab | catalog | `mcp/gitlab` — `GITLAB_API_URL=https://gitlab.dlr.de/api/v4`; requires `--pass-environment` in bridge cmd | -| | kroki | static | `kroki-mcp:8000` (own stack) | -| | markitdown | catalog | `mcp/markitdown` | -| | sequential-thinking | catalog | `mcp/sequentialthinking` | -| | time | spawn | `mcp-server-time` | -| | docling | static | `docling-mcp:8000` | -| | coder | static | `coder-mcp:8000` | -| | shepard | static | `https://shepard.nuclide.systems/v2/mcp` | -| **research** | crawl4ai | spawn | `unclecode/crawl4ai` | -| | paper-search | catalog | `mcp/paper-search` — Docker MCP Catalog; replaces old `python:3.12-slim` inline approach | -| | searxng | spawn | `isokoliuk/mcp-searxng` | -| | wikipedia-mcp | catalog | | -| | youtube-transcript | catalog | `mcp/youtube-transcript` | -| **personal** | bluesky | static | `ariel-mcp:8000` | -| | obsidian | spawn | `mcp-obsidian-bridge` + mcpvault; vault at Nextcloud `fkrebs@nucli.de/files/Notizen` (UNAS mount) | -| | gotify | spawn | | -| | home-assistant | static | `http://192.168.1.60:9583/private_...` (HA add-on) | -| | immich | spawn | | -| | memos | spawn | uses `MEMOS_TOKEN` from `.env` | -| | n8n | static | `https://n8n.nuclide.systems/mcp-server/http` | -| | nextcloud | spawn | `ghcr.io/cbcoutinho/nextcloud-mcp-server` — `single_user_basic` mode, `NEXTCLOUD_VERIFY_SSL=false`; app-password generated via `occ` for `fkrebs@nucli.de` 2026-05-22 | -| | unifi | spawn | | -| **image** | comfyui | static | `comfyui-mcp:8000` (own stack, see [comfyui.md](comfyui.md)) | -| **storage** | upload-artifact | static | `upload-artifact-mcp:8000` | -| **infra** | gitea | static | `gitea-mcp:8000` — Gitea API MCP server | -| | paperless | static | `paperless-mcp:8000` — Paperless-ngx MCP (`@nloui/paperless-mcp` via mcp-proxy) | -| | proxmox | static | `proxmox-mcp:8000` — read-only PVE API (`PVEAuditor` role, token `root@pam!mcp`) | +## Server inventory (as of 2026-05-26) -> **Catalog bridge env var passthrough**: `mcp-proxy`'s `stdio_client` uses a sanitized env (`get_default_environment()`), stripping custom vars. For catalog servers that need env vars (e.g. `gitlab`), the bridge command **must** include `--pass-environment` so Docker CLI can resolve `-e KEY` passthrough from the bridge container's env. +**29 connected clients, ~760 tools total.** -> **Server count**: 30 total (27 from 2026-05-22 + gitea, paperless, proxmox added 2026-05-23). Gateway updated to persist `enabled` flag across restarts. +| Client name | Upstream | Notes | +|---|---|---| +| `bluesky` | `http://ariel-mcp:8000/mcp` | | +| `coder` | `http://coder-mcp:8000/mcp` | | +| `comfyui` | `http://comfyui-mcp:8000/mcp` | Intel Arc image gen | +| `context7` | `http://mcp-context7:8000/mcp` | | +| `crawl4ai` | `http://mcp-crawl4ai:11235/mcp/sse` (SSE) | | +| `docling` | `http://docling-mcp:8000/mcp` | PDF→Markdown | +| `fetch` | `http://mcp-fetch:8000/mcp` | | +| `git` | `http://mcp-git:8000/mcp` | | +| `gitea` | `http://gitea-mcp:8000/mcp` | | +| `gitlab` | `http://mcp-gitlab:8000/mcp` | `GITLAB_API_URL=https://gitlab.dlr.de/api/v4` | +| `gotify` | `http://mcp-gotify:8000/mcp` | | +| `home_assistant` | `http://192.168.1.60:9583/private_ehnWeRl2G3De6NnbcN7teQ` | HA add-on; no TLS | +| `immich` | `http://mcp-immich:8000/mcp` | | +| `kroki` | `http://kroki-mcp:8000/mcp` | Diagram rendering | +| `markitdown` | `http://mcp-markitdown:8000/mcp` | | +| `memos` | `http://mcp-memos:8000/mcp` | | +| `nextcloud` | `http://mcp-nextcloud:8000/mcp` | | +| `ntfy` | `http://mcp-ntfy:8000/mcp` | | +| `obsidian` | `http://mcp-obsidian:8000/mcp` | Vault at Nextcloud/UNAS | +| `paper_search` | `http://mcp-paper-search:8000/mcp` | | +| `paperless` | `http://paperless-mcp:8000/mcp` | | +| `proxmox` | `http://mcp-proxmox:8000/mcp` | Read-only (`PVEAuditor`) | +| `searxng` | `http://mcp-searxng:8000/mcp` | | +| `sequential_thinking` | `http://mcp-sequential-thinking:8000/mcp` | | +| `time` | `http://mcp-time:8000/mcp` | | +| `unifi` | `http://mcp-unifi:8000/mcp` | | +| `upload_artifact` | `http://upload-artifact-mcp:8000/mcp` | S3 via Garage | +| `wikipedia` | `http://mcp-wikipedia-mcp:8000/mcp` | | +| `youtube_transcript` | `http://mcp-youtube-transcript:8000/mcp` | | -## Client registration +### Not yet connected -### Easy path — `/mcp-config` download +| Client | Reason | +|---|---| +| `n8n` — `https://n8n.nuclide.systems/mcp-server/http` | Streamable-HTTP transport: POST returns SSE stream, Bifrost HTTP client times out. | +| `shepard` — `https://shepard.nuclide.systems/v2/mcp` | Same streamable-HTTP issue. Bearer token stored in `/tmp/migrate_mcp_oauth.py`. | -The gateway hands back a ready-to-use config blob for the signed-in user: +## Client configuration -``` -GET https://mcp.nuclide.systems/mcp-config?format=claude # default -GET https://mcp.nuclide.systems/mcp-config?format=cursor -GET https://mcp.nuclide.systems/mcp-config?format=raw # token + URLs -``` +### Claude Code -- Requires Pocket-ID session cookie or existing Bearer (303 → `/login` otherwise). -- Mints a fresh long-lived gateway token (`mcp_`) bound to the user; persisted in `gateway_tokens.json` with `purpose=client-config:` so it's distinguishable from OAuth-issued tokens and revocable. -- Returns JSON with `Content-Disposition: attachment`. - -**Claude Code workflow:** -1. Sign in at `https://mcp.nuclide.systems`. -2. Click **Claude config** in the top nav → browser downloads `nuclide-mcp.claude.json`. -3. Merge the `mcpServers` block into `~/.claude/settings.json`. Restart Claude Code. - -The downloaded file has one entry per enabled MCP server, with `type: http` + `url: https://mcp.nuclide.systems//mcp` + `Authorization: Bearer ...` header. No per-server config needed. - -### Manual - -**LobeChat** — Settings → Skills → Skill Store → Custom → Import JSON: +Add to `~/.claude.json` or project `.mcp.json`: ```json -{ "mcpServers": { "comfyui": { "type": "http", - "url": "https://mcp.nuclide.systems/comfyui/mcp", - "auth": { "type": "bearer", "accessToken": "" } } } } +{ + "mcpServers": { + "nuclide": { + "type": "http", + "url": "https://ai.nuclide.systems/mcp", + "headers": { + "Authorization": "Bearer sk-bf-bfc19117-4c46-4d48-9b10-85d78b1ae2b3" + } + } + } +} ``` -(For ComfyUI specifically, LobeChat can also connect straight to `http://comfyui-mcp:8000/mcp` internally — see [comfyui.md](comfyui.md) — no token.) +### Claude.ai -**Claude.ai** — add `https://mcp.nuclide.systems//mcp` as a custom connector; auth is discovered via the protected-resource metadata. - -### Token rotation - -Each `/mcp-config` download mints a NEW token. Old tokens stay valid until removed from `gateway_tokens.json` (manual prune; future: admin UI button). +Settings → Integrations → Add MCP server: +- URL: `https://ai.nuclide.systems/mcp` +- Header: `Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52` ## Ops ```bash -cd /opt/stacks/ai/mcp-gateway && docker compose up -d --build -docker logs mcp-gateway +# On CT 104 +cd /opt/stacks/ai/bifrost +docker compose up -d --force-recreate + +docker logs bifrost -f + +# Inspect config DB +sqlite3 data/config.db '.tables' +sqlite3 data/config.db 'SELECT name, auth_type, allow_on_all_virtual_keys FROM config_mcp_clients;' + +# Count active tools via API +curl -s http://localhost:14003/mcp \ + -H "Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52" \ + -H "Content-Type: application/json" \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' | python3 -m json.tool | grep '"name"' | wc -l ``` -UI: `https://mcp.nuclide.systems/ui` (Pocket-ID-gated; `/login` to sign in). -Models UI: `https://mcp.nuclide.systems/ui/models` — sortable model catalog + service assignment overrides. -Model assignments data: `mcp-gateway/model_assignments.json` (bind-mounted rw). API: `GET /api/model-assignments`, `PATCH /api/model-assignments/{service_key}`. - -Config: `mcp-gateway/config.json` (live config, persisted). Timestamped backups created on every edit (`config.json.bak.YYYYMMDD-HHMMSS`). - -### Add or enable a server - -1. Edit `mcp-gateway/config.json` — add an `mcpServers.` entry (or set `enabled: true`). -2. Restart the gateway: `docker restart mcp-gateway`. -3. Spawn servers come up on demand via `POST /api/servers//start` (bearer); or `POST /api/servers/bulk/start` for all. - -### Disable a server - -Set `mcpServers..enabled = false` and add `disabled_reason`. Stop+rm the container (`docker stop mcp- && docker rm mcp-`); the gateway will not re-spawn it. - -### UI check (Playwright) - -Live UI is auth-gated, so the local check serves the template + mocks `/api/servers`: +### Add a new MCP client ```bash -uv run --with playwright -- python -m playwright install chromium # once -uv run --with playwright -- python ai/mcp-gateway/ui_check.py # -> /tmp/mcp_ui.png +curl -s -X POST http://localhost:14003/api/mcp/client \ + -H "Authorization: Bearer sk-tapirnase" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "", + "connection_type": "http", + "connection_string": "http://:8000/mcp", + "auth_type": "none", + "allow_on_all_virtual_keys": true + }' ``` -## Model management (added 2026-05-23) +Wait ~10 s for tool discovery, then enable tools via `PUT /api/mcp/client/` with `tools_to_execute`. +See `/tmp/add_all_mcp_clients.py` on CT 104 for a complete example. -`model_assignments.json` tracks the model each AI-consuming service uses, plus the full 35-model catalog with capability metadata (vision, tools, context window, latency, cost tier). +### Key requirements for tools to appear in `/mcp tools/list` -**Services tracked:** karakeep_text, karakeep_image, paperless_ai, lobechat_default, nextcloud_assistant, home_assistant_voice. +1. `auth_type=none` (not `per_user_oauth`) +2. `allow_on_all_virtual_keys=true` +3. `tools_to_execute_json` populated (tool names without client prefix) +4. VK must have `sk-bf-` prefix -**Catalog models:** 35 (trimmed from 58 on 2026-05-23; dropped paid API models superseded by free SAIA equivalents). +## Legacy mcp-gateway (FastAPI / DinD) -**Model evaluator agent** in `agents.json` — can be triggered via `POST /api/agents/model-evaluator/run` to audit assignments vs SAIA free baseline. +**Status: pending decommission.** Container `mcp-gateway` still running at `192.168.1.40:8080` / `https://mcp.nuclide.systems`. Stack: `ai/mcp-gateway/`. -**Env file writes**: the PATCH endpoint updates `model_assignments.json` and attempts to rewrite the service's `.env` file in-place. Because host paths (e.g. `/opt/stacks/karakeep/.env`) are not bind-mounted into the gateway container, `env_written` will be `false` — apply manually and `docker compose up -d --force-recreate `. +Before removing: +1. Confirm Zoraxy route `mcp.nuclide.systems` is no longer referenced anywhere (explicit user confirmation required before touching Zoraxy CT 108). +2. Stop container: rename compose file to `docker-compose.yml.disabled`, update `/opt/stacks/CLAUDE.md`. +3. Remove Pocket-ID client `e73bb7b9` (mcp-gateway) and `78c78998` (Claude MCP callback to mcp.nuclide.systems) after confirming no active tokens. +4. Remove Zoraxy route `mcp.nuclide.systems` → `192.168.1.40:8080`. -**Nextcloud assistant** uses `integration_openai` app pointing at `https://ai.nuclide.systems/v1` (LiteLLM). No env file — config is in the Nextcloud DB, changed via `occ`: -```bash -# On CT 105 (192.168.1.41): -docker exec nextcloud-aio-nextcloud php occ config:app:set integration_openai default_completion_model_id --value= -# Clear stale model cache after changes: -docker exec nextcloud-aio-nextcloud php occ config:app:delete integration_openai models -``` -Current assignments (2026-05-23): completion=`qwen3.5-122b-a10b` (vision+tools+65K, DE/EN), TTS=`tts-1-de`, STT=`whisper-large-v3-turbo`, T2I=`saia-flux`. +## Bifrost OIDC client (Pocket ID) -## Verified end-to-end +Used for `per_user_oauth` flows (not currently active — all clients use `auth_type=none`): -- `/health` 200 (no auth); `/.well-known/oauth-protected-resource` JSON (no auth). -- `/mcp.json`, `//mcp` → 401 without token; correct `WWW-Authenticate`. -- Static upstreams (`bluesky`, `coder`, `comfyui`, `docling`, `kroki`, `upload-artifact`) — endpoints reachable from inside `ai-internal` (406/405 to plain GET, which is the correct MCP behavior). -- All spawn/catalog containers up (`docker ps mcp-*`): 16/17 (nextcloud disabled; git/gitlab/paper-search added 2026-05-22). +| Field | Value | +|---|---| +| Client ID | `ec0d15e6-e86d-49b0-ac12-cdfb5afc9086` | +| Authorize URL | `https://id.nuclide.systems/authorize` | +| Token URL | `https://id.nuclide.systems/api/oidc/token` | +| `mcp_external_client_url` | `https://ai.nuclide.systems` |