diff --git a/services/mcp-gateway.md b/services/mcp-gateway.md index 62bb411..3cc4fe4 100644 --- a/services/mcp-gateway.md +++ b/services/mcp-gateway.md @@ -88,6 +88,55 @@ Settings → Integrations → Add MCP server: - URL: `https://ai.nuclide.systems/mcp` - Header: `Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52` +## LLM Inference & Governance + +Bifrost proxies LLM inference at `/v1` (OpenAI-compatible). `enforce_auth_on_inference=1` — all `/v1` calls require a valid `sk-bf-*` VK. + +### Providers + +| Provider | Internal name | Base URL | Notes | +|---|---|---|---| +| SAIA (GPU cluster) | `openai` | `https://chat-ai.academiccloud.de/v1` | Rate limited (see below) | +| Google Gemini | `gemini` | default | | +| Mistral AI | `mistral` | default | | +| Cerebras | `cerebras` | default | | +| claude-max-bridge | `openrouter` | `http://claude-max-bridge:8000` | Claude Opus/Sonnet/Haiku via max subscription | + +### SAIA rate limits + +SAIA enforces per-account quotas. Bifrost is configured with a global provider-level limit (`config_providers.rate_limit_id='saia-minute'`): + +| Window | SAIA limit | Bifrost enforcement | +|---|---|---| +| Per minute | 30 req | ✅ active (`saia-minute` row) | +| Per hour | 200 req | row exists (`saia-hour`), not linked | +| Per day | 1000 req | row exists (`saia-day`), not linked | +| Per month | 3000 req | not trackable across restarts | + +**Note**: Bifrost only supports one rate limit window per provider. The minute window is linked because it provides burst protection. The `saia-hour` / `saia-day` rows are in `governance_rate_limits` and can be linked by updating `config_providers SET rate_limit_id='saia-hour'` if needed. + +To change the active window: +```bash +sqlite3 /opt/stacks/ai/bifrost/data/config.db \ + "UPDATE config_providers SET rate_limit_id='saia-day' WHERE name='openai';" +docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate +``` + +**Rate limit API**: `GET /api/governance/rate-limits` is read-only. POST/PUT return 405. Use direct SQLite to create new entries. + +### Virtual key governance + +All VKs (`governance_virtual_key_provider_configs`) have: +- `allow_all_keys=1` — set via SQL (Bifrost API PUT silently ignores this field) +- `allowed_models` — explicit JSON model list as text in DB (SQL NULL = deny all with enforce_auth_on_inference=1) + +If models stop working after a Bifrost upgrade/restore, re-run `/tmp/fix_vk_final.py` on CT 104 and then: +```bash +sqlite3 /opt/stacks/ai/bifrost/data/config.db \ + "UPDATE governance_virtual_key_provider_configs SET allow_all_keys=1;" +docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate +``` + ## Ops ```bash @@ -111,8 +160,12 @@ curl -s http://localhost:14003/mcp \ ### Add a new MCP client ```bash +curl -sc /tmp/bfcookies http://localhost:14003/api/session/login \ + -H "Content-Type: application/json" \ + -d '{"username":"fkrebs","password":"tapirnase"}' > /dev/null + curl -s -X POST http://localhost:14003/api/mcp/client \ - -H "Authorization: Bearer sk-tapirnase" \ + -b /tmp/bfcookies \ -H "Content-Type: application/json" \ -d '{ "name": "",