docs: add SAIA rate limits, Bifrost governance, and admin auth notes to mcp-gateway.md
- SAIA rate limits: minute=30/1m active at provider level; hour/day rows exist but not linked - VK governance: allow_all_keys SQL workaround, allowed_models must be non-null JSON text - Fix stale sk-tapirnase reference in add-MCP-client example (now uses cookie auth) - Add LLM provider table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+54
-1
@@ -88,6 +88,55 @@ Settings → Integrations → Add MCP server:
|
||||
- URL: `https://ai.nuclide.systems/mcp`
|
||||
- Header: `Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52`
|
||||
|
||||
## LLM Inference & Governance
|
||||
|
||||
Bifrost proxies LLM inference at `/v1` (OpenAI-compatible). `enforce_auth_on_inference=1` — all `/v1` calls require a valid `sk-bf-*` VK.
|
||||
|
||||
### Providers
|
||||
|
||||
| Provider | Internal name | Base URL | Notes |
|
||||
|---|---|---|---|
|
||||
| SAIA (GPU cluster) | `openai` | `https://chat-ai.academiccloud.de/v1` | Rate limited (see below) |
|
||||
| Google Gemini | `gemini` | default | |
|
||||
| Mistral AI | `mistral` | default | |
|
||||
| Cerebras | `cerebras` | default | |
|
||||
| claude-max-bridge | `openrouter` | `http://claude-max-bridge:8000` | Claude Opus/Sonnet/Haiku via max subscription |
|
||||
|
||||
### SAIA rate limits
|
||||
|
||||
SAIA enforces per-account quotas. Bifrost is configured with a global provider-level limit (`config_providers.rate_limit_id='saia-minute'`):
|
||||
|
||||
| Window | SAIA limit | Bifrost enforcement |
|
||||
|---|---|---|
|
||||
| Per minute | 30 req | ✅ active (`saia-minute` row) |
|
||||
| Per hour | 200 req | row exists (`saia-hour`), not linked |
|
||||
| Per day | 1000 req | row exists (`saia-day`), not linked |
|
||||
| Per month | 3000 req | not trackable across restarts |
|
||||
|
||||
**Note**: Bifrost only supports one rate limit window per provider. The minute window is linked because it provides burst protection. The `saia-hour` / `saia-day` rows are in `governance_rate_limits` and can be linked by updating `config_providers SET rate_limit_id='saia-hour'` if needed.
|
||||
|
||||
To change the active window:
|
||||
```bash
|
||||
sqlite3 /opt/stacks/ai/bifrost/data/config.db \
|
||||
"UPDATE config_providers SET rate_limit_id='saia-day' WHERE name='openai';"
|
||||
docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate
|
||||
```
|
||||
|
||||
**Rate limit API**: `GET /api/governance/rate-limits` is read-only. POST/PUT return 405. Use direct SQLite to create new entries.
|
||||
|
||||
### Virtual key governance
|
||||
|
||||
All VKs (`governance_virtual_key_provider_configs`) have:
|
||||
- `allow_all_keys=1` — set via SQL (Bifrost API PUT silently ignores this field)
|
||||
- `allowed_models` — explicit JSON model list as text in DB (SQL NULL = deny all with enforce_auth_on_inference=1)
|
||||
|
||||
If models stop working after a Bifrost upgrade/restore, re-run `/tmp/fix_vk_final.py` on CT 104 and then:
|
||||
```bash
|
||||
sqlite3 /opt/stacks/ai/bifrost/data/config.db \
|
||||
"UPDATE governance_virtual_key_provider_configs SET allow_all_keys=1;"
|
||||
docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate
|
||||
```
|
||||
|
||||
## Ops
|
||||
|
||||
```bash
|
||||
@@ -111,8 +160,12 @@ curl -s http://localhost:14003/mcp \
|
||||
### Add a new MCP client
|
||||
|
||||
```bash
|
||||
curl -sc /tmp/bfcookies http://localhost:14003/api/session/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"username":"fkrebs","password":"tapirnase"}' > /dev/null
|
||||
|
||||
curl -s -X POST http://localhost:14003/api/mcp/client \
|
||||
-H "Authorization: Bearer sk-tapirnase" \
|
||||
-b /tmp/bfcookies \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "<name>",
|
||||
|
||||
Reference in New Issue
Block a user