docs: add SAIA rate limits, Bifrost governance, and admin auth notes to mcp-gateway.md
- SAIA rate limits: minute=30/1m active at provider level; hour/day rows exist but not linked - VK governance: allow_all_keys SQL workaround, allowed_models must be non-null JSON text - Fix stale sk-tapirnase reference in add-MCP-client example (now uses cookie auth) - Add LLM provider table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+54
-1
@@ -88,6 +88,55 @@ Settings → Integrations → Add MCP server:
|
|||||||
- URL: `https://ai.nuclide.systems/mcp`
|
- URL: `https://ai.nuclide.systems/mcp`
|
||||||
- Header: `Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52`
|
- Header: `Authorization: Bearer sk-bf-279d3ecc-9031-41ff-a582-ecf3dca61d52`
|
||||||
|
|
||||||
|
## LLM Inference & Governance
|
||||||
|
|
||||||
|
Bifrost proxies LLM inference at `/v1` (OpenAI-compatible). `enforce_auth_on_inference=1` — all `/v1` calls require a valid `sk-bf-*` VK.
|
||||||
|
|
||||||
|
### Providers
|
||||||
|
|
||||||
|
| Provider | Internal name | Base URL | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| SAIA (GPU cluster) | `openai` | `https://chat-ai.academiccloud.de/v1` | Rate limited (see below) |
|
||||||
|
| Google Gemini | `gemini` | default | |
|
||||||
|
| Mistral AI | `mistral` | default | |
|
||||||
|
| Cerebras | `cerebras` | default | |
|
||||||
|
| claude-max-bridge | `openrouter` | `http://claude-max-bridge:8000` | Claude Opus/Sonnet/Haiku via max subscription |
|
||||||
|
|
||||||
|
### SAIA rate limits
|
||||||
|
|
||||||
|
SAIA enforces per-account quotas. Bifrost is configured with a global provider-level limit (`config_providers.rate_limit_id='saia-minute'`):
|
||||||
|
|
||||||
|
| Window | SAIA limit | Bifrost enforcement |
|
||||||
|
|---|---|---|
|
||||||
|
| Per minute | 30 req | ✅ active (`saia-minute` row) |
|
||||||
|
| Per hour | 200 req | row exists (`saia-hour`), not linked |
|
||||||
|
| Per day | 1000 req | row exists (`saia-day`), not linked |
|
||||||
|
| Per month | 3000 req | not trackable across restarts |
|
||||||
|
|
||||||
|
**Note**: Bifrost only supports one rate limit window per provider. The minute window is linked because it provides burst protection. The `saia-hour` / `saia-day` rows are in `governance_rate_limits` and can be linked by updating `config_providers SET rate_limit_id='saia-hour'` if needed.
|
||||||
|
|
||||||
|
To change the active window:
|
||||||
|
```bash
|
||||||
|
sqlite3 /opt/stacks/ai/bifrost/data/config.db \
|
||||||
|
"UPDATE config_providers SET rate_limit_id='saia-day' WHERE name='openai';"
|
||||||
|
docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate
|
||||||
|
```
|
||||||
|
|
||||||
|
**Rate limit API**: `GET /api/governance/rate-limits` is read-only. POST/PUT return 405. Use direct SQLite to create new entries.
|
||||||
|
|
||||||
|
### Virtual key governance
|
||||||
|
|
||||||
|
All VKs (`governance_virtual_key_provider_configs`) have:
|
||||||
|
- `allow_all_keys=1` — set via SQL (Bifrost API PUT silently ignores this field)
|
||||||
|
- `allowed_models` — explicit JSON model list as text in DB (SQL NULL = deny all with enforce_auth_on_inference=1)
|
||||||
|
|
||||||
|
If models stop working after a Bifrost upgrade/restore, re-run `/tmp/fix_vk_final.py` on CT 104 and then:
|
||||||
|
```bash
|
||||||
|
sqlite3 /opt/stacks/ai/bifrost/data/config.db \
|
||||||
|
"UPDATE governance_virtual_key_provider_configs SET allow_all_keys=1;"
|
||||||
|
docker compose -f /opt/stacks/ai/bifrost.yml up -d --force-recreate
|
||||||
|
```
|
||||||
|
|
||||||
## Ops
|
## Ops
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -111,8 +160,12 @@ curl -s http://localhost:14003/mcp \
|
|||||||
### Add a new MCP client
|
### Add a new MCP client
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
curl -sc /tmp/bfcookies http://localhost:14003/api/session/login \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"username":"fkrebs","password":"tapirnase"}' > /dev/null
|
||||||
|
|
||||||
curl -s -X POST http://localhost:14003/api/mcp/client \
|
curl -s -X POST http://localhost:14003/api/mcp/client \
|
||||||
-H "Authorization: Bearer sk-tapirnase" \
|
-b /tmp/bfcookies \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d '{
|
-d '{
|
||||||
"name": "<name>",
|
"name": "<name>",
|
||||||
|
|||||||
Reference in New Issue
Block a user