docs/ideas/litellm-claude-max-bridge.md

# Design: Claude Max Subscription → LiteLLM Gateway Bridge

**Status**: Proposal — not implemented  
**Context**: LiteLLM at `https://ai.nuclide.systems` currently has no Anthropic models. Claude Max subscription (claude.ai) provides high-rate access to Sonnet 4.5/4.6 and Opus 4.7 but is decoupled from Anthropic API billing. This doc explores bridging the two.

## Problem

Claude Max and the Anthropic API are separate products with separate billing:

| | Claude Max | Anthropic API |
|---|---|---|
| Auth | OAuth / browser session | API key |
| Billing | Flat monthly subscription | Per-token |
| Rate limits | 5h rolling windows, model-specific | Tier-based RPM/TPM |
| Access | claude.ai web + Claude Code CLI | Any HTTP client |

The goal is to surface Max-subscription capacity through LiteLLM so that LobeHub, n8n, Coder workspaces, and other internal tools can call `claude-sonnet-4-6` at zero marginal cost and fall back to paid providers only when Max limits are hit.

## Approaches

### A — Session Cookie Reverse-Engineering (not recommended)

Several community projects (e.g. `claude-unofficial-api`) scrape the claude.ai WebSocket/HTTP protocol and expose an OpenAI-compatible endpoint. LiteLLM would point at this as a custom `openai/` provider.

**Pros**: Exposes the full web model lineup; streaming works.  
**Cons**: Violates Anthropic ToS; breaks on any claude.ai front-end change; auth flow requires persisting browser cookies; no multimodal or tool-use parity guarantees.

**Verdict**: Avoid. Fragile and non-compliant.

---

### B — Claude Code CLI Bridge (recommended)

Claude Code CLI (`claude`) is already installed on CT 104 and authenticated with the Max subscription via `~/.claude/`. It ships a `--print` / `--output-format stream-json` mode designed for non-interactive use, and Anthropic explicitly supports programmatic use of the CLI.

A small **`claude-max-bridge`** service wraps this CLI as an OpenAI-compatible HTTP endpoint. LiteLLM registers it as a custom `openai/` base URL. No ToS issues — this is the supported surface.

```
LobeHub / n8n / Coder / Claude Code
          │
          ▼
    LiteLLM Gateway  (ai.nuclide.systems)
          │  model: claude-sonnet-4-6  →  openai/claude-sonnet-4-6
          │  api_base: http://claude-max-bridge:8000
          ▼
    claude-max-bridge  (new container, CT 104 ai-internal)
          │  subprocess:  claude --model ... --print --output-format stream-json
          ▼
    ~/.claude/  (Max subscription session)
          │
          ▼
    Anthropic (claude.ai)
```

**Pros**:
- Uses the officially supported programmatic interface
- Auth is already set up; no cookie management
- `claude` CLI handles retries, token limits, context window management
- Subprocess overhead is ~200–400 ms cold; warm invocations faster

**Cons**:
- One subprocess per request — cannot multiplex a single session (unlike streaming HTTP)
- CLI is tied to the single authenticated user; no multi-user isolation
- Max rate limits apply per-account, same pool as interactive use
- Claude Code SDK (TypeScript) is cleaner but adds Node dependency

---

### C — Official API + Budget Cap (stop-gap)

Add Anthropic API key to LiteLLM with a hard budget cap (e.g. $20/month). Use it for Claude-specific features (tool use, long context) and let the existing free SAIA/Gemini fallback chain absorb general-purpose traffic.

**Pros**: Zero implementation work; full API feature parity.  
**Cons**: Still costs money; no benefit from Max subscription.

**Verdict**: Valid fallback if B proves too complex, or as a complement for tool-heavy workloads that need the official API surface.

---

## Recommended Architecture (Option B)

### claude-max-bridge service

```
/opt/stacks/ai/claude-max-bridge/
  Dockerfile
  server.py          # FastAPI, ~150 lines
  docker-compose.yml (or entry in ai/docker-compose.yml)
```

**Dockerfile** — reuse the existing `claude` CLI install:
```dockerfile
FROM python:3.12-slim
RUN pip install fastapi uvicorn
# Mount ~/.claude from host; claude binary from host PATH or copied in
COPY server.py /app/server.py
CMD ["uvicorn", "app.server:app", "--host", "0.0.0.0", "--port", "8000"]
```

**server.py** — OpenAI `/v1/chat/completions` shim:
```python
# POST /v1/chat/completions
# Translates messages[] → claude --print --model ... --output-format stream-json
# Streams NDJSON lines back as SSE (text/event-stream)
# Maps finish_reason, usage tokens from CLI output headers
```

Key translation notes:
- `messages` → write to a temp file, pass via `--input-file` (avoids shell quoting issues)
- `stream: true` → parse `stream-json` lines, emit `data: {...}` SSE chunks
- `stream: false` → buffer all chunks, return single response object
- `max_tokens`, `temperature`, `system` → map to `--max-tokens`, `--temperature`, `--system`
- Tool use: **not supported** in first iteration; return 501 for requests with `tools`
- Model name passthrough: `claude-sonnet-4-6` → `--model claude-sonnet-4-6`

### LiteLLM config additions

```yaml
model_list:
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"          # bridge ignores it; LiteLLM requires a value
      stream_timeout: 120
      timeout: 120

  - model_name: claude-opus-4-7
    litellm_params:
      model: openai/claude-opus-4-7
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"
      stream_timeout: 180
      timeout: 180

  - model_name: claude-haiku-4-5
    litellm_params:
      model: openai/claude-haiku-4-5-20251001
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"
      stream_timeout: 60
      timeout: 60
```

Add to `router_settings.fallbacks` — Max hits rate limit → fall to SAIA:
```yaml
fallbacks:
  - claude-sonnet-4-6: [qwen3.5-397b-a17b]
  - claude-opus-4-7:   [qwen3.5-397b-a17b, mistral-large-latest]
  - claude-haiku-4-5:  [qwen3.5-122b-a10b, cerebras-llama-3.1-8b]
```

### Rate limit handling

The bridge should detect the CLI's rate-limit exit code / stderr message and return HTTP 429. LiteLLM's router will then trigger the fallback chain. No custom logic needed in LiteLLM itself.

Max limits as of 2026 (approximate, vary by plan tier):
- Sonnet 4.6: ~50 messages / 5-hour window at Pro; higher at Max
- Opus 4.7: ~10–20 messages / 5-hour window
- Haiku 4.5: effectively unlimited under Max

## Volume estimate

Typical homelab workloads (LobeHub chat, n8n automations, Coder coding assist) generate maybe 200–500 completions/day. Max's 5-hour windows reset 4–5× daily, so the practical limit is not usually hit unless Opus is used heavily.

## Risks

| Risk | Mitigation |
|---|---|
| CLI API changes between Claude Code releases | Pin the `claude` binary version; watch for breaking changes in release notes |
| Max rate limit shared with interactive use | Monitor via `claude usage`; bridge adds `X-Max-Usage` header from CLI output |
| Auth session expiry | Bridge returns 401 on auth failure; detect and alert via Gotify |
| Subprocess latency (cold start) | Keep a warm subprocess pool (1–2 persistent processes) using `--interactive` + message passing, or accept the 200–400 ms overhead |
| No tool use support | Fallback chain routes tool-use requests to API providers (Gemini, Mistral) |

## Implementation plan

1. Write `server.py` shim (~150 lines) + Dockerfile
2. Build image on CT 104, add to `ai/docker-compose.yml`
3. Bind-mount `~/.claude` read-only into the container
4. Add model entries to `litellm-config/config.yaml`
5. Test streaming via `curl` against the bridge directly
6. Test end-to-end via LiteLLM → LobeHub
7. Watch `docker logs claude-max-bridge` for rate-limit / auth errors for 48h

## Open questions

- Does `claude --print` support all message roles (`system`, `user`, `assistant` turns)? Needs verification with multi-turn conversation format.
- Should the bridge be on CT 104 (access to `~/.claude`) or CT 109 ops (remote docker socket to CT 104)? CT 104 is simpler for v1.
- Is there an official Claude Code SDK for Python? (TypeScript SDK exists at `@anthropic-ai/claude-code`; Python wrapper may be cleaner than shelling out.)