docs/ideas/litellm-claude-max-bridge.md

Design: Claude Max Subscription → LiteLLM Gateway Bridge

Status: Proposal — not implemented
Context: LiteLLM at https://ai.nuclide.systems currently has no Anthropic models. Claude Max subscription (claude.ai) provides high-rate access to Sonnet 4.5/4.6 and Opus 4.7 but is decoupled from Anthropic API billing. This doc explores bridging the two.

Problem

Claude Max and the Anthropic API are separate products with separate billing:

	Claude Max	Anthropic API
Auth	OAuth / browser session	API key
Billing	Flat monthly subscription	Per-token
Rate limits	5h rolling windows, model-specific	Tier-based RPM/TPM
Access	claude.ai web + Claude Code CLI	Any HTTP client

The goal is to surface Max-subscription capacity through LiteLLM so that LobeHub, n8n, Coder workspaces, and other internal tools can call claude-sonnet-4-6 at zero marginal cost and fall back to paid providers only when Max limits are hit.

Approaches

A — Session Cookie Reverse-Engineering (not recommended)

Several community projects (e.g. claude-unofficial-api) scrape the claude.ai WebSocket/HTTP protocol and expose an OpenAI-compatible endpoint. LiteLLM would point at this as a custom openai/ provider.

Pros: Exposes the full web model lineup; streaming works.
Cons: Violates Anthropic ToS; breaks on any claude.ai front-end change; auth flow requires persisting browser cookies; no multimodal or tool-use parity guarantees.

Verdict: Avoid. Fragile and non-compliant.

---

B — Claude Code CLI Bridge (recommended)

Claude Code CLI (claude) is already installed on CT 104 and authenticated with the Max subscription via ~/.claude/. It ships a --print / --output-format stream-json mode designed for non-interactive use, and Anthropic explicitly supports programmatic use of the CLI.

A small claude-max-bridge service wraps this CLI as an OpenAI-compatible HTTP endpoint. LiteLLM registers it as a custom openai/ base URL. No ToS issues — this is the supported surface.

LobeHub / n8n / Coder / Claude Code
          │
          ▼
    LiteLLM Gateway  (ai.nuclide.systems)
          │  model: claude-sonnet-4-6  →  openai/claude-sonnet-4-6
          │  api_base: http://claude-max-bridge:8000
          ▼
    claude-max-bridge  (new container, CT 104 ai-internal)
          │  subprocess:  claude --model ... --print --output-format stream-json
          ▼
    ~/.claude/  (Max subscription session)
          │
          ▼
    Anthropic (claude.ai)

Pros:

• Uses the officially supported programmatic interface
• Auth is already set up; no cookie management
• claude CLI handles retries, token limits, context window management
• Subprocess overhead is ~200–400 ms cold; warm invocations faster

Cons:

• One subprocess per request — cannot multiplex a single session (unlike streaming HTTP)
• CLI is tied to the single authenticated user; no multi-user isolation
• Max rate limits apply per-account, same pool as interactive use
• Claude Code SDK (TypeScript) is cleaner but adds Node dependency

---

C — Official API + Budget Cap (stop-gap)

Add Anthropic API key to LiteLLM with a hard budget cap (e.g. $20/month). Use it for Claude-specific features (tool use, long context) and let the existing free SAIA/Gemini fallback chain absorb general-purpose traffic.

Pros: Zero implementation work; full API feature parity.
Cons: Still costs money; no benefit from Max subscription.

Verdict: Valid fallback if B proves too complex, or as a complement for tool-heavy workloads that need the official API surface.

---

Recommended Architecture (Option B)

claude-max-bridge service

/opt/stacks/ai/claude-max-bridge/
  Dockerfile
  server.py          # FastAPI, ~150 lines
  docker-compose.yml (or entry in ai/docker-compose.yml)

Dockerfile — reuse the existing claude CLI install:

FROM python:3.12-slim
RUN pip install fastapi uvicorn
# Mount ~/.claude from host; claude binary from host PATH or copied in
COPY server.py /app/server.py
CMD ["uvicorn", "app.server:app", "--host", "0.0.0.0", "--port", "8000"]

server.py — OpenAI /v1/chat/completions shim:

# POST /v1/chat/completions
# Translates messages[] → claude --print --model ... --output-format stream-json
# Streams NDJSON lines back as SSE (text/event-stream)
# Maps finish_reason, usage tokens from CLI output headers

Key translation notes:

• messages → write to a temp file, pass via --input-file (avoids shell quoting issues)
• stream: true → parse stream-json lines, emit data: {...} SSE chunks
• stream: false → buffer all chunks, return single response object
• max_tokens, temperature, system → map to --max-tokens, --temperature, --system
• Tool use: not supported in first iteration; return 501 for requests with tools
• Model name passthrough: claude-sonnet-4-6 → --model claude-sonnet-4-6

LiteLLM config additions

model_list:
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"          # bridge ignores it; LiteLLM requires a value
      stream_timeout: 120
      timeout: 120

  - model_name: claude-opus-4-7
    litellm_params:
      model: openai/claude-opus-4-7
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"
      stream_timeout: 180
      timeout: 180

  - model_name: claude-haiku-4-5
    litellm_params:
      model: openai/claude-haiku-4-5-20251001
      api_base: http://claude-max-bridge:8000
      api_key: "dummy"
      stream_timeout: 60
      timeout: 60

Add to router_settings.fallbacks — Max hits rate limit → fall to SAIA:

fallbacks:
  - claude-sonnet-4-6: [qwen3.5-397b-a17b]
  - claude-opus-4-7:   [qwen3.5-397b-a17b, mistral-large-latest]
  - claude-haiku-4-5:  [qwen3.5-122b-a10b, cerebras-llama-3.1-8b]

Rate limit handling

The bridge should detect the CLI's rate-limit exit code / stderr message and return HTTP 429. LiteLLM's router will then trigger the fallback chain. No custom logic needed in LiteLLM itself.

Max limits as of 2026 (approximate, vary by plan tier):

• Sonnet 4.6: ~50 messages / 5-hour window at Pro; higher at Max
• Opus 4.7: ~10–20 messages / 5-hour window
• Haiku 4.5: effectively unlimited under Max

Volume estimate

Typical homelab workloads (LobeHub chat, n8n automations, Coder coding assist) generate maybe 200–500 completions/day. Max's 5-hour windows reset 4–5× daily, so the practical limit is not usually hit unless Opus is used heavily.

Risks

Risk	Mitigation
CLI API changes between Claude Code releases	Pin the claude binary version; watch for breaking changes in release notes
Max rate limit shared with interactive use	Monitor via claude usage; bridge adds X-Max-Usage header from CLI output
Auth session expiry	Bridge returns 401 on auth failure; detect and alert via Gotify
Subprocess latency (cold start)	Keep a warm subprocess pool (1–2 persistent processes) using --interactive + message passing, or accept the 200–400 ms overhead
No tool use support	Fallback chain routes tool-use requests to API providers (Gemini, Mistral)

Implementation plan

1. Write server.py shim (~150 lines) + Dockerfile
2. Build image on CT 104, add to ai/docker-compose.yml
3. Bind-mount ~/.claude read-only into the container
4. Add model entries to litellm-config/config.yaml
5. Test streaming via curl against the bridge directly
6. Test end-to-end via LiteLLM → LobeHub
7. Watch docker logs claude-max-bridge for rate-limit / auth errors for 48h

Open questions

• Does claude --print support all message roles (system, user, assistant turns)? Needs verification with multi-turn conversation format.
• Should the bridge be on CT 104 (access to ~/.claude) or CT 109 ops (remote docker socket to CT 104)? CT 104 is simpler for v1.
• Is there an official Claude Code SDK for Python? (TypeScript SDK exists at @anthropic-ai/claude-code; Python wrapper may be cleaner than shelling out.)