# ComfyUI → LobeChat via MCP

Image generation (FLUX.1-schnell GGUF on the Intel Arc iGPU) exposed to LobeChat
as an MCP tool. Chosen over LobeChat's native ComfyUI provider because that
provider is hardcoded to non-GGUF nodes and is not configurable without forking
LobeChat (see `docs/mcp-gateway-requirements.md` research notes).

## Components

- **`ai/mcp-servers/comfyui/`** — purpose-built MCP server (async queue edition)
  - `server.py` — FastMCP, streamable-HTTP on `:8000` at `/mcp`. Six tools:
    - `generate_image(prompt, width, height, steps, seed)` — txt2img; returns `job_id` immediately
    - `img2img(prompt, images, strength, steps, seed)` — img2img for one or more images; one job per image; images can be URLs, base64 data URIs, or `job:<id>` references
    - `get_job_status(job_ids)` — returns status text + inline PNG for completed jobs
    - `list_queue(limit)` — lists all jobs in the in-process registry
    - `cancel_job(job_id)` — cancels queued/running job (user must confirm first)
    - `list_recent_images(n)` — shows n most recent completed images for reference/chaining
  - Background polling thread updates job state every 3 s via ComfyUI `/history`.
  - `flux-gguf-api.json` — txt2img workflow; nodes 4=prompt, 6=size, 8=steps/seed.
  - `flux-img2img-api.json` — img2img workflow; node 11=LoadImage, 12=VAEEncode, 8=KSampler with denoise.
  - `Dockerfile`, `requirements.txt` (`mcp[cli]`, `httpx`).
- **`ai/comfyui-mcp.yml`** — compose; container `comfyui-mcp` on
  `shared_backend` (reaches `comfyui:8188`; reachable by `lobehub`, which is on
  `shared_backend`). Host port `18003:8000` for testing/Zoraxy.

## Async workflow

```
generate_image("a red apple")  →  "Job submitted: txt-1a8bbeda"   (< 1 s)
                                   [ComfyUI rendering... ~90-300 s]
get_job_status(["txt-1a8bbeda"])  →  status text + inline PNG image

# Chain: use previous output as img2img input (no bytes through LLM)
img2img("add a blue bowl", images=["job:txt-1a8bbeda"], strength=0.6)
  →  "img-2b9ccefa"
```

## Verified

- All 6 tools discovered via MCP `tools/list`.
- `generate_image` returns in < 0.1 s (non-blocking); job shows in `list_queue` as "running".
- Previous end-to-end: `generate_image` → `get_job_status` → inline PNG ~94 s bare, ~160 s via MCP.
- LobeChat v2.1.58 renders MCP `image` blocks (uploads to Garage S3 → inline `![](url)`).

## Known characteristics / limitations

- **In-memory registry**: job state is lost on container restart. Resubmit if needed.
- **No auth** on the MCP server (internal `shared_backend` only). Host port
  18003 is LAN-exposed and unauthenticated — fine for a trusted homelab LAN.
- ~90-300 s/image latency; use `get_job_status()` to poll — never block waiting.

## Final hookup (manual — LobeChat UI/DB, no server-mode config path)

LobeChat → **Settings → Skills (Tools) → Skill Store → Custom → Import JSON**:

```json
{
  "mcpServers": {
    "comfyui-flux": {
      "type": "http",
      "url": "http://comfyui-mcp:8000/mcp"
    }
  }
}
```

Then enable the `comfyui-flux` skill in an agent/chat and ask the model to
"generate an image of …". Images appear inline in the conversation.

## Ops

- Build/deploy: `cd /opt/stacks/ai && docker compose -f comfyui-mcp.yml up -d --build`
- Logs: `docker logs comfyui-mcp`
- The workflow is the single source of truth in `flux-gguf-api.json`; keep it in
  sync with `ai/comfyui/workflows/flux-schnell-api.json` if the graph changes.