ComfyUI → LobeChat via MCP

Image generation (FLUX.1-schnell GGUF on the Intel Arc iGPU) exposed to LobeChat as an MCP tool. Chosen over LobeChat's native ComfyUI provider because that provider is hardcoded to non-GGUF nodes and is not configurable without forking LobeChat (see docs/mcp-gateway-requirements.md research notes).

Components

ai/mcp-servers/comfyui/ — purpose-built MCP server (async queue edition)
- server.py — FastMCP, streamable-HTTP on :8000 at /mcp. Six tools:
  - generate_image(prompt, width, height, steps, seed) — txt2img; returns job_id immediately
  - img2img(prompt, images, strength, steps, seed) — img2img for one or more images; one job per image; images can be URLs, base64 data URIs, or job:<id> references
  - get_job_status(job_ids) — returns status text + inline PNG for completed jobs
  - list_queue(limit) — lists all jobs in the in-process registry
  - cancel_job(job_id) — cancels queued/running job (user must confirm first)
  - list_recent_images(n) — shows n most recent completed images for reference/chaining
- Background polling thread updates job state every 3 s via ComfyUI /history.
- flux-gguf-api.json — txt2img workflow; nodes 4=prompt, 6=size, 8=steps/seed.
- flux-img2img-api.json — img2img workflow; node 11=LoadImage, 12=VAEEncode, 8=KSampler with denoise.
- Dockerfile, requirements.txt (mcp[cli], httpx).
ai/comfyui-mcp.yml — compose; container comfyui-mcp on shared_backend (reaches comfyui:8188; reachable by lobehub, which is on shared_backend). Host port 18003:8000 for testing/Zoraxy.

Async workflow

generate_image("a red apple")  →  "Job submitted: txt-1a8bbeda"   (< 1 s)
                                   [ComfyUI rendering... ~90-300 s]
get_job_status(["txt-1a8bbeda"])  →  status text + inline PNG image

# Chain: use previous output as img2img input (no bytes through LLM)
img2img("add a blue bowl", images=["job:txt-1a8bbeda"], strength=0.6)
  →  "img-2b9ccefa"

Verified

All 6 tools discovered via MCP tools/list.
generate_image returns in < 0.1 s (non-blocking); job shows in list_queue as "running".
Previous end-to-end: generate_image → get_job_status → inline PNG ~94 s bare, ~160 s via MCP.
LobeChat v2.1.58 renders MCP image blocks (uploads to Garage S3 → inline ![](url)).

Known characteristics / limitations

In-memory registry: job state is lost on container restart. Resubmit if needed.
No auth on the MCP server (internal shared_backend only). Host port 18003 is LAN-exposed and unauthenticated — fine for a trusted homelab LAN.
~90-300 s/image latency; use get_job_status() to poll — never block waiting.

Final hookup (manual — LobeChat UI/DB, no server-mode config path)

LobeChat → Settings → Skills (Tools) → Skill Store → Custom → Import JSON:

{
  "mcpServers": {
    "comfyui-flux": {
      "type": "http",
      "url": "http://comfyui-mcp:8000/mcp"
    }
  }
}

Then enable the comfyui-flux skill in an agent/chat and ask the model to "generate an image of …". Images appear inline in the conversation.

Ops

Build/deploy: cd /opt/stacks/ai && docker compose -f comfyui-mcp.yml up -d --build
Logs: docker logs comfyui-mcp
The workflow is the single source of truth in flux-gguf-api.json; keep it in sync with ai/comfyui/workflows/flux-schnell-api.json if the graph changes.

3.4 KiB Raw Blame History