Skip to content

# Cloud GPU Extension — Scaleway L40S

Seamless on-demand GPU (FLUX.1-dev, video, multi-user) via a Scaleway L40S instance. The instance auto-starts on first request and shuts down after 45 min idle. Everything personal stays on the NUC.

## Pricing (current as of May 2026)

Resource | Rate | Notes
--- | --- | ---
L40S-1-48G compute | €1.40/hour | PAR-2, billed per minute
Block volume 200 GB | €16/month | SSD, keeps models across restarts
Flexible IP | €0.004/hour | Static IP for WireGuard endpoint
Snapshots | €0.000044/GB/h | Only needed for image backups

### Realistic monthly cost

Usage pattern | Compute | Storage | Total
--- | --- | --- | ---
Weekend sessions (8 h/week) | €45 | €16 | ~€61/month
Daily 1–2 h | €63–126 | €16 | ~€79–142/month
Heavy (4 h/day) | €168 | €16 | ~€184/month
Always-on (don't) | €1,008 | €16 | €1,024/month

The on-demand proxy below makes "daily 1–2 h" the natural default — you just click generate, it starts automatically.

## Architecture

    NUC (home, always-on)                        Scaleway PAR-2 (on demand)
    ─────────────────────────────                ───────────────────────────────
    LobeChat  ──▶  gpu-proxy:8190 ─── wg1 ──▶   ComfyUI  :8188
    comfyui-mcp ──▶   (Docker)                   Ollama   :11434  (optional)
                          │
                          ├─ start/stop via Scaleway API
                          └─ idle watchdog (45 min → stop)

`gpu-proxy` is a small Docker service that: 1. Forwards requests to the GPU instance 2. Auto-starts the Scaleway instance if it's stopped (cold-start ~90 s) 3. Shuts it down after 45 min with no traffic

## Prerequisites

    # Scaleway CLI
    curl -s https://raw.githubusercontent.com/scaleway/scaleway-cli/master/scripts/get.sh | sh
    scw init   # enter API key + project ID

## Step 1 — Persistent block volume (models live here)

    # Create 200 GB SSD volume in PAR-2
    scw block volume create name=nuclide-gpu-models size=200GB zone=fr-par-2
    # Note the volume ID: vol-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Models are kept on this volume. The compute instance can be deleted and recreated freely.

## Step 2 — Create the L40S instance with cloud-init

Save as `/opt/stacks/scripts/gpu-cloud-init.yaml`:

    #cloud-config
    package_update: true
    packages:
      - wireguard-tools
      - docker.io
      - docker-compose-v2
      - nvidia-driver-545
      - nvidia-container-toolkit

    write_files:
      - path: /etc/wireguard/wg0.conf
        permissions: '0600'
        content: |
          [Interface]
          Address = 10.200.0.1/24
          ListenPort = 51820
          PrivateKey = SCALEWAY_WG_PRIVKEY

          [Peer]
          # NUC
          PublicKey = NUC_WG_PUBKEY
          AllowedIPs = 10.200.0.2/32
          PersistentKeepalive = 25

      - path: /opt/gpu/docker-compose.yml
        content: |
          services:
            comfyui:
              image: yanwk/comfyui-boot:cu124
              container_name: comfyui
              ports:
                - "10.200.0.1:8188:8188"
              volumes:
                - /mnt/models:/root/ComfyUI/models
                - comfyui_config:/root/ComfyUI
              environment:
                - CLI_ARGS=--listen 0.0.0.0
              deploy:
                resources:
                  reservations:
                    devices:
                      - driver: nvidia
                        count: all
                        capabilities: [gpu]
              restart: unless-stopped

            ollama:
              image: ollama/ollama:latest
              container_name: ollama
              ports:
                - "10.200.0.1:11434:11434"
              volumes:
                - /mnt/models/ollama:/root/.ollama
              deploy:
                resources:
                  reservations:
                    devices:
                      - driver: nvidia
                        count: all
                        capabilities: [gpu]
              restart: unless-stopped

          volumes:
            comfyui_config:

    runcmd:
      # Mount the models volume (will be /dev/sdb or similar)
      - mkdir -p /mnt/models
      - |
        DISK=$(lsblk -ndo NAME,SIZE | awk '$2=="200G"{print "/dev/"$1}' | head -1)
        if [ -n "$DISK" ]; then
          blkid "$DISK" || mkfs.ext4 "$DISK"
          echo "$DISK /mnt/models ext4 defaults 0 2" >> /etc/fstab
          mount "$DISK" /mnt/models
        fi
      - mkdir -p /mnt/models/ollama
      # Enable WireGuard
      - systemctl enable --now wg-quick@wg0
      # Configure nvidia-container-toolkit
      - nvidia-ctk runtime configure --runtime=docker
      - systemctl restart docker
      # Start GPU services
      - cd /opt/gpu && docker compose up -d

Before using this file, replace: - `SCALEWAY_WG_PRIVKEY` → output of `wg genkey` (run on Scaleway instance side first) - `NUC_WG_PUBKEY` → output of `cat /etc/wireguard/nuc_wg_pub` (see Step 3)

Launch the instance:

    # Generate WireGuard keys first (do this on the NUC)
    wg genkey | tee /etc/wireguard/scaleway_wg_priv | wg pubkey > /etc/wireguard/scaleway_wg_pub
    wg genkey | tee /etc/wireguard/nuc_wg_priv | wg pubkey > /etc/wireguard/nuc_wg_pub

    # Fill in cloud-init.yaml with the keys, then:
    INSTANCE_ID=$(scw instance server create \
      type=L40S-1-48G \
      image=ubuntu_jammy_gpu \
      zone=fr-par-2 \
      name=nuclide-gpu \
      cloud-init=@/opt/stacks/scripts/gpu-cloud-init.yaml \
      output=json | jq -r '.id')

    echo "Instance ID: $INSTANCE_ID"

    # Attach the models volume
    scw instance server attach-volume \
      server-id=$INSTANCE_ID \
      volume-id=vol-xxxxxxxx \
      zone=fr-par-2

    # Allocate a Flexible IP (static IP that survives instance restarts)
    FLEXIP_ID=$(scw instance ip create zone=fr-par-2 output=json | jq -r '.id')
    scw instance server attach-flexible-ip \
      server-id=$INSTANCE_ID \
      ip-id=$FLEXIP_ID \
      zone=fr-par-2

    FLEXIP=$(scw instance ip get $FLEXIP_ID zone=fr-par-2 output=json | jq -r '.address')
    echo "Scaleway public IP: $FLEXIP"

## Step 3 — WireGuard on the NUC

    # /etc/wireguard/wg1.conf  (separate from any existing VPN tunnel)
    cat > /etc/wireguard/wg1.conf << EOF
    [Interface]
    Address = 10.200.0.2/24
    PrivateKey = $(cat /etc/wireguard/nuc_wg_priv)

    [Peer]
    # Scaleway GPU
    PublicKey = $(cat /etc/wireguard/scaleway_wg_pub)
    Endpoint = ${FLEXIP}:51820
    AllowedIPs = 10.200.0.1/32
    PersistentKeepalive = 25
    EOF

    chmod 600 /etc/wireguard/wg1.conf
    systemctl enable --now wg-quick@wg1

Test when the instance is running:

    ping 10.200.0.1          # WireGuard tunnel
    curl http://10.200.0.1:8188   # ComfyUI

## Step 4 — gpu-proxy (seamless on-demand startup)

This Docker service runs on the NUC. It proxies to the GPU instance and auto-starts/stops it.

### `ai/gpu-proxy/docker-compose.yml`

    services:
      gpu-proxy:
        build: .
        container_name: gpu-proxy
        environment:
          - SCW_SECRET_KEY=${SCW_SECRET_KEY}
          - SCW_PROJECT_ID=${SCW_PROJECT_ID}
          - SCW_INSTANCE_ID=${SCW_INSTANCE_ID}
          - SCW_ZONE=fr-par-2
          - GPU_HOST=10.200.0.1
          - COMFYUI_PORT=8188
          - OLLAMA_PORT=11434
          - IDLE_TIMEOUT=2700   # 45 min
        ports:
          - "8190:8190"   # ComfyUI proxy
          - "8191:8191"   # Ollama proxy
        restart: unless-stopped
        networks:
          - shared_backend

    networks:
      shared_backend:
        external: true

### `ai/gpu-proxy/proxy.py`

    """
    On-demand GPU proxy. Auto-starts the Scaleway instance on first request,
    shuts it down after IDLE_TIMEOUT seconds of inactivity.
    """
    import asyncio, os, time, httpx, subprocess
    from fastapi import FastAPI, Request
    from fastapi.responses import StreamingResponse, JSONResponse

    app = FastAPI()

    SCW_KEY      = os.environ["SCW_SECRET_KEY"]
    SCW_PROJECT  = os.environ["SCW_PROJECT_ID"]
    INSTANCE_ID  = os.environ["SCW_INSTANCE_ID"]
    ZONE         = os.environ.get("SCW_ZONE", "fr-par-2")
    GPU_HOST     = os.environ.get("GPU_HOST", "10.200.0.1")
    COMFYUI_PORT = int(os.environ.get("COMFYUI_PORT", 8188))
    OLLAMA_PORT  = int(os.environ.get("OLLAMA_PORT", 11434))
    IDLE_TIMEOUT = int(os.environ.get("IDLE_TIMEOUT", 2700))

    SCW_API = f"https://api.scaleway.com/instance/v1/zones/{ZONE}"
    HEADERS = {"X-Auth-Token": SCW_KEY, "Content-Type": "application/json"}

    _state = {"last_activity": 0.0, "starting": False, "up": False}
    _lock  = asyncio.Lock()

    async def _scw(method: str, path: str, **kwargs):
        async with httpx.AsyncClient() as c:
            r = await c.request(method, f"{SCW_API}{path}", headers=HEADERS, **kwargs)
            r.raise_for_status()
            return r.json()

    async def _instance_state() -> str:
        d = await _scw("GET", f"/servers/{INSTANCE_ID}")
        return d["server"]["state"]   # running | stopped | stopping | starting

    async def _start_instance():
        await _scw("POST", f"/servers/{INSTANCE_ID}/action", json={"action": "poweron"})

    async def _stop_instance():
        await _scw("POST", f"/servers/{INSTANCE_ID}/action", json={"action": "poweroff"})

    async def _wait_ready(host: str, port: int, timeout=180) -> bool:
        deadline = time.time() + timeout
        while time.time() < deadline:
            try:
                async with httpx.AsyncClient(timeout=3) as c:
                    await c.get(f"http://{host}:{port}/")
                    return True
            except Exception:
                await asyncio.sleep(5)
        return False

    async def ensure_up(port: int) -> bool:
        async with _lock:
            if _state["up"]:
                _state["last_activity"] = time.time()
                return True
            if _state["starting"]:
                return False   # caller will retry
            _state["starting"] = True

        try:
            state = await _instance_state()
            if state != "running":
                print(f"[gpu-proxy] instance {state} → starting")
                await _start_instance()
                # wait for API to report running
                for _ in range(60):
                    await asyncio.sleep(5)
                    if await _instance_state() == "running":
                        break
            # wait for service to respond
            if await _wait_ready(GPU_HOST, port):
                async with _lock:
                    _state["up"] = True
                    _state["last_activity"] = time.time()
                print("[gpu-proxy] GPU instance ready")
                return True
            return False
        finally:
            async with _lock:
                _state["starting"] = False

    async def idle_watchdog():
        while True:
            await asyncio.sleep(60)
            async with _lock:
                if not _state["up"]:
                    continue
                idle = time.time() - _state["last_activity"]
            if idle > IDLE_TIMEOUT:
                print(f"[gpu-proxy] idle {idle:.0f}s → stopping instance")
                try:
                    await _stop_instance()
                    async with _lock:
                        _state["up"] = False
                        _state["last_activity"] = 0.0
                except Exception as e:
                    print(f"[gpu-proxy] stop error: {e}")

    @app.on_event("startup")
    async def startup():
        asyncio.create_task(idle_watchdog())
        # Probe: is the instance already running from a previous session?
        try:
            if await _instance_state() == "running":
                if await _wait_ready(GPU_HOST, COMFYUI_PORT, timeout=10):
                    async with _lock:
                        _state["up"] = True
                        _state["last_activity"] = time.time()
                    print("[gpu-proxy] GPU already up on startup")
        except Exception:
            pass

    async def _proxy(request: Request, host: str, port: int):
        _state["last_activity"] = time.time()
        ready = await ensure_up(port)
        if not ready:
            # Starting up — keep trying for up to 3 min
            for _ in range(36):
                await asyncio.sleep(5)
                if _state["up"]:
                    break
            else:
                return JSONResponse({"error": "GPU instance failed to start"}, 503)

        url = f"http://{host}:{port}{request.url.path}"
        if request.url.query:
            url += f"?{request.url.query}"
        body = await request.body()
        client = httpx.AsyncClient(timeout=httpx.Timeout(None, connect=10))
        req = client.build_request(request.method, url,
                                   headers={k: v for k, v in request.headers.items()
                                            if k.lower() not in {"host", "content-length"}},
                                   content=body)
        try:
            resp = await client.send(req, stream=True)
        except httpx.ConnectError:
            await client.aclose()
            return JSONResponse({"error": "GPU not reachable"}, 502)

        async def stream():
            async for chunk in resp.aiter_raw():
                yield chunk
            await resp.aclose()
            await client.aclose()

        return StreamingResponse(stream(), status_code=resp.status_code,
                                 headers=dict(resp.headers),
                                 media_type=resp.headers.get("content-type"))

    @app.api_route("/status", methods=["GET"])
    async def status():
        try:
            scw_state = await _instance_state()
        except Exception as e:
            scw_state = f"error: {e}"
        return {"instance": scw_state, "proxy_up": _state["up"],
                "idle_s": int(time.time() - _state["last_activity"]) if _state["up"] else None}

    @app.api_route("/{path:path}", methods=["GET","POST","PUT","DELETE","OPTIONS","PATCH"])
    async def comfyui_proxy(request: Request, path: str):
        return await _proxy(request, GPU_HOST, COMFYUI_PORT)

### `ai/gpu-proxy/Dockerfile`

    FROM python:3.12-slim
    RUN pip install fastapi uvicorn httpx
    COPY proxy.py /app/proxy.py
    WORKDIR /app
    EXPOSE 8190 8191
    CMD ["uvicorn", "proxy:app", "--host", "0.0.0.0", "--port", "8190"]

## Step 5 — Wire up NUC services

Add to `ai/.env`:

    SCW_SECRET_KEY=<your-scaleway-api-key>
    SCW_PROJECT_ID=<your-project-id>
    SCW_INSTANCE_ID=<instance-id-from-step-2>

In `ai/lobehub.yml`:

    - 'COMFYUI_BASE_URL=http://gpu-proxy:8190'

In `ai/mcp-gateway/server.py` (SERVERS dict):

    "comfyui": {
        "static": True,
        "upstream": "http://gpu-proxy:8190",
        "group": "image",
    },

For Immich ML (optional), add to `immich/docker-compose.yml`:

    environment:
      - IMMICH_MACHINE_LEARNING_URL=http://gpu-proxy:8192   # add a third port for ML

## Step 6 — Deploy

    # Deploy the proxy
    cd /opt/stacks/ai/gpu-proxy
    docker compose up -d

    # Restart LobeChat and MCP gateway to pick up new env
    cd /opt/stacks/ai
    docker compose -f lobehub.yml up -d --force-recreate lobe
    cd mcp-gateway && docker compose up -d --force-recreate mcp-gateway

### What the user experience looks like

1. Open LobeChat, generate an image → first request triggers auto-start
2. "Starting GPU…" — ComfyUI returns a brief wait (up to 90 s cold start)
3. Images generate normally; the proxy keeps the instance alive
4. After 45 min with no requests → instance stops automatically
5. `/status` endpoint on port 8190: `{"instance":"stopped","proxy_up":false}`

## Quick-start / stop scripts

    # /usr/local/bin/nuclide-gpu (NUC helper)
    #!/bin/bash
    ZONE=fr-par-2
    ID=$(docker exec gpu-proxy env | grep SCW_INSTANCE_ID | cut -d= -f2)
    case "$1" in
      start)  scw instance server action action=poweron  server-id=$ID zone=$ZONE ;;
      stop)   scw instance server action action=poweroff server-id=$ID zone=$ZONE ;;
      status) curl -s http://localhost:8190/status | python3 -m json.tool ;;
      *)      echo "Usage: nuclide-gpu start|stop|status" ;;
    esac

## Scaleway firewall (security groups)

On the Scaleway instance, only expose WireGuard. All services bind to `10.200.0.1` (WireGuard IP):

    ufw default deny incoming
    ufw allow 51820/udp        # WireGuard from anywhere
    ufw allow from 10.200.0.0/24   # full access over tunnel
    ufw enable

## Model storage layout (on the 200 GB volume)

    /mnt/models/
    ├── checkpoints/        # FLUX.1-dev, SDXL, etc.
    ├── vae/
    ├── loras/
    ├── controlnet/
    ├── ollama/             # Ollama model blobs
    └── upscale_models/

FLUX.1-dev (FP16): ~24 GB
FLUX.1-schnell (NF4): ~8 GB
Ollama llama3:70b (Q4): ~40 GB — fits alongside FLUX on 200 GB with room for LoRAs.

To pre-download models after first boot:

    ssh root@10.200.0.1  # via WireGuard
    docker exec comfyui python3 -c "
    from huggingface_hub import hf_hub_download
    hf_hub_download('black-forest-labs/FLUX.1-dev', 'flux1-dev.safetensors',
                    local_dir='/root/ComfyUI/models/checkpoints')
    "