# Cloud GPU Extension — Scaleway L40S

Seamless on-demand GPU (FLUX.1-dev, video, multi-user) via a Scaleway L40S instance.
The instance auto-starts on first request and shuts down after 45 min idle.
Everything personal stays on the NUC.

---

## Pricing (current as of May 2026)

| Resource | Rate | Notes |
|---|---|---|
| L40S-1-48G compute | **€1.40/hour** | PAR-2, billed per minute |
| Block volume 200 GB | **€16/month** | SSD, keeps models across restarts |
| Flexible IP | €0.004/hour | Static IP for WireGuard endpoint |
| Snapshots | €0.000044/GB/h | Only needed for image backups |

### Realistic monthly cost

| Usage pattern | Compute | Storage | Total |
|---|---|---|---|
| Weekend sessions (8 h/week) | €45 | €16 | **~€61/month** |
| Daily 1–2 h | €63–126 | €16 | **~€79–142/month** |
| Heavy (4 h/day) | €168 | €16 | **~€184/month** |
| Always-on (don't) | €1,008 | €16 | €1,024/month |

The on-demand proxy below makes "daily 1–2 h" the natural default — you just click generate, it starts automatically.

---

## Architecture

```
NUC (home, always-on)                        Scaleway PAR-2 (on demand)
─────────────────────────────                ───────────────────────────────
LobeChat  ──▶  gpu-proxy:8190 ─── wg1 ──▶   ComfyUI  :8188
comfyui-mcp ──▶   (Docker)                   Ollama   :11434  (optional)
                      │
                      ├─ start/stop via Scaleway API
                      └─ idle watchdog (45 min → stop)
```

`gpu-proxy` is a small Docker service that:
1. Forwards requests to the GPU instance
2. Auto-starts the Scaleway instance if it's stopped (cold-start ~90 s)
3. Shuts it down after 45 min with no traffic

---

## Prerequisites

```bash
# Scaleway CLI
curl -s https://raw.githubusercontent.com/scaleway/scaleway-cli/master/scripts/get.sh | sh
scw init   # enter API key + project ID
```

---

## Step 1 — Persistent block volume (models live here)

```bash
# Create 200 GB SSD volume in PAR-2
scw block volume create name=nuclide-gpu-models size=200GB zone=fr-par-2
# Note the volume ID: vol-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
```

Models are kept on this volume. The compute instance can be deleted and recreated freely.

---

## Step 2 — Create the L40S instance with cloud-init

Save as `/opt/stacks/scripts/gpu-cloud-init.yaml`:

```yaml
#cloud-config
package_update: true
packages:
  - wireguard-tools
  - docker.io
  - docker-compose-v2
  - nvidia-driver-545
  - nvidia-container-toolkit

write_files:
  - path: /etc/wireguard/wg0.conf
    permissions: '0600'
    content: |
      [Interface]
      Address = 10.200.0.1/24
      ListenPort = 51820
      PrivateKey = SCALEWAY_WG_PRIVKEY

      [Peer]
      # NUC
      PublicKey = NUC_WG_PUBKEY
      AllowedIPs = 10.200.0.2/32
      PersistentKeepalive = 25

  - path: /opt/gpu/docker-compose.yml
    content: |
      services:
        comfyui:
          image: yanwk/comfyui-boot:cu124
          container_name: comfyui
          ports:
            - "10.200.0.1:8188:8188"
          volumes:
            - /mnt/models:/root/ComfyUI/models
            - comfyui_config:/root/ComfyUI
          environment:
            - CLI_ARGS=--listen 0.0.0.0
          deploy:
            resources:
              reservations:
                devices:
                  - driver: nvidia
                    count: all
                    capabilities: [gpu]
          restart: unless-stopped

        ollama:
          image: ollama/ollama:latest
          container_name: ollama
          ports:
            - "10.200.0.1:11434:11434"
          volumes:
            - /mnt/models/ollama:/root/.ollama
          deploy:
            resources:
              reservations:
                devices:
                  - driver: nvidia
                    count: all
                    capabilities: [gpu]
          restart: unless-stopped

      volumes:
        comfyui_config:

runcmd:
  # Mount the models volume (will be /dev/sdb or similar)
  - mkdir -p /mnt/models
  - |
    DISK=$(lsblk -ndo NAME,SIZE | awk '$2=="200G"{print "/dev/"$1}' | head -1)
    if [ -n "$DISK" ]; then
      blkid "$DISK" || mkfs.ext4 "$DISK"
      echo "$DISK /mnt/models ext4 defaults 0 2" >> /etc/fstab
      mount "$DISK" /mnt/models
    fi
  - mkdir -p /mnt/models/ollama
  # Enable WireGuard
  - systemctl enable --now wg-quick@wg0
  # Configure nvidia-container-toolkit
  - nvidia-ctk runtime configure --runtime=docker
  - systemctl restart docker
  # Start GPU services
  - cd /opt/gpu && docker compose up -d
```

**Before using this file**, replace:
- `SCALEWAY_WG_PRIVKEY` → output of `wg genkey` (run on Scaleway instance side first)
- `NUC_WG_PUBKEY` → output of `cat /etc/wireguard/nuc_wg_pub` (see Step 3)

Launch the instance:

```bash
# Generate WireGuard keys first (do this on the NUC)
wg genkey | tee /etc/wireguard/scaleway_wg_priv | wg pubkey > /etc/wireguard/scaleway_wg_pub
wg genkey | tee /etc/wireguard/nuc_wg_priv | wg pubkey > /etc/wireguard/nuc_wg_pub

# Fill in cloud-init.yaml with the keys, then:
INSTANCE_ID=$(scw instance server create \
  type=L40S-1-48G \
  image=ubuntu_jammy_gpu \
  zone=fr-par-2 \
  name=nuclide-gpu \
  cloud-init=@/opt/stacks/scripts/gpu-cloud-init.yaml \
  output=json | jq -r '.id')

echo "Instance ID: $INSTANCE_ID"

# Attach the models volume
scw instance server attach-volume \
  server-id=$INSTANCE_ID \
  volume-id=vol-xxxxxxxx \
  zone=fr-par-2

# Allocate a Flexible IP (static IP that survives instance restarts)
FLEXIP_ID=$(scw instance ip create zone=fr-par-2 output=json | jq -r '.id')
scw instance server attach-flexible-ip \
  server-id=$INSTANCE_ID \
  ip-id=$FLEXIP_ID \
  zone=fr-par-2

FLEXIP=$(scw instance ip get $FLEXIP_ID zone=fr-par-2 output=json | jq -r '.address')
echo "Scaleway public IP: $FLEXIP"
```

---

## Step 3 — WireGuard on the NUC

```bash
# /etc/wireguard/wg1.conf  (separate from any existing VPN tunnel)
cat > /etc/wireguard/wg1.conf << EOF
[Interface]
Address = 10.200.0.2/24
PrivateKey = $(cat /etc/wireguard/nuc_wg_priv)

[Peer]
# Scaleway GPU
PublicKey = $(cat /etc/wireguard/scaleway_wg_pub)
Endpoint = ${FLEXIP}:51820
AllowedIPs = 10.200.0.1/32
PersistentKeepalive = 25
EOF

chmod 600 /etc/wireguard/wg1.conf
systemctl enable --now wg-quick@wg1
```

Test when the instance is running:
```bash
ping 10.200.0.1          # WireGuard tunnel
curl http://10.200.0.1:8188   # ComfyUI
```

---

## Step 4 — gpu-proxy (seamless on-demand startup)

This Docker service runs on the NUC. It proxies to the GPU instance and auto-starts/stops it.

### `ai/gpu-proxy/docker-compose.yml`

```yaml
services:
  gpu-proxy:
    build: .
    container_name: gpu-proxy
    environment:
      - SCW_SECRET_KEY=${SCW_SECRET_KEY}
      - SCW_PROJECT_ID=${SCW_PROJECT_ID}
      - SCW_INSTANCE_ID=${SCW_INSTANCE_ID}
      - SCW_ZONE=fr-par-2
      - GPU_HOST=10.200.0.1
      - COMFYUI_PORT=8188
      - OLLAMA_PORT=11434
      - IDLE_TIMEOUT=2700   # 45 min
    ports:
      - "8190:8190"   # ComfyUI proxy
      - "8191:8191"   # Ollama proxy
    restart: unless-stopped
    networks:
      - shared_backend

networks:
  shared_backend:
    external: true
```

### `ai/gpu-proxy/proxy.py`

```python
"""
On-demand GPU proxy. Auto-starts the Scaleway instance on first request,
shuts it down after IDLE_TIMEOUT seconds of inactivity.
"""
import asyncio, os, time, httpx, subprocess
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse, JSONResponse

app = FastAPI()

SCW_KEY      = os.environ["SCW_SECRET_KEY"]
SCW_PROJECT  = os.environ["SCW_PROJECT_ID"]
INSTANCE_ID  = os.environ["SCW_INSTANCE_ID"]
ZONE         = os.environ.get("SCW_ZONE", "fr-par-2")
GPU_HOST     = os.environ.get("GPU_HOST", "10.200.0.1")
COMFYUI_PORT = int(os.environ.get("COMFYUI_PORT", 8188))
OLLAMA_PORT  = int(os.environ.get("OLLAMA_PORT", 11434))
IDLE_TIMEOUT = int(os.environ.get("IDLE_TIMEOUT", 2700))

SCW_API = f"https://api.scaleway.com/instance/v1/zones/{ZONE}"
HEADERS = {"X-Auth-Token": SCW_KEY, "Content-Type": "application/json"}

_state = {"last_activity": 0.0, "starting": False, "up": False}
_lock  = asyncio.Lock()


async def _scw(method: str, path: str, **kwargs):
    async with httpx.AsyncClient() as c:
        r = await c.request(method, f"{SCW_API}{path}", headers=HEADERS, **kwargs)
        r.raise_for_status()
        return r.json()


async def _instance_state() -> str:
    d = await _scw("GET", f"/servers/{INSTANCE_ID}")
    return d["server"]["state"]   # running | stopped | stopping | starting


async def _start_instance():
    await _scw("POST", f"/servers/{INSTANCE_ID}/action", json={"action": "poweron"})


async def _stop_instance():
    await _scw("POST", f"/servers/{INSTANCE_ID}/action", json={"action": "poweroff"})


async def _wait_ready(host: str, port: int, timeout=180) -> bool:
    deadline = time.time() + timeout
    while time.time() < deadline:
        try:
            async with httpx.AsyncClient(timeout=3) as c:
                await c.get(f"http://{host}:{port}/")
                return True
        except Exception:
            await asyncio.sleep(5)
    return False


async def ensure_up(port: int) -> bool:
    async with _lock:
        if _state["up"]:
            _state["last_activity"] = time.time()
            return True
        if _state["starting"]:
            return False   # caller will retry
        _state["starting"] = True

    try:
        state = await _instance_state()
        if state != "running":
            print(f"[gpu-proxy] instance {state} → starting")
            await _start_instance()
            # wait for API to report running
            for _ in range(60):
                await asyncio.sleep(5)
                if await _instance_state() == "running":
                    break
        # wait for service to respond
        if await _wait_ready(GPU_HOST, port):
            async with _lock:
                _state["up"] = True
                _state["last_activity"] = time.time()
            print("[gpu-proxy] GPU instance ready")
            return True
        return False
    finally:
        async with _lock:
            _state["starting"] = False


async def idle_watchdog():
    while True:
        await asyncio.sleep(60)
        async with _lock:
            if not _state["up"]:
                continue
            idle = time.time() - _state["last_activity"]
        if idle > IDLE_TIMEOUT:
            print(f"[gpu-proxy] idle {idle:.0f}s → stopping instance")
            try:
                await _stop_instance()
                async with _lock:
                    _state["up"] = False
                    _state["last_activity"] = 0.0
            except Exception as e:
                print(f"[gpu-proxy] stop error: {e}")


@app.on_event("startup")
async def startup():
    asyncio.create_task(idle_watchdog())
    # Probe: is the instance already running from a previous session?
    try:
        if await _instance_state() == "running":
            if await _wait_ready(GPU_HOST, COMFYUI_PORT, timeout=10):
                async with _lock:
                    _state["up"] = True
                    _state["last_activity"] = time.time()
                print("[gpu-proxy] GPU already up on startup")
    except Exception:
        pass


async def _proxy(request: Request, host: str, port: int):
    _state["last_activity"] = time.time()
    ready = await ensure_up(port)
    if not ready:
        # Starting up — keep trying for up to 3 min
        for _ in range(36):
            await asyncio.sleep(5)
            if _state["up"]:
                break
        else:
            return JSONResponse({"error": "GPU instance failed to start"}, 503)

    url = f"http://{host}:{port}{request.url.path}"
    if request.url.query:
        url += f"?{request.url.query}"
    body = await request.body()
    client = httpx.AsyncClient(timeout=httpx.Timeout(None, connect=10))
    req = client.build_request(request.method, url,
                               headers={k: v for k, v in request.headers.items()
                                        if k.lower() not in {"host", "content-length"}},
                               content=body)
    try:
        resp = await client.send(req, stream=True)
    except httpx.ConnectError:
        await client.aclose()
        return JSONResponse({"error": "GPU not reachable"}, 502)

    async def stream():
        async for chunk in resp.aiter_raw():
            yield chunk
        await resp.aclose()
        await client.aclose()

    return StreamingResponse(stream(), status_code=resp.status_code,
                             headers=dict(resp.headers),
                             media_type=resp.headers.get("content-type"))


@app.api_route("/status", methods=["GET"])
async def status():
    try:
        scw_state = await _instance_state()
    except Exception as e:
        scw_state = f"error: {e}"
    return {"instance": scw_state, "proxy_up": _state["up"],
            "idle_s": int(time.time() - _state["last_activity"]) if _state["up"] else None}


@app.api_route("/{path:path}", methods=["GET","POST","PUT","DELETE","OPTIONS","PATCH"])
async def comfyui_proxy(request: Request, path: str):
    return await _proxy(request, GPU_HOST, COMFYUI_PORT)
```

### `ai/gpu-proxy/Dockerfile`

```dockerfile
FROM python:3.12-slim
RUN pip install fastapi uvicorn httpx
COPY proxy.py /app/proxy.py
WORKDIR /app
EXPOSE 8190 8191
CMD ["uvicorn", "proxy:app", "--host", "0.0.0.0", "--port", "8190"]
```

---

## Step 5 — Wire up NUC services

Add to `ai/.env`:
```bash
SCW_SECRET_KEY=<your-scaleway-api-key>
SCW_PROJECT_ID=<your-project-id>
SCW_INSTANCE_ID=<instance-id-from-step-2>
```

In `ai/lobehub.yml`:
```yaml
- 'COMFYUI_BASE_URL=http://gpu-proxy:8190'
```

In `ai/mcp-gateway/server.py` (SERVERS dict):
```python
"comfyui": {
    "static": True,
    "upstream": "http://gpu-proxy:8190",
    "group": "image",
},
```

For Immich ML (optional), add to `immich/docker-compose.yml`:
```yaml
environment:
  - IMMICH_MACHINE_LEARNING_URL=http://gpu-proxy:8192   # add a third port for ML
```

---

## Step 6 — Deploy

```bash
# Deploy the proxy
cd /opt/stacks/ai/gpu-proxy
docker compose up -d

# Restart LobeChat and MCP gateway to pick up new env
cd /opt/stacks/ai
docker compose -f lobehub.yml up -d --force-recreate lobe
cd mcp-gateway && docker compose up -d --force-recreate mcp-gateway
```

### What the user experience looks like

1. Open LobeChat, generate an image → first request triggers auto-start
2. "Starting GPU…" — ComfyUI returns a brief wait (up to 90 s cold start)
3. Images generate normally; the proxy keeps the instance alive
4. After 45 min with no requests → instance stops automatically
5. `/status` endpoint on port 8190: `{"instance":"stopped","proxy_up":false}`

---

## Quick-start / stop scripts

```bash
# /usr/local/bin/nuclide-gpu (NUC helper)
#!/bin/bash
ZONE=fr-par-2
ID=$(docker exec gpu-proxy env | grep SCW_INSTANCE_ID | cut -d= -f2)
case "$1" in
  start)  scw instance server action action=poweron  server-id=$ID zone=$ZONE ;;
  stop)   scw instance server action action=poweroff server-id=$ID zone=$ZONE ;;
  status) curl -s http://localhost:8190/status | python3 -m json.tool ;;
  *)      echo "Usage: nuclide-gpu start|stop|status" ;;
esac
```

---

## Scaleway firewall (security groups)

On the Scaleway instance, only expose WireGuard. All services bind to `10.200.0.1` (WireGuard IP):

```bash
ufw default deny incoming
ufw allow 51820/udp        # WireGuard from anywhere
ufw allow from 10.200.0.0/24   # full access over tunnel
ufw enable
```

---

## Model storage layout (on the 200 GB volume)

```
/mnt/models/
├── checkpoints/        # FLUX.1-dev, SDXL, etc.
├── vae/
├── loras/
├── controlnet/
├── ollama/             # Ollama model blobs
└── upscale_models/
```

FLUX.1-dev (FP16): ~24 GB  
FLUX.1-schnell (NF4): ~8 GB  
Ollama llama3:70b (Q4): ~40 GB — fits alongside FLUX on 200 GB with room for LoRAs.

To pre-download models after first boot:
```bash
ssh root@10.200.0.1  # via WireGuard
docker exec comfyui python3 -c "
from huggingface_hub import hf_hub_download
hf_hub_download('black-forest-labs/FLUX.1-dev', 'flux1-dev.safetensors',
                local_dir='/root/ComfyUI/models/checkpoints')
"
```