# Portainer BE (migrated from Arcane 2026-05-26) Docker management UI with Portainer Business Edition license. ## Stack - **Server**: CT 109 ("ops", 192.168.1.8), `/opt/stacks/monitoring/docker-compose.yml`, port **9000** - **Image**: `portainer/portainer-ee:2.39.2` - **Data volume**: `monitoring_portainer_data` → `/var/lib/docker/volumes/monitoring_portainer_data/_data` - **URL**: `http://192.168.1.8:9000` (LAN-only; no Zoraxy route — access via LAN/SSH tunnel) - **Admin user**: `fkrebs` / `tapirnase` ## License - **Edition**: Business Edition (BE) — 3 nodes free for personal use - **License key**: `3-QIORiAXMuBSgeYdePYgh1nuqRkvx/XWyu5D/+MQlVpvSng2CXtCG4V78212HEleOIWnIV0kK5IkpEaifea3b8NGU6o2STA0c/XXj150c/v5XSguhchCmqiXyWXDj2/+r` - **Expires**: 2027-05-25 (365 days from issue date 2026-05-26) - **Nodes**: 3 licensed (CT 104, CT 109 local, + 1 more) To re-apply license (e.g. after fresh install): ```bash curl -X POST http://192.168.1.8:9000/api/auth \ -H "Content-Type: application/json" \ -d '{"username":"fkrebs","password":"tapirnase"}' | python3 -c 'import sys,json; print(json.load(sys.stdin)["jwt"])' # then: curl -X POST http://192.168.1.8:9000/api/licenses/add \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{"license":""}' ``` ## Agents (Portainer environments) | Host | IP | Port | Status | |---|---|---|---| | CT 109 local | unix:///var/run/docker.sock | — | built-in | | CT 104 | 192.168.1.40 | 9001 | `/opt/stacks/portainer-agent.yml` | | CT 105 | 192.168.1.5 | 9001 | `/opt/stacks/ops-agents/docker-compose.yml` | | CT 111 | 192.168.1.42 | 9001 | `/opt/stacks/ops-agents/docker-compose.yml` | | CT 112 | 192.168.1.7 | 9001 | `/opt/stacks/ops-agents/docker-compose.yml` | | CT 113 | 192.168.1.6 | 9001 | `/opt/stacks/db/docker-compose.yml` | Add each as a Portainer Agent environment: `http://:9001`. ## OIDC (Pocket-ID) **Status: pending** — requires manual OIDC client creation in Pocket-ID web UI first. 1. Go to `https://id.nuclide.systems/settings/admin/oidc-clients` 2. Create client: - Name: `portainer` - Redirect URIs: `http://192.168.1.8:9000/` - Note the **Client ID** and **Client Secret** 3. Configure in Portainer → Settings → Authentication → OAuth 2.0: - Authorization URL: `https://id.nuclide.systems/authorize` - Access Token URL: `https://id.nuclide.systems/api/oidc/token` - Resource URL: `https://id.nuclide.systems/api/oidc/userinfo` - Redirect URL: `http://192.168.1.8:9000/` - Client ID / Secret from step 2 - User Identifier: `email` - Scopes: `openid profile email` - Logout URL: `https://id.nuclide.systems/logout` - Enable "Automatic user provisioning" Or via API once client credentials are known: ```bash JWT=$(curl -s -X POST http://192.168.1.8:9000/api/auth \ -H "Content-Type: application/json" \ -d '{"username":"fkrebs","password":"tapirnase"}' | python3 -c 'import sys,json; print(json.load(sys.stdin)["jwt"])') curl -X PUT http://192.168.1.8:9000/api/settings \ -H "Authorization: Bearer $JWT" \ -H "Content-Type: application/json" \ -d '{ "AuthenticationMethod": 3, "OAuthSettings": { "ClientID": "", "ClientSecret": "", "AuthorizationURI": "https://id.nuclide.systems/authorize", "AccessTokenURI": "https://id.nuclide.systems/api/oidc/token", "ResourceURI": "https://id.nuclide.systems/api/oidc/userinfo", "RedirectURI": "http://192.168.1.8:9000/", "LogoutURI": "https://id.nuclide.systems/logout", "UserIdentifier": "email", "Scopes": "openid profile email", "OAuthAutoCreateUsers": true, "SSO": true } }' ``` ## Backup - **Local S3**: daily at 01:15 UTC via `portainer-backup.timer` on CT 109 - Script: `/usr/local/sbin/portainer-backup.py` - Bucket: `ct109-portainer-backup` on Garage (CT 104:10004) - Key ID: `GKd4511c4a01155ebbc37aa7ff` - Retention: 7 daily tarballs - **Offsite**: CT 103 `walg-offsite-sync.sh` syncs `ct109-portainer-backup` → `jottacloud:WAL-G/ct109-portainer-backup/` daily at 02:30 Restore: ```bash # Download latest from Garage aws --endpoint-url http://192.168.1.40:10004 s3 ls s3://ct109-portainer-backup/ aws --endpoint-url http://192.168.1.40:10004 s3 cp s3://ct109-portainer-backup/portainer-YYYY-MM-DD.tar.gz . tar xzf portainer-YYYY-MM-DD.tar.gz # Replace /var/lib/docker/volumes/monitoring_portainer_data/_data/ with extracted portainer_data/ ``` ## Ops ```bash # CT 109 cd /opt/stacks/monitoring docker compose up -d --force-recreate portainer docker logs portainer -f # Trigger manual backup python3 /usr/local/sbin/portainer-backup.py ``` ## Observability Portainer metrics scraped by Prometheus on CT 109 at `/api/metrics`. - **Auth**: `X-API-Key: ptr_tJhUVPuut6wreG6yTkuRZcfM0Rlu8Jehbx7+LHG4unc=` (prometheus-scrape API token, user fkrebs) - **Prometheus job**: `portainer` in `/opt/stacks/monitoring/prometheus/prometheus.yml` - **Grafana dashboard**: `http://192.168.1.8:3000/d/portainer-be/portainer-be` - Environment count, health status table, CPU/memory per environment, auth attempt rate - **Alert rules**: `/opt/stacks/monitoring/prometheus/rules/portainer.yml` - `PortainerEnvironmentUnhealthy` — env status == 2 for >2m → warning - `PortainerDown` — scrape target unreachable for >1m → critical Key metrics: | Metric | Description | |---|---| | `portainer_environment_count` | Total environments by type (docker/k8s/swarm) | | `portainer_environment_status` | 1=healthy, 2=unhealthy per environment | | `portainer_environment_resource_usage` | CPU/memory % per environment | | `portainer_authentication_total_status` | Auth success/fail counters | ## Arcane decommission (2026-05-26) Arcane was replaced by Portainer on 2026-05-26: - Arcane server (CT 109 `/opt/stacks/arcane/docker-compose.yml`) → renamed `.DECOMMISSIONED-2026-05-26` - Arcane agents removed from: CT 104 ops-agents, CT 105 ops-agents, CT 111 ops-agents, CT 112 ops-agents, CT 113 db stack - Arcane OIDC client `81cf4ed0-ea48-4df7-9c2d-cc1704b060f9` in Pocket-ID → to be deleted manually - Zoraxy route `arcane.nuclide.systems` → still exists, pending explicit confirmation to remove