ops: Loki/Alloy fleet enabled; Backrest+WAL-G+Gitea Gotify hooks; Immich WAL fix

- Alloy now active on CT 102, 103, 104, 109, 101, 105, 113, nuc (mix of
  Docker container + systemd binary); all hosts shipping journal/docker
  logs to Loki on CT 109. CT 103 alloy newly enabled.
- Backrest plans: actionGotify hooks on SUCCESS (prio 3) + ERROR/WARNING
  (prio 8). Gotify app 'Backrest', token AtIMGOYcZsb5HfO.
- WAL-G staleness monitor on CT 103 (hourly): pg_stat_archiver SSH/exec
  against CT 113 + CT 104 immich_postgres. Found Immich WAL push failing
  for 42h — root cause: garage container on bridge network not
  shared_backend (Garage 2026-05-28 incident aftermath). Reconnected to
  shared_backend; image pinned v0.9.4 in compose to match running.
- Gitea daily digest at 08:00 on PVE host nuc — yesterday's commits.
- Closes fkrebs/n8n-flows #2, #5, #6.

services-overview.md

@@ -67,6 +67,7 @@ Every service running in the homelab, with its access URL(s) and host. **Externa
 | Wetty (SSH-in-browser) | — | <http://192.168.1.8:4090> | CT 109 | — |
 | Backrest (backups UI) | — | <http://192.168.1.3:9898> | CT 103 | [[backrest-ct103]] |
 | Diun (image update notifier) | — | logs only (`docker logs diun`) | CT 109 | — |
+| Alloy (log/metric collector) | — | `:12345` (per host) | CT 104, 109, 102, 103, 101, 105, 113, nuc | — |
 
 ## Network / infra
 

services/backrest.md

@@ -181,18 +181,20 @@ restic -r rclone:jottacloud:services restore latest \
   --include /mnt/pve/unas/services/vaultwarden
 ```
 
-### Monitoring (planned)
+### Monitoring ✅ DEPLOYED 2026-05-28
 
-In Backrest UI → each repo → Hooks:
+Gotify hooks on all 3 plans (services / video-projects / media). Gotify app "Backrest" token `AtIMGOYcZsb5HfO`.
 
-- `CONDITION_BACKUP_ERROR` / `CONDITION_CHECK_ERROR` → POST Gotify priority 8
+- `CONDITION_SNAPSHOT_SUCCESS` → Gotify priority 3 (✓ snapshot summary)
-- `CONDITION_BACKUP_SUCCESS` → POST Gotify priority 3
+- `CONDITION_SNAPSHOT_ERROR` + `CONDITION_SNAPSHOT_WARNING` → Gotify priority 8 (✗ FAILED)
 
-```
+Hooks use Backrest's native `actionGotify` (not `actionCommand`) — no shell required, templating via Go text/template with `{{ .Plan.Id }}`, `{{ .SnapshotStats.FilesNew }}`, `{{ formatSizeBytes .SnapshotStats.DataAdded }}`, `{{ .Duration }}`, `{{ .Error }}`.
-curl -s -X POST 'http://gotify:80/message?token=TOKEN' \
+
-  -H 'Content-Type: application/json' \
+**WAL-G staleness monitor** (`/usr/local/sbin/walg-staleness-check.sh` on CT 103, hourly timer): SSH-then-`docker exec psql` to each Postgres host and queries `pg_stat_archiver` (last_archived_time, last_failed_time, failed_count, failed_wal). Alerts if archive >26h old OR a recent failure is more recent than the last success. Gotify token `Ag570SdxcXcpp51`. Hosts: CT 113 `postgres`, CT 104 `immich_postgres`. Closes `fkrebs/n8n-flows#6`.
-  -d '{"title":"Backrest: {{.Plan}}","message":"{{.Summary}}","priority":3}'
+
-```
+> **Incident found by this monitor (2026-05-28)**: Immich `wal-g wal-push` had been failing with exit 124 (timeout) for 42 hours. Root cause: `garage` container was recreated on Docker `bridge` network instead of `shared_backend` during the 2026-05-28 Garage incident, making `garage:3900` unresolvable from `immich_postgres`. Fix: `docker network connect shared_backend garage`; compose image pinned to v0.9.4 to match running.
+
+**Gitea daily digest** (`/usr/local/sbin/gitea-digest.sh` + `gitea-digest.timer` on PVE host nuc, 08:00 daily): walks all `fkrebs/*` repos, finds yesterday's commits, posts summary to Gotify priority 3 (token `A.qpozt3ZQGgEMG`). Closes `fkrebs/n8n-flows#5`.
 
 ### Applying a new config