fkrebs/docs
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b68ab51e0ed4389b989fd21f063244f0aabb2560
docs/services/homelab-architecture/index.html
T

2742 lines
70 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="Single source of truth for the homelab">
<link rel="canonical" href="http://192.168.1.8:13080/services/homelab-architecture/">
<link rel="prev" href="../../infra/proxmox-memory-audit/">
<link rel="next" href="../dev-environment/">
<link rel="icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.6">
<title>Homelab architecture - nuclide.systems docs</title>
<link rel="stylesheet" href="../../assets/stylesheets/main.484c7ddc.min.css">
<link rel="stylesheet" href="../../assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="blue">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#nuclide-ecosystem-homelab-architecture" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="../.." title="nuclide.systems docs" class="md-header__button md-logo" aria-label="nuclide.systems docs" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
nuclide.systems docs
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Homelab architecture
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="blue" aria-hidden="true" type="radio" name="__palette" id="__palette_0">
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="blue" aria-hidden="true" type="radio" name="__palette" id="__palette_1">
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
<div class="md-search__suggest" data-md-component="search-suggest"></div>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="../.." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="../../CHANGELOG/" class="md-tabs__link">
Changelog
</a>
</li>
<li class="md-tabs__item">
<a href="../../RESUME/" class="md-tabs__link">
Resume
</a>
</li>
<li class="md-tabs__item">
<a href="../../ct-inventory/" class="md-tabs__link">
CT inventory
</a>
</li>
<li class="md-tabs__item">
<a href="../../infra/proxmox-state/" class="md-tabs__link">
Infra
</a>
</li>
<li class="md-tabs__item md-tabs__item--active">
<a href="./" class="md-tabs__link">
Services
</a>
</li>
<li class="md-tabs__item">
<a href="../../security/data-leak-audit-comparison/" class="md-tabs__link">
Security & audits
</a>
</li>
<li class="md-tabs__item">
<a href="../../ideas/stack-ideas/" class="md-tabs__link">
Ideas
</a>
</li>
<li class="md-tabs__item">
<a href="../../history/traefik-migration/" class="md-tabs__link">
History
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="../.." title="nuclide.systems docs" class="md-nav__button md-logo" aria-label="nuclide.systems docs" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
nuclide.systems docs
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../CHANGELOG/" class="md-nav__link">
<span class="md-ellipsis">
Changelog
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../RESUME/" class="md-nav__link">
<span class="md-ellipsis">
Resume
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../ct-inventory/" class="md-nav__link">
<span class="md-ellipsis">
CT inventory
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_5" >
<label class="md-nav__link" for="__nav_5" id="__nav_5_label" tabindex="0">
<span class="md-ellipsis">
Infra
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_5">
<span class="md-nav__icon md-icon"></span>
Infra
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../infra/proxmox-state/" class="md-nav__link">
<span class="md-ellipsis">
Proxmox state
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/portmap/" class="md-nav__link">
<span class="md-ellipsis">
Port map
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/storage/" class="md-nav__link">
<span class="md-ellipsis">
Storage
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/volumes/" class="md-nav__link">
<span class="md-ellipsis">
Volumes
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/docker-networks/" class="md-nav__link">
<span class="md-ellipsis">
Docker networks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/connection-hosts/" class="md-nav__link">
<span class="md-ellipsis">
Connection hosts
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../infra/proxmox-memory-audit/" class="md-nav__link">
<span class="md-ellipsis">
Memory audit
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--active md-nav__item--section md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_6" checked>
<label class="md-nav__link" for="__nav_6" id="__nav_6_label" tabindex="">
<span class="md-ellipsis">
Services
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_6_label" aria-expanded="true">
<label class="md-nav__title" for="__nav_6">
<span class="md-nav__icon md-icon"></span>
Services
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
<span class="md-ellipsis">
Homelab architecture
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
<span class="md-ellipsis">
Homelab architecture
</span>
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#topology" class="md-nav__link">
<span class="md-ellipsis">
Topology
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#docker-networks" class="md-nav__link">
<span class="md-ellipsis">
Docker networks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#ai-agent-platform-the-core-optstacksai" class="md-nav__link">
<span class="md-ellipsis">
AI / Agent platform (the core, /opt/stacks/ai)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#data-layer" class="md-nav__link">
<span class="md-ellipsis">
Data layer
</span>
</a>
<nav class="md-nav" aria-label="Data layer">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#database-design-standards" class="md-nav__link">
<span class="md-ellipsis">
Database design / standards
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#nfs-storage-strategy-corrected-2026-05-19" class="md-nav__link">
<span class="md-ellipsis">
NFS / storage strategy (corrected 2026-05-19)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#services" class="md-nav__link">
<span class="md-ellipsis">
Services
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#operational-rules-conventions" class="md-nav__link">
<span class="md-ellipsis">
Operational rules / conventions
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#open-roadmap" class="md-nav__link">
<span class="md-ellipsis">
Open / roadmap
</span>
</a>
<nav class="md-nav" aria-label="Open / roadmap">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#observability-lxc-planned-scoped-from-this-sessions-incidents" class="md-nav__link">
<span class="md-ellipsis">
Observability LXC (planned — scoped from this session's incidents)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#topology-at-a-glance" class="md-nav__link">
<span class="md-ellipsis">
Topology — at a glance
</span>
</a>
<nav class="md-nav" aria-label="Topology — at a glance">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#physical-ct-layout" class="md-nav__link">
<span class="md-ellipsis">
Physical / CT layout
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#auth-plane-pocket-id-is-the-universal-idp" class="md-nav__link">
<span class="md-ellipsis">
Auth plane — Pocket-ID is the universal IdP
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#mcp-plane-bifrost-host-child-workers" class="md-nav__link">
<span class="md-ellipsis">
MCP plane — Bifrost host, child workers
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#data-plane-what-lives-where" class="md-nav__link">
<span class="md-ellipsis">
Data plane — what lives where
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../dev-environment/" class="md-nav__link">
<span class="md-ellipsis">
Dev environment (Coder + Gitea)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../mcp-gateway/" class="md-nav__link">
<span class="md-ellipsis">
MCP gateway
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../mcp-servers/" class="md-nav__link">
<span class="md-ellipsis">
MCP servers
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../pocket-id/" class="md-nav__link">
<span class="md-ellipsis">
Pocket-ID (OIDC)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../backrest/" class="md-nav__link">
<span class="md-ellipsis">
Backrest (backups)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../databases/" class="md-nav__link">
<span class="md-ellipsis">
Databases
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../secrets-manager/" class="md-nav__link">
<span class="md-ellipsis">
Secrets manager
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../comfyui/" class="md-nav__link">
<span class="md-ellipsis">
ComfyUI
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../zoraxy/" class="md-nav__link">
<span class="md-ellipsis">
Zoraxy
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../adguard-dns/" class="md-nav__link">
<span class="md-ellipsis">
AdGuard DNS
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../cloud-gpu/" class="md-nav__link">
<span class="md-ellipsis">
Cloud GPU
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../arcane/" class="md-nav__link">
<span class="md-ellipsis">
Arcane
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../doc-ingestion/" class="md-nav__link">
<span class="md-ellipsis">
Doc ingestion pipeline
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../llm-benchmark/" class="md-nav__link">
<span class="md-ellipsis">
LLM benchmark
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_7" >
<label class="md-nav__link" for="__nav_7" id="__nav_7_label" tabindex="0">
<span class="md-ellipsis">
Security & audits
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_7_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_7">
<span class="md-nav__icon md-icon"></span>
Security & audits
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../security/data-leak-audit-comparison/" class="md-nav__link">
<span class="md-ellipsis">
Comparison
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../security/data-leak-audit-2026-05-20-tr004-cloud-sandbox/" class="md-nav__link">
<span class="md-ellipsis">
2026-05-20 · cloud-sandbox breach
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../security/data-leak-audit-2026-05-21-tr004-artifacts/" class="md-nav__link">
<span class="md-ellipsis">
2026-05-21 · artifacts (clean)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../security/audit-claude-code-meta/" class="md-nav__link">
<span class="md-ellipsis">
Self-audit (Claude Code)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../security/transcripts/audit-chat-2026-05-21/" class="md-nav__link">
<span class="md-ellipsis">
Transcript (audit session)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_8" >
<label class="md-nav__link" for="__nav_8" id="__nav_8_label" tabindex="0">
<span class="md-ellipsis">
Ideas
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_8_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_8">
<span class="md-nav__icon md-icon"></span>
Ideas
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../ideas/stack-ideas/" class="md-nav__link">
<span class="md-ellipsis">
Stack ideas
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_9" >
<label class="md-nav__link" for="__nav_9" id="__nav_9_label" tabindex="0">
<span class="md-ellipsis">
History
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_9_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_9">
<span class="md-nav__icon md-icon"></span>
History
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../history/traefik-migration/" class="md-nav__link">
<span class="md-ellipsis">
Traefik (abandoned 2026-05-16)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../history/traefik-migration-docker-labels/" class="md-nav__link">
<span class="md-ellipsis">
Traefik labels (abandoned)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../history/mcp-gateway-requirements/" class="md-nav__link">
<span class="md-ellipsis">
MCP gateway requirements (superseded)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../history/scrubbing-list-2026-05-17/" class="md-nav__link">
<span class="md-ellipsis">
Scrubbing list (2026-05-17)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../history/case-study/" class="md-nav__link">
<span class="md-ellipsis">
Case study
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#topology" class="md-nav__link">
<span class="md-ellipsis">
Topology
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#docker-networks" class="md-nav__link">
<span class="md-ellipsis">
Docker networks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#ai-agent-platform-the-core-optstacksai" class="md-nav__link">
<span class="md-ellipsis">
AI / Agent platform (the core, /opt/stacks/ai)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#data-layer" class="md-nav__link">
<span class="md-ellipsis">
Data layer
</span>
</a>
<nav class="md-nav" aria-label="Data layer">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#database-design-standards" class="md-nav__link">
<span class="md-ellipsis">
Database design / standards
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#nfs-storage-strategy-corrected-2026-05-19" class="md-nav__link">
<span class="md-ellipsis">
NFS / storage strategy (corrected 2026-05-19)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#services" class="md-nav__link">
<span class="md-ellipsis">
Services
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#operational-rules-conventions" class="md-nav__link">
<span class="md-ellipsis">
Operational rules / conventions
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#open-roadmap" class="md-nav__link">
<span class="md-ellipsis">
Open / roadmap
</span>
</a>
<nav class="md-nav" aria-label="Open / roadmap">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#observability-lxc-planned-scoped-from-this-sessions-incidents" class="md-nav__link">
<span class="md-ellipsis">
Observability LXC (planned — scoped from this session's incidents)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#topology-at-a-glance" class="md-nav__link">
<span class="md-ellipsis">
Topology — at a glance
</span>
</a>
<nav class="md-nav" aria-label="Topology — at a glance">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#physical-ct-layout" class="md-nav__link">
<span class="md-ellipsis">
Physical / CT layout
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#auth-plane-pocket-id-is-the-universal-idp" class="md-nav__link">
<span class="md-ellipsis">
Auth plane — Pocket-ID is the universal IdP
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#mcp-plane-bifrost-host-child-workers" class="md-nav__link">
<span class="md-ellipsis">
MCP plane — Bifrost host, child workers
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#data-plane-what-lives-where" class="md-nav__link">
<span class="md-ellipsis">
Data plane — what lives where
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1 id="nuclide-ecosystem-homelab-architecture">Nuclide Ecosystem — Homelab Architecture<a class="headerlink" href="#nuclide-ecosystem-homelab-architecture" title="Permanent link">&para;</a></h1>
<p>Living architecture reference for the <code>/opt/stacks</code> homelab. ~66 containers
across ~23 compose stacks. Principle: <strong>self-host everything</strong>, OIDC SSO,
<code>*.nuclide.systems</code> via one reverse proxy.</p>
<h2 id="topology">Topology<a class="headerlink" href="#topology" title="Permanent link">&para;</a></h2>
<ul>
<li><strong>NUC 14 Pro</strong> (<code>192.168.1.40</code>) — primary Docker host, all <code>/opt/stacks/*</code>.
Itself a <strong>Proxmox LXC (CTID 104)</strong> on node <code>nuc</code>; local data on ZFS
<code>rpool/data/subvol-104-disk-0</code>. Mounts the UNAS NFS at <code>/mnt/pve/unas</code>.</li>
<li><strong>Proxmox VE</strong> — host <strong><code>192.168.1.20:8006</code></strong>, single node <strong><code>nuc</code></strong>
(Intel Core Ultra 7 155H, 22 threads, 64G RAM, PVE 9.1.11). API access via
<code>root@pam!mcp</code> token (now <code>PVEAuditor</code>, read-only). <strong>9 guests (+ 2 planned)</strong> — canonical roster in <a href="../../ct-inventory/"><code>ct-inventory.md</code></a>:</li>
<li><code>100</code> qemu <strong>haos</strong> — Home Assistant OS VM (4c/16G) → <code>.60</code></li>
<li><code>101</code> lxc <strong>shepard</strong> — secondary Docker host (12c/16G/107G) → <code>.49</code></li>
<li><code>102</code> lxc <strong>dns</strong> — <strong>AdGuard Home</strong> (network DNS + ad/tracker blocking,
2c/0.5G). UniFi DHCP has <code>dhcpd_dns_enabled: false</code>; clients resolve via
the gateway <code>.1</code>, <strong>and the UDM forwards DNS upstream to AdGuard</strong>
(confirmed) — so ad/tracker blocking is network-wide despite DHCP not
handing out AdGuard's IP directly.</li>
<li><code>103</code> lxc <strong>backrest</strong> — Restic/Backrest backups (1c/0.5G)</li>
<li><code>104</code> lxc <strong>docker</strong> — <strong>the primary <code>/opt/stacks</code> host</strong> (16c/48G/200G) → <code>.40</code></li>
<li><code>105</code> lxc <strong>nextcloud</strong> — Nextcloud (4c/8G/107G) → <code>.41</code></li>
<li><code>108</code> lxc <strong>zoraxy</strong> — reverse proxy (2c/2G) → <code>.4</code></li>
<li><code>110</code> lxc <strong>id</strong> — Pocket-ID OIDC IdP (1c/1G/4G) → <code>.5</code>; migrated off 104 on 2026-05-20</li>
<li><code>111</code> lxc <strong>dev</strong> — Coder + Gitea (12c/32G/60G) → <code>.42</code>; new 2026-05-20</li>
<li><code>109</code> lxc <strong>ops</strong> <code>192.168.1.8</code> — Prometheus + Grafana + Loki + Alloy + pve-exporter (2026-05-23); future: Arcane, Dozzle, Homarr, Tinyauth</li>
<li><code>113</code> lxc <strong>db</strong> — postgres 17 + pgAdmin + WAL-G → Garage S3 + Arcane edge agent → <code>.6</code>; provisioned 2026-05-21</li>
<li>Storages: <code>local</code> (dir), <code>local-zfs</code> (zfspool, ~1.9T), <code>unas</code> (nfs, ~20T).</li>
<li><strong>Ubiquiti UniFi</strong> — UDM "Home" (UDMA6A8, UCG Fiber) gateway + 4× U7-series
APs. Controller at <strong><code>https://192.168.1.1</code></strong> (UniFi OS 5.0.16, SSO + MFA
enforced). Single site "Default". MCP access: <code>ghcr.io/enuno/unifi-mcp-server</code>
(197 tools, Network App API key); Site Manager/cloud tools need
<code>UNIFI_SITE_MANAGER_ENABLED</code>, all local API tools work.</li>
<li><strong>LAN</strong> "Default" — <code>192.168.1.1/24</code>, corporate, no VLAN. DHCP pool
<code>.100–.250</code> (24 h lease), domain <code>localdomain</code>. Static infra lives
<code>.1–.99</code> (outside the pool). WANs: "Internet 1" (primary), "Secondary"
(WAN2, disabled). 47 clients (17 wired, 30 wireless).</li>
<li><strong>WiFi</strong> — single SSID <strong>"nuclide"</strong>, WPA2 with WPA3-transition, bands
2.4/5/6 GHz, not guest, no per-SSID VLAN, no L2 isolation. <strong>Flat
network</strong>: servers, IoT, consoles, phones all share one L2 segment.</li>
<li><strong>Firewall</strong> — zone-based (Internal/External/DMZ/Gateway/VPN/Hotspot),
almost entirely system-predefined rules. Port forwards: <strong>80/443 →
192.168.1.4</strong> (Zoraxy); <strong>15001 TCP/UDP → 192.168.1.40</strong> (Traccar watch
protocol — already at UDM level, no Zoraxy stream proxy needed).</li>
<li>⚠️ <strong>Non-UniFi gear = topology blind spots</strong> (UniFi can't see/manage):<ul>
<li><strong>D-Link DGS-1210-28P</strong> 28-port PoE switch at <code>.10</code> (HW F5, fw
<strong>6.32.008</strong>, S/N TM0I533010201) — unmanaged by UniFi; its ports/
port-clients don't appear in UniFi topology. HTTP-only mgmt
(<code>admin</code>/shared <code>tapirnase</code> pw, RSA-login). <strong>Audit 2026-05-19
(read-only):</strong></li>
<li><strong>SNTP was broken → FIXED 2026-05-19.</strong> Real root cause was <strong>DNS</strong>:
servers were <code>0/1/2.de.pool.ntp.org</code> but the switch has no DNS
resolver (<code>DNS_STATE=2</code>) so it couldn't resolve them. (The default
gateway was <em>fine</em> — an active static default route <code>0.0.0.0/0 →
.1</code> exists; the empty <code>Default_Route_Gateway=[]</code> is just the unused
<em>interface-level</em> gateway field — misleading.) Clock had been stuck
at <strong>31/08/2025</strong> (~8.5 mo). Fix: repointed to the <strong>pinned NTP
source standard</strong> (see below), all by IP, DNS-free. <strong>Verified
synced</strong> 2026-05-19 — clock corrected to live time (CEST/UTC+2).</li>
<li><strong>HTTPS/SSL disabled</strong> — admin creds cross the flat LAN in clear.
<em>Pending</em> (see roadmap: mgmt-TLS certs for Proxmox + D-Link).</li>
<li><strong>SNMP agent enabled</strong> on the flat LAN (community strings unverified).
<em>Pending.</em></li>
<li><strong>Trusted-Host mgmt allowlist disabled</strong> — admin UI reachable from
any LAN device incl. IoT. <em>Pending:</em> enable allowing <code>192.168.1.0/24</code>.</li>
<li>⚠️ The switch is a <strong>transparent L2 switch</strong> — every device on it
<strong>is fully visible to UniFi as a client</strong> (MAC/IP via UDM DHCP/ARP);
only the <em>wired port-topology through it</em> is invisible (needs UniFi
adoption, impossible here). This is unlike the RE700X repeater.</li>
<li>🔴 <strong>The D-Link is the network CORE / single point of failure.</strong>
FDB-table analysis (2026-05-19): the <strong>UDM uplinks on port 26</strong>,
the <strong>entire Proxmox host (NUC .40 + Nextcloud .41 + HA .60 + all
VMs/LXCs) is on port 10</strong>, and the <strong>U7 APs uplink on ports 3 &amp; 16</strong>
(their wireless clients appear via those ports). Nearly all wired
<em>and</em> wireless traffic transits this unmanaged switch → a D-Link
failure = total LAN+WAN outage. Amplifies the mgmt-hardening,
firmware-currency, monitoring and segmentation items below.</li>
<li><strong>UniFi↔D-Link DHCP/LLDP interop = fully compatible</strong> (audited
2026-05-19, read-only):<ul>
<li>DHCP: <code>BOOTP_Relay_State=0</code> <strong>and</strong> <code>DHCP_BOOTP_Local_Relay_
Status=0</code> → switch inserts <strong>no Option 82</strong>; UDM DHCP server
gets clean untagged requests. (<code>Option82_State=1</code> is moot —
only applies when relay/local-relay is active; both are off.)
DHCP Server Screening has no entries → doesn't block the UDM
(also = no rogue-DHCP protection: security note, not interop).</li>
<li>LLDP: D-Link LLDP enabled, <strong>all 28 ports TX+RX</strong> (<code>Port_Basic_
Setting</code> mode 3); stats show 25 inserts / 16 ageouts → LLDP
frames <em>are</em> exchanged &amp; parsed with UniFi gear (no TLV
incompat). "0 neighbors" earlier was just a point-in-time
aged-out snapshot. UniFi still won't <em>draw</em> topology for an
unadopted switch — cosmetic only.</li>
<li>RSTP: both run 802.1w (standard, interoperable). Tuning
opportunity (not a fault): set a deliberate root-bridge
priority — currently all default 32768 so root = lowest MAC
(arbitrary). Make the D-Link (physical core) or UDM root.</li>
<li>Flow control: D-Link global FC <code>=2</code> (off/auto) — matches
UniFi's default (802.3x disabled). Compatible.</li>
</ul>
</li>
<li><strong>TP-Link RE700X</strong> WiFi extender at <code>.187</code> (MACs <code>…1a:8a:0f/10/11</code>) —
bridges/NATs devices behind it. <strong>The Klipper 3D printer at <code>.189</code>
sits behind the RE700X</strong>, so UniFi only sees the repeater, never <code>.189</code>.</li>
<li><strong>DNS:</strong> UniFi DHCP has <code>dhcpd_dns_enabled: false</code> (clients get the
gateway <code>.1</code> as resolver, not AdGuard directly) — see DNS note below.</li>
</ul>
</li>
</ul>
<p><strong>Infrastructure IPs known from UniFi:</strong>
| IP | Name / Hostname | Notes |
|---|---|---|
| .1 | UDM Home (UDMA6A8) | Gateway, UCG Fiber, fw 5.0.16 |
| .4 | Zoraxy LXC 108 | Reverse proxy |
| .20 | Proxmox node <code>nuc</code> | PVE 9.1.11 |
| .40 | NUC 14 Pro / Docker LXC 104 | Main host, Proxmox OUI |
| .41 | Nextcloud LXC 105 | Named "NextCloud", Proxmox OUI |
| .49 | Shepard Docker host | Secondary Docker host |
| .50 | U7-Pro-Wall AP | Hallway/entry |
| .51 | U7 In-Wall AP | In-wall, UAPA6A5 |
| .52 | U7 Mesh (Schlafzimmer) | Bedroom |
| .53 | U7 Mesh (Esszimmer) | Dining room |
| .60 | Home Assistant OS VM | haos CTID 100 |
| .62 | Siemens oven | BSH Hausgeräte, WiFi |
| .66 | Tibber Pulse | Energy monitor (Espressif) |
| .10 | DGS-1210-28P | D-Link 28-port PoE switch (NOT UniFi-managed) |
| .124 | L0018 | Wired, unknown device |
| .143 | — | Sony Interactive Ent. (PlayStation) |
| .164 | C100_7614A4 | Tapo C100 camera (TP-Link) |
| .169 | awtrix_fcf0bc | AWTRIX LED clock (Espressif) |
| .187 | RE700X | TP-Link WiFi extender — NATs devices behind it |
| .189 | — | Klipper 3D printer (behind RE700X, invisible to UniFi) |
| .192 | REDMI-Note-15-Pro-5G | Xiaomi phone |
| .241 | VS9-EU-MNA3478A | Dyson purifier/fan |
- <strong>Zoraxy</strong> reverse proxy at <code>192.168.1.4:8000</code> — wildcard <code>*.nuclide.systems</code>
cert + rules. Source of truth: <code>proxy/zoraxy/routes.json</code> (+ idempotent
<code>scripts/zoraxy_sync.py --apply</code>). Cross-host, so no Docker labels.
- <strong>Ubiquiti UNAS</strong> — NFS server <code>192.168.1.31:/var/nfs/shared/storage</code>
(mounted <code>/mnt/pve/unas</code>, ~19T). Bulk/storage (arr media, qdrant, configs);
several stacks bind data dirs here. ⚠️ SQLite-on-NFS is fragile here
(see n8n note under Operational rules).
- <strong>Auth</strong> — <strong>PocketID</strong> (<code>id.nuclide.systems</code>, OIDC, SQLite) is the
<strong>universal SSO/IdP for the entire ecosystem</strong> — effectively every service
authenticates via PocketID OIDC (LiteLLM, Nextcloud, Coder, Gitea,
Vaultwarden, etc.). Single sign-on everywhere; one identity source to
secure/audit. Vaultwarden remains the lone gap as of 2026-05-20.
Open WebUI OIDC not yet wired (as of 2026-05-26).</p>
<h2 id="docker-networks">Docker networks<a class="headerlink" href="#docker-networks" title="Permanent link">&para;</a></h2>
<p><code>ai-internal</code> (AI/MCP plane) · <code>shared_backend</code> (cross-stack DB/S3) ·
plus per-stack: <code>arr-stack_default</code>,
<code>immich_default</code>, <code>karakeep_default</code>, <code>homepage_default</code>, <code>vpn_default</code>.
The MCP gateway bridges <code>ai-internal</code> + <code>shared_backend</code>.</p>
<h2 id="ai-agent-platform-the-core-optstacksai">AI / Agent platform (the core, <code>/opt/stacks/ai</code>)<a class="headerlink" href="#ai-agent-platform-the-core-optstacksai" title="Permanent link">&para;</a></h2>
<ul>
<li><strong>Bifrost</strong> (<code>ai.nuclide.systems</code>, :14003) — unified LLM gateway + MCP host.
Stack <code>ai/bifrost/</code>. Auth via <code>sk-bf-</code> VKs. LLM inference at <code>/v1</code>; MCP at
<code>/mcp</code> (29 clients, ~760 tools as of 2026-05-26). Proxies LLM requests to
LiteLLM internally.</li>
<li><strong>LiteLLM</strong> (:14000, <strong>internal only</strong> — no Zoraxy route) — model gateway,
backend for Bifrost. ~28 curated models (provider allowlist), SAIA-terminal
<strong>failovers</strong> + <code>num_retries</code>, Redis completion cache, Gemini €10/30d
<code>provider_budget_config</code>. Config: <code>ai/litellm-config/config.yaml</code> (+ DB
overlay <code>LiteLLM_Config</code>). Reachable as <code>http://litellm:4000</code> inside Docker.</li>
<li><strong>Open WebUI</strong> (<code>chat.nuclide.systems</code>, :14002) — chat UI. Stack
<code>ai/open-webui.yml</code>. Uses Qdrant + TEI for RAG. OIDC via Pocket-ID (not
yet wired as of 2026-05-26).</li>
<li><strong>syncstack</strong> (<code>ai/syncstack.py</code>, cron <code>/etc/cron.d/syncstack</code> 15 min) —
<strong>model syncer/optimizer ONLY</strong> (curation/allowlist → DB, model-health → ntfy).
One-shot container.</li>
<li>~~<strong>mcp-gateway</strong>~~ — <strong>DECOMMISSIONED 2026-05-26.</strong> Was FastAPI/DinD on
<code>:8080</code> at <code>mcp.nuclide.systems</code>. Compose renamed <code>.DECOMMISSIONED-2026-05-26</code>.
MCP is now served by Bifrost at <code>https://ai.nuclide.systems/mcp</code>.</li>
<li><strong>MCP servers</strong> (Bifrost-managed, formerly gateway-managed): time, home-assistant,
kroki, coder, fetch, sequential-thinking, gotify, memos, docling, markitdown,
crawl4ai, context7, wikipedia, papersearch, comfyui, bluesky, obsidian,
immich, n8n, nextcloud, paperless, unifi, upload-artifact, gitea, proxmox,
shepard, git, gitlab, youtube-transcript, searxng (30 servers). 29 clients,
~760 tools. See <a href="../mcp-servers/"><code>services/mcp-servers.md</code></a>.</li>
<li><strong>Coder</strong> (<code>dev.nuclide.systems</code>, CT 111) — replaced Daytona as the sandbox /
dev-environment runtime. Templates: <code>python-uv</code> (persistent, GPU passthrough,
baked LiteLLM env + Claude Code), <code>mcp-sandbox</code> (ephemeral, sci stack
pre-baked). See <a href="../dev-environment/"><code>services/dev-environment.md</code></a>.</li>
<li>~~<strong>LobeChat</strong>~~ — <strong>DECOMMISSIONED 2026-05-26.</strong> Was at <code>chat.nuclide.systems</code>
(:14001). Compose renamed <code>.DECOMMISSIONED-lobehub-2026-05-26.yml</code>. Replaced
by Open WebUI.</li>
<li><strong>ComfyUI</strong> (img gen, Intel Arc iGPU), <strong>Kroki</strong>, <strong>searxng</strong> (self-hosted
meta-search).</li>
</ul>
<h2 id="data-layer">Data layer<a class="headerlink" href="#data-layer" title="Permanent link">&para;</a></h2>
<ul>
<li><strong>shared-postgres</strong> (pg16, <code>shared_backend</code>, data on local ZFS) — the
<strong>primary/standard DB for all deployments</strong>. Tenants: LiteLLM, paperless,
<strong>memos</strong> (migrated 2026-05-19), n8n (added 2026-05-19). Superuser
<code>postgres</code> via unix socket; per-app dedicated role+db (role owns its db,
password in the app stack's <code>.env</code>). Tuned 2026-05-19 (3G limit,
shared_buffers 768M, max_conn 200).
⚠️ <strong>n8n was NOT successfully migrated</strong> — n8n's <code>export/import</code> CLI only
covers workflows+credentials, <strong>not users/settings/SSO</strong>, so the PG
cutover lost the owner account + OIDC. <strong>Reverted to local-disk SQLite</strong>
(tier-1 still satisfied, like PocketID). Compounding: unpinned
<code>n8nio/n8n:latest</code> had drifted 2.7→<strong>2.20.11</strong>, breaking the custom OIDC
<code>hooks.js</code> (hardcoded old <code>/usr/local/lib</code> module paths) → n8n crash-looped.
Fixed: image pinned to <code>2.20.11</code>, <code>hooks.js</code> paths patched for 2.20.11
pnpm layout (<code>/usr/lib/node_modules/n8n/...</code>), <strong>OIDC hook re-enabled
2026-05-19</strong> — <strong>verified</strong>: <code>/auth/oidc/login</code> → 302 to PocketID
(client <code>33135ad4</code>, correct redirect/scope/state/nonce). <strong>Lessons:
(1) only a FULL pgloader migration of <em>all</em>
tables is complete — partial <code>export/import</code> is not; (2) pin critical
images — <code>latest</code> drift is a real outage cause.</strong>
Migration note: services without a native full SQLite→PG export use
<strong>pgloader data-only of ALL tables</strong> into the app-built schema (exclude
only the app's own migration-tracking table), app role temp-SUPERUSER for
the load (FK/trigger disable) then reverted.</li>
<li><strong>lobe-postgres</strong> (pg17/ParadeDB) — LobeChat (decommissioned 2026-05-26; DB retained pending cleanup). <strong>lobe-redis</strong> — LiteLLM cache (LobeChat decommissioned). <strong>Garage S3</strong> (<code>garage:3900</code>, ext <code>s3.nuclide.systems</code> /
<code>:10004</code>) — lobe-files, memos, <strong>WAL-G PG backups</strong>. <strong>Qdrant</strong> (vector,
on UNAS) — unused yet (future RAG/mem0). pgAdmin (internal).</li>
<li><strong>Backups</strong>: WAL-G v3 → Garage for all 3 PG instances (shared-postgres,
lobe-postgres, immich_postgres); daily 3am cron (<code>/etc/cron.d/pg-backup</code>).
<strong>Incident + fix (2026-05-19):</strong> Garage stored <code>meta</code>+<code>data</code> on the UNAS
NFS → NFS stalls hung <code>wal-g wal-push</code> → archivers hung since 2026-05-18
14:51 (<code>failed_count=0</code> = hung not failing), zero base backups, stalled
WAL recycled (that window unrecoverable). <strong>Resolved:</strong> Garage moved to
local disk (<code>/opt/stacks/shared-db/garage/{meta,data}</code>); all 3 archivers
drained; <code>archive_command</code> hardened to <code>timeout 60 wal-g wal-push %p</code>
(fail-fast vs infinite hang); fresh base backups taken for all 3. Extra
safety net: local logical dumps in <code>/opt/stacks/backups/shared-pg/</code>.
Lesson: Garage metadata is fsync/lock-heavy — <strong>never</strong> on NFS, same rule
as SQLite.</li>
</ul>
<h3 id="database-design-standards">Database design / standards<a class="headerlink" href="#database-design-standards" title="Permanent link">&para;</a></h3>
<ul>
<li><strong>Default = Postgres on <code>shared-postgres</code>.</strong> Any app needing persistence
that supports Postgres gets a dedicated <code>role</code>+<code>database</code> there. SQLite is
allowed <strong>only</strong> when the app has no Postgres support, and then <strong>only on
local disk</strong> — never the UNAS NFS share (broken POSIX/SMB file locking;
caused the n8n outage + PocketID latent risk).</li>
<li><strong>Provisioning pattern:</strong> <code>CREATE ROLE &lt;app&gt; LOGIN PASSWORD …; CREATE
DATABASE &lt;app&gt; OWNER &lt;app&gt;;</code> → set the app's <code>DB_*</code> env, password lives in
that stack's <code>.env</code> (vault migration is the long-term plan).</li>
<li><strong>Why standardize:</strong> unified, durable <strong>WAL-G PITR backup</strong> — the single
biggest reason. One backup story instead of N un-backed-up SQLite files.</li>
<li><strong>Migration reality:</strong> few apps have a native SQLite→PG data port. n8n does
(<code>export/import</code> CLI). Most (PocketID, Vaultwarden, traccar, …) do not —
switching <code>DB_PROVIDER</code> starts a <em>fresh</em> DB; data port needs
<code>pgloader</code>/app-specific tooling. <strong>Memos</strong> done via pgloader (data-only,
app-built schema, role temp-SUPERUSER for the load). When no safe port
exists, fall back to local-disk SQLite (PocketID) until a port is built.</li>
</ul>
<h3 id="nfs-storage-strategy-corrected-2026-05-19">NFS / storage strategy (corrected 2026-05-19)<a class="headerlink" href="#nfs-storage-strategy-corrected-2026-05-19" title="Permanent link">&para;</a></h3>
<ul>
<li>The Docker host is an <strong>unprivileged Proxmox LXC (104)</strong> → it <strong>cannot do
in-container NFS mounts</strong> (kernel denies; tested — <code>operation not
permitted</code>). That's why Proxmox NFS-mounts at the host and bind-mounts
<code>/mnt/pve/unas</code> in. <strong>Consequence:</strong> per-stack "Docker NFS volumes at
v4.1" (the old tier-3 idea) is <strong>infeasible</strong> here without
<code>pct set 104 --features mount=nfs</code> + a full LXC reboot (all stacks down).</li>
<li><strong>Tier 1 is the rule and does the real work:</strong> DBs/metadata/lock- or
fsync-heavy stores → local ZFS or shared-postgres, <strong>never NFS</strong> (any
version — SQLite/LMDB/sled are unsafe on NFS regardless of tuning).</li>
<li><strong>NFSv4 is NOT available — the Ubiquiti UNAS Pro is NFSv3-only</strong> (current
firmware; verified 2026-05-19 on node <code>nuc</code>: <code>showmount -e</code> works but
<code>mount :/ -o vers=4.1</code> fails server-side <code>No such file or directory</code>, i.e.
no v4 pseudo-root). So tier-2 (host v3→v4.1) and tier-3 (Docker NFS v4.1
volumes) are <strong>both dead ends</strong> — no v4 to upgrade to, no reboot worth
doing for it. Real export path is <code>/volume/&lt;uuid&gt;/.srv/.unifi-drive/
storage/.data</code> (Proxmox <code>unas</code> storage uses the <code>/var/nfs/shared/storage</code>
alias, works on v3 — leave it).</li>
<li><strong>Therefore tier-1 is the whole strategy.</strong> NFSv3 stays for bulk/
sequential data (fine for that). Optional marginal v3 tuning: <code>nconnect=4</code>
for throughput — not required.</li>
<li><strong>Tier-1 complete (2026-05-19):</strong> all SQLite-on-NFS backlog resolved —
ntfy, Karakeep, arcane → local disk; <strong>Vaultwarden → shared-postgres</strong>
(pgloader, 2054 rows, 0 errors, NFS kept for attachments/rsa_key.pem);
traccar data → local disk. PocketID stays on local SQLite (no native
PG migration path, deferred).</li>
</ul>
<h2 id="services">Services<a class="headerlink" href="#services" title="Permanent link">&para;</a></h2>
<p>Nextcloud (LXC .41), <strong>Paperless-ngx</strong> (+paperless-ai, tika/gotenberg),
<strong>Immich</strong> (server/ML/redis/postgres/power-tools), <strong>Memos</strong>, <strong>Karakeep</strong>
(+chrome), <strong>Vaultwarden</strong>, <strong>n8n</strong>, <strong>Home Assistant</strong> (.60, ~2492 entities),
<strong>ntfy</strong> (<code>homelab-ai</code> topic — model-health + agent alerts), <strong>traccar</strong> (GPS, HTTP :15000 / watch :15001),
<strong>arr-stack</strong> (prowlarr/shelfarr/flaresolverr behind gluetun VPN), <strong>streamio</strong>,
<strong>Arcane</strong> (Docker mgmt), <strong>Dozzle</strong> (logs), <strong>Homepage</strong> (dashboard, 6 groups),
<strong>Daytona OIDC adapter</strong> (Keycloak→PocketID PKCE proxy for the VS Code ext).</p>
<h2 id="operational-rules-conventions">Operational rules / conventions<a class="headerlink" href="#operational-rules-conventions" title="Permanent link">&para;</a></h2>
<ul>
<li><strong>Separation</strong>: <code>syncstack</code> = model curation; <code>bifrost</code> = MCP + LLM gateway (mcp-gateway decommissioned 2026-05-26).</li>
<li><strong>Self-host first</strong>: external/cloud only when no self-hosted form exists.</li>
<li><strong>Pinned NTP source standard</strong> — all network gear/infra should point at
the <em>same fixed-IP, DNS-free</em> NTP sources (devices like the D-Link can't
resolve hostnames; pinned IPs avoid silent DNS-based SNTP failure):</li>
<li><code>192.53.103.108</code> — PTB ptbtime1 (DE national time)</li>
<li><code>192.53.103.104</code> — PTB ptbtime2 (DE national time)</li>
<li><code>162.159.200.123</code> — Cloudflare NTP anycast (fallback)
Applied to the D-Link switch 2026-05-19. UDM + other infra should converge
on the same set (a self-hosted LAN NTP server is a roadmap option, but the
standard stays "same pinned sources" everywhere).</li>
<li>Run Python with <code>uv run</code> (not python3) in <code>/opt/stacks</code>.</li>
<li>Zoraxy self-service: edit <code>routes.json</code> → <code>scripts/zoraxy_sync.py --apply</code>.</li>
<li>Secrets in <code>ai/.env</code> (plaintext — <strong>env-audit/secret-vault is an open
hardening task</strong>; Vaultwarden available). Note: a UniFi MFA JWT has leaked
into <code>homepage/config/logs/homepage.log</code> — rotate + scrub when hardening.
⚠️ <strong>Shared-password reuse:</strong> <code>tapirnase</code> is reused as the WiFi PSK, the
LiteLLM master key root (<code>sk-tapirnase</code>), and the D-Link switch admin pw —
single sniff/leak has broad blast radius. Rotate per-service when hardening
(noted 2026-05-19, rotation deferred per user).</li>
<li><strong>Proxmox/UniFi safety</strong>: any write/change to the Proxmox host or Ubiquiti
network gear requires explicit confirmation + a stated risk assessment
first; read-only queries are fine. High blast radius (foundation layer).</li>
<li><strong>SQLite-on-NFS is unsafe here</strong>: the UNAS NFS share has unreliable file
locking (true for SMB/CIFS too). Any SQLite app must use <code>shared-postgres</code>
or local disk, never the NFS share. <strong>n8n</strong> hit this (<code>Database connection
timed out</code>) and was <strong>migrated to <code>shared-postgres</code></strong> (DB on local ZFS;
data dir moved to local <code>/opt/stacks/n8n/data</code>; old NFS <code>database.sqlite</code>
kept as rollback). Audit other SQLite stacks for NFS-backed data dirs.</li>
<li>Naming debt: "syncstack" has outgrown its name — rename once the agent-platform identity firms up. mcp-gateway renamed/decommissioned 2026-05-26.</li>
</ul>
<h2 id="open-roadmap">Open / roadmap<a class="headerlink" href="#open-roadmap" title="Permanent link">&para;</a></h2>
<p>mem0 vs Qdrant for agent memory (deferred); Agent teams/orchestration +
expose Agent Operator as MCP; S3/Immich/n8n/Paperless/Proxmox MCP servers
(in progress); <strong>UniFi MCP</strong> — COMPLETE 2026-05-19 (<code>ghcr.io/enuno/unifi-mcp-server</code>,
197 tools, Network App API key, <code>http</code> transport; local API fully working).
<strong>Tier-1 SQLite-off-NFS: COMPLETE</strong> — all services off NFS for DB/metadata.
Note: Traccar watch protocol port 15001 is already forwarded at the UDM level —
no Zoraxy stream proxy needed. Plus: env → secret vault. Wire Open WebUI OIDC
via Pocket-ID once Bifrost auth is settled.</p>
<p><strong>Management-plane TLS (planned):</strong> issue/trust proper certs for admin-UI
auth on the <strong>Proxmox</strong> host and the <strong>D-Link DGS-1210</strong> (currently HTTP-only
on the D-Link → admin creds in clear on the flat LAN; Proxmox self-signed).
Brings switch/hypervisor mgmt onto the <code>*.nuclide.systems</code> PKI like the rest.</p>
<p><strong>Network segmentation (planned):</strong> the LAN is flat — servers, IoT (Siemens
oven, Dyson, Tapo cam, AWTRIX), consoles and phones all on one L2 (<code>192.168.1.0/24</code>,
no VLAN, no L2 isolation). Plan an <strong>IoT VLAN</strong> (+ matching firewall zone) so
untrusted appliances can't reach the server/Proxmox subnet. Complications to
design around: the non-UniFi <strong>D-Link DGS-1210-28P</strong> switch (<code>.10</code>) and
<strong>TP-Link RE700X</strong> extender (<code>.187</code>, NATs the Klipper printer <code>.189</code>) won't
honour UniFi VLAN tags natively — segmentation needs a plan for the wired
trunk through the D-Link and the repeater's bridge mode.</p>
<h3 id="observability-lxc-planned-scoped-from-this-sessions-incidents">Observability LXC (planned — scoped from this session's incidents)<a class="headerlink" href="#observability-lxc-planned-scoped-from-this-sessions-incidents" title="Permanent link">&para;</a></h3>
<p><strong>Why it's now a priority:</strong> the WAL-G archiver was hung <strong>silently for
~13 h</strong> (<code>failed_count=0</code>, zero base backups) and would never have been
noticed; NFS stalls and SQLite-on-NFS damage are likewise silent. Monitoring
must target <em>exactly these silent-failure classes</em>.
- <strong>Placement:</strong> its <strong>own LXC on node <code>nuc</code></strong>, NOT inside LXC 104 — 104
hosts everything, so the monitor must survive/alert when 104 is down.
- <strong>Stack:</strong> VictoriaMetrics (or Prometheus) + Grafana + Loki + Alertmanager
→ <strong>ntfy <code>homelab-ai</code></strong> (already the alert channel).
- <strong>Exporters/probes:</strong> node_exporter (per host + key LXCs), postgres_exporter
×3 (shared/lobe/immich), cAdvisor/docker, blackbox (HTTP + TLS-expiry for
Zoraxy wildcard), <strong>pve-exporter</strong> (use the <code>root@pam!mcp</code> PVEAuditor
token), and a <strong>custom WAL-G/archiver textfile collector</strong>:
<code>pg_stat_archiver</code> (last_archived age, <code>failed_count</code>), <code>.ready</code> backlog,
and <code>wal-g backup-list</code> newest-base age — per PG instance.
- <strong>Alerts (priority order, derived from real incidents):</strong>
1. WAL archiver stalled (last_archived age &gt; 15 m) or newest base backup
&gt; 26 h, any PG instance.
2. NFS mount on <code>/mnt/pve/unas</code> unresponsive / high op latency.
3. <strong>Config-drift guard:</strong> any <code>*.db</code>/<code>*.sqlite*</code> appears under
<code>/mnt/pve/unas</code> (catches a regression of the tier-1 rule).
4. Container unhealthy/restart-looping &gt; 5 m (the gluetun pattern).
5. local-zfs <code>rpool</code> or NFS pool &gt; 85 %; 6. TLS cert &lt; 14 d.</p>
<h2 id="topology-at-a-glance">Topology — at a glance<a class="headerlink" href="#topology-at-a-glance" title="Permanent link">&para;</a></h2>
<h3 id="physical-ct-layout">Physical / CT layout<a class="headerlink" href="#physical-ct-layout" title="Permanent link">&para;</a></h3>
<pre class="mermaid"><code>flowchart TB
subgraph Host["Proxmox host · 192.168.1.20 · Intel Core Ultra 7 155H · 64 GiB"]
direction TB
haos["VM 100 · haos&lt;br/&gt;(.60)&lt;br/&gt;Home Assistant"]
shepard["CT 101 · shepard&lt;br/&gt;(.49)&lt;br/&gt;Shepard product stack"]
dns["CT 102 · dns&lt;br/&gt;(.2)&lt;br/&gt;AdGuard Home"]
backrest["CT 103 · backrest&lt;br/&gt;(.3)&lt;br/&gt;Backrest / restic"]
docker104["&lt;b&gt;CT 104 · docker&lt;/b&gt;&lt;br/&gt;(.40) · 16c/48G/200G&lt;br/&gt;~65 containers · Intel Arc passthrough"]
nc["CT 105 · nextcloud&lt;br/&gt;(.41)&lt;br/&gt;Nextcloud AIO (NFS)"]
zoraxy["CT 108 · zoraxy&lt;br/&gt;(.4)&lt;br/&gt;reverse proxy + ACME"]
obs["CT 109 · ops&lt;br/&gt;(.8)&lt;br/&gt;Prometheus · Grafana · Loki · Alloy · pve-exporter"]
id["CT 110 · id&lt;br/&gt;(.5)&lt;br/&gt;Pocket-ID (moved here 2026-05-20)"]
dev["CT 111 · dev&lt;br/&gt;(.42) · 12c/32G/60G&lt;br/&gt;Coder + Gitea + workspaces · Intel Arc"]
db["CT 113 · db&lt;br/&gt;(.6) · provisioned 2026-05-21&lt;br/&gt;shared Postgres + pgAdmin"]
end
UNAS[("UNAS&lt;br/&gt;192.168.1.31&lt;br/&gt;NFSv3")]
UDM[["UDM-SE · 192.168.1.1&lt;br/&gt;UniFi gateway · DNS → AdGuard"]]
Inet([Internet · ACME challenges · jottacloud · LiteLLM upstreams])
UDM &lt;--&gt; Host
UDM &lt;--&gt; Inet
UNAS &lt;--&gt; shepard
UNAS &lt;--&gt; backrest
UNAS &lt;--&gt; docker104
UNAS &lt;--&gt; nc
UNAS &lt;--&gt; dev
classDef planned stroke-dasharray:5 5,fill:#222,stroke:#aaa,color:#aaa
class db planned</code></pre>
<h3 id="auth-plane-pocket-id-is-the-universal-idp">Auth plane — Pocket-ID is the universal IdP<a class="headerlink" href="#auth-plane-pocket-id-is-the-universal-idp" title="Permanent link">&para;</a></h3>
<p>Every web service that supports OIDC federates against Pocket-ID. Coder/Gitea/Vaultwarden access through Zoraxy; Zoraxy + Tinyauth fronts the non-OIDC-native ones.</p>
<pre class="mermaid"><code>flowchart LR
user(["fkrebs · browser / VS Code / Claude"]) --&gt; zx[Zoraxy&lt;br/&gt;CT 108]
zx --&gt; coder[Coder · CT 111]
zx --&gt; gitea[Gitea · CT 111]
zx --&gt; nc2[Nextcloud · CT 105]
zx --&gt; immich[Immich · CT 104]
zx --&gt; owui[Open WebUI · CT 104]
zx --&gt; n8n[n8n · CT 104]
zx --&gt; bifrost[Bifrost · CT 104]
zx --&gt; vw[Vaultwarden · CT 104]
coder --&gt; pid[(Pocket-ID&lt;br/&gt;CT 110)]
gitea --&gt; pid
nc2 --&gt; pid
immich --&gt; pid
owui -.-&gt;|OIDC not yet wired| pid
n8n --&gt; pid
bifrost -.-&gt;|VK auth, not OIDC| pid
vw -.-&gt;|via Tinyauth&lt;br/&gt;when CT 109 lands| pid
classDef pending stroke-dasharray:4 4,color:#888
class vw pending</code></pre>
<h3 id="mcp-plane-bifrost-host-child-workers">MCP plane — Bifrost host, child workers<a class="headerlink" href="#mcp-plane-bifrost-host-child-workers" title="Permanent link">&para;</a></h3>
<p>Bifrost on CT 104 is the MCP host; ~30 child MCP servers run on the same docker network (<code>ai-internal</code>). The old mcp-gateway FastAPI/DinD container was decommissioned 2026-05-26.</p>
<pre class="mermaid"><code>flowchart LR
client[["Claude Code / Cursor / Open WebUI"]]
gw["Bifrost&lt;br/&gt;(CT 104)&lt;br/&gt;ai.nuclide.systems/mcp"]
client -- Bearer VK --&gt; gw
subgraph "CT 104 · ai-internal docker net"
direction TB
coderm[coder-mcp]
immm[mcp-immich]
n8nm[mcp-n8n]
fetch[mcp-fetch]
time[mcp-time]
cw[mcp-crawl4ai]
seq[mcp-sequential-thinking]
ham[home-assistant-mcp]
kr[kroki-mcp]
others[...18 more]
end
gw --&gt; coderm
gw --&gt; immm
gw --&gt; n8nm
gw --&gt; fetch
gw --&gt; time
gw --&gt; cw
gw --&gt; seq
gw --&gt; ham
gw --&gt; kr
gw --&gt; others
coderm -.spawns/controls.-&gt; CoderWS[(Coder workspaces&lt;br/&gt;CT 111)]
ham -.bridges to.-&gt; HAOS[(Home Assistant&lt;br/&gt;VM 100)]</code></pre>
<h3 id="data-plane-what-lives-where">Data plane — what lives where<a class="headerlink" href="#data-plane-what-lives-where" title="Permanent link">&para;</a></h3>
<pre class="mermaid"><code>flowchart TB
subgraph Tier1["Tier-1 / latency-sensitive · local NVMe"]
pid_d[Pocket-ID sqlite · CT 110]
immich_pg[Immich Postgres · CT 104]
shared_pg[shared-postgres · CT 104]
coder_pg[coder-db · CT 111]
gitea_pg[gitea-db · CT 111]
garage["Garage S3 · CT 104&lt;br/&gt;(moved off NFS 2026-05-19)"]
n8n_local["n8n data · CT 104&lt;br/&gt;(reverted from NFS 2026-05-19)"]
end
subgraph Bulk["Bulk · UNAS NFSv3"]
media["Immich media, Paperless docs,&lt;br/&gt;arr-stack media, audiobooks"]
coder_homes[Coder workspace homes · /mnt/pve/unas/services/coder]
gitea_data[Gitea repos · /mnt/pve/unas/services/gitea]
vw_data["Vaultwarden data&lt;br/&gt;(tier-1 leak — plan to move local)"]
end
subgraph BulkCIFS["Bulk · UNAS CIFS (Nextcloud only)"]
nc_data[Nextcloud user files]
end
subgraph Backup["Off-host backup"]
jottacloud[(jottacloud&lt;br/&gt;via Backrest)]
end
Tier1 -. WAL-G .-&gt; garage
Bulk -. only media/data-dir .-&gt; jottacloud
classDef gap fill:#5a2a2a,stroke:#c44,color:#fcc
class vw_data,jottacloud gap</code></pre>
<p>The red blocks above are gaps: Vaultwarden is on NFS when it shouldn't be; off-host backup currently covers only one UNAS path, not service data. See <code>stacks/storage.md</code> for the verified state and the cleanup TODO list.</p>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "../..", "features": ["navigation.tabs", "navigation.sections", "navigation.expand", "navigation.top", "search.highlight", "search.suggest", "content.code.copy"], "search": "../../assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink