... managed with Flux, Renovate, and Forgejo Actions
This is a mono repository for my home infrastructure and Kubernetes cluster. I try to keep everything as code, from the operating system on the nodes down to the individual application releases, including the two hosts that run outside the cluster.
- Talos Linux — immutable, API-driven OS that runs nothing but Kubernetes.
- Flux — reconciles the cluster against this repository.
- Renovate — opens pull requests for dependency updates across the whole repo.
- Ansible and doco-cd — provision and deploy the off-cluster hosts.
My cluster runs on three mini PCs with Talos, all of them control plane. It is hyper-converged, workloads and block storage share the same resources on the nodes, while a separate ZFS server provides NFS shares and holds the backups.
- Networking: cilium provides eBPF networking, BGP and LB-IPAM, while envoy-gateway handles ingress through the Gateway API and towonel tunnels public traffic in from the VPS. multus puts a few pods on a real VLAN for mDNS, and external-dns keeps Cloudflare and UniFi records in sync.
- Security & Secrets: cert-manager manages certificates and authentik handles SSO. For secrets I use external-secrets with 1Password Connect, so none live in Git.
- Storage & Backups: rook-ceph provides RBD and CephFS volumes, with kopiur backing them up to the NAS and offsite. Databases run on CloudNativePG and caches on Dragonfly.
- Observability: VictoriaMetrics and VictoriaLogs store metrics and logs, with dashboards from grafana-operator, health checks from gatus, and alerts sent to Pushover and karma.
- Automation & CI/CD: Forgejo hosts this repository and its Actions runners, and konflate renders every pull request through Flux to post the diff.
Flux watches the kubernetes folder and applies what is in it. It searches kubernetes/apps for the top level kustomization.yaml in each directory, which usually holds a namespace and one or more Flux kustomizations (ks.yaml). Those apply the HelmRelease and the other resources for the application.
Renovate watches the entire repository for dependency updates and opens a PR when it finds one. When the PR is merged Flux applies the change to the cluster.
📁 ansible # provisioning for the off-cluster hosts
📁 bootstrap # bringing a fresh cluster up from nothing
📁 docker # compose stacks for the edge VPS and the GPU box
📁 kubernetes # the cluster itself
📁 talos # machine config rendering📁 kubernetes
├── 📁 apps # applications, grouped by namespace
├── 📁 components # reusable kustomize components
└── 📁 flux # flux configurationA HelmRelease can depend on other releases, and a Kustomization on other kustomizations. In the example below paperless is not installed or upgraded until the Ceph cluster is healthy.
graph TD
A>Kustomization: rook-ceph] -->|Creates| B[HelmRelease: rook-ceph]
A>Kustomization: rook-ceph] -->|Creates| C[HelmRelease: rook-ceph-cluster]
C>HelmRelease: rook-ceph-cluster] -->|Depends on| B>HelmRelease: rook-ceph]
D>Kustomization: paperless-ngx] -->|Creates| E(HelmRelease: paperless-ngx)
E>HelmRelease: paperless-ngx] -->|Depends on| C>HelmRelease: rook-ceph-cluster]
Lenovo M920Q (Core i5-8500T) × 3 · 64 GB DDR4 · Talos / Kubernetes
- OS & etcd — 480 GB Micron 5300/5400 PRO SATA SSD (power-loss protection)
- Rook-Ceph — 1 TB Samsung 990 PRO NVMe (2280)
- Miroir — 256 GB SK hynix BC511 NVMe, replicated with DRBD
- Network — 1 G onboard (disabled) + Mellanox 10/25 G SFP+
MS-02 Ultra (Core Ultra 9 285HX) · 96 GB RAM DDR5 · Ubuntu
- GPU — NVIDIA RTX PRO 4000 Blackwell SFF, 24 GB
- Disk — 2 TB Samsung 9100 PRO NVMe
Ugreen DXP4800 Plus · 32 GB DDR5 · TrueNAS SCALE / ZFS
- Boot — 128 GB NVMe
- home-pool — 7.25 TB
- 2 × 4 TB Seagate IronWolf CMR — mirror
- 2 × 4 TB Seagate IronWolf CMR — mirror
- fast-pool — 236 GB
- 2 × 256 GB Samsung PM981 NVMe — mirror
- Network — 2.5 G + 10 G
- Cloud Gateway Fiber — router · 2×10 G SFP+, 1×10 G RJ45, 4×2.5 G RJ45
- Switch Aggregation — 8×10 G SFP+
- Switch Pro Max 16 PoE — 12×1 G PoE+ (30 W), 4×2.5 G PoE++ (60 W), 2×10 G SFP+
While most of my infrastructure is self-hosted, I rely on the cloud for a few parts of the setup. This keeps me from dealing with chicken and egg scenarios, and means alerting still reaches me when the cluster is offline.
| Service | Use | Cost |
|---|---|---|
| 1Password | Secrets with External Secrets | ~€50/yr |
| Cloudflare | Domain and DNS | Free |
| GitHub | Public mirror of this repository | Free |
| Hetzner | Edge VPS and offsite backups | ~€10/mo |
| Pushover | Kubernetes alerts and app notifications | $5 OTP |
| Resend | Outbound email | Free |
| Total: ~€14/mo |
A lot of what is here came from the Home Operations community and from the repositories of onedr0p, bjw-s, buroa and eleboucher. There is a template over at onedr0p/cluster-template if you want to follow along with some of the practices used here.
See LICENSE.