Cluster Networking
Relevant source files
- .taskfiles/Flux/Taskfile.yaml
- .taskfiles/bootstrap/Taskfile.yaml
- kubernetes/apps/ai/toolhive/config/grafanadashboard.yaml
- kubernetes/apps/kube-system/cilium/app/grafanadashboard.yaml
- kubernetes/apps/kube-system/cilium/app/helm/values.yaml
- kubernetes/apps/kube-system/cilium/app/helmrelease.yaml
- kubernetes/apps/kube-system/cilium/app/httproute.yaml
- kubernetes/apps/kube-system/cilium/app/kustomization.yaml
- kubernetes/apps/network/envoy-gateway/app/envoy.yaml
- kubernetes/apps/network/envoy-gateway/app/helmrelease.yaml
- kubernetes/apps/network/envoy-gateway/app/kustomization.yaml
- kubernetes/apps/network/envoy-gateway/app/scaledobject.yaml
- kubernetes/apps/security/tetragon/app/helmrelease.yaml
- kubernetes/apps/security/trivy-operator-polr-adapter/app/helmrelease.yaml
- scripts/bootstrap-cluster.sh
- scripts/render-machine-config.sh
The cluster networking stack provides a high-performance, secure, and observable foundation for all workloads. It leverages Cilium as the CNI (Container Network Interface) for eBPF-powered routing and security, Envoy Gateway for modern ingress management, and a split-brain DNS architecture to handle internal and external resolution.
System Architecture
The networking stack is divided into four primary layers: the CNI fabric, the ingress gateway, the DNS resolution layer, and multi-interface VLAN networking.
Networking Stack Overview
[Flowchart Diagram]
Sources:kubernetes/apps/network/envoy-gateway/app/envoy.yaml45-165kubernetes/apps/kube-system/cilium/app/helmrelease.yaml58-65
Cilium CNI and Network Policies [#4.1]
Cilium is the core networking provider, configured in Native Routing mode <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/helmrelease.yaml#L90-L90" min=90 file-path="kubernetes/apps/kube-system/cilium/app/helmrelease.yaml">Hii</FileRef> to eliminate encapsulation overhead. It replaces kube-proxy entirely <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/helmrelease.yaml#L58-L58" min=58 file-path="kubernetes/apps/kube-system/cilium/app/helmrelease.yaml">Hii</FileRef>, utilizing eBPF for service load balancing with the Maglev algorithm <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/helmrelease.yaml#L63-L63" min=63 file-path="kubernetes/apps/kube-system/cilium/app/helmrelease.yaml">Hii</FileRef>.
Key features include:
- L2 Announcements: Cilium advertises LoadBalancer IPs on the local network via ARP
<FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/helmrelease.yaml#L60-L61" min=60 max=61 file-path="kubernetes/apps/kube-system/cilium/app/helmrelease.yaml">Hii</FileRef>. - Direct Server Return (DSR): Optimized traffic flow where return traffic bypasses the load balancer
<FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/helmrelease.yaml#L64-L64" min=64 file-path="kubernetes/apps/kube-system/cilium/app/helmrelease.yaml">Hii</FileRef>. - Observability: Hubble provides deep visibility into flow logs and network performance, with dedicated Grafana dashboards
<FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/kube-system/cilium/app/grafanadashboard.yaml#L1-L33" min=1 max=33 file-path="kubernetes/apps/kube-system/cilium/app/grafanadashboard.yaml">Hii</FileRef>.
For details, see Cilium CNI and Network Policies.
Envoy Gateway and Ingress [#4.2]
The cluster utilizes Envoy Gateway as the implementation for the Kubernetes Gateway API. The architecture separates traffic into two distinct Gateway instances: envoy-external for public-facing services and envoy-internal for local-only access <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/network/envoy-gateway/app/envoy.yaml#L49-L95" min=49 max=95 file-path="kubernetes/apps/network/envoy-gateway/app/envoy.yaml">Hii</FileRef>.
Gateway Implementation Mapping
[Flowchart Diagram]
Sources:kubernetes/apps/network/envoy-gateway/app/envoy.yaml34-43kubernetes/apps/network/envoy-gateway/app/envoy.yaml167-248
The configuration includes advanced traffic management such as Zstd/Brotli compression<FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/network/envoy-gateway/app/envoy.yaml#L207-L212" min=207 max=212 file-path="kubernetes/apps/network/envoy-gateway/app/envoy.yaml">Hii</FileRef>, custom Lua scripts for blocking specific user-agents <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/network/envoy-gateway/app/kustomization.yaml#L12-L17" min=12 max=17 file-path="kubernetes/apps/network/envoy-gateway/app/kustomization.yaml">Hii</FileRef>, and automated scaling via KEDA based on Gatus health probes <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/network/envoy-gateway/app/scaledobject.yaml#L23-L28" min=23 max=28 file-path="kubernetes/apps/network/envoy-gateway/app/scaledobject.yaml">Hii</FileRef>.
For details, see Envoy Gateway and Ingress.
DNS Architecture and External DNS [#4.3]
A split-brain DNS strategy ensures that internal services resolve to local IPs while public services are reachable via the internet.
- Cloudflare: Manages public records and provides a secure entry point via Cloudflare Tunnels.
- ExternalDNS: Automatically synchronizes
GatewayandHTTPRoutehostnames to DNS providers. - CoreDNS: Handles cluster-internal
.svc.cluster.localresolution.
The system uses DNSEndpoint CRDs and annotations on Gateway resources to manage IP-to-hostname mappings <FileRef file-url="https://github.com/chaijunkin/home-ops/blob/b5f8d898/kubernetes/apps/network/envoy-gateway/app/envoy.yaml#L50-L108" min=50 max=108 file-path="kubernetes/apps/network/envoy-gateway/app/envoy.yaml">Hii</FileRef>.
For details, see DNS Architecture and External DNS.
Multus CNI and VLAN Networking [#4.4]
While Cilium handles the primary pod network, Multus CNI allows pods to attach to multiple network interfaces. This is critical for home automation workloads (like Home Assistant) that need direct access to isolated VLANs (IoT, No-Internet, etc.) without routing through the cluster’s primary gateway.
This is achieved through NetworkAttachmentDefinitions using the macvlan driver, allowing pods to appear as first-class citizens on specific physical VLAN tags.
For details, see Multus CNI and VLAN Networking.
Configuration Summary
| Component | Role | Primary Config |
|---|---|---|
| Cilium | CNI / eBPF Fabric | kubernetes/apps/kube-system/cilium/app/helmrelease.yaml |
| Envoy Gateway | Ingress Controller | kubernetes/apps/network/envoy-gateway/app/envoy.yaml |
| KEDA | Gateway Autoscaling | kubernetes/apps/network/envoy-gateway/app/scaledobject.yaml |
| Hubble | Network Observability | kubernetes/apps/kube-system/cilium/app/httproute.yaml |
Sources:kubernetes/apps/kube-system/cilium/app/helmrelease.yaml1-118kubernetes/apps/network/envoy-gateway/app/envoy.yaml1-270kubernetes/apps/network/envoy-gateway/app/scaledobject.yaml1-87