Proxmox and Talos Node Provisioning

Relevant source files

This page details the automated provisioning of the home-ops infrastructure layer, focusing on the lifecycle of Talos Linux nodes running on a Proxmox hypervisor. The process integrates Terraform for hardware virtualization, Jinja2/OpenTofu templating for immutable OS configuration, and the TUPPR controller for rolling cluster upgrades.

Infrastructure Lifecycle Overview

The provisioning workflow follows a “Virtualization-as-Code” pattern where the Proxmox hypervisor is treated as a target for the bpg/proxmox Terraform provider.

Provisioning Data Flow

The following diagram illustrates the relationship between configuration files and the resulting infrastructure entities.

Diagram: Configuration to Provisioning Mapping

[Flowchart Diagram]

Sources: infrastructure/terraform/proxmox/talos/virtual-machines.tf1-149 infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl1-159 infrastructure/terraform/proxmox/talos_cluster.auto.tfvars1-31

Proxmox VM Provisioning

VMs are managed using the bpg/proxmox provider infrastructure/terraform/proxmox/providers.tf23-26 The configuration utilizes a q35 machine type and virtio-scsi-single hardware for optimal performance infrastructure/terraform/proxmox/talos/virtual-machines.tf12-14

Hardware and iGPU Passthrough

For workloads requiring hardware acceleration (e.g., Plex/Jellyfin), Intel iGPU passthrough is configured at the VM level.

Host PCI Mapping: The hostpci block maps the physical device 0000:00:02.0 to the VM infrastructure/terraform/proxmox/talos/virtual-machines.tf122-148
Kernel Modules: Talos is configured to load the i915 and nbd modules via the machine configuration infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl2-5

Storage and Network

Disks: Primary disks are provisioned using the disk dynamic block, referencing Talos images downloaded directly to Proxmox storage infrastructure/terraform/proxmox/talos/virtual-machines.tf63-92
Networking: Configured via network_device blocks with support for specific MAC addresses and VLAN tags infrastructure/terraform/proxmox/talos/virtual-machines.tf53-60

Sources: infrastructure/terraform/proxmox/talos/virtual-machines.tf1-150 infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl1-87

Talos Machine Configuration

Talos Linux is configured through immutable YAML files generated from Terraform templates (.tftpl). These templates allow for dynamic injection of variables such as IP addresses, disk paths, and Kubernetes versions.

Configuration Components

Component	File Path	Description
Common	`common.yaml.tftpl`	Shared sysctls, kernel modules, and kubelet feature gates infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl1-159
Control Plane	`control-plane.yaml.tftpl`	VIP configuration, etcd settings, and API server manifests infrastructure/terraform/proxmox/talos/machine-config/control-plane.yaml.tftpl1-40
Worker	`worker.yaml.tftpl`	Worker-specific node labels and hardware passthrough settings

Key System Optimizations

The common.yaml.tftpl includes several high-performance sysctl tunings for 10Gb/s networking and PostgreSQL workloads:

Networking: net.core.rmem_max and net.core.wmem_max are set to 67108864 to support high-throughput QUIC/Cloudflared traffic infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl69-70
Storage: NFS mount options are optimized with nconnect=8 and nfsvers=4.2infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl53-57
Database: vm.nr_hugepages is set to 1024 for PostgreSQL performance infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl83

Sources: infrastructure/terraform/proxmox/talos/machine-config/common.yaml.tftpl1-83 infrastructure/terraform/proxmox/talos/machine-config/control-plane.yaml.tftpl1-40

Talos Image Factory and Bootstrap

The cluster uses custom Talos images generated via the Talos Image Factory. The specific image schematic (drivers, extensions) is defined in schematic.yaml.

Bootstrap Process

The scripts/bootstrap-cluster.sh script handles the initial transition from bare VMs to a functional cluster:

Apply Config: Renders machine configurations and applies them via talosctl apply-config --insecurescripts/bootstrap-cluster.sh10-62
Bootstrap: Initiates the etcd cluster on a designated controller node scripts/bootstrap-cluster.sh65-80
Kubeconfig: Fetches the admin credentials once the API server is up scripts/bootstrap-cluster.sh83-95
Resource Handover: Applies CRDs and initial Helm releases via helmfile before Flux takes over scripts/bootstrap-cluster.sh115-172

Sources: scripts/bootstrap-cluster.sh1-198 infrastructure/terraform/proxmox/talos/virtual-machines.tf86-91

Rolling Upgrades with TUPPR

Cluster maintenance is automated using TUPPR (Talos/Kubernetes Upgrade PR), which manages rolling upgrades for both the Talos OS and the Kubernetes components.

Upgrade Orchestration

TUPPR utilizes custom resources to define upgrade targets and health check policies.

Diagram: TUPPR Upgrade Logic

[Flowchart Diagram]

Configuration Entities

TalosUpgrade: Defined in talosupgrade.yaml, specifying the target version (e.g., v1.13.0) and health checks to ensure VolSync is not currently synchronizing data during a reboot kubernetes/apps/system-upgrade/tuppr/upgrades/talosupgrade.yaml1-21
Versions Source: The versions.env file acts as the single source of truth for KUBERNETES_VERSION and TALOS_VERSION, monitored by Renovate kubernetes/apps/system-upgrade/versions.env1-5

Sources: kubernetes/apps/system-upgrade/tuppr/upgrades/talosupgrade.yaml1-21 kubernetes/apps/system-upgrade/versions.env1-5 kubernetes/apps/system-upgrade/tuppr/app/ocirepository.yaml1-14

Cloudjur

Explorer

Proxmox-and-Talos-Node-Provisioning