From 9930e3dc09d39a28d7fd5eb985b9c60f6d8a2fb6 Mon Sep 17 00:00:00 2001 From: Pratik Mahalle Date: Mon, 24 Nov 2025 19:28:22 +0530 Subject: [PATCH] docs(capm3): add Metal3 custom resources docs Signed-off-by: Pratik Mahalle --- docs/user-guide/src/SUMMARY.md | 6 +- docs/user-guide/src/capm3/custom_resources.md | 49 +++ docs/user-guide/src/capm3/metal3data.md | 191 +++++++++++ .../src/capm3/metal3datatemplate.md | 321 ++++++++++++++++++ 4 files changed, 565 insertions(+), 2 deletions(-) create mode 100644 docs/user-guide/src/capm3/custom_resources.md create mode 100644 docs/user-guide/src/capm3/metal3data.md create mode 100644 docs/user-guide/src/capm3/metal3datatemplate.md diff --git a/docs/user-guide/src/SUMMARY.md b/docs/user-guide/src/SUMMARY.md index df22c7aa..216a8f4d 100644 --- a/docs/user-guide/src/SUMMARY.md +++ b/docs/user-guide/src/SUMMARY.md @@ -39,15 +39,17 @@ - [Old Installation Process](ironic/ironic_installation.md) - [Cluster-api-provider-metal3](capm3/introduction.md) - [Install Metal³ provider](capm3/installation_guide.md) + - [Custom Resources](capm3/custom_resources.md) + - [Metal3DataTemplate](capm3/metal3datatemplate.md) + - [Metal3Data](capm3/metal3data.md) - [Features](capm3/features.md) - [Remediation](capm3/remediaton.md) - [Node Reuse](capm3/node_reuse.md) - [Pivoting](capm3/pivoting.md) - [Automated cleaning](capm3/automated_cleaning.md) - - [Label synchronization](capm3/label_sync.md) + - [Label Sync](capm3/label_sync.md) - [Data sources](capm3/data_sources.md) - [ClusterClass](capm3/clusterclass.md) - - [Failure Domain](capm3/failure_domain.md) - [Ip-address-manager](ipam/introduction.md) - [Install Ip-address-manager](ipam/ipam_installation.md) - [Troubleshooting FAQ](troubleshooting.md) diff --git a/docs/user-guide/src/capm3/custom_resources.md b/docs/user-guide/src/capm3/custom_resources.md new file mode 100644 index 00000000..dbef7c1a --- /dev/null +++ b/docs/user-guide/src/capm3/custom_resources.md @@ -0,0 +1,49 @@ +# Custom Resources + +Cluster API Provider Metal3 (CAPM3) extends the Kubernetes API with several custom +resources that enable the management of bare metal infrastructure through the +Cluster API framework. These resources provide the necessary abstractions for +defining, configuring, and managing bare metal hosts and their associated +metadata. + +## Overview + +The custom resources in CAPM3 are designed to work together to provide a complete +solution for bare metal cluster management: + +- **Metal3DataTemplate**: Defines templates for generating metadata and network + configuration for bare metal hosts +- **Metal3Data**: Represents the rendered data instances created from templates +- **Metal3Machine**: Represents a bare metal machine in the cluster +- **Metal3Cluster**: Represents a bare metal cluster + +## Resource Relationships + +```mermaid +Metal3Cluster + ↓ +Metal3MachineTemplate + ↓ +Metal3Machine + ↓ +Metal3DataClaim + ↓ +Metal3Data ← Metal3DataTemplate +``` + +## Usage Patterns + +### Basic Usage + +1. Create a `Metal3DataTemplate` with your desired metadata and network + configuration +2. Reference the template in your `Metal3MachineTemplate` +3. CAPM3 automatically creates `Metal3Data` instances and renders the + configuration + +### Advanced Usage + +- Use IP pools for dynamic IP address allocation +- Configure complex network topologies with bonds, VLANs, and multiple + interfaces +- Implement custom metadata generation based on host properties diff --git a/docs/user-guide/src/capm3/metal3data.md b/docs/user-guide/src/capm3/metal3data.md new file mode 100644 index 00000000..0ef799fb --- /dev/null +++ b/docs/user-guide/src/capm3/metal3data.md @@ -0,0 +1,191 @@ +# Metal3Data + +The `Metal3Data` resource represents the rendered data instances created from +`Metal3DataTemplate` objects. It contains the host-specific configuration data +that has been generated for a particular bare metal host and links to the +associated secrets. + +## Overview + +A `Metal3Data` object is created automatically by the CAPM3 controller when a +`Metal3Machine` references a `Metal3DataTemplate`. It contains: + +- **Index**: A unique index assigned to the host +- **Claim reference**: Link to the `Metal3DataClaim` that requested this data +- **Secret references**: Links to the generated metadata and network data + secrets +- **Template reference**: Link to the `Metal3DataTemplate` that was used + +## API Reference + +```yaml +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: Metal3Data +metadata: + name: - + namespace: + ownerReferences: + - apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + controller: true + kind: Metal3DataTemplate + name: +spec: + templateReference: # Optional + index: + claim: + name: + namespace: + metaData: + name: -metadata- + namespace: + networkData: + name: -networkdata- + namespace: + template: + name: + namespace: +status: + ready: + error: + errorMessage: "" +``` + +## Lifecycle + +### Creation Process + +1. **Claim Creation**: When a `Metal3Machine` references a `Metal3DataTemplate`, + a `Metal3DataClaim` is created +2. **Index Assignment**: The controller selects the lowest available index for + the new claim +3. **Data Generation**: A `Metal3Data` object is created with the assigned + index +4. **Secret Generation**: The controller renders the template and creates + metadata and network data secrets +5. **Status Update**: The `ready` status is set to `true` when all secrets are + created successfully + +### Template Update Behavior + +**Important**: If the `Metal3DataTemplate` object is updated, the generated +secrets will not be updated automatically. This behavior is intentional to +allow for reprovisioning of the nodes in the exact same state as they were +initially provisioned. + +To apply template updates to existing nodes, it is necessary to perform a +rolling upgrade of all nodes that reference the updated template. + +### Index Management + +Indexes are managed automatically by the controller: + +- **Starting Point**: Indexes always start from 0 +- **Increment**: Each new index increments by 1 +- **Availability Check**: The controller selects the lowest available index not + in use +- **Conflict Resolution**: If a conflict occurs during creation, the controller + retries with a new index + +### Naming Convention + +The `Metal3Data` object name follows the pattern: + +```text +- +``` + +For example: + +- Template: `worker-template` +- Index: `0` +- Result: `worker-template-0` + +## Secret Generation + +The controller generates two types of secrets for each `Metal3Data` instance: + +### Metadata Secret + +Contains host-specific metadata in YAML format. + +**Naming**: `-metadata-` + +**Content**: Rendered metadata based on the template configuration + +### Network Data Secret + +Contains network configuration in JSON format following the +[Nova network_data.json format](https://docs.openstack.org/nova/latest/user/metadata.html#openstack-format-metadata). + +**Naming**: `-networkdata-` + +**Content**: Rendered network configuration based on the template + +## Complete Example + +```yaml +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: Metal3Data +metadata: + name: worker-template-0 + namespace: default + ownerReferences: + - apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + controller: true + kind: Metal3DataTemplate + name: worker-template +spec: + index: 0 + claim: + name: worker-0 + namespace: default + metaData: + name: worker-0-metadata-0 + namespace: default + networkData: + name: worker-0-networkdata-0 + namespace: default + template: + name: worker-template + namespace: default +status: + ready: true + error: false + errorMessage: "" +``` + +## Usage Patterns + +### Basic Usage + +1. **Create a Metal3DataTemplate** with your desired configuration +2. **Reference the template** in your Metal3MachineTemplate +3. **Create Metal3Machines** - CAPM3 automatically creates Metal3Data + instances +4. **Access the secrets** for provisioning BareMetalHosts + +### Manual Secret Creation + +If a `Metal3Machine` is created without a `dataTemplate` but with `metaData` or +`networkData` fields set, the controller will: + +- Look for existing secrets with the specified names +- Set the status fields accordingly +- Start BareMetalHost provisioning when secrets are available + +### Dynamic Secret Creation + +When a `Metal3Machine` references a `dataTemplate`: + +1. A `Metal3DataClaim` is created automatically +2. The claim controller creates a `Metal3Data` instance +3. The `Metal3Data` controller generates the required secrets +4. The `Metal3Machine` controller uses the secrets for provisioning + +### Hybrid Configuration + +You can mix template-based and manual configuration: + +- Set `dataTemplate` for one type of data (e.g., network data) +- Set `metaData` or `networkData` directly for the other type +- The manual configuration overrides the template for that specific secret diff --git a/docs/user-guide/src/capm3/metal3datatemplate.md b/docs/user-guide/src/capm3/metal3datatemplate.md new file mode 100644 index 00000000..103d20ab --- /dev/null +++ b/docs/user-guide/src/capm3/metal3datatemplate.md @@ -0,0 +1,321 @@ +# Metal3DataTemplate + +The `Metal3DataTemplate` resource defines templates for generating metadata and +network configuration for bare metal hosts in a Cluster API Provider Metal3 +(CAPM3) cluster. It serves as a blueprint for creating host-specific +configuration data that will be rendered into Kubernetes secrets. + +## Overview + +A `Metal3DataTemplate` contains: + +- **Metadata templates**: Templates for generating host-specific metadata +- **Network data templates**: Templates for generating network configuration +- **Index management**: Configuration for managing host indexes and naming + +The template is reconciled by its own controller, which adds labels pointing to +the `Metal3Cluster` that has nodes linking to this object. + +## API Reference + +```yaml +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: Metal3DataTemplate +metadata: + name: + namespace: + ownerReferences: + - apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + controller: true + kind: Metal3Cluster + name: +spec: + templateReference: # Optional + metaData: + # Metadata template configuration + networkData: + # Network data template configuration +status: + indexes: + "": "" + dataNames: + "": "" + lastUpdated: "" +``` + +## Metadata Specifications + +The `metaData` field contains templates that will render data in different ways. +The following types of objects are available: + +### Strings + +Renders the given string as a value in the metadata. + +```yaml +metaData: + strings: + - key: hostname + value: worker-node +``` + +### Object Names + +Renders the name of the object that matches the specified type. + +```yaml +metaData: + objectNames: + - key: machine_name + object: machine + - key: bmh_name + object: baremetalhost +``` + +### Indexes + +Renders the index of the current object with configurable offset and step. + +```yaml +metaData: + indexes: + - key: node_index + offset: 0 + step: 1 + prefix: "worker-" + suffix: "-node" +``` + +### IP Pool References + +Renders values from IP Pool objects defined in the +[IP Address Manager](https://github.com/metal3-io/ip-address-manager). + +```yaml +metaData: + ipAddressesFromIPPool: + - key: ip_address + name: pool-1 + prefixesFromIPPool: + - key: network_prefix + name: pool-1 + gatewaysFromIPPool: + - key: gateway + name: pool-1 + dnsServersFromIPPool: + - key: dns_servers + name: pool-1 +``` + +### Host Interface References + +Renders the MAC address of a BareMetalHost interface. + +```yaml +metaData: + fromHostInterfaces: + - key: primary_mac + interface: "eth0" +``` + +### Label and Annotation References + +Renders content from labels or annotations on objects. + +```yaml +metaData: + fromLabels: + - key: environment + object: machine + label: env + fromAnnotations: + - key: rack_location + object: machine + annotation: rack +``` + +## Network Data Specifications + +The `networkData` field defines network configuration templates that follow the +[Nova network_data.json format](https://docs.openstack.org/nova/latest/user/metadata.html#openstack-format-metadata). + +### Links Configuration + +Defines layer 2 interfaces including ethernets, bonds, and VLANs. + +```yaml +networkData: + links: + ethernets: + - type: "phy" + id: "enp1s0" + mtu: 1500 + macAddress: + fromAnnotation: + object: machine + annotation: primary-mac + bonds: + - id: "bond0" + mtu: 1500 + macAddress: + string: "XX:XX:XX:XX:XX:XX" + bondMode: "802.3ad" + bondLinks: + - enp1s0 + - enp2s0 + vlans: + - id: "vlan1" + mtu: 1500 + macAddress: + string: "YY:YY:YY:YY:YY:YY" + vlanID: 1 + vlanLink: bond0 +``` + +#### Ethernet Types + +- `bridge` +- `dvs` +- `hw_veb` +- `hyperv` +- `ovs` +- `tap` +- `vhostuser` +- `vif` +- `phy` + +#### Bond Modes + +- `802.3ad` +- `balance-rr` +- `active-backup` +- `balance-xor` +- `broadcast` +- `balance-tlb` +- `balance-alb` + +#### MAC Address Sources + +- `string`: Direct MAC address string +- `fromAnnotation`: MAC from object annotation +- `fromHostInterface`: MAC from BareMetalHost interface + +### Networks Configuration + +Defines layer 3 networks with various IP addressing schemes. + +```yaml +networkData: + networks: + ipv4DHCP: + - id: "provisioning" + link: "bond0" + ipv4: + - id: "Baremetal" + link: "vlan1" + ipAddressFromIPPool: pool-1 + routes: + - network: "0.0.0.0" + netmask: 0 + gateway: + fromIPPool: pool-1 + services: + dns: + - "8.8.4.4" + dnsFromIPPool: pool-1 + ipv6DHCP: + - id: "provisioning6" + link: "bond0" + ipv6SLAAC: + - id: "provisioning6slaac" + link: "bond0" + ipv6: + - id: "Baremetal6" + link: "vlan1" + ipAddressFromIPPool: pool6-1 + routes: + - network: "0::0" + netmask: 0 + gateway: + string: "2001:0db8:85a3::8a2e:0370:1" + services: + dns: + - "2001:4860:4860::8844" + dnsFromIPPool: pool6-1 +``` + +### Services Configuration + +Defines DNS services. + +```yaml +networkData: + services: + dns: + - "8.8.8.8" + - "2001:4860:4860::8888" + dnsFromIPPool: pool-1 +``` + +## Complete Example + +```yaml +apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 +kind: Metal3DataTemplate +metadata: + name: worker-template + namespace: default + ownerReferences: + - apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 + controller: true + kind: Metal3Cluster + name: my-cluster +spec: + templateReference: worker-v1 + metaData: + strings: + - key: hostname + value: worker + indexes: + - key: node_index + offset: 0 + step: 1 + prefix: "worker-" + suffix: "-node" + ipAddressesFromIPPool: + - key: ip + name: worker-pool + fromHostInterfaces: + - key: mac + interface: "eth0" + networkData: + links: + ethernets: + - type: "phy" + id: "eth0" + mtu: 1500 + macAddress: + fromHostInterface: "eth0" + networks: + ipv4: + - id: "provisioning" + link: "eth0" + ipAddressFromIPPool: worker-pool + routes: + - network: "0.0.0.0" + netmask: 0 + gateway: + fromIPPool: worker-pool + services: + dns: + - "8.8.8.8" + - "8.8.4.4" +status: + indexes: + "0": "worker-0" + "1": "worker-1" + dataNames: + "worker-0": worker-template-0 + "worker-1": worker-template-1 + lastUpdated: "2023-01-01T00:00:00Z" +```