Skip to content

Commit 3b167c0

Browse files
rebtoorrebtoor
committed
[cleanup_openstack] enhance cleanup for infrastructure reuse
Enhance the cleanup_openstack role to support infrastructure reuse by cleaning up OpenStack resources while preserving the OpenShift cluster infrastructure. This enables faster test cycles by avoiding full infrastructure reprovisioning. Key improvements: - Add OpenStack API resource cleanup (servers, networks, volumes, etc.) before CR deletion to prevent orphaned resources - Add direct CR deletion from cluster for comprehensive cleanup - Add storage cleanup (PVCs, secrets, ConfigMaps, PVs) - Add optional namespace cleanup - Preserve infrastructure operators (NMState, MetalLB, OLM) for reuse - Make role self-contained with all required defaults included New playbooks: - cleanup-openstack-for-reuse.yml: Main playbook for infrastructure reuse - ci/playbooks/clean-config-drives-only.yml: Clean config drive ISOs New task files: - cleanup_crs_direct.yaml: Delete OpenStack CRs directly from cluster - cleanup_openstack_api.yaml: Delete OpenStack API resources - cleanup_storage.yaml: Clean up PVCs, secrets, ConfigMaps, and PVs - cleanup_namespaces.yaml: Optionally delete empty namespaces - common.yaml: Common variables to eliminate code duplication Role improvements: - Include all required defaults from kustomize_deploy and deploy_bmh roles to make cleanup_openstack self-contained and usable from any playbook location without cross-role dependencies - Fix execution order: API resources deleted first while control plane is running, then CRs are deleted - Refactor CR deletion patterns using loops to reduce duplication - Add configurable variables for granular cleanup control - Update README with comprehensive documentation - Update dictionary with new terms Related: OSPRH-21759 Signed-off-by: Roberto Alfieri <[email protected]>
1 parent ff8eb72 commit 3b167c0

File tree

13 files changed

+941
-24
lines changed

13 files changed

+941
-24
lines changed

‎ci/playbooks/clean-config-drives-only.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
# This playbook cleans only config drive ISO files to allow infrastructure reuse.
18+
# It does NOT clean libvirt VMs or other resources - only config drives.
19+
# This is needed when reusing infrastructure to avoid conflicts with existing
20+
# ISO files that might be attached to VMs.
21+
22+
- name: Clean config drives for infrastructure reuse
23+
hosts: "{{ cifmw_target_host | default('localhost') }}"
24+
gather_facts: true
25+
tasks:
26+
- name: Cleanup config_drive workdir
27+
ansible.builtin.include_role:
28+
name: config_drive
29+
tasks_from: cleanup.yml
30+
31+
- name: Remove ISO files from workload directory
32+
when: cifmw_libvirt_manager_basedir is defined
33+
ansible.builtin.find:
34+
paths: "{{ cifmw_libvirt_manager_basedir }}/workload"
35+
patterns: "*.iso"
36+
register: _iso_files
37+
failed_when: false
38+
39+
- name: Delete ISO files from workload directory
40+
when:
41+
- cifmw_libvirt_manager_basedir is defined
42+
- _iso_files.files | default([]) | length > 0
43+
ansible.builtin.file:
44+
path: "{{ item.path }}"
45+
state: absent
46+
loop: "{{ _iso_files.files | default([]) }}"
47+
failed_when: false
48+
# Note: This may fail if ISO is attached to a running VM, but that's okay
49+
# The config_drive role will handle the case where ISO doesn't exist

‎cleanup-openstack-for-reuse.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
# This playbook cleans up OpenStack resources while preserving the OpenShift
18+
# cluster infrastructure for reuse. It removes:
19+
# - All OpenStack CRs (ControlPlane, DataPlane, etc.)
20+
# - Storage resources (PVCs, secrets, ConfigMaps)
21+
# - Optionally: OpenStack API resources (servers, networks, volumes, etc.)
22+
#
23+
# Usage examples:
24+
#
25+
# Basic cleanup (removes OpenStack CRs and storage, keeps cluster):
26+
# ansible-playbook -i inventory.yml cleanup-openstack-for-reuse.yml
27+
#
28+
# Skip API resource cleanup (if needed):
29+
# ansible-playbook -i inventory.yml cleanup-openstack-for-reuse.yml \
30+
# -e cleanup_api_resources=false
31+
#
32+
# Aggressive cleanup (removes everything including namespaces):
33+
# ansible-playbook -i inventory.yml cleanup-openstack-for-reuse.yml \
34+
# -e cleanup_api_resources=true \
35+
# -e cleanup_namespaces=true \
36+
# -e force_remove_finalizers=true
37+
38+
- name: Clean OpenStack deployment for infrastructure reuse
39+
hosts: "{{ target_host | default('localhost') }}"
40+
gather_facts: true
41+
vars:
42+
# By default, clean OpenStack CRs, storage, and API resources but keep OpenShift cluster
43+
# Set to false to skip OpenStack API resource cleanup
44+
cifmw_cleanup_openstack_delete_api_resources: "{{ cleanup_api_resources | default(true) }}"
45+
# Set to true to delete namespaces (use with caution)
46+
cifmw_cleanup_openstack_delete_namespaces: "{{ cleanup_namespaces | default(false) }}"
47+
# Set to true to force remove finalizers from stuck CRs
48+
cifmw_cleanup_openstack_force_remove_finalizers: "{{ force_remove_finalizers | default(false) }}"
49+
tasks:
50+
- name: Cleanup OpenStack deployment
51+
ansible.builtin.include_role:
52+
name: cleanup_openstack
53+
54+
- name: Display cleanup summary
55+
ansible.builtin.debug:
56+
msg: >-
57+
OpenStack cleanup completed. The OpenShift cluster is now ready for reuse.
58+
59+
Cleaned resources:
60+
- OpenStack CRs (ControlPlane, DataPlane, etc.)
61+
- Storage resources (PVCs, secrets, ConfigMaps)
62+
- OpenStack API resources (servers, networks, volumes, etc.)
63+
- Artifacts and logs
64+
{% if cifmw_cleanup_openstack_delete_namespaces %}
65+
- OpenStack namespaces (if empty)
66+
{% endif %}
67+
68+
The cluster infrastructure is preserved and ready for a new deployment.

‎docs/dictionary/en-custom.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ buildah
5555
buildpkgs
5656
cacert
5757
cacheable
58+
certmanager
5859
catalogsource
5960
cci
6061
ccitredhat
@@ -138,6 +139,7 @@ deepscrub
138139
delorean
139140
deployer
140141
deprovision
142+
deprovisioned
141143
deps
142144
dest
143145
dev
@@ -185,6 +187,7 @@ extraRPMs
185187
ezzmy
186188
favorit
187189
fbqufbqkfbzxrja
190+
finalizers
188191
fci
189192
fdp
190193
fedoraproject
@@ -299,6 +302,7 @@ kvm
299302
lacp
300303
lajly
301304
LDAP
305+
Lifecycle
302306
ldp
303307
libguestfs
304308
libvirt
@@ -415,8 +419,12 @@ openstack
415419
openstackclient
416420
openstackcontrolplane
417421
openstackdataplane
422+
openstackdataplanedeployment
423+
OpenStackDataPlaneDeployment
418424
openstackdataplanenodeset
419425
openstackdataplanenodesets
426+
openstackdataplaneservice
427+
OpenStackDataPlaneService
420428
openstackprovisioner
421429
openstacksdk
422430
openstackversion
@@ -443,6 +451,8 @@ passwd
443451
passwordless
444452
pastebin
445453
pem
454+
persistentvolumes
455+
PersistentVolumes
446456
pkgs
447457
pki
448458
png
@@ -468,6 +478,7 @@ pubkey
468478
publicdomain
469479
pullsecret
470480
pvs
481+
PVCs
471482
pwd
472483
pxe
473484
py
@@ -491,6 +502,7 @@ readmes
491502
readthedocs
492503
reauthenticate
493504
rebaser
505+
reusability
494506
redfish
495507
redhat
496508
refspec

‎roles/cleanup_openstack/README.md‎

Lines changed: 106 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,115 @@
11
# cleanup_openstack
22

3-
Cleans up openstack resources created by CIFMW by deleting CRs
3+
Cleans up OpenStack resources created by CIFMW while preserving the OpenShift cluster infrastructure for reuse. This role removes OpenStack-specific resources (CRs, API resources, storage) but keeps infrastructure operators and cluster components intact.
44

55
## Privilege escalation
66
None
77

88
## Parameters
9-
As this role is for cleanup it utilizes default vars from other roles which can be referenced at their role readme page: kustomize_deploy, deploy_bmh
9+
10+
### Cleanup Behavior
1011

1112
* `cifmw_cleanup_openstack_detach_bmh`: (Boolean) Detach BMH when cleaning flag, this is used to avoid deprovision when is not required. Default: `true`
13+
14+
* `cifmw_cleanup_openstack_delete_crs_direct`: (Boolean) Delete OpenStack CRs directly from cluster (not just from files). This ensures all OpenStackControlPlane, OpenStackDataPlaneDeployment, OpenStackDataPlaneNodeSet, and other CRs are removed. Default: `true`
15+
16+
* `cifmw_cleanup_openstack_delete_api_resources`: (Boolean) Delete OpenStack API resources (servers, networks, volumes, flavors, security groups, etc.) using the OpenStack client. This requires either an openstackclient pod in the cluster or openstackclient installed locally. Default: `true`
17+
18+
* `cifmw_cleanup_openstack_delete_storage`: (Boolean) Delete PVCs, secrets, ConfigMaps, and release PersistentVolumes. Default: `true`
19+
20+
* `cifmw_cleanup_openstack_delete_namespaces`: (Boolean) Delete OpenStack namespaces if they are empty. Use with caution as this will remove the namespace entirely. Default: `false`
21+
22+
* `cifmw_cleanup_openstack_force_remove_finalizers`: (Boolean) Force remove finalizers from stuck OpenStackControlPlane CRs. Use only if CRs are stuck in terminating state. Default: `false`
23+
24+
* `cifmw_cleanup_openstack_cloud_name`: (String) OpenStack cloud name to use for API cleanup. Default: `default`
25+
26+
### Path Configuration
27+
28+
The role includes default values for paths used by the `kustomize_deploy` and `deploy_bmh` roles. These can be overridden if needed:
29+
30+
* `cifmw_kustomize_deploy_basedir`: Base directory for kustomize deployment artifacts. Default: `{{ cifmw_basedir | default(ansible_user_dir ~ '/ci-framework-data') }}`
31+
32+
* `cifmw_kustomize_deploy_kustomizations_dest_dir`: Directory containing kustomization files. Default: `{{ cifmw_kustomize_deploy_basedir }}/artifacts/kustomize_deploy`
33+
34+
* `cifmw_kustomize_deploy_namespace`: OpenStack namespace. Default: `openstack`
35+
36+
* `cifmw_kustomize_deploy_operators_namespace`: OpenStack operators namespace. Default: `openstack-operators`
37+
38+
* `cifmw_deploy_bmh_basedir`: Base directory for BMH artifacts. Default: `{{ cifmw_basedir | default(ansible_user_dir ~ '/ci-framework-data') }}`
39+
40+
* `cifmw_deploy_bmh_dest_dir`: Directory containing BMH CRs. Default: `{{ cifmw_deploy_bmh_basedir }}/artifacts/deploy_bmh`
41+
42+
* `cifmw_deploy_bmh_namespace`: Namespace for BaremetalHost resources. Default: `openshift-machine-api`
43+
44+
### OpenShift Cluster Access
45+
46+
* `cifmw_openshift_kubeconfig`: (String) Path to kubeconfig file. Optional - will be inherited from `openshift_login` role if available, or defaults to `~/.kube/config`.
47+
48+
* `cifmw_openshift_token`: (String) OpenShift API token. Optional.
49+
50+
* `cifmw_openshift_context`: (String) OpenShift context to use. Optional.
51+
52+
### Architecture Variables (Optional)
53+
54+
These are only needed if you deployed using architecture-based automation. If not provided, cleanup will skip architecture-specific tasks:
55+
56+
* `cifmw_architecture_repo`: (String) Path to architecture repository. Optional.
57+
58+
* `cifmw_architecture_scenario`: (String) Scenario name used during deployment. Optional.
59+
60+
* `cifmw_architecture_automation_file`: (String) Direct path to automation YAML file. Optional - overrides repo+scenario.
61+
62+
**Note**: This role is self-contained and does not require the `kustomize_deploy`, `deploy_bmh`, or `openshift_login` roles to be present. All necessary default values are included in this role's `defaults/main.yaml`. Architecture variables are optional and only needed for architecture-based deployments.
63+
64+
## What gets cleaned up
65+
66+
### Always cleaned (when enabled):
67+
- OpenStack CRs (OpenStackControlPlane, OpenStackDataPlaneDeployment, OpenStackDataPlaneNodeSet, OpenStackDataPlaneService, OpenStackClient, OpenStackVersion)
68+
- Bare Metal Hosts (detached, not deprovisioned)
69+
- OpenStack deployment CRs from kustomize files
70+
- OpenStack API resources (servers, networks, volumes, flavors, security groups, etc.)
71+
- PVCs, secrets, ConfigMaps in OpenStack namespace
72+
- PersistentVolumes in Released state
73+
- Certificates and Issuers (cert-manager)
74+
- Artifacts, logs, and test directories
75+
76+
### Optionally cleaned:
77+
- Namespaces (if empty)
78+
79+
## What is preserved
80+
81+
The following infrastructure components are **NOT** deleted to preserve cluster reusability:
82+
- NMState operator (network management)
83+
- MetalLB operator (load balancing)
84+
- OLM (Operator Lifecycle Manager)
85+
- OpenShift cluster operators
86+
- Cluster-level infrastructure resources
87+
88+
## Usage
89+
90+
Basic cleanup (removes OpenStack CRs and storage, keeps OpenShift cluster):
91+
```yaml
92+
- name: Cleanup OpenStack
93+
include_role:
94+
name: cleanup_openstack
95+
```
96+
97+
Disable API resource cleanup (if needed):
98+
```yaml
99+
- name: Cleanup OpenStack without API resources
100+
include_role:
101+
name: cleanup_openstack
102+
vars:
103+
cifmw_cleanup_openstack_delete_api_resources: false
104+
```
105+
106+
Aggressive cleanup (removes everything including namespaces):
107+
```yaml
108+
- name: Aggressive cleanup
109+
include_role:
110+
name: cleanup_openstack
111+
vars:
112+
cifmw_cleanup_openstack_delete_api_resources: true
113+
cifmw_cleanup_openstack_delete_namespaces: true
114+
cifmw_cleanup_openstack_force_remove_finalizers: true
115+
```

‎roles/cleanup_openstack/defaults/main.yaml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,76 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
# Cleanup behavior flags
118
cifmw_cleanup_openstack_detach_bmh: true
19+
# Delete OpenStack CRs directly from cluster (not just from files)
20+
cifmw_cleanup_openstack_delete_crs_direct: true
21+
# Delete OpenStack API resources (servers, networks, volumes, etc.)
22+
cifmw_cleanup_openstack_delete_api_resources: true
23+
# Delete PVCs, secrets, and storage resources
24+
cifmw_cleanup_openstack_delete_storage: true
25+
# Delete namespaces if empty (use with caution)
26+
cifmw_cleanup_openstack_delete_namespaces: false
27+
# Force remove finalizers from stuck CRs
28+
cifmw_cleanup_openstack_force_remove_finalizers: false
29+
# OpenStack cloud name for API cleanup
30+
cifmw_cleanup_openstack_cloud_name: default
31+
32+
# Variables from kustomize_deploy role
33+
# These are needed for cleanup to locate deployment artifacts and namespaces
34+
cifmw_kustomize_deploy_basedir: >-
35+
{{
36+
cifmw_basedir | default(ansible_user_dir ~ '/ci-framework-data')
37+
}}
38+
39+
cifmw_kustomize_deploy_kustomizations_dest_dir: >-
40+
{{
41+
[
42+
cifmw_kustomize_deploy_basedir,
43+
'artifacts',
44+
'kustomize_deploy'
45+
] | path_join
46+
}}
47+
48+
cifmw_kustomize_deploy_namespace: openstack
49+
cifmw_kustomize_deploy_operators_namespace: openstack-operators
50+
51+
# Variables from deploy_bmh role
52+
# These are needed for cleanup to locate and detach baremetal hosts
53+
cifmw_deploy_bmh_basedir: >-
54+
{{
55+
cifmw_basedir | default(ansible_user_dir ~ '/ci-framework-data')
56+
}}
57+
58+
cifmw_deploy_bmh_dest_dir: >-
59+
{{
60+
[
61+
cifmw_deploy_bmh_basedir,
62+
'artifacts',
63+
'deploy_bmh'
64+
] | path_join
65+
}}
66+
67+
cifmw_deploy_bmh_namespace: openshift-machine-api
68+
69+
# Variables for OpenShift cluster access
70+
# Default kubeconfig path (can be overridden by cifmw_openshift_kubeconfig or cifmw_openshift_login_kubeconfig)
71+
cifmw_cleanup_openstack_kubeconfig_default: "{{ ansible_user_dir }}/.kube/config"
72+
73+
# Architecture variables (optional - if not provided, cleanup will skip architecture-specific tasks)
74+
# cifmw_architecture_repo: Path to architecture repository (if using architecture-based deployment)
75+
# cifmw_architecture_scenario: Scenario name (if using architecture-based deployment)
76+
# cifmw_architecture_automation_file: Direct path to automation file (overrides repo+scenario)

‎roles/cleanup_openstack/tasks/cleanup_crs.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
- name: Cleaning operators resources
99
kubernetes.core.k8s:
10-
kubeconfig: "{{ cifmw_openshift_kubeconfig }}"
10+
kubeconfig: "{{ _k8s_kubeconfig }}"
1111
api_key: "{{ cifmw_openshift_token | default(omit) }}"
1212
context: "{{ cifmw_openshift_context | default(omit) }}"
1313
state: absent

0 commit comments

Comments
 (0)