fix(azure): Enable storage account and private endpoint reuse for AzureFileShare on re-runs #4154
Conversation
…reFileShare on re-runs
fix(azure): Enable storage account and private endpoint reuse for AzureFileShare on re-runs
Problem:
When LISA was re-run with preserved resource groups (using deploy:false),
the AzureFileShare feature would create new storage accounts and file shares
instead of reusing existing ones, causing resource tracking and cleanup issues.
Additionally, re-runs would fail with IndexError when private endpoints
already existed.
Changes:
1. features.py - _initialize_fileshare_information():
- Added logic to scan for existing 'lisasc*' storage accounts in the
resource group and reuse them instead of generating new random names
- Added import for get_storage_client
2. common.py - create_update_private_endpoints():
- Added check for existing private endpoints before attempting creation
- Added new helper function _get_private_endpoint_ip() that extracts
IP address using two methods:
a) From custom_dns_configs (original approach)
b) From network interface (fallback for existing endpoints)
- Fixes IndexError when private endpoint already exists but
custom_dns_configs is empty
3. storage.py - verify_cifs_basic():
- Fixed test to pass allow_shared_key_access=True since the test uses
key-based authentication (credentials stored in /etc/smbcredentials/)
- Added TODO comment to flag test for future validation with these changes
Tested:
- verify_cifs_basic: PASSED (fresh deployment)
- verify_azure_file_share: Requires validation with re-run scenario
Impact:
- Enables proper resource reuse when re-running LISA with preserved environments
- Prevents orphaned storage accounts on failed/retried test runs
- Fixes private endpoint IP retrieval for existing endpoints
There was a problem hiding this comment.
Pull request overview
This pull request enables resource reuse for AzureFileShare operations when LISA is re-run with preserved resource groups. The changes address issues where storage accounts and private endpoints were being recreated instead of reused, and fixes an IndexError when retrieving IP addresses from existing private endpoints.
Key changes include:
- Storage account reuse logic to scan for existing "lisasc*" accounts before creating new ones
- Private endpoint IP retrieval improvements with fallback methods and existence checking
- Test fix to explicitly pass allow_shared_key_access=True for key-based authentication
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| lisa/sut_orchestrator/azure/features.py | Adds storage account reuse logic by scanning existing accounts with "lisasc" prefix; adds get_storage_client import |
| lisa/sut_orchestrator/azure/common.py | Adds private endpoint existence check and _get_private_endpoint_ip helper with fallback IP retrieval methods (custom_dns_configs and network interface) |
| lisa/microsoft/testsuites/core/storage.py | Fixes test to pass allow_shared_key_access=True; adds TODO comment noting the test doesn't use private endpoints |
Key Test Cases:
verify_cifs_basic|verify_azure_file_share
Impacted LISA Features:
AzureFileShare, Nfs
Tested Azure Marketplace Images:
- canonical 0001-com-ubuntu-server-jammy 22_04-lts-gen2 latest
- debian debian-12 12-gen2 latest
- redhat rhel 95_gen2 latest
| self._storage_account_name = account.name | ||
| break | ||
| else: | ||
| random_str = generate_random_chars( | ||
| string.ascii_lowercase + string.digits, 10 | ||
| ) | ||
| self._storage_account_name = f"lisasc{random_str}" |
There was a problem hiding this comment.
The storage account reuse logic conflicts with the cleanup logic in delete_azure_fileshare (line 3863-3870), which always deletes the storage account. If one test/node reuses a storage account that another test/node is still using or might need, the delete operation will cause failures.
This creates a conflict between the goal of reusing storage accounts across re-runs and the cleanup behavior. Consider:
- Only deleting file shares, not the storage account, when reuse is enabled
- Implementing reference counting or a cleanup flag to determine when it's safe to delete the storage account
- Adding configuration to control whether storage accounts should be preserved for reuse
| self._storage_account_name = account.name | |
| break | |
| else: | |
| random_str = generate_random_chars( | |
| string.ascii_lowercase + string.digits, 10 | |
| ) | |
| self._storage_account_name = f"lisasc{random_str}" | |
| self._storage_account_name = account.name | |
| self._reuse_storage_account = True | |
| break | |
| else: | |
| random_str = generate_random_chars( | |
| string.ascii_lowercase + string.digits, 10 | |
| ) | |
| self._storage_account_name = f"lisasc{random_str}" | |
| self._reuse_storage_account = False |
| ) | ||
| for account in storage_accounts: | ||
| if account.name.startswith("lisasc"): | ||
| self._storage_account_name = account.name |
There was a problem hiding this comment.
if reuse the existing one, there will be some issues when storage account is deleted in delete_storage_account
There was a problem hiding this comment.
Good catch, I will add flags and logic to delete_storage_account to ensure resources (selective ) are correctly deleted only and only when created by LISA and keep_environment: no
Proposed input:
Deploy : False
Keep_environment: no
resource_group_name: Specified
Proposed Behavior:
If Storage account exists//Storage account has file shares other than what is created: Do not delete,
If file shares from parameters already exists: Do not delete
If Private endpoint already exists: Do not delete
Private DNS zone: No action
2 participants
fix(azure): Enable storage account and private endpoint reuse for AzureFileShare on re-runs
Problem:
When LISA was re-run with preserved resource groups (using deploy:false), the AzureFileShare feature would create new storage accounts and file shares instead of reusing existing ones, causing resource tracking and cleanup issues. Additionally, re-runs would fail with IndexError when private endpoints already existed.
Changes:
features.py - _initialize_fileshare_information():
common.py - create_update_private_endpoints():
storage.py - verify_cifs_basic():
Tested:
Impact: