[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #6865

Allenz5 · 2025-06-09T20:37:43Z

Description of changes

Extend AllocationStrategy to include capacity-optimized-prioritized and prioritized options
Extend instances_validators.py::InstancesAllocationStrategyValidator to verify that the new AllocationStrategy works correctly with the specified CapacityType

Tests

Extend test_instances_validators.py::test_instances_allocation_strategy_validator to include new test cases

References

Checklist

Make sure you are pointing to the right branch.
If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
Check all commits' messages are clear, describing what and why vs how.
Make sure to have added unit tests or integration tests to cover the new/modified code.
Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…d to AllocationStrategy configuration; Add new configuration of enable_single_availability_zone; Signed-off-by: Hanxuan Zhang <[email protected]>

… develop

…gy validator

… enable_single_availability_zone

…nfig.cluster_config to pcluster.config.common

…ingle_availability_zone

…fig.py to common.py

…ster_config.py to common.py" This reverts commit 13d8cfa.

…dator in cluster_config.py and add registration tests

…ility_zone from False to None

… to slurm.full_config.snapshot.yaml

cli/src/pcluster/config/cluster_config.py

cli/src/pcluster/schemas/cluster_schema.py

cli/src/pcluster/validators/instances_validators.py

...ster/templates/test_cluster_stack/test_cluster_config_limits/slurm.full_config.snapshot.yaml

cli/tests/pcluster/validators/test_all_validators.py

himani2411 · 2025-06-10T17:24:54Z

I would suggest to add a title to the PR too

… develop

…eValidator registration

…tionStrategyValidator

…tor registration test

…gration test

himani2411 · 2025-07-07T20:21:24Z

Update the PR description to maintain consistency on what has been removed from the PR

cli/src/pcluster/config/common.py

...working/test_cluster_networking/test_cluster_with_subnet_prioritization/pcluster.config.yaml

tests/integration-tests/tests/networking/test_cluster_networking.py

...working/test_cluster_networking/test_cluster_with_subnet_prioritization/pcluster.config.yaml

tests/integration-tests/tests/networking/test_cluster_networking.py

…ack to cluster_config.py

… develop

…ion test config file

cli/src/pcluster/validators/instances_validators.py

himani2411 · 2025-07-09T22:02:48Z

@Allenz5 in follow up PR please add the integration tests as part of our daily integration test set (develop.yaml)

…ioritized Allocation Strategy (aws#6865) * [Subnet Prioritization] Add prioritized|capacity-optimized-prioritized to AllocationStrategy configuration; Add new configuration of enable_single_availability_zone; Signed-off-by: Hanxuan Zhang <[email protected]> * [Subnet Prioritization] Add test cases for instance allocation strategy validator * [Subnet Prioritization] Update the default value and update policy of enable_single_availability_zone * [Subnet Prioritization] Move AllocationStrategy Enum from pcluster.config.cluster_config to pcluster.config.common * [Subnet Prioritization] Add validator and validator test for enable_single_availability_zone * [Subnet Prioritization] Move AllocationStrategy Enum from cluster_config.py to common.py * Revert "[Subnet Prioritization] Move AllocationStrategy Enum from cluster_config.py to common.py" This reverts commit 13d8cfa. * [Subnet Prioritization] Register enable_single_availability_zone_validator in cluster_config.py and add registration tests * [Subnet Prioritization] Change default value of enable_single_availability_zone from False to None * [Subnet Prioritization] Add enable_single_availability_zone parameter to slurm.full_config.snapshot.yaml * [Subnet Prioritization] Fix format issues * [Subnet Prioritization] Update CHANGELOG.md * [Subnet Prioritization] Remove duplicated AllocationStrategy Enum * [Subnet Prioritization] Remove duplicated EnableSingleAvailabilityZoneValidator registration * [Subnet Prioritization] Update the failure message of InstancesAllocationStrategyValidator * [Subnet Prioritization] Update enable_single_availability_zone_validator registration test * [Subnet Prioritization] Update format * [Subnet Prioritization] Fix EnableSingleAvailabilityZoneValidator * [Subnet Prioritization] Fix format issue * [Subnet Prioritization] Add integration test for subnet prioritization * [Subnet Prioritization] Update integration test for subnet prioritization * [Subnet Prioritization] Remove EnableSingleAvailabilityZone parameter from configuration * [Subnet Prioritization] Update Integration Test * [Subnet Prioritization] Fix format issues * [Subnet Prioritization] Remove EnableSingleAvailabilityZone from integration test * [Subnet Prioritization] Move AllocationStrategy Enum from common.py back to cluster_config.py * [Subnet Prioritization] Update integration test and format * [Subnet Prioritization] Remove custom hardcode settings from integration test config file * [Subnet Prioritization] Update format in instances_validators.py --------- Signed-off-by: Hanxuan Zhang <[email protected]>

gmarciani · 2025-07-14T16:02:06Z

CHANGELOG.md


 **CHANGES**
 - Ubuntu 20.04 is no longer supported.
+- Support prioritized and capacity-optimized-prioritized Allocation Strategy


Same as https://github.com/aws/aws-parallelcluster-node/pull/671/files#r2205253483

gmarciani · 2025-07-14T16:03:36Z

cli/src/pcluster/validators/instances_validators.py

    def _validate(self, compute_resource_name: str, capacity_type: Enum, allocation_strategy: Enum, **kwargs):
-        """On-demand Capacity type only supports "lowest-price" allocation strategy."""
+        """On-demand Capacity type only supports "lowest-price" and "prioritized" allocation strategy."""
+        valid_on_demand_allocation_strategy = {


[CodeStyle] This is a constant dictionary, let's defined it in a common script, ideally where the constants about strategies are defined. There is not reason to redefine it every time this function gets called.

gmarciani · 2025-07-14T16:05:08Z

cli/src/pcluster/validators/instances_validators.py

            )
+        if (
+            capacity_type == cluster_config.CapacityType.SPOT
+            and allocation_strategy == cluster_config.AllocationStrategy.PRIORITIZED


We could take the chance here to generalize the check the same way we did for on demand, i,.e,. define in a constant the valid strategies for spot and check against that constant.

gmarciani · 2025-07-14T16:05:53Z

cli/src/pcluster/validators/instances_validators.py

+            self._add_failure(
+                f"Compute Resource {compute_resource_name} is using a SPOT CapacityType but the "
+                f"Allocation Strategy specified is {allocation_strategy.value}. SPOT CapacityType can only use "
+                f"'{cluster_config.AllocationStrategy.LOWEST_PRICE.value}', "


Better to define the valid strategies for spot into a constant and concatenate those into the message

gmarciani · 2025-07-14T16:07:13Z

...dators/test_all_validators/test_slurm_validators_are_called_with_correct_argument/slurm.yaml

        - Name: compute_resource2
          InstanceType: c4.2xlarge
    - Name: queue2
+      AllocationStrategy: "prioritized"


nit: double quotes are not required in yaml

Allenz5 and others added 13 commits June 3, 2025 14:56

[Subnet Prioritization] Add prioritized|capacity-optimized-prioritize…

bcb3346

…d to AllocationStrategy configuration; Add new configuration of enable_single_availability_zone; Signed-off-by: Hanxuan Zhang <[email protected]>

Merge branch 'develop' of github.com:Allenz5/aws-parallelcluster into…

bd77a73

… develop

Merge branch 'aws:develop' into develop

96d1e01

[Subnet Prioritization] Add test cases for instance allocation strate…

1360560

…gy validator

[Subnet Prioritization] Update the default value and update policy of…

8cf10aa

… enable_single_availability_zone

[Subnet Prioritization] Move AllocationStrategy Enum from pcluster.co…

7f303dd

…nfig.cluster_config to pcluster.config.common

[Subnet Prioritization] Add validator and validator test for enable_s…

26ddfec

…ingle_availability_zone

[Subnet Prioritization] Move AllocationStrategy Enum from cluster_con…

13d8cfa

…fig.py to common.py

Revert "[Subnet Prioritization] Move AllocationStrategy Enum from clu…

f8de4a5

…ster_config.py to common.py" This reverts commit 13d8cfa.

[Subnet Prioritization] Register enable_single_availability_zone_vali…

c7889cd

…dator in cluster_config.py and add registration tests

[Subnet Prioritization] Change default value of enable_single_availab…

b6506cf

…ility_zone from False to None

[Subnet Prioritization] Add enable_single_availability_zone parameter…

a3b7672

… to slurm.full_config.snapshot.yaml

Merge branch 'aws:develop' into develop

33d3fd1

Allenz5 requested review from a team as code owners June 9, 2025 20:37

github-advanced-security bot found potential problems Jun 9, 2025

View reviewed changes

cli/src/pcluster/config/cluster_config.py Fixed Show fixed Hide fixed

himani2411 reviewed Jun 9, 2025

View reviewed changes

cli/src/pcluster/config/cluster_config.py Outdated Show resolved Hide resolved

himani2411 reviewed Jun 9, 2025

View reviewed changes

cli/src/pcluster/schemas/cluster_schema.py Outdated Show resolved Hide resolved

himani2411 reviewed Jun 9, 2025

View reviewed changes

cli/src/pcluster/validators/instances_validators.py Outdated Show resolved Hide resolved

himani2411 reviewed Jun 9, 2025

View reviewed changes

...ster/templates/test_cluster_stack/test_cluster_config_limits/slurm.full_config.snapshot.yaml Outdated Show resolved Hide resolved

himani2411 reviewed Jun 9, 2025

View reviewed changes

cli/tests/pcluster/validators/test_all_validators.py Outdated Show resolved Hide resolved

Allenz5 changed the title ~~[Subnet Prioritization]~~ [Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy Jun 11, 2025

Allenz5 added 7 commits June 11, 2025 15:55

[Subnet Prioritization] Fix format issues

406d5e8

Merge branch 'develop' of github.com:Allenz5/aws-parallelcluster into…

dee33ca

… develop

[Subnet Prioritization] Update CHANGELOG.md

f2a0c7e

[Subnet Prioritization] Remove duplicated AllocationStrategy Enum

2e94485

[Subnet Prioritization] Remove duplicated EnableSingleAvailabilityZon…

d2c2ba6

…eValidator registration

[Subnet Prioritization] Update the failure message of InstancesAlloca…

db7a9c5

…tionStrategyValidator

[Subnet Prioritization] Update enable_single_availability_zone_valida…

99773c4

…tor registration test

Allenz5 and others added 4 commits July 7, 2025 14:58

[Subnet Prioritization] Update Integration Test

ca4f2e7

[Subnet Prioritization] Fix format issues

c033811

[Subnet Prioritization] Remove EnableSingleAvailabilityZone from inte…

48ccdd0

…gration test

Merge branch 'develop' into develop

1f37961