Introduce volume allocation algorithm global configuration #10696

sudo87 · 2025-04-11T05:21:25Z

Description

This PR introduces separate global configuration for Volume allocation algorithm.

Volume allocation has been using vm.allocation.algorithm config value. It can be confusing and limitting user to use same allocation algorithm for both VM and Volume.
This new configuration will allow vm and volume allocators to use different algoritms independently without being tied together.

Doc PR apache/cloudstack-documentation#499

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Local dev testing is done by deployment of instances against simulator after setting different values for "volume.allocation.algorithm".

How did you try to break this feature and the system with this change?

Volume allocation has been using vm.allocation.algorithm config value. This change will allow them to be configured in isolation.

codecov · 2025-04-11T05:29:18Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 3.99%. Comparing base (6fdaf51) to head (f396fa9).
Report is 62 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (6fdaf51) and HEAD (f396fa9). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (6fdaf51) HEAD (f396fa9)

unittests 1 0

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #10696       +/-   ##
=============================================
- Coverage     16.41%    3.99%   -12.42%     
=============================================
  Files          5702      401     -5301     
  Lines        503405    32654   -470751     
  Branches      60976     5791    -55185     
=============================================
- Hits          82626     1306    -81320     
+ Misses       411594    31198   -380396     
+ Partials       9185      150     -9035

Flag	Coverage Δ
uitests	`3.99% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

rajujith · 2025-04-11T05:33:13Z

@blueorangutan package

blueorangutan · 2025-04-11T05:34:03Z

@rajujith a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-11T06:40:26Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13019

DaanHoogland

clgtm

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

.../src/main/java/org/apache/cloudstack/storage/allocator/ClusterScopeStoragePoolAllocator.java

DaanHoogland · 2025-04-11T09:44:22Z

@blueorangutan package

blueorangutan · 2025-04-11T09:46:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

api/src/main/java/com/cloud/deploy/DeploymentClusterPlanner.java

blueorangutan · 2025-04-11T10:54:30Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13020

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java

sudo87 · 2025-04-14T12:10:24Z

@blueorangutan package

blueorangutan · 2025-04-14T12:12:04Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-14T13:38:49Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13046

sudo87 · 2025-04-15T03:37:00Z

@blueorangutan test

blueorangutan · 2025-04-15T03:38:04Z

@sudo87 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

weizhouapache · 2025-04-15T13:09:08Z

code lgtm

However, I think we need to think of the upgrade
for example, now the vm/volume allocation algorithm is set to firstfit, when uprgade to a new version with this PR, the vm allocation algorithm is still firstfit, but volume allocation algorithm will be the default value random.

@sudo87

blueorangutan · 2025-04-15T19:23:19Z

[SF] Trillian test result (tid-12988)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54138 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10696-t12988-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2025-05-12T07:19:26Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13338

borisstoyanov

LGTM, testing has passed.

DaanHoogland · 2025-05-12T09:42:59Z

@blueorangutan test

blueorangutan · 2025-05-12T09:44:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

github-actions · 2025-05-12T15:48:05Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

blueorangutan · 2025-05-13T02:14:51Z

[SF] Trillian test result (tid-13274)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 56331 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10696-t13274-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

github-actions · 2025-05-13T04:45:27Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

rohityadavcloud · 2025-05-14T07:28:19Z

@blueorangutan package

blueorangutan · 2025-05-14T07:30:03Z

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-05-14T08:56:45Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13377

sudo87 · 2025-05-15T06:39:07Z

@blueorangutan test

blueorangutan · 2025-05-15T06:40:03Z

@sudo87 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-05-15T23:11:44Z

[SF] Trillian test result (tid-13314)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 56564 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10696-t13314-kvm-ol8.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_deployVMInSharedNetwork	`Failure`	72.14	test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown	`Error`	73.23	test_network.py

)

Introduce volume allocation algorithm global configuration.

b28db5d

Volume allocation has been using vm.allocation.algorithm config value. This change will allow them to be configured in isolation.

boring-cyborg bot added component:api component:orchestration component:storage labels Apr 11, 2025

sudo87 requested review from shwstppr and harikrishna-patnala April 11, 2025 05:22

rajujith self-requested a review April 11, 2025 05:32

DaanHoogland approved these changes Apr 11, 2025

View reviewed changes

sudo87 added 2 commits April 11, 2025 14:44

Address review comments

f6e7fa6

format import statements

9c3d381

harikrishna-patnala reviewed Apr 11, 2025

View reviewed changes

api/src/main/java/com/cloud/deploy/DeploymentClusterPlanner.java Show resolved Hide resolved

Pearl1594 added this to the 4.21.0 milestone Apr 11, 2025

make config dynamic in nature to avoid need of mgmt server restart

33f9e6f

DaanHoogland reviewed Apr 14, 2025

View reviewed changes

...rage/src/main/java/org/apache/cloudstack/storage/allocator/AbstractStoragePoolAllocator.java Outdated Show resolved Hide resolved

Revert method signature change done for unit test

c3006bf

Add upgrade patch for volume.allocation.algorithm

5c10036

borisstoyanov approved these changes May 12, 2025

View reviewed changes

github-actions bot added the status:has-conflicts label May 12, 2025

boring-cyborg bot added component:marvin Python Warning... Python code Ahead! labels May 13, 2025

github-actions bot removed the status:has-conflicts label May 13, 2025

sudo87 force-pushed the volumeAllocator branch from 58c893d to 958b066 Compare May 13, 2025 04:45

boring-cyborg bot added component:agent component:integration labels May 13, 2025

github-actions bot added the status:has-conflicts label May 13, 2025

resolve merge conflict with main

f396fa9

sudo87 force-pushed the volumeAllocator branch from 958b066 to f396fa9 Compare May 13, 2025 04:59

github-actions bot removed the status:has-conflicts label May 13, 2025

DaanHoogland merged commit d5ba23c into apache:main May 16, 2025
21 of 26 checks passed

DaanHoogland deleted the volumeAllocator branch May 16, 2025 12:06

sureshanaparti added the type:improvement label Jun 4, 2025

sureshanaparti added this to Apache CloudStack 4.21.0 Jun 5, 2025

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Jun 19, 2025

Introduce volume allocation algorithm global configuration (apache#10696

55aa2b5

)

sureshanaparti moved this to Done in Apache CloudStack 4.21.0 Jul 14, 2025

Introduce volume allocation algorithm global configuration #10696

Introduce volume allocation algorithm global configuration #10696

Uh oh!

Conversation

sudo87 commented Apr 11, 2025 • edited by rohityadavcloud Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rajujith commented Apr 11, 2025

Uh oh!

blueorangutan commented Apr 11, 2025

Uh oh!

blueorangutan commented Apr 11, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DaanHoogland commented Apr 11, 2025

Uh oh!

blueorangutan commented Apr 11, 2025

Uh oh!

Uh oh!

blueorangutan commented Apr 11, 2025

Uh oh!

Uh oh!

sudo87 commented Apr 14, 2025

Uh oh!

blueorangutan commented Apr 14, 2025

Uh oh!

blueorangutan commented Apr 14, 2025

Uh oh!

sudo87 commented Apr 15, 2025

Uh oh!

blueorangutan commented Apr 15, 2025

Uh oh!

weizhouapache commented Apr 15, 2025

Uh oh!

blueorangutan commented Apr 15, 2025

Uh oh!

blueorangutan commented May 12, 2025

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented May 12, 2025

Uh oh!

blueorangutan commented May 12, 2025

Uh oh!

github-actions bot commented May 12, 2025

Uh oh!

blueorangutan commented May 13, 2025

Uh oh!

github-actions bot commented May 13, 2025

Uh oh!

rohityadavcloud commented May 14, 2025

Uh oh!

blueorangutan commented May 14, 2025

Uh oh!

blueorangutan commented May 14, 2025

Uh oh!

sudo87 commented May 15, 2025

Uh oh!

blueorangutan commented May 15, 2025

Uh oh!

blueorangutan commented May 15, 2025

sudo87 commented Apr 11, 2025 •

edited by rohityadavcloud

Loading

codecov bot commented Apr 11, 2025 •

edited

Loading