Skip to content

Add config keys for controlling public/private template secondary storage replica counts #12877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Damans227 wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Add config keys for controlling public/private template secondary storage replica counts #12877

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
40b9e4d
Add config keys for controlling public/private template secondary sto…
Mar 23, 2026
f997444
Allow configurable secondary storage replica count for public and pri…
Mar 23, 2026
0708236
Add unit tests
Mar 23, 2026
e5bfeab
Refactor secondary storage replica limit handling to use zone-specifi…
Damans227 Apr 15, 2026
6dbeb52
Refactor as per pr comments
Damans227 Apr 15, 2026
bbc0560
Implement secondary storage copy limit check for templates based on z…
Damans227 Apr 29, 2026
3b69432
Enhance secondary storage copy limit check to consider image stores b…
Damans227 Apr 29, 2026
c60d927
Improve logging for template store cleanup by adding debug informatio…
Damans227 Apr 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions engine/components-api/src/main/java/com/cloud/template/TemplateManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@
public interface TemplateManager {
static final String AllowPublicUserTemplatesCK = "allow.public.user.templates";
static final String TemplatePreloaderPoolSizeCK = "template.preloader.pool.size";
static final String PublicTemplateSecStorageCopyCK = "secstorage.public.template.copy.max";
static final String PrivateTemplateSecStorageCopyCK = "secstorage.private.template.copy.max";

static final ConfigKey<Boolean> AllowPublicUserTemplates = new ConfigKey<Boolean>("Advanced", Boolean.class, AllowPublicUserTemplatesCK, "true",
"If false, users will not be able to create public Templates.", true, ConfigKey.Scope.Account);
Expand All @@ -64,6 +66,18 @@ public interface TemplateManager {
true,
ConfigKey.Scope.Global);

ConfigKey<Integer> PublicTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
PublicTemplateSecStorageCopyCK, "0",
"Maximum number of secondary storage pools to which a public template is copied. " +
"0 means copy to all secondary storage pools (default behavior).",
true, ConfigKey.Scope.Zone);

ConfigKey<Integer> PrivateTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
PrivateTemplateSecStorageCopyCK, "1",
"Maximum number of secondary storage pools to which a private template is copied. " +
"Default is 1 to preserve existing behavior.",
true, ConfigKey.Scope.Zone);
Comment on lines +69 to +79
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config key descriptions don’t mention that the limits are evaluated per-zone (and the keys themselves are zone-scoped). Consider clarifying the wording (e.g., “per zone”) so operators understand how replica limits are applied across multiple zones/image stores.

Copilot uses AI. Check for mistakes.

static final String VMWARE_TOOLS_ISO = "vmware-tools.iso";
static final String XS_TOOLS_ISO = "xs-tools.iso";

Expand Down
47 changes: 45 additions & 2 deletions .../storage/image/src/main/java/org/apache/cloudstack/storage/image/TemplateServiceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,40 @@ public void handleSysTemplateDownload(HypervisorType hostHyper, Long dcId) {
}
}

private boolean hasReachedSecStorageCopyLimit(VMTemplateVO template, long zoneId) {
boolean isPrivate = !template.isPublicTemplate() && !template.isFeatured()
&& !TemplateType.SYSTEM.equals(template.getTemplateType());
int copyLimit = isPrivate
? TemplateManager.PrivateTemplateSecStorageCopy.valueIn(zoneId)
: TemplateManager.PublicTemplateSecStorageCopy.valueIn(zoneId);
if (copyLimit <= 0) {
return false;
}
List<DataStore> stores = _storeMgr.getImageStoresByScope(new ZoneScope(zoneId));
if (stores == null || stores.isEmpty()) {
return false;
}
int count = 0;
for (DataStore ds : stores) {
List<TemplateDataStoreVO> rows = _vmTemplateStoreDao.listByTemplateStore(template.getId(), ds.getId());
if (rows == null) {
continue;
}
for (TemplateDataStoreVO row : rows) {
State st = row.getState();
Status ds_state = row.getDownloadState();
if (st != State.Failed && st != State.Destroyed
&& ds_state != Status.ABANDONED && ds_state != Status.DOWNLOAD_ERROR) {
count++;
break;
}
}
}
logger.debug("Template [{}] secstorage copy check in zone [{}]: count={}, limit={}",
template.getUniqueName(), zoneId, count, copyLimit);
return count >= copyLimit;
}

protected boolean shouldDownloadTemplateToStore(VMTemplateVO template, DataStore store) {
Long zoneId = store.getScope().getScopeId();
DataStore directedStore = _tmpltMgr.verifyHeuristicRulesForZone(template, zoneId);
Expand All @@ -304,6 +338,12 @@ protected boolean shouldDownloadTemplateToStore(VMTemplateVO template, DataStore
return false;
}

if (zoneId != null && hasReachedSecStorageCopyLimit(template, zoneId)) {
logger.info("Skipping sync of template [{}] to image store [{}]: zone [{}] has reached the configured copy limit.",
template.getUniqueName(), store.getName(), zoneId);
return false;
}

if (template.isPublicTemplate()) {
logger.debug("Download of template [{}] to image store [{}] cannot be skipped, as it is public.", template.getUniqueName(),
store.getName());
Expand Down Expand Up @@ -531,10 +571,13 @@ public void handleTemplateSync(DataStore store) {
&& tmpltStore.getState() == State.Ready
&& tmpltStore.getInstallPath() == null) {
logger.info("Keep fake entry in template store table for migration of previous NFS to object store");
} else {
} else if (tmpltStore.getDownloadState() == VMTemplateStorageResourceAssoc.Status.DOWNLOADED
|| tmpltStore.getState() == State.Ready) {
logger.info("Removing leftover template {} entry from template store table", tmplt);
// remove those leftover entries
_vmTemplateStoreDao.remove(tmpltStore.getId());
} else {
logger.debug("Template {} entry on store {} is in pre-download state ({}/{}); not treating as leftover.",
tmplt, store, tmpltStore.getState(), tmpltStore.getDownloadState());
}
}
}
Expand Down
19 changes: 10 additions & 9 deletions server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@
import java.util.ArrayList;
import java.util.Collections;
import java.util.Date;
import java.util.HashSet;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Set;
import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.stream.Collectors;

Expand Down Expand Up @@ -292,9 +292,10 @@ protected void createTemplateWithinZones(TemplateProfile profile, VMTemplateVO t

if (imageStore == null) {
List<DataStore> imageStores = getImageStoresThrowsExceptionIfNotFound(zoneId, profile);
standardImageStoreAllocation(imageStores, template);
standardImageStoreAllocation(imageStores, template, zoneId);
} else {
validateSecondaryStorageAndCreateTemplate(List.of(imageStore), template, null);
int copyLimit = getSecStorageCopyLimit(template, zoneId);
validateSecondaryStorageAndCreateTemplate(List.of(imageStore), template, new HashMap<>(), copyLimit);
}
}
}
Expand All @@ -307,17 +308,17 @@ protected List<DataStore> getImageStoresThrowsExceptionIfNotFound(long zoneId, T
return imageStores;
}

protected void standardImageStoreAllocation(List<DataStore> imageStores, VMTemplateVO template) {
Set<Long> zoneSet = new HashSet<Long>();
protected void standardImageStoreAllocation(List<DataStore> imageStores, VMTemplateVO template, long zoneId) {
int copyLimit = getSecStorageCopyLimit(template, zoneId);
Collections.shuffle(imageStores);
validateSecondaryStorageAndCreateTemplate(imageStores, template, zoneSet);
validateSecondaryStorageAndCreateTemplate(imageStores, template, new HashMap<>(), copyLimit);
}

protected void validateSecondaryStorageAndCreateTemplate(List<DataStore> imageStores, VMTemplateVO template, Set<Long> zoneSet) {
protected void validateSecondaryStorageAndCreateTemplate(List<DataStore> imageStores, VMTemplateVO template, Map<Long, Integer> zoneCopyCount, int copyLimit) {
for (DataStore imageStore : imageStores) {
Long zoneId = imageStore.getScope().getScopeId();

if (!isZoneAndImageStoreAvailable(imageStore, zoneId, zoneSet, isPrivateTemplate(template))) {
if (!isZoneAndImageStoreAvailable(imageStore, zoneId, zoneCopyCount, copyLimit)) {
continue;
}

Expand Down
29 changes: 14 additions & 15 deletions server/src/main/java/com/cloud/template/TemplateAdapterBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,9 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;

import javax.inject.Inject;

Expand Down Expand Up @@ -169,7 +167,13 @@ protected DataStore verifyHeuristicRulesForZone(VMTemplateVO template, Long zone
return heuristicRuleHelper.getImageStoreIfThereIsHeuristicRule(zoneId, heuristicType, template);
}

protected boolean isZoneAndImageStoreAvailable(DataStore imageStore, Long zoneId, Set<Long> zoneSet, boolean isTemplatePrivate) {
protected int getSecStorageCopyLimit(VMTemplateVO template, long zoneId) {
return isPrivateTemplate(template)
? TemplateManager.PrivateTemplateSecStorageCopy.valueIn(zoneId)
: TemplateManager.PublicTemplateSecStorageCopy.valueIn(zoneId);
}

protected boolean isZoneAndImageStoreAvailable(DataStore imageStore, Long zoneId, Map<Long, Integer> zoneCopyCount, int copyLimit) {
if (zoneId == null) {
logger.warn(String.format("Zone ID is null, cannot allocate ISO/template in image store [%s].", imageStore));
return false;
Expand All @@ -191,19 +195,13 @@ protected boolean isZoneAndImageStoreAvailable(DataStore imageStore, Long zoneId
return false;
}

if (zoneSet == null) {
logger.info(String.format("Zone set is null; therefore, the ISO/template should be allocated in every secondary storage of zone [%s].", zone));
return true;
}

if (isTemplatePrivate && zoneSet.contains(zoneId)) {
logger.info(String.format("The template is private and it is already allocated in a secondary storage in zone [%s]; therefore, image store [%s] will be skipped.",
zone, imageStore));
int currentCount = zoneCopyCount.getOrDefault(zoneId, 0);
if (copyLimit > 0 && currentCount >= copyLimit) {
logger.info("Copy limit of {} reached for zone [{}]; skipping image store [{}].", copyLimit, zone, imageStore);
return false;
}

logger.info(String.format("Private template will be allocated in image store [%s] in zone [%s].", imageStore, zone));
zoneSet.add(zoneId);
zoneCopyCount.put(zoneId, currentCount + 1);
return true;
}

Expand All @@ -212,12 +210,13 @@ protected boolean isZoneAndImageStoreAvailable(DataStore imageStore, Long zoneId
* {@link TemplateProfile#getZoneIdList()}.
*/
protected void postUploadAllocation(List<DataStore> imageStores, VMTemplateVO template, List<TemplateOrVolumePostUploadCommand> payloads) {
Set<Long> zoneSet = new HashSet<>();
Map<Long, Integer> zoneCopyCount = new HashMap<>();
Collections.shuffle(imageStores);
for (DataStore imageStore : imageStores) {
Long zoneId_is = imageStore.getScope().getScopeId();
int copyLimit = zoneId_is == null ? 0 : getSecStorageCopyLimit(template, zoneId_is);

if (!isZoneAndImageStoreAvailable(imageStore, zoneId_is, zoneSet, isPrivateTemplate(template))) {
if (!isZoneAndImageStoreAvailable(imageStore, zoneId_is, zoneCopyCount, copyLimit)) {
continue;
}

Expand Down
4 changes: 3 additions & 1 deletion server/src/main/java/com/cloud/template/TemplateManagerImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2493,7 +2493,9 @@ public ConfigKey<?>[] getConfigKeys() {
return new ConfigKey<?>[] {AllowPublicUserTemplates,
TemplatePreloaderPoolSize,
ValidateUrlIsResolvableBeforeRegisteringTemplate,
TemplateDeleteFromPrimaryStorage};
TemplateDeleteFromPrimaryStorage,
PublicTemplateSecStorageCopy,
PrivateTemplateSecStorageCopy};
}

public List<TemplateAdapter> getTemplateAdapters() {
Expand Down
Loading
Loading