Skip to content

test(func): improve parallelism #10934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
DariuszPorowski wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

test(func): improve parallelism #10934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
669d417
fix: improve parallelism in functional test commands
DariuszPorowski Dec 12, 2025
5762a76
fix: replace static sleep with dynamic wait for cleanup completion in…
DariuszPorowski Dec 12, 2025
5030b25
fix: enable fast cleanup for functional tests and optimize parallelis…
DariuszPorowski Dec 12, 2025
ec753bd
fix: disable fast cleanup for long-running Azure tests
DariuszPorowski Dec 12, 2025
ed4bc3f
fix: remove unnecessary parallelism flags from functional test commands
DariuszPorowski Dec 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions build/test.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Expand Down Expand Up @@ -86,76 +86,76 @@ test-functional-ucp: test-functional-ucp-noncloud test-functional-ucp-cloud ## R

.PHONY: test-functional-ucp-noncloud
test-functional-ucp-noncloud: ## Runs UCP functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/ucp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 5 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/ucp/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 4 $(GOTEST_OPTS)

.PHONY: test-functional-ucp-cloud
test-functional-ucp-cloud: ## Runs UCP functional tests that require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/ucp/cloud/... -timeout ${TEST_TIMEOUT} -v -parallel 5 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/ucp/cloud/... -timeout ${TEST_TIMEOUT} -v -p 2 $(GOTEST_OPTS)

.PHONY: test-functional-kubernetes
test-functional-kubernetes: test-functional-kubernetes-noncloud ## Runs all Kubernetes functional tests
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/kubernetes/... -timeout ${TEST_TIMEOUT} -v -parallel 5 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/kubernetes/... -timeout ${TEST_TIMEOUT} -v -p 1 $(GOTEST_OPTS)

.PHONY: test-functional-kubernetes-noncloud
test-functional-kubernetes-noncloud: ## Runs Kubernetes functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/kubernetes/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 5 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/kubernetes/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 1 $(GOTEST_OPTS)

.PHONY: test-functional-corerp
test-functional-corerp: test-functional-corerp-noncloud test-functional-corerp-cloud ## Runs all Core RP functional tests (both cloud and non-cloud)

.PHONY: test-functional-corerp-noncloud
test-functional-corerp-noncloud: ## Runs corerp functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/corerp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 10 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/corerp/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 3 $(GOTEST_OPTS)

.PHONY: test-functional-corerp-cloud
test-functional-corerp-cloud: ## Runs corerp functional tests that require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/corerp/cloud/... -timeout ${TEST_TIMEOUT} -v -parallel 10 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/corerp/cloud/... -timeout ${TEST_TIMEOUT} -v -p 2 $(GOTEST_OPTS)

.PHONY: test-functional-msgrp
test-functional-msgrp: test-functional-msgrp-noncloud ## Runs all Messaging RP functional tests (both cloud and non-cloud)

.PHONY: test-functional-msgrp-noncloud
test-functional-msgrp-noncloud: ## Runs Messaging RP functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/messagingrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 2 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/messagingrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 2 $(GOTEST_OPTS)

.PHONY: test-functional-cli
test-functional-cli: test-functional-cli-noncloud ## Runs all cli functional tests (both cloud and non-cloud)

.PHONY: test-functional-cli-noncloud
test-functional-cli-noncloud: ## Runs cli functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/cli/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 10 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/cli/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 4 $(GOTEST_OPTS)

.PHONY: test-functional-daprrp
test-functional-daprrp: test-functional-daprrp-noncloud ## Runs all Dapr RP functional tests (both cloud and non-cloud)

.PHONY: test-functional-daprrp-noncloud
test-functional-daprrp-noncloud: ## Runs Dapr RP functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/daprrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 3 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/daprrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 2 $(GOTEST_OPTS)

.PHONY: test-functional-datastoresrp
test-functional-datastoresrp: test-functional-datastoresrp-noncloud ## Runs all Datastores RP functional tests (non-cloud)

.PHONY: test-functional-datastoresrp-noncloud
test-functional-datastoresrp-noncloud: ## Runs Datastores RP functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/datastoresrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 3 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/datastoresrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 3 $(GOTEST_OPTS)

.PHONY: test-functional-dynamicrp-noncloud
test-functional-dynamicrp-noncloud: ## Runs Dynamic RP functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/dynamicrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 1 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/dynamicrp/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 1 -p 1 $(GOTEST_OPTS)

.PHONY: test-functional-upgrade
test-functional-upgrade: test-functional-upgrade-noncloud ## Runs all Upgrade functional tests

.PHONY: test-functional-upgrade-noncloud
test-functional-upgrade-noncloud: ## Runs Upgrade functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/upgrade/... -timeout ${TEST_TIMEOUT} -v -parallel 1 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/upgrade/... -timeout ${TEST_TIMEOUT} -v -parallel 1 -p 1 $(GOTEST_OPTS)

.PHONY: test-functional-samples
test-functional-samples: test-functional-samples-noncloud ## Runs all Samples functional tests

.PHONY: test-functional-samples-noncloud
test-functional-samples-noncloud: ## Runs Samples functional tests that do not require cloud resources
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/samples/noncloud/... -timeout ${TEST_TIMEOUT} -v -parallel 5 $(GOTEST_OPTS)
CGO_ENABLED=1 $(GOTEST_TOOL) ./test/functional-portable/samples/noncloud/... -timeout ${TEST_TIMEOUT} -v -p 4 $(GOTEST_OPTS)

.PHONY: test-validate-bicep
test-validate-bicep: ## Validates that all .bicep files compile cleanly
Expand All @@ -175,7 +175,7 @@ oav-installed:
@echo "$(ARROW) OK"

# TODO re-enable https://github.com/radius-project/radius/issues/5091
.PHONY: test-ucp-spec-examples
.PHONY: test-ucp-spec-examples
test-ucp-spec-examples: oav-installed ## Validates UCP examples conform to UCP OpenAPI Spec
# @echo "$(ARROW) Testing x-ms-examples conform to ucp spec..."
# oav validate-example swagger/specification/ucp/resource-manager/UCP/preview/2023-10-01-preview/openapi.json
Expand Down
12 changes: 10 additions & 2 deletions test/functional-portable/upgrade/upgrade_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,11 @@ func Test_PreflightContainer_PreflightDisabled(t *testing.T) {
_ = exec.Command(cleanupCommand[0], cleanupCommand[1:]...).Run() // Ignore errors during cleanup

// Wait for cleanup to complete
time.Sleep(3 * time.Second)
require.Eventually(t, func() bool {
cmd := exec.Command("helm", "status", "radius", "--namespace", radiusNamespace)
err := cmd.Run()
return err != nil
}, 30*time.Second, 1*time.Second, "Radius release was not uninstalled")

t.Log("Installing Radius with preflight disabled using helm")
installCommand := []string{
Expand Down Expand Up @@ -180,7 +184,11 @@ func Test_PreflightContainer_JobConfiguration(t *testing.T) {
_ = exec.Command(cleanupCommand[0], cleanupCommand[1:]...).Run() // Ignore errors during cleanup

// Wait for cleanup to complete
time.Sleep(3 * time.Second)
require.Eventually(t, func() bool {
cmd := exec.Command("helm", "status", "radius", "--namespace", radiusNamespace)
err := cmd.Run()
return err != nil
}, 30*time.Second, 1*time.Second, "Radius release was not uninstalled")

// Use local registry image for testing
t.Log("Installing Radius with custom preflight configuration using helm")
Expand Down