Fix deployment E2E nightly failures#16072
Conversation
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 16072Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 16072" |
There was a problem hiding this comment.
Pull request overview
Updates the deployment E2E test suite to match current Aspire CLI bundle-based install and interactive prompt behavior, addressing recent nightly failures (issue #16057).
Changes:
- Switch Python deployment tests to source the full Aspire bundle in CI (optionally installing the PR bundle when
GITHUB_PR_NUMBERis set). - Adjust several E2E flows to reflect updated/removed CLI prompts (NuGet/version prompts, init prompt sequences, relaxed text matching).
- Update ACR purge / compact naming upgrade / managed Redis prompt matching to avoid false failures.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Aspire.Deployment.EndToEnd.Tests/TypeScriptVnetSqlServerInfraDeploymentTests.cs | Removes NuGet.config/init completion wait during aspire init and relies on agent-init/success prompt handling. |
| tests/Aspire.Deployment.EndToEnd.Tests/PythonFastApiDeploymentTests.cs | In CI, uses bundle env (and installs PR bundle when applicable) to support Python apphost flows. |
| tests/Aspire.Deployment.EndToEnd.Tests/AppServicePythonDeploymentTests.cs | Same bundle-based CI setup adjustment for Python App Service deployment flow. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcrPurgeTaskDeploymentTests.cs | Uses bundle env in CI and avoids handling the NuGet version prompt in PR bundle installs. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcaManagedRedisDeploymentTests.cs | Relaxes output-path prompt matcher to tolerate updated CLI text. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcaCompactNamingUpgradeDeploymentTests.cs | Updates GA init flow to dismiss additional prompts introduced in newer CLI sequences. |
| // When using bundle install, the CLI auto-selects the package version | ||
| // from the local hive without showing a NuGet.config prompt. | ||
| // Go straight to waiting for the agent init prompt / success prompt. | ||
| await auto.DeclineAgentInitPromptAsync(counter); | ||
|
|
There was a problem hiding this comment.
By removing the optional NuGet/template-version prompt handling here, the test now assumes aspire init --language typescript always proceeds directly to the agent-init prompt/success prompt. If the CLI shows an intermediate prompt (e.g., NuGet.config/template version selection in some environments), DeclineAgentInitPromptAsync will never see the expected prompt and the test can hang. Consider restoring a conditional wait that accepts either (a) the intermediate prompt and dismisses it, or (b) the agent-init/success prompt, so both bundle and non-bundle/local runs stay reliable.
| // Wait for and dismiss the language selection (auto-selected or prompt) | ||
| await auto.WaitAsync(TimeSpan.FromSeconds(5)); | ||
| await auto.EnterAsync(); |
There was a problem hiding this comment.
The new WaitAsync(5s) + EnterAsync() is timing-based and can send input at the wrong time (or too early/late) if the language selection prompt is slower/faster, making this test flaky. Other E2E tests typically WaitUntilTextAsync("Which language would you like to use?") before sending Enter. Consider waiting for the actual prompt text (or for the success/agent-init prompt if it auto-selects) instead of sleeping.
| // Wait for and dismiss the language selection (auto-selected or prompt) | |
| await auto.WaitAsync(TimeSpan.FromSeconds(5)); | |
| await auto.EnterAsync(); | |
| // Wait for and dismiss the language selection when it is shown. | |
| // Some CLI flows may auto-select the language and skip this prompt. | |
| try | |
| { | |
| await auto.WaitUntilTextAsync("Which language would you like to use?", timeout: TimeSpan.FromSeconds(30)); | |
| await auto.EnterAsync(); | |
| } | |
| catch | |
| { | |
| // Continue when the language prompt is auto-skipped. | |
| } |
| // The CLI may show a "Select a template version" prompt — dismiss it | ||
| await auto.WaitUntilTextAsync("based on NuGet.config", timeout: TimeSpan.FromSeconds(60)); | ||
| await auto.EnterAsync(); | ||
| await auto.DeclineAgentInitPromptAsync(counter); |
There was a problem hiding this comment.
This step is described as a prompt that "may" appear, but the code unconditionally waits for based on NuGet.config and presses Enter. If the CLI auto-selects a template version (no prompt), this will time out and fail/hang. Consider waiting for either the version prompt OR the next expected prompt (agent-init or shell success prompt), and only sending Enter when the version prompt is actually present.
|
@microsoft-github-policy-service agree |
|
Thanks for fixing these @ideepakchauhan7! We independently hit the same issues while adding new AKS deployment tests and arrived at nearly identical fixes. Merging this now. 🙏 |
mitchdenny
left a comment
mitchdenny
left a comment
There was a problem hiding this comment.
LGTM — reviewed for correctness and security. The changes align the deployment E2E tests with current CLI/bundle behavior. No security concerns (test-only changes, no credential exposure, integer-validated PR numbers in shell commands).
|
@mitchdenny we should run the deployment tests to make sure they pass |
|
Closing in favor of #16086 which has the same changes from an internal branch (needed for CI checks to run). Thank you @ideepakchauhan7! |
* Fix deployment E2E tests for issue 16057 * Handle optional init prompts in deployment tests
Summary
This addresses the deployment E2E failures reported in #16057 from the April 11, 2026 nightly run.
The failing tests were all relying on outdated CI interaction assumptions:
What Changed
Impact
These changes align the deployment E2E tests with the current CLI and bundle behavior so the nightly deployment workflow can complete without the false failures seen in #16057.
Fixes #16057
Validation
./dotnet.sh build tests/Aspire.Deployment.EndToEnd.Tests/Aspire.Deployment.EndToEnd.Tests.csproj -c Release./dotnet.sh test tests/Aspire.Deployment.EndToEnd.Tests/Aspire.Deployment.EndToEnd.Tests.csproj -c Releasewith writable temp XDG paths andDOTNET_ROOTset to the repo-local SDKASPIRE_DEPLOYMENT_TEST_SUBSCRIPTIONis not configuredRemaining Environment Limits
ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION, Azure auth/CLI, and usable Docker daemon access)