Fix deployment E2E nightly failures#16086
Closed
mitchdenny wants to merge 2 commits into
Closed
Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 16086Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 16086" |
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Deployment E2E test suite to align with recent Aspire CLI interactive prompt and bundle/layout behaviors, addressing nightly failures in deployment scenarios.
Changes:
- Updates
aspire initprompt-handling logic in TypeScript VNet SQL infra and compact naming upgrade tests (template version prompt + agent-init prompt sequencing). - Switches Python deployment tests’ CI setup to use the bundle environment (and optionally install from PR artifacts).
- Adjusts a few prompt matchers/flows (ACR purge prompt handling, managed Redis output-path prompt text).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Aspire.Deployment.EndToEnd.Tests/TypeScriptVnetSqlServerInfraDeploymentTests.cs | Refines aspire init --language typescript prompt handling to accommodate newer CLI prompt sequences. |
| tests/Aspire.Deployment.EndToEnd.Tests/PythonFastApiDeploymentTests.cs | In CI, sources bundle environment and optionally installs from PR artifacts for Python template flows. |
| tests/Aspire.Deployment.EndToEnd.Tests/AppServicePythonDeploymentTests.cs | Same CI bundle setup adjustment as Python FastAPI test, for App Service scenario. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcrPurgeTaskDeploymentTests.cs | Updates CI bundle setup and modifies aspire add prompt handling for PR vs non-PR runs. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcaManagedRedisDeploymentTests.cs | Relaxes output-path prompt matcher to tolerate minor CLI text changes. |
| tests/Aspire.Deployment.EndToEnd.Tests/AcaCompactNamingUpgradeDeploymentTests.cs | Reworks GA aspire init automation to handle multiple possible prompts before success. |
Comments suppressed due to low confidence (1)
tests/Aspire.Deployment.EndToEnd.Tests/AcrPurgeTaskDeploymentTests.cs:109
- The version-selection handling is now skipped when
GetPrNumber() > 0, but the prompt itself is detected via the very specific string"(based on NuGet.config)". That text can change depending on channel/source details, and skipping the prompt entirely is risky ifaspire addstill prompts (the command would block and the nextWaitForSuccessPromptAsyncwould time out). Consider waiting for the actual selection prompt (e.g., "Select a version of …") and only sending Enter when that prompt is present, rather than branching on PR number / matching the NuGet.config suffix.
await auto.TypeAsync("aspire add Aspire.Hosting.Azure.AppContainers");
await auto.EnterAsync();
if (DeploymentE2ETestHelpers.IsRunningInCI && DeploymentE2ETestHelpers.GetPrNumber() <= 0)
{
await auto.WaitUntilTextAsync("(based on NuGet.config)", timeout: TimeSpan.FromSeconds(60));
await auto.EnterAsync();
}
Comment on lines
+89
to
+95
| var waitingForTemplateVersionPrompt = new CellPatternSearcher() | ||
| .Find("NuGet.config"); | ||
| var waitingForInitComplete = new CellPatternSearcher() | ||
| .Find("Aspire initialization complete"); | ||
| var waitingForAgentInitPrompt = new CellPatternSearcher() | ||
| .Find("configure AI agent environments"); | ||
| var waitingForSuccessPrompt = new CellPatternSearcher() | ||
| .FindPattern(counter.Value.ToString()) | ||
| .RightText(" OK] $ "); |
There was a problem hiding this comment.
The "template version prompt" matcher is currently Find("NuGet.config"), which is a brittle signal (it depends on the channel/source text shown in the selection list). If the CLI shows the template-version selection but without the "NuGet.config" suffix, this wait will never detect it and aspire init will hang until the 2-minute timeout. Consider matching the actual prompt text (e.g., "Select a template version:") instead of relying on "NuGet.config" appearing somewhere in the output.
Comment on lines
+101
to
+109
| var waitingForLanguageSelectionPrompt = new CellPatternSearcher() | ||
| .Find("Which language would you like to use?"); | ||
| var waitingForTemplateVersionPrompt = new CellPatternSearcher() | ||
| .Find("NuGet.config"); | ||
| var waitingForAgentInitPrompt = new CellPatternSearcher() | ||
| .Find("configure AI agent environments"); | ||
| var waitingForSuccessPrompt = new CellPatternSearcher() | ||
| .FindPattern(counter.Value.ToString()) | ||
| .RightText(" OK] $ "); |
There was a problem hiding this comment.
waitingForTemplateVersionPrompt is searching for "NuGet.config", but the template-version prompt itself has a stable header (e.g., "Select a template version:"). Depending on the configured channel/source, the selection items may not include "NuGet.config", which would make this state machine miss the prompt and time out while aspire init is still waiting for input. Prefer matching the prompt header text rather than a source-detail substring.
Member
Author
|
/deployment-test |
Contributor
|
🚀 Deployment tests starting on PR #16086... This will deploy to real Azure infrastructure. Results will be posted here when complete. |
Contributor
|
🎬 CLI E2E Test Recordings — 68 recordings uploaded (commit View recordings
📹 Recordings uploaded automatically from CI run #24301402310 |
Member
Author
Contributor
|
❌ Deployment E2E Tests failed — 24 passed, 6 failed, 0 cancelled View test results and recordings
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #16072 from internal branch to get CI checks running. Credit to @ideepakchauhan7 for the original fix.
Fixes #16057