Add VS Code extension E2E test architecture

.github/agents/agentic-workflows.agent.md

@@ -176,3 +176,4 @@ gh aw compile --validate
 - Follow security best practices: minimal permissions, explicit network access, no template injection
 - **Network configuration**: Use ecosystem identifiers (`node`, `python`, `go`, etc.) or explicit FQDNs in `network.allowed`. Bare shorthands like `npm` or `pypi` are **not** valid. See https://github.com/github/gh-aw/blob/v0.72.0/.github/aw/network.md for the full list of valid ecosystem identifiers and domain patterns.
 - **Single-file output**: When creating a workflow, produce exactly **one** workflow `.md` file. Do not create separate documentation files (architecture docs, runbooks, usage guides, etc.). If documentation is needed, add a brief `## Usage` section inside the workflow file itself.
+- **User-facing changelog/release-note output**: Workflows that generate changelogs, release notes, status reports intended for release communication, or related summaries must explicitly exclude non-user-facing issues and PRs. Do not include agentic workflow, automation, CI/build, test-only, dependency-bump, release-engineering, changelog-generation, or repository-maintenance changes unless they directly change the released product experience for users.

.github/workflows/extension-changelog.md

@@ -238,12 +238,20 @@ enrich when helpful (for example `is:pr is:merged repo:microsoft/aspire` queries
 to confirm PR titles and labels). You may also read the changed file list /
 diff under `extension/` to understand what actually changed.
 
+Exclude anything that is not user-facing. A change is user-facing only when
+someone using the VS Code extension can observe it in commands, settings,
+debugging, tree views, walkthroughs, diagnostics, packaging/installation,
+security, compatibility, or performance. Do not mention issues or PRs that only
+affect repository operation or the engineering process.
+
 Exclude noise that a user-facing changelog should not mention:
 
 - Dependency-bump / Dependabot PRs unless they fix a user-visible security issue.
 - Build, CI, and test-only changes with no user-facing effect.
 - Internal refactors with no behavior change.
 - Version-bump and release-prep commits (including this PR's own commit).
+- Agentic workflow, automation, changelog-generation, and repository maintenance
+  changes unless they directly change the released extension experience.
 
 ## Step 5: Learn the existing changelog format
 

.github/workflows/milestone-changelog.md

@@ -838,15 +838,25 @@ rather than creating a new one:
 
 ### 5f. Filtering rules
 
+Apply a strict user-facing filter before creating or updating a changelog entry.
+A change is user-facing only when someone using Aspire can observe it in product
+behavior, supported APIs, CLI commands, dashboard/extension UX, templates,
+integrations, documented configuration, security posture, or meaningful
+compatibility/performance behavior. Do not create entries for issues or PRs that
+only affect repository operation or the engineering process.
+
 - **Include**: new features, notable bug fixes, breaking changes, performance
-  improvements, new integrations, new resource types, and notable engineering or
-  workflow changes that have clear developer or release impact.
+  improvements, new integrations, new resource types, and security or
+  compatibility changes that have clear user or application-developer impact.
 - **Exclude**:
   - Internal refactoring, test-only changes, trivial fixes.
   - Dependency version bumps, documentation-only changes.
   - Routine CI/build maintenance with no meaningful user or developer impact.
-- When in doubt about whether a change is notable, include it — it can always be
-  removed via a comment later.
+  - Agentic workflow, automation, release-engineering, changelog-generation, and
+    repository maintenance changes unless the change has a direct user-visible
+    effect in a released Aspire product.
+- When in doubt about whether a change is user-facing, exclude it and record the
+  reason in the PR tracker. It can be restored via editorial feedback later.
 
 ### 5g. Match documentation PRs
 

.github/workflows/release-notes-generate.md

@@ -201,12 +201,22 @@ type (Features / Fixes / etc.), group by type. Use PR labels (`area-*`,
 `feat`, `fix`, `breaking-change`, `security`) as hints but defer to the
 existing structure.
 
+Exclude anything that is not user-facing. A change is user-facing only when
+someone using Aspire can observe it in product behavior, supported APIs, CLI
+commands, dashboard/extension UX, templates, integrations, documented
+configuration, security posture, or meaningful compatibility/performance
+behavior. Do not mention issues or PRs that only affect repository operation or
+the engineering process.
+
 Exclude noise that the existing notes also tend to exclude:
 
 - Dependabot / dependency-bump PRs unless flagged as security.
 - Branch / build / CI infrastructure changes that aren't user-facing.
 - Internal refactors with no behavior change.
 - Test-only changes.
+- Agentic workflow, automation, release-engineering, changelog-generation, and
+  repository maintenance changes unless the change has a direct user-visible
+  effect in a released Aspire product.
 
 If the change set is large (more than ~80 PRs), summarize aggressively and
 keep only user-facing entries; the notes should be scannable.

.github/workflows/tests.yml

@@ -229,6 +229,47 @@ jobs:
     with:
       versionOverrideArg: ${{ inputs.versionOverrideArg }}
 
+  extension_e2e_changes:
+    name: Detect VS Code extension E2E changes
+    runs-on: ubuntu-latest
+    outputs:
+      run_extension_e2e: ${{ steps.changed.outputs.run_extension_e2e }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+
+      - name: Check changed paths
+        id: changed
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          base='${{ github.event.pull_request.base.sha || github.event.before }}'
+          head='${{ github.sha }}'
+
+          if [ -z "$base" ] || [[ "$base" =~ ^0+$ ]]; then
+            echo "No usable base SHA is available; running extension E2E tests."
+            echo "run_extension_e2e=true" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+
+          if ! git cat-file -e "$base^{commit}" 2>/dev/null; then
+            echo "Base SHA $base is not available after checkout; running extension E2E tests."
+            echo "run_extension_e2e=true" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+
+          changed_files="$(git diff --name-only "$base" "$head")"
+          echo "$changed_files"
+
+          if echo "$changed_files" | grep -E -q '^(extension/|src/Aspire\.Cli/|src/Aspire/Cli/|src/Aspire\.Hosting/Cli/|tests/Aspire\.Cli\.Tests/|tests/Aspire\.Cli\.EndToEnd\.Tests/|tests/Aspire\.Hosting\.Tests/Cli/|eng/scripts/get-aspire-cli|\.github/workflows/(tests|extension-e2e-tests|build-packages|build-cli-native-archives)\.yml$)'; then
+            echo "run_extension_e2e=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "run_extension_e2e=false" >> "$GITHUB_OUTPUT"
+          fi
+
   cli_starter_validation_windows:
     name: Aspire CLI Starter Validation (${{ matrix.os.name }})
     runs-on: ${{ matrix.os.runner }}
@@ -310,7 +351,7 @@ jobs:
           if-no-files-found: ignore
 
   extension_tests_win:
-    name: Run VS Code extension tests (Windows)
+    name: Run VS Code extension unit tests (Windows)
     runs-on: windows-latest
     env:
       NPM_REGISTRY: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public-npm/npm/registry/
@@ -357,9 +398,9 @@ jobs:
 
           # Seed Corepack's cache from the internal dnceng npm feed. Corepack's
           # built-in `corepack prepare --activate` would download Yarn 1.x from
-          # registry.yarnpkg.com (hardcoded in Corepack 0.34's config.json and
-          # not redirectable via COREPACK_NPM_REGISTRY), bypassing the dnceng
-          # mirror this workflow is supposed to validate. The shared
+          # its hardcoded upstream endpoint rather than the configured internal
+          # feed, bypassing the dependency source this workflow is supposed to
+          # validate. The shared
           # prepareCorepackYarn.mjs script does the equivalent via `npm pack`
           # against $NPM_REGISTRY, then drops the same on-disk layout Corepack
           # would have written.
@@ -372,8 +413,7 @@ jobs:
         # yarn.lock that carry a tarball URL — `npm:` aliases don't).
         # CONTRIBUTING.MD asks contributors to regenerate yarn.lock through the
         # internal dotnet-public-npm feed; an allowlist catches drift to any
-        # other public mirror (npmmirror.com, jsr.io, github.com tarballs, ...)
-        # rather than only npmjs.org and yarnpkg.com.
+        # other public mirror or tarball host.
         run: |
           node -e "const fs = require('fs'); const allow = 'pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public-npm'; const bad = fs.readFileSync('yarn.lock', 'utf8').split(/\r?\n/).filter(l => /^\s*resolved\s+\x22/.test(l)).filter(l => !l.includes(allow)); if (bad.length) { throw new Error('extension/yarn.lock contains resolved entries outside the internal dotnet-public-npm feed. Regenerate it through the internal feed before restoring. First offender -> ' + bad[0]); }"
       - name: Install dependencies
@@ -445,6 +485,12 @@ jobs:
       - name: Validate seeded cache via yarn install
         run: corepack yarn install --frozen-lockfile --non-interactive
 
+  extension_e2e_tests:
+    name: Run VS Code extension E2E tests
+    if: ${{ needs.extension_e2e_changes.outputs.run_extension_e2e == 'true' }}
+    uses: ./.github/workflows/extension-e2e-tests.yml
+    needs: [extension_e2e_changes, build_packages, build_cli_archive_linux, build_cli_archive_windows, extension_tests_win, extension_bootstrap_linux]
+
   typescript_sdk_tests:
     name: TypeScript SDK Unit Tests
     uses: ./.github/workflows/typescript-sdk-tests.yml
@@ -469,7 +515,9 @@ jobs:
       prepare_winget_installer_artifacts,
       prepare_homebrew_installer_artifacts,
       build_cli_e2e_image,
+      extension_e2e_changes,
       extension_tests_win,
+      extension_e2e_tests,
       extension_bootstrap_linux,
       cli_starter_validation_windows,
       typescript_sdk_tests,
@@ -550,6 +598,8 @@ jobs:
         #   tests, so an empty matrix and 'skipped' result are expected.
         # - cli_starter_validation_windows: this job only runs for pull requests and is expected to
         #   be skipped for other workflow events.
+        # - extension_e2e_tests: this job only runs when the PR/push changes the VS Code extension,
+        #   Aspire CLI, or the extension E2E workflow wiring.
         # - polyglot_validation: this job runs for pull requests and main branch builds; it is
         #   expected to be skipped for other workflow events.
         # All other jobs in this gate are required, and a 'skipped' result is treated as a failure.
@@ -559,6 +609,8 @@ jobs:
                contains(needs.*.result, 'cancelled') ||
                (github.event_name == 'pull_request' &&
                 (needs.extension_tests_win.result == 'skipped' ||
+                 (needs.extension_e2e_changes.outputs.run_extension_e2e == 'true' &&
+                  needs.extension_e2e_tests.result == 'skipped') ||
                  needs.cli_starter_validation_windows.result == 'skipped' ||
                  needs.typescript_sdk_tests.result == 'skipped' ||
                  needs.typescript_api_compat.result == 'skipped' ||
@@ -576,6 +628,8 @@ jobs:
                  needs.polyglot_validation.result == 'skipped')) ||
                (github.event_name != 'pull_request' &&
                 (needs.extension_tests_win.result == 'skipped' ||
+                 (needs.extension_e2e_changes.outputs.run_extension_e2e == 'true' &&
+                  needs.extension_e2e_tests.result == 'skipped') ||
                  needs.typescript_sdk_tests.result == 'skipped' ||
                  needs.build_cli_archive_macos_x64.result == 'skipped' ||
                  needs.prepare_winget_installer_artifacts.result == 'skipped' ||

.gitignore

@@ -196,6 +196,11 @@ tools/ReleaseNotes/release-notes-*.md
 extension/dist/
 extension/.localization/
 extension/out/
+extension/.test-artifacts/
+extension/.test-extensions/
+extension/.test-results/
+extension/.test-storage/
+extension/.test-workspaces/
 extension/node_modules/
 extension/.corepack-cache/
 **/.vscode-test/

extension/.mocharc.e2e.js

@@ -0,0 +1,39 @@
+'use strict';
+
+const path = require('path');
+
+const resultsDir = process.env.ASPIRE_EXTENSION_E2E_RESULTS_DIR || path.join('.test-results', 'e2e');
+const extesterModulePath = process.env.ASPIRE_EXTENSION_E2E_EXTESTER_MODULE ?? 'vscode-extension-tester';
+const { VSBrowser } = require(extesterModulePath);
+
+function getScreenshotName(test) {
+  return (test?.fullTitle?.() ?? test?.title ?? 'failed-test')
+    .replace(/[^a-z0-9._-]+/gi, '-')
+    .replace(/^-+|-+$/g, '')
+    .slice(0, 120) || 'failed-test';
+}
+
+module.exports = {
+  ui: 'tdd',
+  timeout: 240000,
+  reporter: path.join(__dirname, 'scripts', 'e2e-mocha-reporter.cjs'),
+  reporterOptions: {
+    output: path.join(resultsDir, 'mocha.json'),
+  },
+  parallel: false,
+  spec: 'out/test-e2e/**/*.e2e.test.js',
+  rootHooks: {
+    async afterEach() {
+      if (this.currentTest?.state !== 'failed') {
+        return;
+      }
+
+      try {
+        await VSBrowser.instance.takeScreenshot(getScreenshotName(this.currentTest));
+      }
+      catch (error) {
+        console.warn(`Failed to capture E2E failure screenshot: ${error instanceof Error ? error.message : String(error)}`);
+      }
+    },
+  },
+};

extension/.vscodeignore

@@ -1,5 +1,16 @@
 .vscode/**
 .vscode-test/**
+.test-*
+.test-*/**
+.test-artifacts/**
+.test-extensions/**
+.test-results/**
+.test-storage/**
+.test-workspaces/**
+.mocharc.e2e.js
+tsconfig.e2e.json
+scripts/run-e2e.js
+test-e2e/**
 out/**
 node_modules/**
 src/**