From 7325b4142c4018441daa78cd3aa0c08c2cdcac28 Mon Sep 17 00:00:00 2001 From: Deyaa Eldeen Date: Fri, 24 Apr 2026 17:25:06 +0000 Subject: [PATCH 1/3] Add spec fidelity, narrative consistency, and convention matching rules to hero-scenarios guidelines Addresses three root causes of issues found during hero scenario review: 1. Spec Fidelity Rules: Example values must match spec types (uuid fields need real UUIDs, url fields need schemes, response shapes must match operation return types, no batching single-item APIs) 2. Narrative-Step Consistency: Every claim in the story/success criteria must have a corresponding API call in the steps 3. Matching Existing Document Conventions: When appending to an existing hero-scenarios.md, adopt its URL path style, ID format, and section structure instead of introducing inconsistencies Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/prompts/hero-scenarios-guidelines.md | 51 ++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/.github/prompts/hero-scenarios-guidelines.md b/.github/prompts/hero-scenarios-guidelines.md index feed640a2ed2..c8673aa228e7 100644 --- a/.github/prompts/hero-scenarios-guidelines.md +++ b/.github/prompts/hero-scenarios-guidelines.md @@ -247,6 +247,57 @@ For each scenario, include: the API references them. For LROs, show the polling header in the initial response. +### Spec Fidelity Rules + +Before writing any request or response example, check the spec +definitions for the operations and models involved. Specifically: + +- **Property types must match the spec.** If a property is typed as + `uuid` (or `format: uuid`), every example value for that property + MUST be a valid UUID (e.g., `10013344-5566-7788-99aa-bbccddeeff00`). + Do NOT invent human-readable prefixed IDs like `cde-...`, `col-...`, + or `dp-...`. If a property is typed as `url` (or `format: uri`), + the value MUST include a scheme (e.g., `https://host.example.com`). +- **Response shapes must match the operation's return type.** If an + operation returns a single object (not an array or paged list), + show a single object. If it returns a paged list, show the `value` + array wrapper. Do NOT show a batch response for an operation that + returns a single item — split into multiple calls instead. +- **One API call per operation invocation.** If you need to perform the + same operation on multiple resources (e.g., ingest two columns, tag + two columns), show each call separately with its own request and + response. Do NOT combine them into a batch unless the API explicitly + supports batch semantics. + +### Narrative–Step Consistency + +Every claim in the scenario description, prerequisites, and success +criteria MUST be backed by a corresponding API call in the steps: + +- If the story says "tags both CustomerEmail and AccountNumber," there + must be two tagging API calls — one for each column. +- If the prerequisites mention a resource ID, that exact ID must appear + in a request body or URL path in the steps. +- The success criteria must only claim outcomes that the steps actually + achieve. Do NOT list a success criterion for an action that was + described in the narrative but omitted from the steps. + +### Matching Existing Document Conventions + +When appending scenarios to an **existing** `hero-scenarios.md`, read +the file first and adopt its conventions: + +- **URL path format**: If existing scenarios use short paths (e.g., + `/dataAssets/query`) without an `api-version` query parameter, use + the same style. If they use full paths with version parameters, + match that instead. Do NOT mix conventions within the same document. +- **ID format**: If existing scenarios use placeholder-style IDs + (e.g., `{domain-id}`), use the same pattern. If they use concrete + example UUIDs, generate concrete UUIDs. +- **Section structure**: Match the heading levels, formatting, and + section ordering (e.g., Story → Prerequisites → Steps → Success + Criteria → Value Delivered) used by existing scenarios. + ## Step 4 — Post the Suggestion **If no `hero-scenarios.md` exists**, use `add-comment` to post a PR comment From c802157db790233671d254b914c4f4be7c9a721f Mon Sep 17 00:00:00 2001 From: Deyaa Eldeen Date: Fri, 24 Apr 2026 17:27:20 +0000 Subject: [PATCH 2/3] Recompile hero-scenarios.lock.yml with updated guidelines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/aw/actions-lock.json | 6 +- .github/workflows/hero-scenarios.lock.yml | 255 ++++++++++++++-------- 2 files changed, 169 insertions(+), 92 deletions(-) diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json index 3f1348d87684..cbb2cfde41d4 100644 --- a/.github/aw/actions-lock.json +++ b/.github/aw/actions-lock.json @@ -25,10 +25,10 @@ "version": "v9.0.0", "sha": "d746ffe35508b1917358783b479e04febd2b8f71" }, - "github/gh-aw-actions/setup@v0.68.1": { + "github/gh-aw-actions/setup@v0.71.0": { "repo": "github/gh-aw-actions/setup", - "version": "v0.68.1", - "sha": "2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc" + "version": "v0.71.0", + "sha": "49157453228f9641824955e35cbeccbca74ee0fd" } } } diff --git a/.github/workflows/hero-scenarios.lock.yml b/.github/workflows/hero-scenarios.lock.yml index 261dca6d3a54..e4830eea8b57 100644 --- a/.github/workflows/hero-scenarios.lock.yml +++ b/.github/workflows/hero-scenarios.lock.yml @@ -1,5 +1,5 @@ -# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"d9ef1562db97408cbeddef2ebba693a4a0e70bcbd53dba32ac11dfd4866f9c58","compiler_version":"v0.68.1","strict":true,"agent_id":"copilot"} -# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"668228422ae6a00e4ad889ee87cd7109ec5666a7","version":"v5.0.4"},{"repo":"actions/cache/save","sha":"668228422ae6a00e4ad889ee87cd7109ec5666a7","version":"v5.0.4"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"373c709c69115d41ff229c7e5df9f8788daa9553","version":"v9"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9"},{"repo":"actions/upload-artifact","sha":"bbbca2ddaa5d8feaa63e36b76fdaad77386f024f","version":"v7"},{"repo":"github/gh-aw-actions/setup","sha":"2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc","version":"v0.68.1"}]} +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"d9ef1562db97408cbeddef2ebba693a4a0e70bcbd53dba32ac11dfd4866f9c58","compiler_version":"v0.71.0","strict":true,"agent_id":"copilot"} +# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/cache/restore","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/cache/save","sha":"27d5ce7f107fe9357f9df03efb73ab90386fccae","version":"v5.0.5"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"373c709c69115d41ff229c7e5df9f8788daa9553","version":"v9"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"49157453228f9641824955e35cbeccbca74ee0fd","version":"v0.71.0"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.28","digest":"sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.25.28@sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.28","digest":"sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.28@sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.28","digest":"sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.25.28@sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.2.30","digest":"sha256:e950e6d39f003862d33bfb8d4eb93e242d919cf6ca874b90728e5e0ea7434c6f","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.2.30@sha256:e950e6d39f003862d33bfb8d4eb93e242d919cf6ca874b90728e5e0ea7434c6f"},{"image":"ghcr.io/github/github-mcp-server:v1.0.0","digest":"sha256:d2550953f8050bc5a1c8f80d1678766f66f60bbfbcd953fdeaf661fe4269bd95","pinned_image":"ghcr.io/github/github-mcp-server:v1.0.0@sha256:d2550953f8050bc5a1c8f80d1678766f66f60bbfbcd953fdeaf661fe4269bd95"},{"image":"node:lts-alpine","digest":"sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f","pinned_image":"node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f"}]} # ___ _ _ # / _ \ | | (_) # | |_| | __ _ ___ _ __ | |_ _ ___ @@ -14,7 +14,7 @@ # \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ # \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ # -# This file was automatically generated by gh-aw (v0.68.1). DO NOT EDIT. +# This file was automatically generated by gh-aw (v0.71.0). DO NOT EDIT. # # To update this file, edit the corresponding .md file and run: # gh aw compile @@ -35,14 +35,22 @@ # - GITHUB_TOKEN # # Custom actions used: -# - actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4 -# - actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4 +# - actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 +# - actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 # - actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 # - actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 # - actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 -# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 -# - actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 -# - github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 +# - actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 +# - actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 +# - github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 +# +# Container images used: +# - ghcr.io/github/gh-aw-firewall/agent:0.25.28@sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a +# - ghcr.io/github/gh-aw-firewall/api-proxy:0.25.28@sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb +# - ghcr.io/github/gh-aw-firewall/squid:0.25.28@sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474 +# - ghcr.io/github/gh-aw-mcpg:v0.2.30@sha256:e950e6d39f003862d33bfb8d4eb93e242d919cf6ca874b90728e5e0ea7434c6f +# - ghcr.io/github/github-mcp-server:v1.0.0@sha256:d2550953f8050bc5a1c8f80d1678766f66f60bbfbcd953fdeaf661fe4269bd95 +# - node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f name: "Hero Scenario Suggestion" "on": @@ -111,7 +119,7 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -124,18 +132,18 @@ jobs: GH_AW_INFO_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'auto' }} GH_AW_INFO_VERSION: "1.0.21" GH_AW_INFO_AGENT_VERSION: "1.0.21" - GH_AW_INFO_CLI_VERSION: "v0.68.1" + GH_AW_INFO_CLI_VERSION: "v0.71.0" GH_AW_INFO_WORKFLOW_NAME: "Hero Scenario Suggestion" GH_AW_INFO_EXPERIMENTAL: "false" GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true" GH_AW_INFO_STAGED: "false" GH_AW_INFO_ALLOWED_DOMAINS: '["defaults"]' GH_AW_INFO_FIREWALL_ENABLED: "true" - GH_AW_INFO_AWF_VERSION: "v0.25.18" + GH_AW_INFO_AWF_VERSION: "v0.25.28" GH_AW_INFO_AWMG_VERSION: "" GH_AW_INFO_FIREWALL_TYPE: "squid" GH_AW_COMPILED_STRICT: "true" - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -154,11 +162,22 @@ jobs: sparse-checkout: | .github .agents + .claude + .codex + .crush + .gemini + .opencode sparse-checkout-cone-mode: true fetch-depth: 1 + - name: Save agent config folders for base branch restoration + env: + GH_AW_AGENT_FOLDERS: ".agents .claude .codex .crush .gemini .github .opencode" + GH_AW_AGENT_FILES: ".crush.json AGENTS.md CLAUDE.md GEMINI.md opencode.jsonc" + # poutine:ignore untrusted_checkout_exec + run: bash "${RUNNER_TEMP}/gh-aw/actions/save_base_github_folders.sh" - name: Check workflow lock file id: check-lock-file - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_WORKFLOW_FILE: "hero-scenarios.lock.yml" GH_AW_CONTEXT_WORKFLOW_REF: "${{ github.workflow_ref }}" @@ -169,9 +188,9 @@ jobs: const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs'); await main(); - name: Check compile-agentic version - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_COMPILED_VERSION: "v0.68.1" + GH_AW_COMPILED_VERSION: "v0.71.0" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -180,7 +199,9 @@ jobs: await main(); - name: Compute current body text id: sanitized - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -254,7 +275,7 @@ jobs: GH_AW_PROMPT_8edf7851920af826_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number || inputs.item_number }} @@ -265,7 +286,7 @@ jobs: const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs'); await main(); - name: Substitute placeholders - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_ALLOWED_EXTENSIONS: '' @@ -329,13 +350,14 @@ jobs: run: bash "${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh" - name: Upload activation artifact if: success() - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: activation path: | /tmp/gh-aw/aw_info.json /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/github_rate_limits.jsonl + /tmp/gh-aw/base if-no-files-found: ignore retention-days: 1 @@ -353,18 +375,21 @@ jobs: GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs GH_AW_WORKFLOW_ID_SANITIZED: heroscenarios outputs: + agentic_engine_timeout: ${{ steps.detect-copilot-errors.outputs.agentic_engine_timeout || 'false' }} checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} effective_tokens: ${{ steps.parse-mcp-gateway.outputs.effective_tokens }} has_patch: ${{ steps.collect_output.outputs.has_patch }} - inference_access_error: ${{ steps.detect-inference-error.outputs.inference_access_error || 'false' }} + inference_access_error: ${{ steps.detect-copilot-errors.outputs.inference_access_error || 'false' }} + mcp_policy_error: ${{ steps.detect-copilot-errors.outputs.mcp_policy_error || 'false' }} model: ${{ needs.activation.outputs.model }} + model_not_supported_error: ${{ steps.detect-copilot-errors.outputs.model_not_supported_error || 'false' }} output: ${{ steps.collect_output.outputs.output }} output_types: ${{ steps.collect_output.outputs.output_types }} setup-trace-id: ${{ steps.setup.outputs.trace-id }} steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -372,9 +397,11 @@ jobs: - name: Set runtime paths id: set-runtime-paths run: | - echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" >> "$GITHUB_OUTPUT" - echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" >> "$GITHUB_OUTPUT" - echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" >> "$GITHUB_OUTPUT" + { + echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" + echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" + echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" + } >> "$GITHUB_OUTPUT" - name: Checkout repository uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -389,7 +416,7 @@ jobs: - name: Create cache-memory directory run: bash "${RUNNER_TEMP}/gh-aw/actions/create_cache_memory_dir.sh" - name: Restore cache-memory file share data - uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4 + uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 with: key: memory-unapproved-e3d939f8-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }} path: /tmp/gh-aw/cache-memory @@ -427,7 +454,7 @@ jobs: id: checkout-pr if: | github.event.pull_request || github.event.issue.pull_request - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} with: @@ -442,7 +469,7 @@ jobs: env: GH_HOST: github.com - name: Install AWF binary - run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.18 + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.28 - name: Parse integrity filter lists id: parse-guard-vars env: @@ -451,7 +478,7 @@ jobs: GH_AW_APPROVAL_LABELS_VAR: ${{ vars.GH_AW_GITHUB_APPROVAL_LABELS || '' }} run: bash "${RUNNER_TEMP}/gh-aw/actions/parse_guard_list.sh" - name: Download container images - run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.18 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.18 ghcr.io/github/gh-aw-firewall/squid:0.25.18 ghcr.io/github/gh-aw-mcpg:v0.2.17 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine + run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.28@sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a ghcr.io/github/gh-aw-firewall/api-proxy:0.25.28@sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb ghcr.io/github/gh-aw-firewall/squid:0.25.28@sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474 ghcr.io/github/gh-aw-mcpg:v0.2.30@sha256:e950e6d39f003862d33bfb8d4eb93e242d919cf6ca874b90728e5e0ea7434c6f ghcr.io/github/github-mcp-server:v1.0.0@sha256:d2550953f8050bc5a1c8f80d1678766f66f60bbfbcd953fdeaf661fe4269bd95 node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f - name: Write Safe Outputs Config run: | mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" @@ -465,7 +492,7 @@ jobs: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 1 comment(s) can be added. Target: ${{ github.event.pull_request.number }}.", + "add_comment": " CONSTRAINTS: Maximum 1 comment(s) can be added. Target: ${{ github.event.pull_request.number }}. Supports reply_to_id for discussion threading.", "add_labels": " CONSTRAINTS: Maximum 1 label(s) can be added. Target: ${{ github.event.pull_request.number }}.", "create_pull_request_review_comment": " CONSTRAINTS: Maximum 5 review comment(s) can be created. Comments will be on the RIGHT side of the diff.", "submit_pull_request_review": " CONSTRAINTS: Maximum 1 review(s) can be submitted." @@ -487,6 +514,10 @@ jobs: "item_number": { "issueOrPRNumber": true }, + "reply_to_id": { + "type": "string", + "maxLength": 256 + }, "repo": { "type": "string", "maxLength": 256 @@ -641,7 +672,7 @@ jobs: } } } - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -697,10 +728,10 @@ jobs: GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} run: | set -eo pipefail - mkdir -p /tmp/gh-aw/mcp-config + mkdir -p "${RUNNER_TEMP}/gh-aw/mcp-config" # Export gateway environment variables for MCP config and gateway script - export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_PORT="8080" export MCP_GATEWAY_DOMAIN="host.docker.internal" MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') echo "::add-mask::${MCP_GATEWAY_API_KEY}" @@ -711,15 +742,19 @@ jobs: export DEBUG="*" export GH_AW_ENGINE="copilot" - export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.17' + MCP_GATEWAY_UID=$(id -u 2>/dev/null || echo '0') + MCP_GATEWAY_GID=$(id -g 2>/dev/null || echo '0') + DOCKER_SOCK_GID=$(stat -c '%g' /var/run/docker.sock 2>/dev/null || echo '0') + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host --add-host host.docker.internal:127.0.0.1 --user '"${MCP_GATEWAY_UID}"':'"${MCP_GATEWAY_GID}"' --group-add '"${DOCKER_SOCK_GID}"' -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.30' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_0e89696a5df24962_EOF | bash "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh" + GH_AW_NODE=$(which node 2>/dev/null || command -v node 2>/dev/null || echo node) + cat << GH_AW_MCP_CONFIG_0e89696a5df24962_EOF | "$GH_AW_NODE" "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.cjs" { "mcpServers": { "github": { "type": "stdio", - "container": "ghcr.io/github/github-mcp-server:v0.32.0", + "container": "ghcr.io/github/github-mcp-server:v1.0.0", "env": { "GITHUB_HOST": "\${GITHUB_SERVER_URL}", "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", @@ -764,6 +799,12 @@ jobs: with: name: activation path: /tmp/gh-aw + - name: Restore agent config folders from base branch + if: steps.checkout-pr.outcome == 'success' + env: + GH_AW_AGENT_FOLDERS: ".agents .claude .codex .crush .gemini .github .opencode" + GH_AW_AGENT_FILES: ".crush.json AGENTS.md CLAUDE.md GEMINI.md opencode.jsonc" + run: bash "${RUNNER_TEMP}/gh-aw/actions/restore_base_github_folders.sh" - name: Clean git credentials continue-on-error: true run: bash "${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh" @@ -789,21 +830,25 @@ jobs: run: | set -o pipefail touch /tmp/gh-aw/agent-step-summary.md + GH_AW_NODE_BIN=$(command -v node 2>/dev/null || true) + export GH_AW_NODE_BIN (umask 177 && touch /tmp/gh-aw/agent-stdio.log) # shellcheck disable=SC1003 - sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.18 --skip-pull --enable-api-proxy \ - -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool write --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --image-tag 0.25.28,squid=sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474,agent=sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a,api-proxy=sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb,cli-proxy=sha256:fdf310e4678ce58d248c466b89399e9680a3003038fd19322c388559016aaac7 --skip-pull --enable-api-proxy \ + -- /bin/bash -c 'GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || echo node)"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool write --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log env: COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_API_KEY: dummy-byok-key-for-offline-mode COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} - COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'claude-sonnet-4.6' }} GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json GH_AW_PHASE: agent GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} - GH_AW_VERSION: v0.68.1 + GH_AW_VERSION: v0.71.0 GITHUB_API_URL: ${{ github.api_url }} GITHUB_AW: true + GITHUB_COPILOT_INTEGRATION_ID: agentic-workflows GITHUB_HEAD_REF: ${{ github.head_ref }} GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} GITHUB_REF_NAME: ${{ github.ref_name }} @@ -815,11 +860,11 @@ jobs: GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com GIT_COMMITTER_NAME: github-actions[bot] XDG_CONFIG_HOME: /home/runner - - name: Detect inference access error - id: detect-inference-error + - name: Detect Copilot errors + id: detect-copilot-errors if: always() continue-on-error: true - run: bash "${RUNNER_TEMP}/gh-aw/actions/detect_inference_access_error.sh" + run: node "${RUNNER_TEMP}/gh-aw/actions/detect_copilot_errors.cjs" - name: Configure Git credentials env: REPO_NAME: ${{ github.repository }} @@ -848,7 +893,7 @@ jobs: bash "${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh" "$GATEWAY_PID" - name: Redact secrets in logs if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -874,7 +919,7 @@ jobs: - name: Ingest agent output id: collect_output if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" @@ -888,7 +933,7 @@ jobs: await main(); - name: Parse agent logs for step summary if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ with: @@ -900,7 +945,7 @@ jobs: - name: Parse MCP Gateway logs for step summary if: always() id: parse-mcp-gateway - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -913,9 +958,9 @@ jobs: env: AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs run: | - # Fix permissions on firewall logs so they can be uploaded as artifacts + # Fix permissions on firewall logs/audit dirs so they can be uploaded as artifacts # AWF runs with sudo, creating files owned by root - sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall 2>/dev/null || true # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) if command -v awf &> /dev/null; then awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" @@ -925,7 +970,7 @@ jobs: - name: Parse token usage for step summary if: always() continue-on-error: true - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -941,7 +986,7 @@ jobs: # Upload repo memory as artifacts for push job - name: Upload repo-memory artifact (default) if: always() - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: repo-memory-default path: /tmp/gh-aw/repo-memory/default @@ -953,7 +998,7 @@ jobs: GH_AW_CACHE_DIR: /tmp/gh-aw/cache-memory run: bash "${RUNNER_TEMP}/gh-aw/actions/commit_cache_memory_git.sh" - name: Upload cache-memory data as artifact - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: always() with: name: cache-memory @@ -961,7 +1006,7 @@ jobs: - name: Upload agent artifacts if: always() continue-on-error: true - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: agent path: | @@ -979,14 +1024,6 @@ jobs: /tmp/gh-aw/agent_output.json /tmp/gh-aw/aw-*.patch /tmp/gh-aw/aw-*.bundle - if-no-files-found: ignore - - name: Upload firewall audit logs - if: always() - continue-on-error: true - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 - with: - name: firewall-audit-logs - path: | /tmp/gh-aw/sandbox/firewall/logs/ /tmp/gh-aw/sandbox/firewall/audit/ if-no-files-found: ignore @@ -1019,7 +1056,7 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -1038,9 +1075,9 @@ jobs: mkdir -p /tmp/gh-aw/ find "/tmp/gh-aw/" -type f -print echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" - - name: Process No-Op Messages + - name: Process no-op messages id: noop - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_NOOP_MAX: "1" @@ -1055,9 +1092,25 @@ jobs: setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); await main(); + - name: Log detection run + id: detection_runs + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Hero Scenario Suggestion" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_detection_runs.cjs'); + await main(); - name: Record missing tool id: missing_tool - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_MISSING_TOOL_CREATE_ISSUE: "true" @@ -1071,7 +1124,7 @@ jobs: await main(); - name: Record incomplete id: report_incomplete - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_REPORT_INCOMPLETE_CREATE_ISSUE: "true" @@ -1086,17 +1139,21 @@ jobs: - name: Handle agent failure id: handle_agent_failure if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_WORKFLOW_NAME: "Hero Scenario Suggestion" GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} GH_AW_WORKFLOW_ID: "hero-scenarios" + GH_AW_ACTION_FAILURE_ISSUE_EXPIRES_HOURS: "168" GH_AW_ENGINE_ID: "copilot" GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }} GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }} + GH_AW_MCP_POLICY_ERROR: ${{ needs.agent.outputs.mcp_policy_error }} + GH_AW_AGENTIC_ENGINE_TIMEOUT: ${{ needs.agent.outputs.agentic_engine_timeout }} + GH_AW_MODEL_NOT_SUPPORTED_ERROR: ${{ needs.agent.outputs.model_not_supported_error }} GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }} GH_AW_STALE_LOCK_FILE_FAILED: ${{ needs.activation.outputs.stale_lock_file_failed }} GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e 📖 *Suggested by [{workflow_name}]({run_url})*\",\"runStarted\":\"📖 [{workflow_name}]({run_url}) is generating hero scenarios for this PR…\",\"runSuccess\":\"📖 [{workflow_name}]({run_url}) posted hero scenarios. ✅\",\"runFailure\":\"📖 [{workflow_name}]({run_url}) {status}. ❌\"}" @@ -1126,11 +1183,12 @@ jobs: contents: read outputs: detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} + detection_reason: ${{ steps.detection_conclusion.outputs.reason }} detection_success: ${{ steps.detection_conclusion.outputs.success }} steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -1155,8 +1213,12 @@ jobs: with: persist-credentials: false # --- Threat Detection --- + - name: Clean stale firewall files from agent artifact + run: | + rm -rf /tmp/gh-aw/sandbox/firewall/logs + rm -rf /tmp/gh-aw/sandbox/firewall/audit - name: Download container images - run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.18 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.18 ghcr.io/github/gh-aw-firewall/squid:0.25.18 + run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.28@sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a ghcr.io/github/gh-aw-firewall/api-proxy:0.25.28@sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb ghcr.io/github/gh-aw-firewall/squid:0.25.28@sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474 - name: Check if detection needed id: detection_guard if: always() @@ -1174,7 +1236,7 @@ jobs: - name: Clear MCP configuration for detection if: always() && steps.detection_guard.outputs.run_detection == 'true' run: | - rm -f /tmp/gh-aw/mcp-config/mcp-servers.json + rm -f "${RUNNER_TEMP}/gh-aw/mcp-config/mcp-servers.json" rm -f /home/runner/.copilot/mcp-config.json rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" - name: Prepare threat detection files @@ -1193,7 +1255,7 @@ jobs: ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true - name: Setup threat detection if: always() && steps.detection_guard.outputs.run_detection == 'true' - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: WORKFLOW_NAME: "Hero Scenario Suggestion" WORKFLOW_DESCRIPTION: "Hero Scenarios: Suggest hero scenarios for API specifications" @@ -1209,12 +1271,17 @@ jobs: run: | mkdir -p /tmp/gh-aw/threat-detection touch /tmp/gh-aw/threat-detection/detection.log + - name: Setup Node.js + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + with: + node-version: '24' + package-manager-cache: false - name: Install GitHub Copilot CLI run: bash "${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh" 1.0.21 env: GH_HOST: github.com - name: Install AWF binary - run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.18 + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.28 - name: Execute GitHub Copilot CLI if: always() && steps.detection_guard.outputs.run_detection == 'true' id: detection_agentic_execution @@ -1223,19 +1290,23 @@ jobs: run: | set -o pipefail touch /tmp/gh-aw/agent-step-summary.md + GH_AW_NODE_BIN=$(command -v node 2>/dev/null || true) + export GH_AW_NODE_BIN (umask 177 && touch /tmp/gh-aw/threat-detection/detection.log) # shellcheck disable=SC1003 - sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,telemetry.enterprise.githubcopilot.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.18 --skip-pull --enable-api-proxy \ - -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,telemetry.enterprise.githubcopilot.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --allow-host-ports 80,443,8080 --image-tag 0.25.28,squid=sha256:844c18280f82cd1b06345eb2f4e91966b34185bfc51c9f237c3e022e848fb474,agent=sha256:a8834e285807654bf680154faa710d43fe4365a0868142f5c20e48c85e137a7a,api-proxy=sha256:93290f2393752252911bd7c39a047f776c0b53063575e7bde4e304962a9a61cb,cli-proxy=sha256:fdf310e4678ce58d248c466b89399e9680a3003038fd19322c388559016aaac7 --skip-pull --enable-api-proxy \ + -- /bin/bash -c 'GH_AW_NODE_EXEC="${GH_AW_NODE_BIN:-}"; if [ -z "$GH_AW_NODE_EXEC" ] || [ ! -x "$GH_AW_NODE_EXEC" ]; then GH_AW_NODE_EXEC="$(command -v node 2>/dev/null || echo node)"; fi; "$GH_AW_NODE_EXEC" ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt-file /tmp/gh-aw/aw-prompts/prompt.txt' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log env: COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_API_KEY: dummy-byok-key-for-offline-mode COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} - COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }} + COPILOT_MODEL: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || 'claude-sonnet-4.6' }} GH_AW_PHASE: detection GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_VERSION: v0.68.1 + GH_AW_VERSION: v0.71.0 GITHUB_API_URL: ${{ github.api_url }} GITHUB_AW: true + GITHUB_COPILOT_INTEGRATION_ID: agentic-workflows GITHUB_HEAD_REF: ${{ github.head_ref }} GITHUB_REF_NAME: ${{ github.ref_name }} GITHUB_SERVER_URL: ${{ github.server_url }} @@ -1248,7 +1319,7 @@ jobs: XDG_CONFIG_HOME: /home/runner - name: Upload threat detection log if: always() && steps.detection_guard.outputs.run_detection == 'true' - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: detection path: /tmp/gh-aw/threat-detection/detection.log @@ -1256,9 +1327,10 @@ jobs: - name: Parse and conclude threat detection id: detection_conclusion if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} + GH_AW_DETECTION_CONTINUE_ON_ERROR: "true" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); @@ -1279,13 +1351,13 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} - name: Check team membership for workflow id: check_membership - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_REQUIRED_ROLES: "admin,maintainer,write" with: @@ -1317,8 +1389,8 @@ jobs: - agent - detection if: > - always() && (needs.detection.result == 'success' || needs.detection.result == 'skipped') && - needs.agent.result == 'success' + always() && (!cancelled()) && (needs.detection.result == 'success' || needs.detection.result == 'skipped') && + needs.agent.result != 'skipped' runs-on: ubuntu-slim permissions: contents: write @@ -1332,7 +1404,7 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -1364,7 +1436,7 @@ jobs: - name: Push repo-memory changes (default) id: push_repo_memory_default if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_TOKEN: ${{ github.token }} GITHUB_RUN_ID: ${{ github.run_id }} @@ -1399,9 +1471,12 @@ jobs: timeout-minutes: 15 env: GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/hero-scenarios" + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens }} GH_AW_ENGINE_ID: "copilot" GH_AW_ENGINE_MODEL: ${{ needs.agent.outputs.model }} + GH_AW_ENGINE_VERSION: "1.0.21" GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e 📖 *Suggested by [{workflow_name}]({run_url})*\",\"runStarted\":\"📖 [{workflow_name}]({run_url}) is generating hero scenarios for this PR…\",\"runSuccess\":\"📖 [{workflow_name}]({run_url}) posted hero scenarios. ✅\",\"runFailure\":\"📖 [{workflow_name}]({run_url}) {status}. ❌\"}" GH_AW_WORKFLOW_ID: "hero-scenarios" GH_AW_WORKFLOW_NAME: "Hero Scenario Suggestion" @@ -1417,7 +1492,7 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -1447,7 +1522,7 @@ jobs: echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" - name: Process Safe Outputs id: process_safe_outputs - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" @@ -1463,10 +1538,12 @@ jobs: await main(); - name: Upload Safe Outputs Items if: always() - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: safe-outputs-items - path: /tmp/gh-aw/safe-output-items.jsonl + path: | + /tmp/gh-aw/safe-output-items.jsonl + /tmp/gh-aw/temporary-id-map.json if-no-files-found: ignore update_cache_memory: @@ -1484,7 +1561,7 @@ jobs: steps: - name: Setup Scripts id: setup - uses: github/gh-aw-actions/setup@2fe53acc038ba01c3bbdc767d4b25df31ca5bdfc # v0.68.1 + uses: github/gh-aw-actions/setup@49157453228f9641824955e35cbeccbca74ee0fd # v0.71.0 with: destination: ${{ runner.temp }}/gh-aw/actions job-name: ${{ github.job }} @@ -1507,7 +1584,7 @@ jobs: fi - name: Save cache-memory to cache (default) if: steps.check_cache_default.outputs.has_content == 'true' - uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4 + uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 with: key: memory-unapproved-e3d939f8-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }} path: /tmp/gh-aw/cache-memory From 61471a2157fccb5d25b30ef37e320660413c599c Mon Sep 17 00:00:00 2001 From: Deyaa Eldeen Date: Fri, 24 Apr 2026 17:31:13 +0000 Subject: [PATCH 3/3] Add Step 3.5: verify each scenario against the spec before posting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a mandatory verification gate between scenario generation (Step 3) and posting (Step 4). The agent must re-read the actual spec files and verify each HTTP call against the definitions — not rely on memory. Per-step checks: operation existence, request/response shape, property type validation (uuid/url/datetime/enum), no invented batching. Per-scenario checks: data flow continuity across steps, narrative completeness (every claim backed by an API call), existing document convention matching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/prompts/hero-scenarios-guidelines.md | 54 ++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/.github/prompts/hero-scenarios-guidelines.md b/.github/prompts/hero-scenarios-guidelines.md index c8673aa228e7..969ee41e10b5 100644 --- a/.github/prompts/hero-scenarios-guidelines.md +++ b/.github/prompts/hero-scenarios-guidelines.md @@ -298,6 +298,60 @@ the file first and adopt its conventions: section ordering (e.g., Story → Prerequisites → Steps → Success Criteria → Value Delivered) used by existing scenarios. +## Step 3.5 — Verify Each Scenario Against the Spec + +After generating scenarios and before posting, walk through every +scenario and verify it against the actual specification files. Do NOT +rely on memory from Step 2 — re-read the relevant spec files during +this step. Do not proceed to Step 4 until every scenario passes. + +### Per-step verification + +For **each HTTP request/response** in the scenario: + +1. **Operation exists.** Search the TypeSpec or OpenAPI files for the + matching route (method + path pattern). If the operation does not + exist in the spec, remove or rewrite the step. +2. **Request shape matches.** Every property in the example request body + must exist in the operation's request model as defined in the spec. + Remove any invented properties. +3. **Response shape matches.** The example response must match the + operation's return type — single object, paged list with `value` + array and `nextLink`, or no body (for `202`/`204`). Do not invent + a response body for operations that return no content. Do not show + a batch/array response for an operation that returns a single item. +4. **Example values match property types.** Re-read the model definition + for each property and verify: + - `format: uuid` → valid UUID (`xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`) + - `format: uri` / `url` type → includes scheme (`https://...`) + - `format: date-time` → ISO 8601 + - enum/union → one of the values defined in the spec + - integer → a number, not a string +5. **No invented batching.** If the scenario performs the same operation + on multiple resources (e.g., tag two columns), verify the spec has a + batch endpoint. If not, split into separate calls — one per resource. + +### Per-scenario verification + +After checking every step, review the scenario as a whole: + +6. **Data flow continuity.** An ID or value returned in one step's + response and used in a later step's request must be identical in + both places — no typos, no mismatched UUIDs. +7. **Narrative completeness.** Re-read the scenario description and + success criteria. Every action described must have a corresponding + API call. Every success criterion must be achievable solely through + the steps shown. Remove any success criteria for actions that were + omitted from the steps. +8. **Existing document conventions.** If appending to an existing + `hero-scenarios.md`, re-read the file and confirm your scenarios + use the same URL path format (short vs. full with `api-version`), + the same ID style (placeholders vs. concrete UUIDs), and the same + section structure. + +If any check fails, fix the scenario and re-verify the fix before +proceeding. + ## Step 4 — Post the Suggestion **If no `hero-scenarios.md` exists**, use `add-comment` to post a PR comment