uucore: stop option parsing at first operand when POSIXLY_CORRECT is set #41757
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: GnuTests | |
| # spell-checker:ignore (abbrev/names) CodeCov gnulib GnuTests | |
| # spell-checker:ignore (jargon) submodules devel | |
| # spell-checker:ignore (libs/utils) chksum dpkg getenforce gperf lcov libexpect limactl pyinotify setenforce shopt valgrind libattr libcap taiki-e zstd cpio | |
| # spell-checker:ignore (options) Ccodegen Coverflow Cpanic Zpanic | |
| # spell-checker:ignore (vars) FILESET SUBDIRS XPASS | |
| # spell-checker:ignore userns nodocs | |
| # * note: to run a single test => `REPO/util/run-gnu-test.sh PATH/TO/TEST/SCRIPT` | |
| on: | |
| pull_request: | |
| push: | |
| branches: | |
| - '*' | |
| ## the reference branch carries no code, only published test results | |
| - '!gnu-reference' | |
| permissions: | |
| contents: read | |
| # End the current execution if there is a new changeset in the PR. | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} | |
| env: | |
| DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} | |
| ## Machine-maintained branch holding the latest GNU test results of the | |
| ## default branch; every PR run is compared against what it carries. | |
| REFERENCE_BRANCH: 'gnu-reference' | |
| TEST_FULL_SUMMARY_FILE: 'gnu-full-result.json' | |
| TEST_ROOT_FULL_SUMMARY_FILE: 'gnu-root-full-result.json' | |
| TEST_STTY_FULL_SUMMARY_FILE: 'gnu-stty-full-result.json' | |
| TEST_SELINUX_FULL_SUMMARY_FILE: 'selinux-gnu-full-result.json' | |
| TEST_SELINUX_ROOT_FULL_SUMMARY_FILE: 'selinux-root-gnu-full-result.json' | |
| TEST_QEMU_FULL_SUMMARY_FILE: 'qemu-gnu-full-result.json' | |
| jobs: | |
| native: | |
| name: Run GNU tests (native) | |
| runs-on: ubuntu-26.04 # some strace? test needs newer glibc | |
| steps: | |
| #### Get the code | |
| - name: Checkout code (uutils) | |
| uses: actions/checkout@v7.0.1 | |
| with: | |
| path: 'uutils' | |
| persist-credentials: false | |
| - uses: Swatinem/rust-cache@v2 | |
| with: | |
| workspaces: "./uutils -> target" | |
| - name: Checkout code (GNU coreutils) | |
| run: (mkdir -p gnu && cd gnu && bash ../uutils/util/fetch-gnu.sh) | |
| #### Build environment setup | |
| - name: Install dependencies | |
| shell: bash | |
| run: | | |
| ## Install dependencies | |
| sudo apt-get update | |
| ## Check that build-gnu.sh works on the non SELinux system by installing libselinux only on lima | |
| sudo apt-get install -y gperf gdb python3-pyinotify valgrind libexpect-perl libacl1-dev libattr1-dev libcap-dev attr quilt | |
| - name: Add various locales | |
| shell: bash | |
| run: | | |
| ## Add various locales | |
| echo "Before:" | |
| locale -a | |
| ## Some tests fail with 'cannot change locale (en_US.ISO-8859-1): No such file or directory' | |
| ## Some others need a French locale | |
| sudo locale-gen | |
| sudo locale-gen --keep-existing fr_FR | |
| sudo locale-gen --keep-existing fr_FR.UTF-8 | |
| sudo locale-gen --keep-existing es_ES.UTF-8 | |
| sudo locale-gen --keep-existing sv_SE | |
| sudo locale-gen --keep-existing sv_SE.UTF-8 | |
| sudo locale-gen --keep-existing en_US | |
| sudo locale-gen --keep-existing en_US.UTF-8 | |
| sudo locale-gen --keep-existing ru_RU.KOI8-R | |
| sudo locale-gen --keep-existing fa_IR.UTF-8 # Iran | |
| sudo locale-gen --keep-existing am_ET.UTF-8 # Ethiopia | |
| sudo locale-gen --keep-existing th_TH.UTF-8 # Thailand | |
| sudo locale-gen --keep-existing zh_CN.GB18030 # China | |
| sudo update-locale | |
| echo "After:" | |
| locale -a | |
| ### Build | |
| - name: Build binaries | |
| shell: bash | |
| run: | | |
| ## Build binaries | |
| cd 'uutils' | |
| env PROFILE=release-small bash util/build-gnu.sh | |
| ### Run tests as user | |
| - name: Run GNU tests | |
| shell: bash | |
| run: | | |
| ## Use unshare | |
| sudo sysctl -w kernel.unprivileged_userns_clone=1 | |
| sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 | |
| ## Run GNU tests | |
| path_GNU='gnu' | |
| path_UUTILS='uutils' | |
| bash "uutils/util/run-gnu-test.sh" | |
| - name: Extract testing info from individual logs into JSON | |
| shell: bash | |
| run : | | |
| path_UUTILS='uutils' | |
| python uutils/util/gnu-json-result.py gnu/tests > ${{ env.TEST_FULL_SUMMARY_FILE }} | |
| ### Run tests as root | |
| - name: Run GNU root tests | |
| shell: bash | |
| run: | | |
| ## Run GNU root tests | |
| path_GNU='gnu' | |
| path_UUTILS='uutils' | |
| bash "uutils/util/run-gnu-test.sh" run-root | |
| - name: Extract testing info from individual logs (run as root) into JSON | |
| shell: bash | |
| run : | | |
| path_UUTILS='uutils' | |
| python uutils/util/gnu-json-result.py gnu/tests > ${{ env.TEST_ROOT_FULL_SUMMARY_FILE }} | |
| ### This shell has been changed from "bash" to this command | |
| ### "script" will start a pty and the -q command removes the "script" initiation log | |
| ### the -e flag makes it propagate the error code and -c runs the command in a pty | |
| ### the primary purpose of this change is to run the tty GNU tests | |
| ### The reason its separated from the rest of the tests is because one test can corrupt the other | |
| ### tests through the use of the shared terminal and it changes the environment that the other | |
| ### tests are run in, which can cause different results. | |
| - name: Run GNU stty tests | |
| shell: 'script -q -e -c "bash {0}"' | |
| run: | | |
| ## Run GNU root tests | |
| path_GNU='gnu' | |
| path_UUTILS='uutils' | |
| bash "uutils/util/run-gnu-test.sh" run-tty | |
| - name: Extract testing info from individual logs (stty) into JSON | |
| shell: bash | |
| run : | | |
| path_UUTILS='uutils' | |
| python uutils/util/gnu-json-result.py gnu/tests > ${{ env.TEST_STTY_FULL_SUMMARY_FILE }} | |
| ### Upload artifacts | |
| - name: Upload full json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: gnu-full-result | |
| path: ${{ env.TEST_FULL_SUMMARY_FILE }} | |
| - name: Upload root json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: gnu-root-full-result | |
| path: ${{ env.TEST_ROOT_FULL_SUMMARY_FILE }} | |
| - name: Upload stty json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: gnu-stty-full-result | |
| path: ${{ env.TEST_STTY_FULL_SUMMARY_FILE }} | |
| - name: Compress test logs | |
| shell: bash | |
| run : | | |
| # Compress logs before upload (fails otherwise) | |
| gzip gnu/tests/*/*.log | |
| - name: Upload test logs | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: test-logs | |
| path: | | |
| gnu/tests/*.log | |
| gnu/tests/*/*.log.gz | |
| selinux: | |
| name: Run GNU tests (SELinux) | |
| runs-on: ubuntu-latest # using VM's glibc | |
| steps: | |
| #### Get the code | |
| - name: Checkout code (uutils) | |
| uses: actions/checkout@v7.0.1 | |
| with: | |
| path: 'uutils' | |
| persist-credentials: false | |
| - name: Checkout code (GNU coreutils) | |
| run: (mkdir -p gnu && cd gnu && bash ../uutils/util/fetch-gnu.sh) | |
| #### Lima build environment setup | |
| - name: Setup Lima | |
| uses: lima-vm/lima-actions/setup@v1 | |
| id: lima-actions-setup | |
| - name: Cache ~/.cache/lima | |
| uses: actions/cache@v6 | |
| with: | |
| path: ~/.cache/lima | |
| key: lima-${{ steps.lima-actions-setup.outputs.version }} | |
| - name: Start Fedora VM with SELinux | |
| run: limactl start --plain --name=default --cpus=4 --disk=40 --memory=8 --network=lima:user-v2 template:fedora | |
| - name: Verify SELinux Status and Configuration | |
| run: | | |
| lima getenforce | |
| lima ls -laZ /etc/selinux | |
| lima sudo sestatus | |
| - name: Install dependencies in VM | |
| run: | | |
| lima sudo dnf -y install --nodocs autoconf bison gperf gcc gdb jq libacl-devel libattr-devel libcap-devel libselinux-devel attr rustup clang-devel automake patch quilt | |
| lima rustup-init -y --profile=minimal --default-toolchain stable | |
| - name: Copy the sources to VM | |
| run: | | |
| rsync -a -e ssh . lima-default:~/work/ | |
| ### Build | |
| - name: Build binaries | |
| run: | | |
| lima bash -c "cd ~/work/uutils/ && SELINUX_ENABLED=1 PROFILE=release-small bash util/build-gnu.sh" | |
| ### Run tests as user | |
| - name: Generate SELinux tests list | |
| run: | | |
| # Find and list all tests that require SELinux | |
| lima bash -c "cd ~/work/gnu/ && grep -l 'require_selinux_' -r tests/ > ~/work/uutils/selinux-tests.txt" | |
| lima bash -c "cd ~/work/uutils/ && cat selinux-tests.txt" | |
| # Count the tests | |
| lima bash -c "cd ~/work/uutils/ && echo 'Found SELinux tests:'; wc -l selinux-tests.txt" | |
| - name: Run GNU SELinux tests | |
| run: | | |
| # Ensure we're running in enforcing mode | |
| lima sudo setenforce 1 | |
| lima getenforce | |
| # Create test files with SELinux contexts for testing | |
| lima sudo mkdir -p /var/test_selinux | |
| lima sudo touch /var/test_selinux/test_file | |
| lima sudo chcon -t etc_t /var/test_selinux/test_file | |
| lima ls -Z /var/test_selinux/test_file # Verify context | |
| lima cat /proc/filesystems | |
| lima bash -c "cd ~/work/uutils/ && bash util/run-gnu-test.sh \$(cat selinux-tests.txt)" | |
| - name: Extract testing info from individual logs into JSON | |
| shell: bash | |
| run : | | |
| lima bash -c "cd ~/work/gnu/ && python3 ../uutils/util/gnu-json-result.py tests > ~/work/${{ env.TEST_SELINUX_FULL_SUMMARY_FILE }}" | |
| ### Run tests as root | |
| - name: Run GNU SELinux root tests | |
| run: | | |
| lima bash -c "cd ~/work/uutils/ && CI=1 bash util/run-gnu-test.sh run-root \$(cat selinux-tests.txt)" | |
| - name: Extract testing info from individual logs (run as root) into JSON | |
| shell: bash | |
| run : | | |
| lima bash -c "cd ~/work/gnu/ && python3 ../uutils/util/gnu-json-result.py tests > ~/work/${{ env.TEST_SELINUX_ROOT_FULL_SUMMARY_FILE }}" | |
| ### Upload artifacts | |
| - name: Collect test logs and test results from VM | |
| run: | | |
| mkdir -p gnu/tests-selinux | |
| # Copy the json output back from the Lima VM to the host | |
| rsync -v -a -e ssh lima-default:~/work/*.json ./ | |
| # Copy the test directory now | |
| rsync -v -a -e ssh lima-default:~/work/gnu/tests/ ./gnu/tests-selinux/ | |
| - name: Upload SELinux json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: selinux-gnu-full-result | |
| path: ${{ env.TEST_SELINUX_FULL_SUMMARY_FILE }} | |
| - name: Upload SELinux root json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: selinux-root-gnu-full-result | |
| path: ${{ env.TEST_SELINUX_ROOT_FULL_SUMMARY_FILE }} | |
| - name: Compress SELinux test logs | |
| shell: bash | |
| run : | | |
| # Compress logs before upload (fails otherwise) | |
| gzip gnu/tests-selinux/*/*.log | |
| - name: Upload SELinux test logs | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: selinux-test-logs | |
| path: | | |
| gnu/tests-selinux/*.log | |
| gnu/tests-selinux/*/*.log.gz | |
| qemu: | |
| name: Run GNU tests (SMACK/ROOTFS) | |
| runs-on: ubuntu-latest # using Arch glibc | |
| steps: | |
| - name: Checkout code (uutils) | |
| uses: actions/checkout@v7.0.1 | |
| with: | |
| path: 'uutils' | |
| persist-credentials: false | |
| - uses: Swatinem/rust-cache@v2 | |
| with: | |
| workspaces: "./uutils -> target" | |
| - name: Checkout code (GNU coreutils) | |
| run: (mkdir -p gnu && cd gnu && bash ../uutils/util/fetch-gnu.sh) | |
| - name: Install dependencies | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y qemu-system-x86 zstd cpio | |
| - name: Run GNU SMACK/ROOTFS tests | |
| run: | | |
| cd uutils | |
| bash util/run-gnu-tests-smack-ci.sh "$GITHUB_WORKSPACE/gnu" "$GITHUB_WORKSPACE/gnu/tests-qemu" | |
| - name: Extract testing info into JSON | |
| run: | | |
| python3 uutils/util/gnu-json-result.py gnu/tests-qemu > ${{ env.TEST_QEMU_FULL_SUMMARY_FILE }} | |
| - name: Upload SMACK/ROOTFS json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: qemu-gnu-full-result | |
| path: ${{ env.TEST_QEMU_FULL_SUMMARY_FILE }} | |
| - name: Compress SMACK/ROOTFS test logs | |
| run: gzip gnu/tests-qemu/*/*.log 2>/dev/null || true | |
| - name: Upload SMACK/ROOTFS test logs | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: qemu-test-logs | |
| path: | | |
| gnu/tests-qemu/*.log | |
| gnu/tests-qemu/*/*.log.gz | |
| aggregate: | |
| needs: [native, selinux, qemu] | |
| permissions: | |
| actions: read # to download this run's own result artifacts | |
| contents: read # for actions/checkout to fetch code, and to read the reference branch | |
| name: Aggregate GNU test results | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Initialize workflow variables | |
| id: vars | |
| shell: bash | |
| run: | | |
| ## VARs setup | |
| outputs() { step_id="${{ github.action }}"; for var in "$@" ; do echo steps.${step_id}.outputs.${var}="${!var}"; echo "${var}=${!var}" >> $GITHUB_OUTPUT; done; } | |
| # | |
| TEST_SUMMARY_FILE='gnu-result.json' | |
| AGGREGATED_SUMMARY_FILE='aggregated-result.json' | |
| outputs TEST_SUMMARY_FILE AGGREGATED_SUMMARY_FILE | |
| - name: Checkout code (uutils) | |
| uses: actions/checkout@v7.0.1 | |
| with: | |
| path: 'uutils' | |
| persist-credentials: false | |
| - name: Fetch reference results (default branch) | |
| shell: bash | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| ## Fetch the reference results published by the last GnuTests run on the | |
| ## default branch. | |
| ## They live on a dedicated, machine-maintained branch rather than in a | |
| ## workflow artifact: an artifact is addressable only by run id, so | |
| ## finding one meant listing workflow runs, and the server-side 'branch' | |
| ## filter of that listing is served by a search index that is only | |
| ## eventually consistent (observed returning nothing newer than a month | |
| ## old while the unfiltered listing was current). A run resolved from a | |
| ## stale listing made every test fixed on the default branch since then | |
| ## look like a fix contributed by the PR, and every test broken since | |
| ## then like a regression introduced by it. A branch has no such index, | |
| ## and no 90-day expiry either. | |
| ## A missing reference is not fatal: the branch does not exist until the | |
| ## first run on the default branch has published to it. | |
| mkdir -p reference/aggregated-result reference/test-summary | |
| ## Resolve the branch tip once and read every file at that commit. The | |
| ## branch is force-pushed by 'publish-reference', so reading each file | |
| ## by branch name could straddle a publish and pair one file with | |
| ## another reference's, comparing against a baseline that never existed. | |
| ## Only a 404 means "no reference yet"; any other failure must not be | |
| ## mistaken for one, or a transient API error would silently skip both | |
| ## comparisons and let a regression through unnoticed. | |
| REFERENCE_SHA='' | |
| ref_err="${RUNNER_TEMP:-/tmp}/reference-ref.err" | |
| for attempt in 1 2 3; do | |
| if REFERENCE_SHA=$(gh api "repos/${{ github.repository }}/git/refs/heads/${REFERENCE_BRANCH}" \ | |
| --jq '.object.sha' 2>"${ref_err}"); then | |
| break | |
| fi | |
| REFERENCE_SHA='' | |
| if grep -q 'HTTP 404' "${ref_err}"; then | |
| ## The branch does not exist until the first run on the default | |
| ## branch has published to it. | |
| break | |
| fi | |
| if [[ "${attempt}" -eq 3 ]]; then | |
| cat "${ref_err}" >&2 | |
| echo "::error ::Could not resolve the '${REFERENCE_BRANCH}' branch; refusing to report an API failure as a missing baseline." | |
| exit 1 | |
| fi | |
| sleep $(( attempt * 5 )) | |
| done | |
| fetch() { | |
| gh api "repos/${{ github.repository }}/contents/${1}?ref=${REFERENCE_SHA}" \ | |
| -H 'Accept: application/vnd.github.raw' > "${2}" 2>/dev/null \ | |
| && [[ -s "${2}" ]] | |
| } | |
| if [[ -n "${REFERENCE_SHA}" ]] \ | |
| && fetch aggregated-result.json reference/aggregated-result/aggregated-result.json \ | |
| && fetch gnu-result.json reference/test-summary/gnu-result.json; then | |
| ## 'meta.json' says which commit the reference was built from; it is | |
| ## informational, so tolerate it being absent or unparsable. | |
| meta=$(gh api "repos/${{ github.repository }}/contents/meta.json?ref=${REFERENCE_SHA}" \ | |
| -H 'Accept: application/vnd.github.raw' 2>/dev/null || true) | |
| published_sha=$(jq -r '.sha // empty' <<< "${meta}" 2>/dev/null || true) | |
| published_at=$(jq -r '.published_at // empty' <<< "${meta}" 2>/dev/null || true) | |
| echo "Reference: ${{ env.DEFAULT_BRANCH }}@${published_sha:-<unknown>} (published ${published_at:-<unknown>}, ${REFERENCE_BRANCH}@${REFERENCE_SHA})" | |
| published_epoch='' | |
| if [[ -n "${published_at}" ]]; then | |
| published_epoch=$(date -u -d "${published_at}" +%s 2>/dev/null || true) | |
| fi | |
| if [[ -n "${published_epoch}" ]]; then | |
| age_days=$(( ( $(date -u +%s) - published_epoch ) / 86400 )) | |
| if [[ "${age_days}" -gt 2 ]]; then | |
| echo "::warning ::The reference results are ${age_days} days old (${published_at}); the comparison below may report unrelated fixes and regressions." | |
| fi | |
| fi | |
| else | |
| rm -f reference/aggregated-result/aggregated-result.json reference/test-summary/gnu-result.json | |
| echo "::warning ::No reference results found on the '${REFERENCE_BRANCH}' branch; the comparisons below will be skipped." | |
| fi | |
| - name: Download full json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: gnu-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Download root json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: gnu-root-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Download stty json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: gnu-stty-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Download selinux json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: selinux-gnu-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Download selinux root json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: selinux-root-gnu-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Download SMACK/ROOTFS json results | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: qemu-gnu-full-result | |
| path: results | |
| merge-multiple: true | |
| - name: Extract/summarize testing info | |
| id: summary | |
| shell: bash | |
| run: | | |
| ## Extract/summarize testing info | |
| outputs() { step_id="${{ github.action }}"; for var in "$@" ; do echo steps.${step_id}.outputs.${var}="${!var}"; echo "${var}=${!var}" >> $GITHUB_OUTPUT; done; } | |
| path_UUTILS='uutils' | |
| json_count=$(ls -l results/*.json | wc -l) | |
| if [[ "$json_count" -ne 6 ]]; then | |
| echo "::error ::Failed to download all results json files (expected 6 files, found $json_count); failing early" | |
| ls -lR results || true | |
| exit 1 | |
| fi | |
| # Look at all individual results and summarize | |
| eval $(python3 uutils/util/analyze-gnu-results.py -o=${{ steps.vars.outputs.AGGREGATED_SUMMARY_FILE }} results/*.json) | |
| if [[ "$TOTAL" -eq 0 || "$TOTAL" -eq 1 ]]; then | |
| echo "::error ::Failed to parse test results from '${{ env.TEST_FULL_SUMMARY_FILE }}'; failing early" | |
| exit 1 | |
| fi | |
| output="GNU tests summary = TOTAL: $TOTAL / PASS: $PASS / FAIL: $FAIL / ERROR: $ERROR / SKIP: $SKIP" | |
| echo "${output}" | |
| if [[ "$FAIL" -gt 0 || "$ERROR" -gt 0 ]]; then | |
| echo "::warning ::${output}" | |
| fi | |
| jq -n \ | |
| --arg date "$(date --rfc-email)" \ | |
| --arg sha "$GITHUB_SHA" \ | |
| --arg total "$TOTAL" \ | |
| --arg pass "$PASS" \ | |
| --arg skip "$SKIP" \ | |
| --arg fail "$FAIL" \ | |
| --arg xpass "$XPASS" \ | |
| --arg error "$ERROR" \ | |
| '{($date): { sha: $sha, total: $total, pass: $pass, skip: $skip, fail: $fail, xpass: $xpass, error: $error, }}' > '${{ steps.vars.outputs.TEST_SUMMARY_FILE }}' | |
| HASH=$(sha1sum '${{ steps.vars.outputs.TEST_SUMMARY_FILE }}' | cut --delim=" " -f 1) | |
| outputs HASH | |
| - name: Upload SHA1/ID of 'test-summary' | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: "${{ steps.summary.outputs.HASH }}" | |
| path: "${{ steps.vars.outputs.TEST_SUMMARY_FILE }}" | |
| - name: Upload test results summary | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: test-summary | |
| path: "${{ steps.vars.outputs.TEST_SUMMARY_FILE }}" | |
| - name: Upload aggregated json results | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: aggregated-result | |
| path: ${{ steps.vars.outputs.AGGREGATED_SUMMARY_FILE }} | |
| - name: Compare test failures VS reference | |
| shell: bash | |
| run: | | |
| ## Compare test failures VS reference using JSON files | |
| REF_SUMMARY_FILE='reference/aggregated-result/aggregated-result.json' | |
| CURRENT_SUMMARY_FILE='${{ steps.vars.outputs.AGGREGATED_SUMMARY_FILE }}' | |
| REPO_DEFAULT_BRANCH='${{ env.DEFAULT_BRANCH }}' | |
| path_UUTILS='uutils' | |
| # Path to ignore file for intermittent issues | |
| IGNORE_INTERMITTENT="uutils/.github/workflows/ignore-intermittent.txt" | |
| # Set up comment directory | |
| COMMENT_DIR="reference/comment" | |
| mkdir -p ${COMMENT_DIR} | |
| echo ${{ github.event.number }} > ${COMMENT_DIR}/NR | |
| COMMENT_LOG="${COMMENT_DIR}/result.txt" | |
| COMPARISON_RESULT=0 | |
| if test -f "${CURRENT_SUMMARY_FILE}"; then | |
| if test -f "${REF_SUMMARY_FILE}"; then | |
| echo "Reference summary SHA1/ID: $(sha1sum -- "${REF_SUMMARY_FILE}")" | |
| echo "Current summary SHA1/ID: $(sha1sum -- "${CURRENT_SUMMARY_FILE}")" | |
| python3 uutils/util/compare_test_results.py \ | |
| --ignore-file "${IGNORE_INTERMITTENT}" \ | |
| --output "${COMMENT_LOG}" \ | |
| "${CURRENT_SUMMARY_FILE}" "${REF_SUMMARY_FILE}" | |
| COMPARISON_RESULT=$? | |
| else | |
| echo "::warning ::Skipping test comparison; no prior reference summary is available at '${REF_SUMMARY_FILE}'." | |
| fi | |
| else | |
| echo "::error ::Failed to find summary of test results (missing '${CURRENT_SUMMARY_FILE}'); failing early" | |
| exit 1 | |
| fi | |
| if [ ${COMPARISON_RESULT} -eq 1 ]; then | |
| echo "ONLY_INTERMITTENT=false" >> $GITHUB_ENV | |
| echo "::error ::Found new non-intermittent test failures" | |
| exit 1 | |
| else | |
| echo "ONLY_INTERMITTENT=true" >> $GITHUB_ENV | |
| echo "::notice ::No new test failures detected" | |
| fi | |
| - name: Upload comparison log (for GnuComment workflow) | |
| if: success() || failure() # run regardless of prior step success/failure | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: comment | |
| path: reference/comment/ | |
| - name: Compare test summary VS reference | |
| if: success() || failure() # run regardless of prior step success/failure | |
| shell: bash | |
| run: | | |
| ## Compare test summary VS reference | |
| REF_SUMMARY_FILE='reference/test-summary/gnu-result.json' | |
| if test -f "${REF_SUMMARY_FILE}"; then | |
| echo "Reference SHA1/ID: $(sha1sum -- "${REF_SUMMARY_FILE}")" | |
| mv "${REF_SUMMARY_FILE}" main-gnu-result.json | |
| python uutils/util/compare_gnu_result.py | |
| else | |
| echo "::warning ::Skipping test summary comparison; no prior reference summary is available." | |
| fi | |
| publish-reference: | |
| ## Publish this run's results as the reference that every later PR run is | |
| ## compared against. Only runs on a push to the default branch, so a fork | |
| ## PR can never reach it. | |
| needs: aggregate | |
| if: > | |
| always() | |
| && github.event_name == 'push' | |
| && github.ref_name == github.event.repository.default_branch | |
| && needs.aggregate.result != 'cancelled' | |
| && needs.aggregate.result != 'skipped' | |
| permissions: | |
| actions: read # to download this run's own result artifacts | |
| contents: write # to push to the reference branch | |
| name: Publish reference GNU test results | |
| runs-on: ubuntu-latest | |
| ## Only one run may publish at a time. The step below reads the currently | |
| ## published run id and then force-pushes; two overlapping runs could | |
| ## interleave those, letting an older one overwrite a newer reference. | |
| concurrency: | |
| group: publish-gnu-reference | |
| cancel-in-progress: false | |
| steps: | |
| - name: Download aggregated json results | |
| continue-on-error: true ## 'aggregate' may have failed before uploading | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: aggregated-result | |
| path: publish | |
| - name: Download test results summary | |
| continue-on-error: true ## 'aggregate' may have failed before uploading | |
| uses: actions/download-artifact@v8 | |
| with: | |
| name: test-summary | |
| path: publish | |
| - name: Publish to the reference branch | |
| shell: bash | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| ## Publish to the reference branch | |
| ## The branch is a single-commit orphan history, force-pushed on every | |
| ## run, so it never accumulates history. Runs on the default branch can | |
| ## finish out of order, so refuse to overwrite a reference published by a | |
| ## *newer* run (run ids are monotonic). | |
| if [[ ! -s publish/aggregated-result.json || ! -s publish/gnu-result.json ]]; then | |
| echo "::warning ::This run produced no results to publish; leaving the '${REFERENCE_BRANCH}' branch untouched." | |
| exit 0 | |
| fi | |
| ## Fail closed: only a 404 means nothing has been published yet. A | |
| ## suppressed error read as run id 0 would let this run force-push over | |
| ## a reference a newer run had already placed. | |
| published_run=0 | |
| meta_err="${RUNNER_TEMP:-/tmp}/published-meta.err" | |
| for attempt in 1 2 3; do | |
| if published_meta=$(gh api "repos/${{ github.repository }}/contents/meta.json?ref=${REFERENCE_BRANCH}" \ | |
| -H 'Accept: application/vnd.github.raw' 2>"${meta_err}"); then | |
| published_run=$(jq -r '.run_id // 0' <<< "${published_meta}" 2>/dev/null || echo 0) | |
| break | |
| fi | |
| if grep -q 'HTTP 404' "${meta_err}"; then | |
| break | |
| fi | |
| if [[ "${attempt}" -eq 3 ]]; then | |
| cat "${meta_err}" >&2 | |
| echo "::error ::Could not read the published reference; refusing to force-push over a possibly newer one." | |
| exit 1 | |
| fi | |
| sleep $(( attempt * 5 )) | |
| done | |
| if [[ "${published_run}" =~ ^[0-9]+$ ]] && (( published_run > GITHUB_RUN_ID )); then | |
| echo "::notice ::Run ${published_run} already published a newer reference; nothing to do." | |
| exit 0 | |
| fi | |
| jq -n --arg sha "${GITHUB_SHA}" \ | |
| --arg run_id "${GITHUB_RUN_ID}" \ | |
| --arg branch "${GITHUB_REF_NAME}" \ | |
| --arg published_at "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ | |
| '{sha: $sha, run_id: $run_id, branch: $branch, published_at: $published_at}' \ | |
| > publish/meta.json | |
| printf '%s\n' \ | |
| '# GNU test reference results' \ | |
| '' \ | |
| 'Machine-maintained branch: the GNU test results of the latest run of' \ | |
| 'the `GnuTests` workflow on the default branch. Every pull request run' \ | |
| 'is compared against the files here, so that it reports only the test' \ | |
| 'changes the pull request is itself responsible for.' \ | |
| '' \ | |
| 'Do not commit here by hand; the branch is force-pushed on every run.' \ | |
| > publish/README.md | |
| cd publish | |
| git init -q -b "${REFERENCE_BRANCH}" | |
| git config user.name 'github-actions[bot]' | |
| git config user.email '41898282+github-actions[bot]@users.noreply.github.com' | |
| git add -A | |
| git commit -q -m "GNU test results for ${GITHUB_SHA}" | |
| git push -q --force \ | |
| "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \ | |
| "HEAD:refs/heads/${REFERENCE_BRANCH}" | |
| echo "::notice ::Published reference results for ${GITHUB_SHA}." |