Skip to content

Conversation

@fjglira
Copy link
Contributor

@fjglira fjglira commented Sep 18, 2025

The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

What type of PR is this?

  • Enhancement / New Feature
  • Bug Fix
  • Refactor
  • Optimization
  • Test
  • Documentation Update

What this PR does / why we need it:

Problem:
The e2e test script was storing the Ginkgo exit code, but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Solution:

  • Added a cleanup() function with trap cleanup EXIT INT TERM to ensure uninstall always runs
  • Moved cleanup logic to the trap so it executes on normal exit, interrupts, and terminations
  • Simplified the main flow to immediately exit with the captured Ginkgo exit code
  • Cleanup errors are suppressed to preserve the original test result

Result:

  • CI now correctly reflects test failures and interruptions
  • JUnit reports are still generated and cleanup always occurs
  • No more false positive test results from incomplete runs

Which issue(s) this PR fixes:

Fixes #1226

Related Issue/PR #

Additional information:

@fjglira fjglira requested a review from a team as a code owner September 18, 2025 20:06
@codecov
Copy link

codecov bot commented Sep 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.54%. Comparing base (749d908) to head (3bbde83).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1227      +/-   ##
==========================================
+ Coverage   77.39%   77.54%   +0.14%     
==========================================
  Files          44       44              
  Lines        2823     2823              
==========================================
+ Hits         2185     2189       +4     
+ Misses        529      526       -3     
+ Partials      109      108       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fjglira fjglira changed the title fix(e2e): Add cleanup trap to prevent false positives in CI test runs WIP fix(e2e): Add cleanup trap to prevent false positives in CI test runs Sep 18, 2025
The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Signed-off-by: Francisco Herrera <[email protected]>
TEMPORARY FAILURE: check failure test on CI

Signed-off-by: Francisco Herrera <[email protected]>
Fix lint messages

Signed-off-by: Francisco Herrera <[email protected]>
This reverts commit 5346be9.

Signed-off-by: Francisco Herrera <[email protected]>
@fjglira fjglira changed the title WIP fix(e2e): Add cleanup trap to prevent false positives in CI test runs fix(e2e): Add cleanup trap to prevent false positives in CI test runs Sep 18, 2025
Copy link
Contributor

@ctartici ctartici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@istio-testing istio-testing merged commit e774b44 into istio-ecosystem:main Sep 19, 2025
17 checks passed
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 19, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 19, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 24, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 24, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
@istio-testing
Copy link
Collaborator

In response to a cherrypick label: new pull request created: #1232

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Falses positives when the e2e test are not completed and result is not being generated

5 participants