Skip to content

chore: enforce incremental code coverage at 80% for PR gating#60

Draft
sungjujin wants to merge 1 commit into
masterfrom
chore/enforce-incremental-code-coverage
Draft

chore: enforce incremental code coverage at 80% for PR gating#60
sungjujin wants to merge 1 commit into
masterfrom
chore/enforce-incremental-code-coverage

Conversation

@sungjujin
Copy link
Copy Markdown

Summary

  • Enforce 80% incremental (patch) code coverage for all PRs as a blocking check
  • Project-level coverage remains informational (70% target, non-blocking) during ramp-up
  • Part of Segment 2025 goal: enforce 80% incremental coverage for 80%+ of repos by end of Q1 2026

Changes

codecov.yml (new)

  • patch.default.informational: false — PRs with <80% incremental coverage will be blocked
  • project.default.informational: true — project-level coverage is advisory only

.github/workflows/ci.yml

  • Added -coverprofile=coverage.out flag to go test to generate coverage report
  • Added codecov/codecov-action@v5 step to upload coverage.out to Codecov

Why Patch Coverage (Not Project Coverage)?

Incremental (patch) coverage is more actionable:

  • Only checks lines added or modified in the PR — not the entire repo
  • New code must be tested; existing legacy code is not penalized
  • Avoids the "coverage ratchet" problem where adding tests to old code becomes a prerequisite to merging new features

Notes

  • After merging to master, the first CI run establishes the coverage baseline; subsequent PRs will be gated against that baseline
  • GitHub branch protection rule for the Codecov status check should be added after merge

Reference Implementation

See table-scanner for the canonical setup.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread .github/workflows/ci.yml
go test -race -v -coverprofile=coverage.out ./...
working-directory: './src/github.com/segmentio/nsq-go'
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep identified an issue in your code:

The codecov/codecov-action is pinned to a mutable version tag (v5) instead of an immutable commit SHA, allowing potential supply chain attacks if the repository is compromised.

More details about this

The codecov/codecov-action is referenced by a version tag (v5) instead of being pinned to a specific commit SHA. This means the action can change without your knowledge—if a malicious actor gains control of the codecov/codecov-action repository, they could push a backdoored version to the v5 tag, and your workflow would automatically use it on the next run.

Exploit scenario:

  1. An attacker compromises the codecov/codecov-action repository on GitHub or tricks the maintainers into merging malicious code
  2. They update the v5 tag to point to their backdoored commit
  3. Your GitHub Actions workflow runs and pulls codecov/codecov-action@v5, which now executes the attacker's code
  4. The malicious action runs with your workflow's credentials and can exfiltrate your coverage.out file, environment variables like GITHUB_TOKEN, or access to your repository

By using a version tag like v5, you're trusting that the tag won't be moved. A full commit SHA (40 hexadecimal characters) is immutable—even if the repository is compromised, the specific code you specified cannot be changed.

To resolve this comment:

✨ Commit fix suggestion

Suggested change
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@b0c41b0cdf2050f1b3a81c9f70294c7c3a0c68f7 # v5.0.0
View step-by-step instructions
  1. Go to the Codecov action repository and find the full commit SHA for the desired release version (e.g., v5): https://github.com/codecov/codecov-action/releases.
  2. Replace uses: codecov/codecov-action@v5 with uses: codecov/codecov-action@<commit-sha> using the full 40-character SHA for the v5 release you want to use.
    For example: uses: codecov/codecov-action@b0c41b0cdf2050f1b3a81c9f70294c7c3a0c68f7
  3. Optionally, add a comment showing the version tag this SHA refers to, like # v5.0.0 for easier maintenance in the future.
    Pinning by commit SHA ensures the action version can't change unexpectedly, reducing supply chain risk.
💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by third-party-action-not-pinned-to-commit-sha.

Need help with this issue? Consult our Semgrep Findings Documentation or ask in #help-appsec on Slack.

You can view more details about this finding in the Semgrep AppSec Platform.

Comment thread .github/workflows/ci.yml
go test -race -v -coverprofile=coverage.out ./...
working-directory: './src/github.com/segmentio/nsq-go'
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep identified an issue in your code:

The codecov-action step uses a mutable tag (@v5) instead of a pinned commit SHA, allowing the action owner to silently change what code executes in your CI/CD pipeline.

More details about this

The step uses codecov/codecov-action@v5, which references a mutable tag rather than a pinned commit SHA. This means the action owner can change what code runs under this tag without your knowledge.

Attack scenario:

  1. An attacker compromises the Codecov GitHub Actions repository or gains access to the v5 tag
  2. They update the v5 tag to point to malicious code
  3. On your next workflow run, the GitHub Actions runner downloads and executes the compromised code with access to your repository's secrets (like GITHUB_TOKEN)
  4. The attacker exfiltrates your repository secrets, coverage reports, or injects code into your build artifacts

This is exactly how the trivy-action and kics-github-action compromises happened. The v5 tag can be silently repointed, making it impossible to detect the compromise until damage is done.

To resolve this comment:

✨ Commit fix suggestion

Suggested change
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@d94fb1c7b16d28912cacee80f0657e33b57cb74e # v5.0.0
View step-by-step instructions
  1. Find the line with uses: codecov/codecov-action@v5.
  2. Visit https://github.com/codecov/codecov-action/releases and identify the preferred stable release for v5.
  3. Copy the full 40-character commit SHA associated with that version.
  4. Change the uses line to use the full commit SHA, for example: uses: codecov/codecov-action@d94fb1c7b16d28912cacee80f0657e33b57cb74e # v5.0.0 (replace with the correct SHA for the version you want).
  5. Optionally add a comment with the original tag name for clarity.

Pinning actions to a commit SHA ensures the workflow always uses the same trusted code and prevents silent upstream changes.

💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by github-actions-mutable-action-tag.

Need help with this issue? Consult our Semgrep Findings Documentation or ask in #help-appsec on Slack.

You can view more details about this finding in the Semgrep AppSec Platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant