Skip to content

chore(ci): reduce Percy visual-snapshot usage - #34366

Draft
claude[bot] wants to merge 3 commits into
developfrom
claude/percy-reduce-usage
Draft

chore(ci): reduce Percy visual-snapshot usage#34366
claude[bot] wants to merge 3 commits into
developfrom
claude/percy-reduce-usage

Conversation

@claude

@claude claude Bot commented Jul 27, 2026

Copy link
Copy Markdown

Requested by Ganesh RK, Jennifer Shehane · Slack thread

  • Closes [na] — CI cost work from the Percy-usage investigation (driven by Jennifer Shehane), greenlit by Ganesh RK in the R&D leadership thread linked above.

Additional details

This PR reduces how often we upload Percy visual snapshots in CI. It is config-only — no product, library, or test code changes, and every functional Cypress run keeps running exactly as before. Only the Percy snapshot layer is affected.

Before → After, in plain terms

  1. Dev-server integration jobs no longer spin up Percy for nothing.

    • Before: run-webpack-dev-server-integration-tests and run-vite-dev-server-integration-tests wrapped their Cypress run in yarn percy exec and carried the test-runner:percy context — but their specs (and the projects they scaffold) never call cy.percySnapshot(), so they captured zero snapshots while still starting a Percy agent and joining the parallel build as empty shards.
    • After: the Percy wrapper/context is removed from both jobs. They run the same yarn cypress:run as before.
  2. Percy snapshots are gated to UI-relevant changes on PRs.

    • Before: Percy snapshotting was tied to the same pipeline parameters that gate the functional UI test jobs. So a PR that only touched a non-UI package which happens to pull in a UI job (e.g. packages/data-context, packages/telemetry) still re-uploaded the full snapshot set.
    • After: a dedicated run-percy pipeline parameter (set by generate-pipeline-parameters.sh) turns Percy on only when a change touches front-end/UI code. On such PRs the UI jobs still run their Cypress tests — they just skip the percy exec layer and upload nothing.

How

  • /.circleci/src/pipeline/@pipeline.yml
    • Removed the percy exec wrapper + PERCY_* env from run-webpack-dev-server-integration-tests and run-vite-dev-server-integration-tests.
    • Added a run-percy boolean pipeline parameter (default true).
    • Simplified percy-finalize to a single halt-if-skipped: run-percy guard (a Percy build now exists exactly when run-percy is true), and dropped the stale dev-server params from it.
  • /.circleci/src/pipeline/workflows/@main.yml and pull-request.yml
    • The six app/launchpad/frontend-shared/reporter component & integration jobs now take percy: << pipeline.parameters.run-percy >> instead of a hard-coded true.
    • Dropped the two dev-server jobs from percy-finalize's requires/context.
  • /.circleci/scripts/generate-pipeline-parameters.sh
    • Computes run-percy = true only for front-end paths: packages/{app,launchpad,reporter,frontend-shared,runner,web-config,icons,resolve-dist} and cli/ (for the static CLI visual snapshots). It stays true for emit_all_true (develop/release, API triggers, run-all-jobs, and global-trigger changes such as deps/config/.circleci).

Expected usage impact

  • Removes the two always-empty dev-server Percy shards from every UI build (overhead cleanup; 0 billed snapshots removed — they weren't producing any).
  • On PRs, skips the full snapshot upload when the change is not UI-relevant but would previously have triggered Percy (e.g. data-context/telemetry-only PRs). The existing halt-based path filtering already skips Percy for pure backend PRs, so this closes the remaining "non-UI change dragged in a UI job" gap.
  • Honest caveat: path filtering only saves when a non-UI change would otherwise have triggered Percy. Front-end PRs, dependency bumps, and config/.circleci changes still snapshot (kept intentionally, so visual coverage is not lost when a change could plausibly affect rendering).
  • develop is intentionally left running full Percy. The path-filter script cannot compute a meaningful changed-file set on a develop push (the merge-base with develop is HEAD), which is exactly why develop uses emit_all_true. Gating develop would need a different diff strategy and is out of scope here.

Steps to test

Because the config source lives in .circleci/src/, run yarn pack-ci --validate locally (or let this PR's own CI pack + validate). Then, to confirm the path-filter behavior of generate-pipeline-parameters.sh:

  • CIRCLE_BRANCH=develop bash .circleci/scripts/generate-pipeline-parameters.shrun-percy: true.
  • On a PR branch, a change under packages/app (or launchpad/reporter/frontend-shared/runner/web-config/icons/resolve-dist/cli) → run-percy: true.
  • On a PR branch, a change under only packages/server / packages/data-context / packages/telemetryrun-percy: false (functional UI jobs may still run, but take no snapshots), and percy-finalize halts.

How has the user experience changed?

No change — this is CI/Percy configuration only. No product or library code is touched.

PR Tasks

  • [na] Is there an associated issue with maintainer approval for PR submission?
  • [na] Have tests been added/updated?
  • [na] Has a PR for user-facing changes been opened in cypress-documentation?
  • [na] Have API changes been updated in the type definitions?

Generated by Claude Code

claude added 2 commits July 27, 2026 19:02
The run-webpack-dev-server-integration-tests and
run-vite-dev-server-integration-tests jobs wrapped their Cypress run in
`yarn percy exec` and carried the test-runner:percy context, but their
specs (npm/{webpack,vite}-dev-server/cypress/e2e) contain no
`cy.percySnapshot()` calls, and neither do the projects they scaffold.
As a result these jobs captured zero Percy snapshots while still spinning
up a Percy agent and joining the parallel build as empty shards.

Strip the Percy wrapper, env vars, and context from both jobs (the
functional `yarn cypress:run` invocation is unchanged) and drop them from
percy-finalize's requires/guard so a dev-server-only PR no longer tries to
finalize a build that was never created.

Note: this removes vestigial Percy overhead; it does not reduce billed
snapshots, because these jobs were not producing any.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnNBfto8Tk3K2wryockMR3
…aram

Percy snapshotting was tied to the same pipeline parameters that gate the
functional UI test jobs, so any change that made an app/launchpad job run
(for example a data-context-only or telemetry-only PR) also re-uploaded the
full snapshot set.

Introduce a dedicated `run-percy` pipeline parameter, computed in
generate-pipeline-parameters.sh, that is true only when a change touches
front-end/UI code (packages/app, launchpad, reporter, frontend-shared,
runner, web-config, icons, resolve-dist, and cli for the static CLI
snapshots). The six app/launchpad/frontend-shared/reporter component &
integration jobs now take `percy: << pipeline.parameters.run-percy >>`
instead of a hard-coded `true`, so on PRs whose changes are not UI-relevant
the jobs still run their Cypress tests but skip the `percy exec` layer and
upload no snapshots.

percy-finalize is simplified to a single `run-percy` guard: a Percy build
now exists exactly when run-percy is true, so this halts finalize cleanly
when no build was produced (and drops the stale dev-server params).

run-percy defaults to true and emit_all_true sets it true, so
develop/release branches, API-triggered pipelines, run-all-jobs, and
global-trigger changes (deps, config, .circleci) keep full visual coverage.
Develop pushes are intentionally left running full Percy: the path-filter
script cannot compute a meaningful changed-file set on a develop push
(merge-base with develop is HEAD), which is why develop uses emit_all_true.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CnNBfto8Tk3K2wryockMR3
@cypress-app-bot

Copy link
Copy Markdown
Collaborator

…e-usage

# Conflicts:
#	.circleci/scripts/generate-pipeline-parameters.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants