ci: run zebra-consensus benches on zebra-touching commits - #48
Open
aphelionz wants to merge 2 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an informational CI workflow to run Zebra consensus Criterion benchmarks on relevant PRs and main pushes.
Changes:
- Adds path-gated and manual workflow triggers.
- Runs available consensus benchmarks with Rust caching.
- Publishes benchmark results to the step summary.
Suppressed comments (5)
.github/workflows/zebra-bench.yml:67
workflow_dispatchis filtered through the same path decision, so manually running this workflow on a commit whose immediate parent changed only documentation silently skips both benches. Since this trigger has no force-run input, special-case manual dispatches as relevant (as the other job-gated workflows do) or remove the trigger.
if [ "$hits" -gt 0 ]; then relevant=true; else relevant=false; fi
.github/workflows/zebra-bench.yml:63
- On a
pushthat contains multiple commits,HEAD^1..HEADonly examines the tip commit. For example, a zebra change followed by a docs-only commit makeshits=0and skips the benches, so a zebra-touching commit is missed. Use the push event'sbeforeSHA (fetching it or using full history), while retaining the merge-parent range for pull requests.
git diff --name-only HEAD^1 HEAD -- "${specs[@]}" > "$RUNNER_TEMP/matched"
.github/workflows/zebra-bench.yml:98
- This
rust-cachesetup does not provide a rolling Criterion baseline: for source-only commits with the same toolchain/job/lockfile, the cache key remains the same, and GitHub cache entries are immutable, so the post-runtarget/criteriondata cannot replace the first cached entry. Later runs therefore compare against stale data rather than the immediately preceding run as the workflow comment claims. Store Criterion data with a per-run key plus a ref-scoped restore key, or otherwise provide an updateable baseline separately from the build cache.
uses: Swatinem/rust-cache@v2
with:
prefix-key: v1-bench
workspaces: |
zebra -> target
.github/workflows/zebra-bench.yml:108
- These benchmark commands do not use
--locked, so a manifest/lockfile mismatch can make Cargo resolve newer compatible dependencies and rewrite the lockfile during the run. The reported timing can then depend on registry state and represent a dependency graph different from the commit; fail loudly with--lockedinstead.
cargo bench -p zebra-consensus --bench script 2>&1 | tee "$RUNNER_TEMP/script.out"
.github/workflows/zebra-bench.yml:124
- This benchmark command has the same reproducibility problem as the script bench: without
--locked, a manifest/lockfile mismatch can resolve newer compatible dependencies during the run, making the timing depend on registry state. Pass--lockedso the benchmark either uses the committed dependency graph or fails.
cargo bench -p zebra-consensus --bench utxo_lookup 2>&1 | tee "$RUNNER_TEMP/utxo.out"
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| ':(glob)zebra/zebra-chain/**' | ||
| ':(glob)zebra/zebra-script/**' | ||
| ':(glob)zebra/Cargo.toml' | ||
| ':(glob)zebra/Cargo.lock' |
Path-gated informational job (not a required check): runs the script-cache and UTXO-lookup criterion benches on PRs and main pushes that touch the benched crates, and posts the timings to the step summary. Bench steps are guarded on their bench files existing, so this lands before #45/#46 and activates as they merge. rust-cache persists target/criterion, so criterion also reports a change estimate against the previous cached run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aphelionz
force-pushed
the
claude/zebra-bench-ci-2a26d1
branch
from
August 26, 2026 13:29
8737529 to
4e5306f
Compare
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs the zebra-consensus criterion benches on every zebra-touching commit (PRs and main pushes), posting the timings to the job's step summary. Informational, not a required check: absolute times on shared runners are noisy, but the ratios the benches watch (cache miss vs hit in #45, serial vs overlapped lookups in #46) are measured within one run on one box.
Follows the z3-regtest conventions: no
paths:trigger filter, a Path gate job diffing the merge commit (manual dispatches bypass it), Swatinem rust-cache on the zebra workspace, saved from main only so per-PR saves cannot evict the smoke/regtest caches. Each bench step is guarded on its bench file existing, so this can merge before #45/#46 and activates as they land.🤖 Generated with Claude Code