Speed up checkpoints v2 pre-push recovery #1208
Open
computermode wants to merge 4 commits into
Open
Conversation
Three independent fixes that drop entiredb's pre-push from ~60s to ~5s: - Probe rotation archive refs with --filter=blob:none and top up only the matched archive's blobs after pick. Falls back to a full fetch on any first-fetch error (e.g. server without uploadpack.allowFilter). - Bound the per-archive ancestry walk to 1s. On repos whose archives are fully disjoint from local /full/current we'd otherwise burn seconds concluding nothing matches. A future /full/root anchor will replace this with a constant-time lookup. - Replace the v1↔v2 cross-check in the migration hint with a single "does v2 /main exist?" ref probe. The hint now only fires when migration was never run; partial migrations are no longer flagged (drift is acceptable post-migration). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Entire-Checkpoint: a15ddf881892
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes v2 checkpoint pre-push recovery by reducing expensive migration checks and limiting how much checkpoint archive data is fetched during rotation recovery.
Changes:
- Replaces v1/v2 checkpoint diffing for the migration hint with a local v2
/mainref existence check. - Changes rotation archive probing to use blobless fetches, then fetch blobs only for the matched archive.
- Adds a wall-clock budget for archive ancestry matching and tests for blobless archive probing behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
cmd/entire/cli/strategy/push_v2.go |
Updates remote rotation archive fetch/recovery to use blobless probing, blob top-up, and ancestry walk budgeting. |
cmd/entire/cli/strategy/push_v2_test.go |
Adds coverage for avoiding blob fetches from unmatched rotation archives. |
cmd/entire/cli/strategy/push_common.go |
Simplifies v2 migration hint detection to probe for the local v2 /main ref. |
cmd/entire/cli/strategy/push_common_test.go |
Updates migration hint tests for the new v2 /main existence behavior. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 37b55be. Configure here.
Trim verbose doc comments and a redundant inline comment in the new rotation-recovery test. No logic changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Entire-Checkpoint: b466825dbff2
3 tasks
Fetch checkpoint recovery refs bloblessly through resolved URLs so pushes do not download transcript blobs or persist partial-clone settings on origin. Replace rotation archive probing with a single wildcard archive fetch, check newest archives first, and use git merge-base under the existing ancestry timeout. Add regression coverage for filtered fetch target resolution, blobless merge recovery, rotation recovery cleanup, and pushability after tree-only merges. Entire-Checkpoint: 0845ffccc4f3
Contributor
Author
|
Bugbot run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

https://entire.io/gh/entireio/cli/trails/374
This PR improves the performance of the pre-push hook for v2-enabled repos with 100+ checkpoints down from ~60s (!) to 5s. Tested with the
entiredbrepo. Observed push times went down from close to a minute to about 5-6 seconds (still unacceptable but much better while the refs fixes are being addressed). Double-checked and ensured checkpoints were pushed up correctly.Replace the v1 <> v2 missing checkpoints check for the migration hint message with a single "does v2 /main exist?" ref probe. The hint now only fires when migration was never run; partial migrations are no longer flagged (a little drift is acceptable post-migration since we fall back to v1 in the UI when we display transcripts for now anyway).
Probe rotation archive refs with
--filter=blob:noneand top up only the matched archive's blobs after pick. Falls back to a full fetch on any first-fetch error (e.g. server without uploadpack.allowFilter).Bind the per-archive ancestry walk to 1s. On repos whose archives are fully disjoint from local /full/current we'd otherwise burn seconds concluding nothing matches. A future /full/root anchor will replace this with a constant-time lookup ( see v2 checkpoints: add deterministic /full/root anchor commit #1201).
Note
Medium Risk
Changes the v2 pre-push recovery fetch/rotation-conflict logic to use explicit filtered fetches and a new archive-selection path, which could affect correctness in edge cases (servers without filter support, unusual ref layouts) but is scoped to checkpoint sync/push flows.
Overview
Speeds up v2 pre-push recovery by switching recovery fetches to probe with
--filter=blob:none(with fallback to unfiltered fetch) so merges can operate on commits/trees without downloading transcript blobs.Adds
remote.ResolveFilteredFetchTargetto always resolve remote names to URLs for explicit filtered fetches, preventing git from persisting promisor/partial-clone settings onto named remotes.Refactors rotation-conflict handling for
v2/full/currentto fetch archived generations via a wildcard refspec, select the related archive usinggit merge-basewith a bounded per-archive walk budget, and improves temp-ref cleanup; updates/adds tests to assert no blob downloads and noremote.origin.promisor/remote.origin.partialclonefilterside effects.Reviewed by Cursor Bugbot for commit 1bc7c4e. Configure here.