Skip to content

chore: Improve canister snapshot download/upload #4406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 28 commits into from
Oct 29, 2025
Merged

chore: Improve canister snapshot download/upload #4406

merged 28 commits into from
Oct 29, 2025

Conversation

@vincent-dfinity
Copy link
Contributor

@vincent-dfinity vincent-dfinity commented Oct 14, 2025 •
edited
Loading

Description

Improve canister snapshot download/upload:

  • Add progress bars to snapshot download/upload
  • Stream snapshot download/upload directly to/from disk.
  • Snapshot download/upload with resuming.
  • Snapshot download/upload with concurrency, default to 3 tasks in parallel.

progress_bar

Fixes # (issue)

SDK-2382
SDK-2383
SDK-2392

How Has This Been Tested?

  • The concurrency is enabled by default to 3 tasks, so it's covered by the existing e2e test.
  • The progress bar and download with resuming are covered by manual testing.

Checklist:

  • The title of this PR complies with Conventional Commits.
  • I have edited the CHANGELOG accordingly.
  • I have made corresponding changes to the documentation.

@vincent-dfinity vincent-dfinity marked this pull request as ready for review October 16, 2025 14:21
@vincent-dfinity vincent-dfinity requested a review from a team as a code owner October 16, 2025 14:21
@vincent-dfinity vincent-dfinity mentioned this pull request Oct 20, 2025
Draft
mraszyk
mraszyk reviewed Oct 20, 2025
View reviewed changes
Copy link
Contributor

@mraszyk mraszyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I didn't do a very thorough review so a second pair of eyes is desirable.

adamspofford-dfinity
adamspofford-dfinity requested changes Oct 20, 2025
View reviewed changes
Copy link
Contributor

@adamspofford-dfinity adamspofford-dfinity left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing tests. In particular please simulate (a) high latency network conditions and (b) a network drop; toxiproxy can be used for this

vincent-dfinity
vincent-dfinity commented Oct 24, 2025
View reviewed changes
@vincent-dfinity vincent-dfinity merged commit 786ca0c into master Oct 29, 2025
313 of 317 checks passed
@vincent-dfinity vincent-dfinity deleted the vincent/SDK-2382 branch October 29, 2025 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mraszyk mraszyk mraszyk left review comments

@adamspofford-dfinity adamspofford-dfinity adamspofford-dfinity approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vincent-dfinity @mraszyk @adamspofford-dfinity