do not overwrite unchanged objects when restoring bucket #29
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a bucket is restored to an earlier state of itself, only a few objects might actually have changed. Changed objects can be identified by comparing the etag of the desired version with the etag of the most recent version. If the etags agree, the object can be skipped. Because the most recent version of any object is always listed first in the response from list_object_versions, its etag is definitely known when handling the version to be restored. Added a test that checks that no new version is created when the object was not changed. Also running the existing test in a configuration that uses the same source and destination bucket without requiring another call of "s3-pit-restore --test". Because this is a frequent use case, it should be tested without special effort.
Contributor
Author
|
My PR still contains an error. I will open another PR once I have fixed it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a bucket is restored to an earlier state of itself, only a few objects might actually have changed. Changed objects can be identified by comparing the etag of the desired version with the etag of the most recent version. If the etags agree, the object can be skipped. Because the most recent version of any object is always listed first in the response from list_object_versions, its etag is definitely known when handling the version to be restored.
Added a test that checks that no new version is created when the object was not changed.
Also running the existing test in a configuration that uses the same source and destination bucket without requiring another call of "s3-pit-restore --test". Because this is a frequent use case, it should be tested without special effort.
This PR solves a similar, but not the same problem as PR #24. PR #24 can also detect objects that need not be updated when source and target bucket differ, but it needs a separate head_object call for each object. This PR addresses the case of restoring a single bucket to a previous state, only, but without a performance penalty.