Skip to content

S3 DeleteObjects Key Validation Fix #3832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

AlexDaines
Copy link
Contributor

Description

Modified the S3 client's DeleteObjects operation to properly handle invalid keys in batch delete requests. Instead of failing the entire operation when any single key is invalid, the client now skips invalid keys and proceeds with deleting valid objects. If all keys in a request are invalid, a clear exception is thrown with an appropriate message.

Motivation and Context

Addresses open issue #3828

This change is required because the current implementation prevents valid keys from being deleted when any key in the batch is invalid (e.g., too long). The S3 DeleteObjects API is designed to support partial successes, but the client-side validation was preventing this behavior by throwing an AmazonS3Exception before the request reached the service.

This fix allows the DeleteObjects operation to behave as expected, processing all valid keys even when some invalid keys are present in the same batch.

Testing

The changes were tested with the following scenarios:

  • Request with only valid keys (verified all objects deleted)
  • Request with mix of valid and invalid keys (verified valid objects deleted)
  • Request with all invalid keys (verified appropriate exception thrown)

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the README document
  • I have added tests to cover my changes
  • All new and existing tests passed

License

  • I confirm that this pull request can be released under the Apache 2 license

@AlexDaines AlexDaines changed the base branch from aws-sdk-net-v3.7-development to development May 20, 2025 20:49
Logger.InfoFormat(message);

ValidationErrors.Add(new DeleteError {
Key = key.Substring(0, Math.Min(key.Length, 128)) + (key.Length > 128 ? "..." : ""),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Min here is not truly required, also does it make sense to trim it it 128 when 1024 length is legit it can be issue when you have common prefix of 128 chars length

});
return;
}
if (key.Length > 1024)
Copy link

@grsw92 grsw92 May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexDaines I understand this change just skips invalid keys and just sends valid ones with the request to the server. I'm wondering shouldn't this behavior be handled on the server side and proper DeleteObjectsResponse be returned from the server in that case?

@peterrsongg peterrsongg requested a review from normj May 21, 2025 17:35
@AlexDaines
Copy link
Contributor Author

Closing -

After investigation, the real issue is that DeleteObjectsResponseUnmarshaller is missing the UnmarshallException override, causing AmazonS3Exception instead of DeleteObjectsException.

New PR (#3839) created with the proper exception handling fix instead of client-side validation.

@AlexDaines AlexDaines closed this May 22, 2025
@dscpinheiro dscpinheiro deleted the fix-s3-deleteobjects-validation branch May 25, 2025 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants