Skip to content

Support trailing checksum in DefaultAwsV4HttpSigner #6296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

dagnir
Copy link
Contributor

@dagnir dagnir commented Jul 23, 2025

Motivation and Context

This commit moves the support for trailing checksums from HttpChecksumStage to the V4 signer signer implementation; this puts it in line with how sync chunked bodies are already handled.

Modifications

Testing

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

dagnir added 2 commits July 23, 2025 16:17
This commit moves the support for trailing checksums from
HttpChecksumStage to the V4 signer signer implementation; this puts it
in line with how sync chunked bodies are already handled.
@zoewangg zoewangg requested a review from Copilot July 24, 2025 16:51
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR moves trailing checksum support from HttpChecksumStage to the V4 signer implementation to align with how sync chunked bodies are handled. The change consolidates checksum handling logic within the signer rather than having it split between different pipeline stages.

  • Removes trailing checksum logic from HttpChecksumStage for async operations
  • Implements comprehensive async chunked encoding support in AwsChunkedV4PayloadSigner
  • Adds new subscriber classes for reactive stream checksum calculation

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
HttpChecksumStage.java Removes async trailing checksum logic that was handled separately
DefaultAwsV4HttpSigner.java Updates async signing flow to support chunked encoding with checksums
AwsChunkedV4PayloadSigner.java Implements async chunked encoding with trailing checksum support
V4PayloadSigner.java Adds async beforeSigning hook for payload preprocessing
SignerUtils.java Adds async content length calculation utilities
LengthCalculatingSubscriber.java New subscriber for calculating content length from reactive streams
UnbufferedChecksumSubscriber.java New subscriber for checksum computation without buffering
ChunkedEncodedPublisher.java Minor improvements to trailer handling and adds getter methods
Various test files TCK compliance tests and unit tests for new subscriber classes
Comments suppressed due to low confidence (1)

core/http-auth-aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/AwsChunkedV4PayloadSigner.java:190

  • [nitpick] This error message is split across lines making it harder to read. Consider consolidating it into a single, well-formatted message.
                                                          // should not happen, this header is added by moveContentLength

Comment on lines +190 to +192
// should not happen, this header is added by moveContentLength
.orElseThrow(() -> new RuntimeException("x-amz-decoded-content-length "
+ "header not present"));
Copy link
Preview

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message should be more descriptive and use a more specific exception type. Consider using IllegalStateException instead of RuntimeException and improve the message clarity.

Suggested change
// should not happen, this header is added by moveContentLength
.orElseThrow(() -> new RuntimeException("x-amz-decoded-content-length "
+ "header not present"));
// This header is expected to be added by moveContentLength. Its absence indicates an unexpected state.
.orElseThrow(() -> new IllegalStateException("Missing required header: x-amz-decoded-content-length. "
+ "This header is necessary for calculating the encoded content length."));

Copilot uses AI. Check for mistakes.

.orElseThrow(() -> new RuntimeException("x-amz-decoded-content-length "
+ "header not present"));

long encodedContentLength = calculateEncodedContentLength(request, decodedContentLength);
Copy link
Preview

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'request' is being passed to calculateEncodedContentLength, but it should be 'requestBuilder' which is the actual SdkHttpRequest.Builder instance being used in this context.

Suggested change
long encodedContentLength = calculateEncodedContentLength(request, decodedContentLength);
long encodedContentLength = calculateEncodedContentLength(requestBuilder, decodedContentLength);

Copilot uses AI. Check for mistakes.


if (checksumAlgorithm != null) {
String checksumHeaderName = checksumHeaderName(checksumAlgorithm);
request.appendHeader(X_AMZ_TRAILER, checksumHeaderName);
Copy link
Preview

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'request' should be 'requestBuilder' to maintain consistency with the rest of the method and ensure the header is added to the correct request builder instance.

Suggested change
request.appendHeader(X_AMZ_TRAILER, checksumHeaderName);
requestBuilder.appendHeader(X_AMZ_TRAILER, checksumHeaderName);

Copilot uses AI. Check for mistakes.

String checksumHeaderName = checksumHeaderName(checksumAlgorithm);
request.appendHeader(X_AMZ_TRAILER, checksumHeaderName);
}
request.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));
Copy link
Preview

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'request' should be 'requestBuilder' to maintain consistency and ensure the header is set on the correct request builder instance.

Suggested change
request.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));
requestBuilder.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));

Copilot uses AI. Check for mistakes.

Comment on lines +200 to +201
request.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));
request.appendHeader(CONTENT_ENCODING, AWS_CHUNKED);
Copy link
Preview

Copilot AI Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'request' should be 'requestBuilder' to maintain consistency and ensure the header is appended to the correct request builder instance.

Suggested change
request.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));
request.appendHeader(CONTENT_ENCODING, AWS_CHUNKED);
requestBuilder.putHeader(Header.CONTENT_LENGTH, Long.toString(encodedContentLength));
requestBuilder.appendHeader(CONTENT_ENCODING, AWS_CHUNKED);

Copilot uses AI. Check for mistakes.

@dagnir
Copy link
Contributor Author

dagnir commented Jul 28, 2025

Closing to break this up into smaller PRs. Addition of truncating content length from upstream publisher adds too much to this PR.

@dagnir dagnir closed this Jul 28, 2025
Copy link

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 28, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant