Skip to content

Add all streaming operation to s3 checksums tests #5998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
L-Applin merged 93 commits into from
Jun 20, 2025
Merged

Add all streaming operation to s3 checksums tests #5998

L-Applin merged 93 commits into from
Jun 20, 2025

Conversation

@L-Applin
Copy link
Contributor

@L-Applin L-Applin commented Mar 31, 2025

Add more test cases to s3 checksums tests:

  • Add all method from RequestBody and ResponseTransformer to the test suite.
  • Add multipart download/upload.
  • Add Transfer-manager based tests.

Prepare for Codebuild execution for the tests suite

L-Applin added 29 commits March 25, 2025 12:22
@L-Applin
add test configs for all streaming type
4ff551c
@L-Applin
buildspec fix
c80568e
@L-Applin
buildspec fix
53b55b3
@L-Applin
check if tests hand without multipart
d892595
@L-Applin
add http-auth-aws-crt for crt sigv4a tests
bd21f3a
@L-Applin
fix request body methods with 'remaining'
bbb5d1e
@L-Applin
test 2MiB instead of 200MiB
c889f6e
@L-Applin
fix for 'remaining' test type
89c5edc
@L-Applin
fix for fromString, reduce logs
3737d0a
@L-Applin
fix for string content length
633e916
@L-Applin
fix for string content length
203b0ed
@L-Applin
test 200MiB large size
babbbdd
@L-Applin
use 80MiB for large files
422c858
@L-Applin
Fix for TM_JAVA
25e3666
@L-Applin
Fix for TM_JAVA
67842a3
@L-Applin
Fix for TM_JAVA
764629a
@L-Applin
Merge remote-tracking branch 'origin/olapplin/s3-checksums-tests' int…
4c13ffb 
…o olapplin/s3-checksums-tests

# Conflicts:
#	test/s3-tests/pom.xml
#	test/s3-tests/src/it/java/software/amazon/awssdk/services/s3/BodyType.java
#	test/s3-tests/src/it/java/software/amazon/awssdk/services/s3/checksum/ChecksumIntegrationTesting.java
@L-Applin
need more memory
2f01bc1
@L-Applin
more fix for TM
3ba77d2
@L-Applin
some more dl stuff
c07dab3
@L-Applin
logs
14c46de
@L-Applin
fix multi
efb4dc8
@L-Applin
fix multi bucket
51950db
@L-Applin
fix multi bucket
692fbe0
@L-Applin
part index starts at 1
3af2ee0
@L-Applin
complete multipart fix
29136b4
@L-Applin
complete multipart fix
994bbc5
@L-Applin
complete multipart fix
7d2f2fb
@L-Applin
complete multipart fix
bf8dd35
@L-Applin L-Applin requested a review from a team as a code owner March 31, 2025 17:07
L-Applin added 17 commits June 19, 2025 12:59
@L-Applin
split upload tests per client type: sync, async, crt, multipart-java.…
f2a8c12 
… Lower large non-file content from 80mib to 60mib to attempt lower build time.
@L-Applin
fix MethodSource method name
87a7900
@L-Applin
fix for MPU tests failing
7732181
@L-Applin
add timeout. Add sync http client with higher conn pool manually and …
1982892 
…close s3 client
@L-Applin
remove unused forced path style from upload configs
69241e4
@L-Applin
set timeout to 30 sec for uploads
84c383b
@L-Applin
try to reduce build time
36f0665
@L-Applin
add log to error
13f5867
@L-Applin
fix async upload without length
42c0bf2
@L-Applin
fix sync upload when missing checksum from response
6af4b85
@L-Applin
fix missing checksum from response
6260cab
@L-Applin
fix missing checksum from response
3483b0e
@L-Applin
increase connection acquisition timeout to fix "failed to acquire con…
a512f92 
…nection" error in sync uploads
@L-Applin
skip RequestBody.fromRemainingByteBuffer with RequestChecksumCalculat…
2ce8c00 
…ion.WHEN_SUPPORTED because of connection acquire timeout, added TOT for investigation
@L-Applin
test cleanup catch exception
a03e166
@L-Applin
use bucket type in upload tests
b1a24c5
@L-Applin
All bucket still cause connection timeouts
4bf3e92
@L-Applin L-Applin enabled auto-merge June 20, 2025 19:11
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 20, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@L-Applin L-Applin added this pull request to the merge queue Jun 20, 2025
Merged via the queue into master with commit d77cb8b Jun 20, 2025
34 of 35 checks passed
@github-actions
Copy link

github-actions bot commented Jun 20, 2025

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 20, 2025
@L-Applin L-Applin deleted the olapplin/s3-checksums-tests branch December 2, 2025 05:14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@zoewangg zoewangg zoewangg approved these changes

Assignees

No one assigned

Labels

changelog-not-required Indicate changelog entry is not required for a specific PR no-api-surface-area-change Indicate there is no API surface area change and thus API surface area review is not required

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@L-Applin @zoewangg