Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support S3AsyncClient from software.amazon.awssdk:s3 #1613

Closed
vlbaluk opened this issue Jan 29, 2024 · 2 comments
Closed

Support S3AsyncClient from software.amazon.awssdk:s3 #1613

vlbaluk opened this issue Jan 29, 2024 · 2 comments

Comments

@vlbaluk
Copy link

vlbaluk commented Jan 29, 2024

Are you going to put S3AsyncClient implementation alongside synchronous in the near future? We will switch to the S3 Async client, but rewriting tests is blocked.😐

@vlbaluk vlbaluk closed this as completed Jan 30, 2024
@afranken
Copy link
Member

afranken commented Feb 6, 2024

@vlbaluk did you get an AsyncClient to work in your tests? It should be easily possible to do so.
I can look into adding Async versions to the test infrastructure in one of the next releases for convenience.

@vlbaluk
Copy link
Author

vlbaluk commented Feb 20, 2024

Hey @afranken, Thank you for asking.

It is not functioning as expected with S3TransferManager, which should enable multipart uploading and downloading by default.

  1. File content on the S3 mock Docker container contains not only data but also metadata, which leads to incorrect downloads.
  2. S3Mock always returns incorrect length for multipart downloads, too.

Experiment, file content is "BBBBB"

  • Async CRT client with multipart and S3TransferManager stores the following file in S3Mock docker container:

BBBBBAsyncAttachment

  • Synchronous client stores correct content:

ffe7b8b720bd49a98902fdb950d5bc97

Code snippet:

store.write("test data".getBytes(UTF_8));
 // under the hood, s3TransferManager.uploadFile(uploadFileRequest) with CRT async client


s3TransferManager.download(downloadFileRequest).completionFuture().join().result().response().contentLength() 
// download returns incorrect content, including additional lines at the beginning and at the end, as on the screen above.  
                

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants