CRT-based S3 client has timeouts with mock when uploading streams of unknown size

[matdue]

Hi,

the AWS CRT-based S3 client reports timeouts when uploading data of unknown size to a bucket in S3Mock. This kind of upload initiates a multipart upload, starting the process with a POST request (which implies a body, although it is empty). It seems that the CRT-based client hangs because the mock does not handle the body.

The error message is the following:

java.lang.IllegalStateException: The service request was not made within 10 seconds of doBlockingWrite being invoked. Make sure to invoke the service request BEFORE invoking doBlockingWrite if your caller is single-threaded.

	at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.waitForSubscriptionIfNeeded(BlockingInputStreamAsyncRequestBody.java:126)
	at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.writeInputStream(BlockingInputStreamAsyncRequestBody.java:90)
	at com.adobe.testing.s3mock.its.CrtAsyncV2IT.testStreamUploadOfUnknownSize(CrtAsyncV2IT.kt:229)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

You can find some code examples at https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_Scenario_UploadStream_section.html

I've created a pull request to fix this issue. Just consume the (empty) body, and CRT client is happy. Other clients are not that picky, as a workaround I'm using a different client in my tests at the moment.

Best regards,
Matthias

[afranken]

thanks for reporting this issue and providing a fix :)
Yeah, CRT client is picky and behaves unexpectedly in some cases, I had to make several fixes and refactorings to accommodate for that.

[afranken]

fixed with #2048

[afranken]

just released 3.10.1 which includes your fix:
https://github.com/adobe/S3Mock/releases/tag/3.10.1