-
Notifications
You must be signed in to change notification settings - Fork 575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Throughput improvements to the AWS SDK S3 client were released in v3.649.0 #6423
Comments
The set of changes were released in https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.649.0. https://www.npmjs.com/package/@aws-sdk/client-s3/v/3.649.0 There is an additional client configuration field related to this. const s3 = new S3Client({
cacheMiddleware: true
});
When, for example, making 400x HeadObject requests with the same S3Client, this can be enabled for a performance boost. The caveat is that modification of the |
The expected performance profile of S3 HeadObject with e.g. parallel batch size 10 and total size 400 objects is that with v3.649.0 and
Using https://github.com/bolt-juri-gavshin/aws-sdk-v3-vs-v2-perf-comparison Note, the node-http2 requestHandler should not be used with S3, I've excluded it from the sample. Disclaimer: due to the greater number of features in the JSv3 SDK and the fact that the v2 SDK sacrifices a lot of correctness (e.g. endpoint resolution) for simplicity, the v2 SDK will still be faster than the v3 SDK in many cases. However, JSv2 has entered maintenance mode and development efforts are focused on this iteration of the AWS SDK for JavaScript. |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread. |
Describe the feature
Investigate and improve throughput performance of the S3Client in this SDK.
Use Case
Improve throughput of parallel or high-volume of operations, including S3 HeadObject.
Proposed Solution
improve request handler creation time
avoid FS / loadNodeConfig calls in the request path
improve endpoint resolution time
defer calls to
socket.setKeepAlive
and other socket event listener attachmentsAcknowledgements
SDK version used
3.600.0+
Environment details (OS name and version, etc.)
Node.js
The text was updated successfully, but these errors were encountered: