(aws-service-catalog): SIGKILL: 9 CustomCDKBucketDeployment Lambda runs out of memory

[schnipseljagd]

Describe the bug

Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdk-accel-assets-XXXX-eu-central-1/d91fe0e02571a1639b83572917ebc528352265bd305ff570f75cae8419dae522.zip', '/tmp/tmpd41vn6fr/contents']' died with <Signals.SIGKILL: 9>. (RequestId: 90349d17-82a5-4ed1-b285-aad7469cb046)

Expected Behavior

I should be able to configure the memory limit for the bucket deployment Lambda used by the ProductStackSynthesizer.

Current Behavior

It is not configurable since it is hard-coded deep in the synthesizer:

aws-cdk/packages/aws-cdk-lib/aws-servicecatalog/lib/private/product-stack-synthesizer.ts

Line 85 in 8e2cbae

?? new BucketDeployment(deploymentScope, deploymentCid, {

Reproduction Steps

You need to try to deploy a big enough CDK Stack using service catalog product stacks.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.134.0 (build 265d769)

Framework Version

No response

Node.js Version

v18.20.0

OS

Linux

Language

TypeScript

Language Version

No response

Other information

No response

[schnipseljagd]

Btw. Increasing the memory limit of the Lambda manually fixes the problem for the time being.

[nmussy]

I don't think increasing the memory allocated to the Lambda is the correct solution here. It's a waste of resources and would not scale with larger file sizes. The file downloaded from S3 should not be fully stored in memory until its download is completed, it should be progressively saved to disk and assembled.

I don't think this is possible with the AWS CLI, but it might be doable with boto3, see Multipart transfers. If not, it could be done with the SDK, see Upload or download large files to and from Amazon S3 using an AWS SDK

EDIT: boto3 should be good to go, and a lot simpler than re-implementing the same behavior with the SDK, see docs

multipart_threshold – The transfer size threshold for which multipart uploads, downloads, and copies will automatically be triggered.

[pahud]

@nmussy Thank you and I agree with you.

But it makes sense to expose the memoryLimit as well. Feel free and welcome to submit a PR to expose the memoryLimit for this.

Please help us prioritize with 👍 .

[nmussy]

I've opened another issue for my solution, so we can keep track of it after this one is fixed by the memoryLimit prop: #29898

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[aws-cdk-automation]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.