(lambda-python): add local bundling to PythonFunction

[relm923]

Description

Similar to the NodeJs Lambda, Python Lambda packaging should allow for local bundling

Use Case

Docker bundling of lambdas is slow and resource intensive. Its valuable to be able to run bundling locally, without Docker.

Proposed Solution

Add a local bundling option similar to NodeJs.

Other information

No response

Acknowledge

• I may be able to implement this feature request
• This feature might incur a breaking change

[corymhall]

As of the latest CDK version (2.4.0/1.38.0) this behavior in PythonFunction is the same as it is for NodejsFunction. You should no longer see bundling for a stack if bundling is skipped. Before 2.4.0 the bundling for PythonFunction actually occurred during the docker build (as part of DockerImage.fromBuild()) which is why bundling was never skipped.

You will still see the docker image being built every time, but it should be much faster than before since bundling is not occurring as part of the build. This is also the current behavior for all of the constructs that use Docker bundling.

I think it would be great if we could skip the docker build as well. We can repurpose this issue to track that if you want.

[relm923]

Thanks for the clarification @corymhall. We are in fact seeing what you described. We leverage NodeJsFunction option for local/non-docker builds which allows us to fully skip bundling there.

Having the ability to fully skip the docker build using BUNDLING_STACKS and/or supporting building completely outside of Docker would be great

[corymhall]

OK maybe I'll repurpose this issue to track allowing local bundling for PythonFunction and I'll use #14747 to track skipping the docker build altogether.

[relm923]

Sounds good!

[relm923]

@corymhall just to clarify - even with aws-cdk-lib v2.4.0 and @aws-cdk/aws-lambda-python-alpha v2.4.0-alpha.0 we're still seeing the Docker image being pulled, pipenv being installed and all the lambda dependencies when running:

CDK_CONTEXT_JSON='{\"aws:cdk:bundling-stacks\":[\"NONE\"]}' cdk synth

Is this expected?

[corymhall]

The aws:cdk:bundling-stacks is internal context that cannot be set by the user. This value is calculated based on the stacks that are provided with the --exclusively option. So if I have 2 stacks StackA and StackB and I run

cdk synth --exclusively StackA

Then aws:cdk:bundling-stacks will be ['StackA'] and bundling will be skipped for StackB

It is currently not possible to skip bundling for a stack that you are synthesizing.

[athewsey]

+1 and is anybody aware of a nice workaround for containerless PythonFunction builds at the moment? I'm trying to get CDK working in an Azure Pipelines environment where the runtime is Linux but DinD is not available (but explicit Docker task steps could be?)... Our stack doesn't have any other need for container building: Just a few Python Lambda functions.

[kirnberger1980]

+1 for this issue

[github-actions]

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

[CoreOxide]

As @evgenyka mentioned, I will open a PR for my RFC (aws/aws-cdk-rfcs#609) under this issue.

[evgenyka]

@CoreOxide please note that #26062 was a previous trial to achieve that, but was abandoned. Pls see the comments there.