Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(CDK watch): skip synthesis if the change is only to an Asset #18045

Open
1 of 2 tasks
skinny85 opened this issue Dec 15, 2021 · 2 comments
Open
1 of 2 tasks

(CDK watch): skip synthesis if the change is only to an Asset #18045

skinny85 opened this issue Dec 15, 2021 · 2 comments
Labels
@aws-cdk/core Related to core CDK functionality effort/large Large work item – several weeks of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. feature-request A feature should be added or improved. p2

Comments

@skinny85
Copy link
Contributor

skinny85 commented Dec 15, 2021

Description

There is a possible performance optimization with cdk watch: if a given change event is for a file that belongs to an Asset, we can skip synthesizing the Cloud Assembly again (we know it will be the same).

Since synthesis is actually the bottleneck for many hotswap deployments (like for Lambda, or for StepFunctions), this might be a significant performance improvement for some use cases, especially in non-NodeJS languages.

Use Case

Hotswapping of resources where cdk synthesis is the performance bottleneck, especially in non-NodeJS languages.

Proposed Solution

The implementation of this would be quite complex. Basically, when we receive a change in cdk watch, we have to:

  1. Check that it belongs to the files/directories of any Asset in any of the Stack(s) we are deploying with watch.
  2. For each of those found Assets, we now have to search the template of the Stack(s) to determine which resources reference that Asset.
  3. For each of those resources, we now have to send them to the hotswap detectors to verify whether they support hotswapping.
  4. If all of them do, we can now perform hotswapping without running cdk synth before.

Other information

No response

Acknowledge

  • I may be able to implement this feature request
  • This feature might incur a breaking change
@skinny85 skinny85 added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. effort/large Large work item – several weeks of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. p2 and removed needs-triage This issue or PR still needs to be triaged. labels Dec 15, 2021
@github-actions github-actions bot added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Dec 15, 2021
@skinny85 skinny85 removed the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Dec 15, 2021
@github-actions github-actions bot added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Dec 21, 2021
@kaizencc kaizencc removed the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Dec 23, 2021
@kaizencc kaizencc removed their assignment Dec 23, 2021
@skinny85
Copy link
Contributor Author

If I have prepared a Proof of Concept of how this functionality would look like. Here's a demo showing it in action: https://youtu.be/mSYJJDdCQzA

The code for the above demo is here. It's very much PoC-level code, but I think gives a good start with how to think about this functionality.

I tried to include ToDos in the code where appropriate, but here's a list of things that need to be fixed before we can think of making this functionality part of the CDK:

  • support old-style synthesis (in addition to new-style synthesis, which works now)
  • think whether no staging of the Assets is really required
  • support bundling in Assets (very important for things like NodeJsFunction)
  • executables in Assets (a feature of new-style synthesis)
  • support for Docker assets (in addition to the file assets that work now)
  • this should probably run the "build" command (if provided) before deployment (this command was introduced in feat(cli): added build field to cdk.json #17176)

Unfortunately, I won't have time to work on this functionality in the near future, so if anyone wants to pick this up (feel free to use the above PoC code as a starting-off point), that would be fantastic!

@phlrnnr
Copy link

phlrnnr commented Feb 6, 2023

This would be an excellent feature enhancement!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality effort/large Large work item – several weeks of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

4 participants