Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): cdk deploy stops early if 2 stacks with a dependency between them share an asset #25719

Merged

Conversation

liamoneill
Copy link
Contributor

@liamoneill liamoneill commented May 24, 2023

Fixes cdk deploy not completing for some apps by removing cycles from core's worker graph between asset publishing steps & stacks. These cycles may be introduced when there dependencies between stacks and these stacks share assets with the same contents.

Additionally fixes a bug in the worker graph's error handling for cycles.

Closes #25714


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented May 24, 2023

@aws-cdk-automation aws-cdk-automation requested a review from a team May 24, 2023 14:40
@github-actions github-actions bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. p2 labels May 24, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation added the pr/needs-cli-test-run This PR needs CLI tests run against it. label May 24, 2023
@rix0rrr
Copy link
Contributor

rix0rrr commented May 25, 2023

Thanks for the deep dive and the quick fix! ❤️

Giving it a look now

@rix0rrr rix0rrr changed the title fix(core): cdk deploy may not dependant stacks have assets with identical contents fix(core): cdk deploy stops early if 2 stacks with a dependency between them share an asset May 25, 2023
@rix0rrr rix0rrr added pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested labels May 25, 2023
@rix0rrr
Copy link
Contributor

rix0rrr commented May 25, 2023

Amazing job on figuring out the issue and solving it! Many thanks again 🙏

@aws-cdk-automation aws-cdk-automation dismissed their stale review May 25, 2023 13:50

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-cli-test-run This PR needs CLI tests run against it. label May 25, 2023
@mergify
Copy link
Contributor

mergify bot commented May 25, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 6955746
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 9e45095 into aws:main May 25, 2023
@mergify
Copy link
Contributor

mergify bot commented May 25, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

mergify bot pushed a commit that referenced this pull request Jun 5, 2023
…25846)

In #25536 we introduced the new work graph orchestration for the CLI which had a bug when the same asset was shared by multiple stacks. #25719 was an attempt at fixing this bug, and while it fixed it for some cases it didn't fix all of them.

The issue is that for cosmetic reasons, asset publishing steps inherit the dependencies of the stacks they are publishing for  (so that the printed output of asset publishing doesn't interfere with the in-place updated progress bar of stack deployment and make it lose track of the terminal state).

There were two bugs:

* These extra dependencies could cause cycles (#25719 addressed direct cycles, where `Publish <--> Stack`, but not indirect cycles, where `Publish -> StackA -> StackB -> Publish`).
* Publishing nodes would overwrite other nodes with the same ID, and the dependency set they would end up with would be somewhat nondeterministic, making this problem hard to isolate.

Fixes #25806.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. p2 pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested pr-linter/exempt-integ-test The PR linter will not require integ test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

‼️ NOTICE: cdk deploy fails when stacks share assets in v2.80.0
3 participants