Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modular toolkit toolchain #1893

Closed
eladb opened this issue Feb 26, 2019 · 0 comments · Fixed by #2636 or MechanicalRock/tech-radar#14 · May be fixed by MechanicalRock/cdk-constructs#5, MechanicalRock/cdk-constructs#6 or MechanicalRock/cdk-constructs#7
Closed

Modular toolkit toolchain #1893

eladb opened this issue Feb 26, 2019 · 0 comments · Fixed by #2636 or MechanicalRock/tech-radar#14 · May be fixed by MechanicalRock/cdk-constructs#5, MechanicalRock/cdk-constructs#6 or MechanicalRock/cdk-constructs#7
Assignees
@eladb
Labels
@aws-cdk/core Related to core CDK functionality effort/medium Medium work item – several days of effort

Comments

@eladb
Copy link
Contributor

eladb commented Feb 26, 2019 •
edited
Loading

Current State

CDK apps can be deployed today using the CDK CLI (or CDK Toolkit). The toolkit is implemented as a monolithic program with no clear boundaries between the various stages. We would like to break up the monolithic process executed by the toolkit in order to synthesize, package and deploy a CDK app.

There a few reasons why we want to do this:

  1. Security: isolate the "deploy" step such that no user code needs to run. This is very important from a security perspective because deployment commonly require administrator privileges on the AWS account, and we need to reduce the attack surface during that time (see feat(app-delivery): continuous delivery for CDK apps #2073, which currently has to run both build and deploy together in the same CodeBuild task).
  2. Modularity: Allow tools to utilize the various steps used to deploy a CDK app inside other tools such as IDEs, deployment tools, etc.
  3. Code Quality: the CLI's code base needs a fresh rewrite, along with complete unit test coverage and this is an opportunity to do that.

Requirements

  • It should be possible to execute each component in the toolchain individually by feeding it the output from the previous step.
  • Given a specific input, the output from each step must be completely reproducible (no side effects).
  • Different components may require different execution environments and/or permissions to run. For example cdk-synth may need to be able to query the target AWS account in order to resolve environmental context, cdk-bundle may need to build docker images, cdk-deploy will need admin permissions in order to deploy the app.
  • It should be possible to invoke each component as a jsii library from all supported languages.
@eladb eladb self-assigned this Feb 26, 2019
This was referenced Feb 26, 2019
Merged
Closed
Merged
@eladb eladb added the @aws-cdk/core Related to core CDK functionality label Feb 28, 2019
sam-goodwin pushed a commit that referenced this issue Mar 1, 2019
feat(core): democratize synthesis and introduce artifacts (#1889)
4ab1cd3 
This change enables each construct in the tree to participate in synthesis
by overriding the synthesize and add "artifacts" and write files to a synthesis session.

At the moment, the only construct that implements synthesize is Stack which emits an `aws:cloudformation:stack` artifact which includes a reference to a template file stored into the synthesis session as well.

`App.run()` traverses the tree and invokes synthesize on all constructs.

When synthesis is finalized, a manifest is produced and stored in the root `manifest.json` file. This is accordance with the draft specification for cloud assemblies as authored by @RomainMuller and included in this change.

For backwards compatibility, and until we implement the required changes in the toolkit, the output also contains `cdk.out` which is 100% compatible with the current output.

Related #1716 
Related #1893
@eladb eladb changed the title Artifacts Artifacts (new cx manifest) Mar 4, 2019
@fulghum fulghum added the effort/medium Medium work item – several days of effort label Mar 4, 2019
@eladb eladb changed the title Artifacts (new cx manifest) Modular toolkit Mar 26, 2019
@eladb eladb changed the title Modular toolkit Modular toolkit lifecycle components Mar 26, 2019
@eladb eladb changed the title Modular toolkit lifecycle components Modular toolkit toolchain Mar 26, 2019
@eladb eladb mentioned this issue Mar 26, 2019
Closed
@eladb eladb mentioned this issue May 24, 2019
Merged
6 tasks
eladb pushed a commit that referenced this issue May 29, 2019
feat: decouple "synth" and "deploy" through cloud assemblies (#2636)
c52bcfc 
Formalize the concept of a cloud assembly to allow decoupling "synth" and "deploy". the main
motivation is to allow "deploy" to run in a controlled environment without needing to execute the app for security purposes.

`cdk synth` now produces a self-contained assembly in the output directory, which we call a "cloud assembly". this directory includes synthesized templates (similar to current behavior) but also a "manifest.json" file and the asset staging directories.

`cdk deploy -a assembly-dir` will now skip synthesis and will directly deploy the assembly.

To that end, we modified the behavior of asset staging such that all synth output always goes to the output directory, which is by default `cdk.out` (can be set with `--output`). if there's a single template, it is printed to stdout, otherwise the name of output directory is printed.

This PR also includes multiple clean ups and deprecations of stale APIs and modules such as:

- `cdk.AppProps` includes various new options.
- An error is thrown if references between stacks that are not in the same app (mostly in test cases).
- Added the token creation stack trace for errors emitted by tokens
- Added `ConstructNode.root` which returns the tree root.
 

**TESTING**: verified that all integration tests passed and added a few tests to verify zip and docker assets as well as cloud assemblies. See: https://github.com/awslabs/cdk-ops/pull/364

Closes #1893 
Closes #2093 
Closes #1954 
Closes #2310 
Related #2073 
Closes #1245
Closes #341
Closes #956
Closes #233

BREAKING CHANGE: *IMPORTANT*: apps created with the CDK version 0.33.0 and above cannot be used with an older CLI version.
- `--interactive` has been removed
- `--numbered` has been removed
- `--staging` is now a boolean flag that indicates whether assets should be copied to the `--output` directory or directly referenced (`--no-staging` is useful for e.g. local debugging with SAM CLI)
- Assets (e.g. Lambda code assets) are now referenced relative to the output directory.
- `SynthUtils.templateForStackName` has been removed (use `SynthUtils.synthesize(stack).template`).
- `cxapi.SynthesizedStack` renamed to `cxapi.CloudFormationStackArtifact` with multiple API changes.
- `cdk.App.run()` now returns a `cxapi.CloudAssembly` instead of `cdk.ISynthesisSession`.
- `cdk.App.synthesizeStack` and `synthesizeStacks` has been removed. The `cxapi.CloudAssembly` object returned from `app.run()` can be used as an alternative to explore a synthesized assembly programmatically (resolves #2016).
- `cdk.CfnElement.creationTimeStamp` may now return `undefined` if there is no stack trace captured.
- `cdk.ISynthesizable.synthesize` now accepts a `cxapi.CloudAssemblyBuilder` instead of `ISynthesisSession`.
- `cdk`: The concepts of a synthesis session and session stores have been removed (`cdk.FileSystemStore`, cdk.InMemoryStore`, `cdk.SynthesisSession`, `cdk.ISynthesisSession` and `cdk.SynthesisOptions`).
- No more support for legacy manifests (`cdk.ManifestOptions` member `legacyManifest` has been removed)
- All support for build/bundle manifest have been removed (`cxapi.BuildManifest`, `cxapi.BuildStep`, etc).
- Multiple deprecations and breaking changes in the `cxapi` module (`cxapi.AppRuntime` has been renamed to `cxapi.RuntimeInfo`, `cxapi.SynthesizeResponse`, `cxapi.SynthesizedStack` has been removed)
- The `@aws-cdk/applet-js` program is no longer available. Use [decdk](https://github.com/awslabs/aws-cdk/tree/master/packages/decdk) as an alternative.
This was referenced Aug 22, 2019
Open
Open
Open
Open
Open
This was referenced Dec 12, 2019
Closed
Closed
Closed
Merged
Closed
Closed
Open
Open
Open
Open
This was referenced Dec 12, 2019
Open
Open
Open
Open
Open
Open
This was referenced Jan 20, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eladb eladb

Labels
@aws-cdk/core Related to core CDK functionality effort/medium Medium work item – several days of effort
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: decouple "synth" and "deploy" through cloud assemblies aws/aws-cdk
[Snyk] Upgrade @aws-cdk/aws-iam from 0.22.0 to 0.39.0 MechanicalRock/tech-radar
[Snyk] Upgrade @aws-cdk/aws-codebuild from 0.24.1 to 0.39.0 MechanicalRock/cdk-constructs
[Snyk] Upgrade @aws-cdk/aws-codepipeline from 0.24.1 to 0.39.0 MechanicalRock/cdk-constructs
[Snyk] Upgrade @aws-cdk/aws-s3 from 0.24.1 to 0.39.0 MechanicalRock/cdk-constructs
[Snyk] Upgrade @aws-cdk/aws-cloudfront from 0.15.2 to 0.39.0 MechanicalRock/cdk-products
[Snyk] Upgrade @aws-cdk/aws-codebuild from 0.15.2 to 0.39.0 MechanicalRock/cdk-products
[Snyk] Upgrade @aws-cdk/aws-codecommit from 0.15.2 to 0.39.0 MechanicalRock/cdk-products
[Snyk] Upgrade @aws-cdk/aws-codepipeline from 0.15.2 to 0.39.0 MechanicalRock/cdk-products
[Snyk] Upgrade @aws-cdk/cdk from 0.24.1 to 0.35.0 MechanicalRock/account-reaper
2 participants
@fulghum @eladb