Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(cli): import is not using a custom qualifier #32278

Open
1 task
moltar opened this issue Nov 25, 2024 · 3 comments
Open
1 task

(cli): import is not using a custom qualifier #32278

moltar opened this issue Nov 25, 2024 · 3 comments
Labels
bug This issue is a bug. effort/medium Medium work item – several days of effort p2 package/tools Related to AWS CDK Tools or CLI

Comments

@moltar
Copy link
Contributor

moltar commented Nov 25, 2024

Describe the bug

Similar to #29179, cdk import ignores qualifier set in the cdk.json context and tries to use defaults.

Which results in a failure.

Supplying the stack name fixes the problem, but it's not a scalable approach.

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.171.0

Expected Behavior

cdk import to respect the qualifier.

Current Behavior

Does not respect the qualifier.

Reproduction Steps

Use a custom qualifier.

Try to do an import with a large template.

Get an error:

❌ my-stack-name failed: Error: Template too large to deploy ("cdk bootstrap" is required)

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.171.0

Framework Version

No response

Node.js Version

22

OS

macOs

Language

TypeScript

Language Version

No response

Other information

No response

@moltar moltar added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 25, 2024
@github-actions github-actions bot added the package/tools Related to AWS CDK Tools or CLI label Nov 25, 2024
@ashishdhingra ashishdhingra added p2 needs-reproduction This issue needs reproduction. and removed needs-triage This issue or PR still needs to be triaged. labels Nov 26, 2024
@ashishdhingra ashishdhingra self-assigned this Nov 26, 2024
@ashishdhingra
Copy link
Contributor

ashishdhingra commented Nov 27, 2024

@moltar Good afternoon. Somehow, I'm unable to reproduce the issue using cdk import to import resources that generate more than 50KB of CloudFormation template. Used version 2.171.0 of aws-cdk-lib and CDK CLI.
STEPS:

  • Bootstrapped CDK account for region ap-south-1 using custom qualifier cdk bootstrap aws://<<ACCOUNT-ID>>/ap-south-1 --qualifier abcdefghi.
  • Created a new CDK project. Modified cdk.json to add "@aws-cdk/core:bootstrapQualifier": "abcdefghi" context key.
  • Using AWS CLI, created 150 test IAM roles using shell script for i in {1..150}; do aws iam create-role --assume-role-policy-document file://comprehend-trust-policy.json --role-name testrole$i; done. Used below policy (taken from https://docs.aws.amazon.com/comprehend/latest/dg/tutorial-reviews-create-role.html):
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "comprehend.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
  • Using shell script for i in {1..150}; do echo "new iam.Role(this, 'testrole$i', { roleName: 'testrole$i', assumedBy: new iam.ServicePrincipal('comprehend.amazonaws.com')});"; done, generated the TypeScript code.
  • Pasted the generated TypeScript code statements into my CDK stack.
  • Ensured that bin/cskstack.ts during app creation, region is set to ap-south-1.
      #!/usr/bin/env node
    import * as cdk from 'aws-cdk-lib';
    import { CdkimportCustomqualifierStack } from '../lib/cdkimport_customqualifier-stack';
    
    const app = new cdk.App();
    new CdkimportCustomqualifierStack(app, 'CdkimportCustomqualifierStack', {
      env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: 'ap-south-1'}, //process.env.CDK_DEFAULT_REGION },
    });
  • Executed command cdk import.
    • In S3 bucket, the template size is 68.6 KB.
    • cdk import is successful.

CLEANUP:
Delete test roles using AWS CLI command for i in {1..150}; do aws iam delete-role --role-name testrole$i; done.

Please let me know if I'm missing anything. Did you also update the version of CDK CLI using command sudo npm install -g aws-cdk@latest?

Thanks,
Ashish

@ashishdhingra ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-reproduction This issue needs reproduction. labels Nov 27, 2024
@moltar
Copy link
Contributor Author

moltar commented Nov 27, 2024

Hi @ashishdhingra thank you for the repro steps.

I think the bootstrap stack name plays a role too.

In your repro, you still used the default one (CDKToolkit) which I'm guessing CDK then uses to find/infer some knowledge.

I use a different stack name. And what fixed the import for me was to explicitly set the stack name as the CLI option.

But IMO that should be unnecessary as there's nothing that CDK should need from that stack. Everything it needs to do the work is based on well known resource names based on the qualifier.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Nov 27, 2024
@ashishdhingra
Copy link
Contributor

RESULT:
Running cdk import fails with the below error:

CdkimportCustomqualifierStack
start: Building deb29ee76febff8330c2b17a6ed5ac0d796273fd2f2fdce616e4622b0abf1274:<<ACCOUNT-ID>>-ap-south-1
success: Built deb29ee76febff8330c2b17a6ed5ac0d796273fd2f2fdce616e4622b0abf1274:<<ACCOUNT-ID>>-ap-south-1
start: Publishing deb29ee76febff8330c2b17a6ed5ac0d796273fd2f2fdce616e4622b0abf1274:<<ACCOUNT-ID>>-ap-south-1
success: Published deb29ee76febff8330c2b17a6ed5ac0d796273fd2f2fdce616e4622b0abf1274:<<ACCOUNT-ID>>-ap-south-1
CdkimportCustomqualifierStack/testrole1/Resource (AWS::IAM::Role): import with RoleName=testrole1 (yes/no) [default: yes]? 
CdkimportCustomqualifierStack/testrole2/Resource (AWS::IAM::Role): import with RoleName=testrole2 (yes/no) [default: yes]? 
...
...
CdkimportCustomqualifierStack/testrole149/Resource (AWS::IAM::Role): import with RoleName=testrole149 (yes/no) [default: yes]? 
CdkimportCustomqualifierStack/testrole150/Resource (AWS::IAM::Role): import with RoleName=testrole150 (yes/no) [default: yes]? 
CdkimportCustomqualifierStack: importing resources into stack...

The template for stack "CdkimportCustomqualifierStack" is 61KiB. Templates larger than 50KiB must be uploaded to S3.
Run the following command in order to setup an S3 bucket in this environment, and then re-deploy:

        $ cdk bootstrap aws://<<ACCOUNT-ID>>/ap-south-1


 ❌  CdkimportCustomqualifierStack failed: Error: Template too large to deploy ("cdk bootstrap" is required)
    at makeBodyParameter (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:607:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async deployStack (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:613:277)
    at async ResourceImporter.importResources (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:818:178840)
    at async ResourceImporter.importResourcesFromMap (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:818:178503)
    at async CdkToolkit.import (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:819:13553)
    at async exec3 (/Users/<<username>>/.nvm/versions/node/v18.20.4/lib/node_modules/aws-cdk/lib/index.js:821:18910)
****************************************************
*** Newer version of CDK is available [2.171.1]  ***
*** Upgrade recommended (npm install -g aws-cdk) ***
****************************************************
Template too large to deploy ("cdk bootstrap" is required)

@ashishdhingra ashishdhingra added the effort/medium Medium work item – several days of effort label Nov 27, 2024
@ashishdhingra ashishdhingra removed their assignment Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. effort/medium Medium work item – several days of effort p2 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

No branches or pull requests

2 participants