ec2: Fetches Incorrect gp2 Volume Type for AL2023 AmazonLinuxImage #27638

DharmSonariya · 2023-10-21T23:48:26Z

Describe the bug

I encountered a validation error when trying to use the ec2.AmazonLinuxImage with AmazonLinuxGeneration.AMAZON_LINUX_2023. It seems the CDK is attempting to fetch an AMI that uses the gp2 volume type for AL2023.

AWS documentation also confirms that AL2023 uses the gp3 volume type by default: link to AWS docs.

There is no AMI al2023-ami with gp2.
aws ssm get-parameters-by-path --path /aws/service/ami-amazon-linux-latest --query "Parameters[].Name"

[
    "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-arm64",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-x86_64",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-minimal-kernel-6.1-arm64",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-minimal-kernel-6.1-x86_64",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-minimal-kernel-default-arm64",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-s3",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-kernel-5.10-hvm-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-minimal-kernel-default-x86_64",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-minimal-hvm-x86_64-s3",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-minimal-pv-x86_64-s3",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-pv-x86_64-s3",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-kernel-5.10-hvm-arm64-gp2",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-kernel-5.10-hvm-x86_64-gp2",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-minimal-hvm-arm64-ebs",
    "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-minimal-hvm-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-minimal-pv-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/amzn-ami-pv-x86_64-ebs",
    "/aws/service/ami-amazon-linux-latest/amzn2-ami-minimal-hvm-x86_64-ebs"
]

Expected Behavior

The CDK should be able to fetch the correct AMI for AmazonLinuxGeneration.AMAZON_LINUX_2023.

Current Behavior

Received a ValidationError as follows:
❌ Deployment failed: Error [ValidationError]: Unable to fetch parameters [/aws/service/ami-amazon-linux-latest/al2023-ami-hvm-x86_64-gp2] from parameter store for this account.

Reproduction Steps

ec2.AmazonLinuxImage(
generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2023,
edition=ec2.AmazonLinuxEdition.STANDARD,
)

Possible Solution

This issue seems similar to a previously resolved bug: #26274 by @pahud

Additional Information/Context

No response

CDK CLI Version

2.102.0 (build 2abc59a)

Framework Version

No response

Node.js Version

v18.18.2

OS

Linux 6.2.0-34-generic #34 x86_64 GNU/Linux

Language

Python

Language Version

Python 3.11.6

Other information

No response

The text was updated successfully, but these errors were encountered:

msambol · 2023-10-22T00:48:01Z

@DharmSonariya I wasn't able to reproduce. Can you confirm the output of cdk version ?

DharmSonariya · 2023-10-22T01:41:55Z

@msambol Thank you for looking into this. Here's the output of the cdk version command from my setup:
2.102.0 (build 2abc59a)
Please let me know if you need any additional information.

msambol · 2023-10-22T01:54:11Z

@DharmSonariya what does your package.json look like?

DharmSonariya · 2023-10-22T13:58:10Z

Hi @msambol,
Certainly! Here's the content of my package.json:

{
    "devDependencies": {
        "aws-cdk": "v2.102.0",
        "@dqbd/tiktoken": "^1.0.7",
        "husky": "^8.0.3"
    },
    "scripts": {
        "prepare": "husky install"
    },
    "dependencies": {}
}

DharmSonariya · 2023-10-22T14:00:34Z

Hi @msambol,
I did some digging into the CDK codebase and noticed something that might be relevant to the bug I'm encountering.

In the aws-cdk/packages/aws-cdk-lib/aws-ec2/lib/machine-image/common.ts file, I couldn't find a value for gp3 under the AmazonLinuxStorage enum:

aws-cdk/packages/aws-cdk-lib/aws-ec2/lib/machine-image/common.ts

Line 8 in 9c0de51

    
           export interface AmazonLinuxImageSsmParameterCommonOptions extends AmazonLinuxImageSsmParameterBaseOptions {

The enum looks like this:

export enum AmazonLinuxStorage {
  /**
   * EBS-backed storage
   */
  EBS = 'ebs',

  /**
   * S3-backed storage
   */
  S3 = 's3',

  /**
   * General Purpose-based storage (recommended)
   */
  GENERAL_PURPOSE = 'gp2',
}

Given the documentation mentions that AL2023 AMIs use Amazon EBS gp3 volumes by default, this could potentially be the root of the issue.

Hope this clue helps in resolving the bug. Let me know if you need any more details or if I can assist further.

tam0ri · 2023-10-24T14:34:58Z

I could reproduce it on my end with the following simple sample code.

import * as cdk from 'aws-cdk-lib';
import {aws_ec2 as ec2} from 'aws-cdk-lib';
import { Construct } from 'constructs';

export class Issue27638Stack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = ec2.Vpc.fromLookup(this, 'VPC', {
      vpcId: "<my vpc id>"
    });

    new ec2.Instance(this, 'targetInstance', {
      vpc: vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE2, ec2.InstanceSize.MICRO),
      machineImage: new ec2.AmazonLinuxImage({ generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2023 }),
    });
  }
}

In AmazonLinuxImage construct's implementation, AMAZON_LINUX_2023 is not handled to be aligned with SSM parameter's naming conventions.

aws-cdk/packages/aws-cdk-lib/aws-ec2/lib/machine-image/machine-image.ts

Lines 486 to 497 in 2abc59a

    
           if (generation === AmazonLinuxGeneration.AMAZON_LINUX_2022) { 
        
             kernel = AmazonLinuxKernel.KERNEL5_X; 
        
             if (props && props.storage) { 
        
               throw new Error('Storage parameter does not exist in smm parameter name for Amazon Linux 2022.'); 
        
             } 
        
             if (props && props.virtualization) { 
        
               throw new Error('Virtualization parameter does not exist in smm parameter name for Amazon Linux 2022.'); 
        
             } 
        
           } else { 
        
             virtualization = (props && props.virtualization) || AmazonLinuxVirt.HVM; 
        
             storage = (props && props.storage) || AmazonLinuxStorage.GENERAL_PURPOSE; 
        
           }

This leads incorrect parameter name for AMAZON_LINUX_2023. We should add the same logic as AMAZON_LINUX_2022. I'll submit PR later.

In the meantime, using latestAmazonLinux2023 method as below is workaround. This method generates correct parameter name.

    new ec2.Instance(this, 'targetInstance', {
      vpc: vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE2, ec2.InstanceSize.MICRO),
      machineImage: ec2.MachineImage.latestAmazonLinux2023(),
    });

DharmSonariya · 2023-10-24T16:51:43Z

Hi @tam0ri,

Thank you so much for taking the time to reproduce the issue and for pinpointing the cause. I really appreciate your thorough analysis.

It's great to hear that a PR will be submitted to address this. I'll definitely try out the latestAmazonLinux2023 method as a temporary workaround. Thanks for suggesting it.

…ter name for AL2023 images (#27698) AmazonLinuxImage construct generates SSM parameter name for Amazon Linux images. The naming convention for Amazon Linux 2023 images is a bit different from Amazon Linux 2. For example, virtualization type (e.g. HVM) or backend storage type (e.g. GP2) are not included in parameter's name for AL2023. AL2: https://github.com/aws/aws-cdk/blob/d0d75478e1cf3bb9a06f33642b9a06fc68d0c99d/packages/aws-cdk-lib/aws-ec2/lib/machine-image/amazon-linux2.ts#L77-L84 AL2023: https://github.com/aws/aws-cdk/blob/d0d75478e1cf3bb9a06f33642b9a06fc68d0c99d/packages/aws-cdk-lib/aws-ec2/lib/machine-image/amazon-linux-2023.ts#L59-L66 Currently, AmazonLinuxImage construct generates incorrect SSM parameter name for AL2023 images, which includes virtualization and storage type in the name. This causes validation error against non-existing parameter name. This PR solves the issue by avoiding to include virtualization and storage in parameter's name when AMAZON_LINUX_2023 is specified as generation. Closes #27638 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

github-actions · 2023-11-10T17:58:57Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

…ter name for AL2023 images (#27698) AmazonLinuxImage construct generates SSM parameter name for Amazon Linux images. The naming convention for Amazon Linux 2023 images is a bit different from Amazon Linux 2. For example, virtualization type (e.g. HVM) or backend storage type (e.g. GP2) are not included in parameter's name for AL2023. AL2: https://github.com/aws/aws-cdk/blob/d0d75478e1cf3bb9a06f33642b9a06fc68d0c99d/packages/aws-cdk-lib/aws-ec2/lib/machine-image/amazon-linux2.ts#L77-L84 AL2023: https://github.com/aws/aws-cdk/blob/d0d75478e1cf3bb9a06f33642b9a06fc68d0c99d/packages/aws-cdk-lib/aws-ec2/lib/machine-image/amazon-linux-2023.ts#L59-L66 Currently, AmazonLinuxImage construct generates incorrect SSM parameter name for AL2023 images, which includes virtualization and storage type in the name. This causes validation error against non-existing parameter name. This PR solves the issue by avoiding to include virtualization and storage in parameter's name when AMAZON_LINUX_2023 is specified as generation. Closes #27638 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

DharmSonariya added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Oct 21, 2023

github-actions bot added the @aws-cdk/aws-ec2 Related to Amazon Elastic Compute Cloud label Oct 21, 2023

indrora added p1 and removed needs-triage This issue or PR still needs to be triaged. labels Oct 23, 2023

tam0ri mentioned this issue Oct 26, 2023

fix(ec2): AmazonLinuxImage construct generates incorrect SSM parameter name for AL2023 images #27698

Merged

github-actions bot mentioned this issue Nov 1, 2023

Monthly issue metrics report - October 2023 #27785

Closed

mergify bot closed this as completed in #27698 Nov 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ec2: Fetches Incorrect gp2 Volume Type for AL2023 AmazonLinuxImage #27638

ec2: Fetches Incorrect gp2 Volume Type for AL2023 AmazonLinuxImage #27638

DharmSonariya commented Oct 21, 2023 •

edited

Loading

msambol commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

msambol commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

tam0ri commented Oct 24, 2023

DharmSonariya commented Oct 24, 2023

github-actions bot commented Nov 10, 2023

ec2: Fetches Incorrect gp2 Volume Type for AL2023 AmazonLinuxImage #27638

ec2: Fetches Incorrect gp2 Volume Type for AL2023 AmazonLinuxImage #27638

Comments

DharmSonariya commented Oct 21, 2023 • edited Loading

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

msambol commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

msambol commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

DharmSonariya commented Oct 22, 2023

tam0ri commented Oct 24, 2023

DharmSonariya commented Oct 24, 2023

github-actions bot commented Nov 10, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

DharmSonariya commented Oct 21, 2023 •

edited

Loading