Problem in understanding stages and assigned versions/aliases

[ssy-lehmann]

AWS SAM Knowledge

Learning phase

Background

Using myself awscli in the past to build up the infrastructure for api gateway, lambda and S3
the advantage (at least) was to have a very similar behavior on building things like
the user does in the AWS Console. I appreciate the idea behind AWS SAM but anyway the
AWS Console UI experience has changed (seems so) looking into AWS SAM...

Looking for anwers

Searching for some answers on how to do it with AWS SAM:
(scenario is a simple hello world as you provide with sam init - working perfectly - of course)

1. I know how to avoid the default stages (ok)
   (using the OpenApiVersion attribute for it 😃 🆗

2. I can additionally deploy further stages 👌
   (using AWS::ApiGateway::Deployment)

3. (It seems) I have no control over the stage I have to name in the api resource itself
   (Using !Ref in StageName on an own stage and an own deployment (stage requires deploymentid)
   result in an error because the tool says that the stage already exists)

   CREATE_FAILED AWS::ApiGateway::Stage HelloWorldApiStage 80ga23f3ge|dev already exists in stack arn:aws:cloudformation:us-
   east-1:987161882288:stack/ssy-tle-stack-hello-world/96fcb740-69a8-11ed-b4e6-0ac54655a7c3
   UPDATE_ROLLBACK_IN_PROGRESS AWS::CloudFormation::Stack ssy-tle-stack-hello-world The following resource(s) failed to create: [HelloWorldApiStage].

   Edit:

   After leaving away StageName I got another error

   CREATE_FAILED AWS::ApiGateway::Stage HelloWorldApiStage 80ga23f3ge|ssytlHello1p6FLNGKoLOS already exists in stack arn:aws:cloudformation:us-
   east-1:987161882288:stack/ssy-tle-stack-hello-world/96fcb740-69a8-11ed-b4e6-0ac54655a7c3
   CREATE_FAILED AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissionStage Resource creation cancelled
   UPDATE_ROLLBACK_IN_PROGRESS AWS::CloudFormation::Stack ssy-tle-stack-hello-world The following resource(s) failed to create: [HelloWorldFunctionHelloWorldPermissionStage,
   HelloWorldApiStage].
   UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS AWS::CloudFormation::Stack ssy-tle-stack

4. I can define an alias for the version on a new deployment including canary 👌

5. I have currently no idea (in AWS SAM) how to tell following (as an example):

   • the dev stage is using the latest version of a lambda function
   • the prod stage is using the "live" alias of a lambda function (not necessarily $LATEST)

   Just to rember that in UI I can define stage variables and in the integration part of a method I can use
   the variables (working same way in aws cli -> apigateway put-integration).

6. Finally - using an alarm like AliasErrorMetricGreaterThanZeroAlarm as documented on official
   AWS documentation the template code references the function by name like

    - Name: FunctionName
      Value: !Ref HelloWorldFunction
   

   Considering to apply same alarm on all lambda functions it seems that I would have
   to duplicate that code? I know there is a global section for providing defaults
   on concrete resources but not an example on how to reuse parametrizable
   template fragments. Any proposes?

   My current template file you will find in the APPENDIX (see below)

Kind Regards,
Thomas

APPENDIX

Using Java 11 for Lamba.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: |
  demo-app
  Sample SAM Template for demo-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 20
    Tracing: Active
  Api:
    TracingEnabled: True

Resources:
  HelloWorldApi:
    Type: AWS::Serverless::Api
    Properties:
      Name: my-demo-api
      Description: A very simple demo api
      StageName: !Ref HelloWorldApiStageDev
      OpenApiVersion: 3.0

  HelloWorldApiStageDev:
    Type: AWS::ApiGateway::Stage
    Properties:
      RestApiId: !Ref HelloWorldApi
      DeploymentId: !Ref DeploymentDev

  DeploymentDev:
    Type: 'AWS::ApiGateway::Deployment'
    Properties:
      RestApiId: !Ref HelloWorldApi
      Description: Deployment to Dev Stage
      StageName: dev

  DeploymentProd:
    Type: 'AWS::ApiGateway::Deployment'
    Properties:
      RestApiId: !Ref HelloWorldApi
      Description: Deployment to Prod Stage
      StageName: Prod

  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      FunctionName: my-lambda-hello-world
      Description: A very simple hello world Lambda function
      CodeUri: HelloWorldFunction
      Handler: helloworld.App::handleRequest
      Runtime: java11
      Architectures:
        - x86_64
      MemorySize: 512
      Timeout: 60

      AutoPublishAlias: live

      # deploying serverless function gradually
      DeploymentPreference:
        Type: Canary10Percent10Minutes
        Alarms:
          # A list of alarms that you want to monitor
          - !Ref AliasErrorMetricGreaterThanZeroAlarm
          - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm

      Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        Variables:
          JAVA_TOOL_OPTIONS: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 # More info about tiered compilation https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-function-performance-for-java/

      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            RestApiId: !Ref HelloWorldApi
            Path: /hello
            Method: get

  AliasErrorMetricGreaterThanZeroAlarm:
    Type: "AWS::CloudWatch::Alarm"
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub "${HelloWorldFunction}:live"
        - Name: FunctionName
          Value: !Ref HelloWorldFunction
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  LatestVersionErrorMetricGreaterThanZeroAlarm:
    Type: "AWS::CloudWatch::Alarm"
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub "${HelloWorldFunction}:live"
        - Name: FunctionName
          Value: !Ref HelloWorldFunction
        - Name: ExecutedVersion
          Value: !GetAtt HelloWorldFunction.Version.Version
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  ApiGatewayAccountConfig:
    Type: "AWS::ApiGateway::Account"
    Properties:
      CloudWatchRoleArn: !GetAtt "ApiGatewayLoggingRole.Arn"

  ApiGatewayLoggingRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - "apigateway.amazonaws.com"
            Action: "sts:AssumeRole"
      Path: "/"
      ManagedPolicyArns:
        - !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs"

Outputs:
  HelloWorldApiProd:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${HelloWorldApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldApiDev:
    Description: "API Gateway endpoint URL for Dev stage for Hello World function"
    Value: !Sub "https://${HelloWorldApi}.execute-api.${AWS::Region}.amazonaws.com/Dev/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

[aahung]

Hi @ssy-lehmann, First thanks for trying SAM. I am trying to understand your questions here. Correct if I misunderstood any.

For the 3, I don't quite get what exactly you tried. You provided a template in the end but two error messages. It would be great if you can paste which template results in which error message. Besides, it will be helpful if you can provide the output of sam validate --debug, which is the out of the AWS SAM.

For 5, as you are using the Api event source, we (SAM) don't really support setting which version of the lambda for Api integration. As a workaround, you can try to use native CFN resource or write your own swagger/openapi to create such lambda integration.

For 6, for native CFN or SAM template, there is no way to parameterize the alarm and automatically apply it to all the lambda functions. You will need put some templating tools such as jinja to generate SAM/CFN template instead.

[ssy-lehmann]

Hi @aahung, thanks for answering.

For the 3, I added the output of sam validate --debug (see APPENDIX A). My intention was to fully specify the stages and deployments myself. In following link https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-generated-resources-api.html I found out that - by specifying the StageName - a stage and a deployment is generated automatically. It seems - maybe I'm wrong - I can't overrule this. However, leaving away StageName in DevStage and DevDeployment I ran into the second error (see 'Edit:' in last post).

Eventuall I cannot use SAM Serverless capabilities with as intended (for stages). I can - of course - define AWS::Apigateway::Resource and AWS::Apigateway::Method instead and then - suddenly - those stage issues are gone but that looks to me more CloudFormation specific and - of course - more verbose; then the version specific call of functions works (besides some permission problems I did have to fix which are not given with AWS::Serverless::API) -> see APPENDIX B

I still could seperate the stuff into different AWS Accounts (ProdAccount, TestAccount, DevAccount) working with just one stage 'Default' (or 'Prod') and all is good but the full API I defined before via awscli was working with three stages in one account and I assumed I could do the same using AWS SAM. I was looking into a Video by Chris Munns with "Deep Dive into AWS SAM" and he put following on his slide (reference: https://www.youtube.com/watch?v=CIdUU6rNdk4):

Unfortunately he was not showing how he has implemented that.

APPENDIX A

sam validate --debug
2022-11-23 08:02:54,426 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-11-23 08:02:54,427 | Using config file: samconfig.toml, config environment: default
2022-11-23 08:02:54,428 | Expand command line arguments to:
2022-11-23 08:02:54,428 | --template_file=C:\dev\poc\aws\sam\demo-app\template.yaml
2022-11-23 08:02:54 Loading policies from IAM...
2022-11-23 08:02:58 Finished loading policies from IAM.
2022-11-23 08:02:58 Template schema validation reported the following errors: [Resources.HelloWorldApi.Properties.OpenApiVersion] 3.0 is not of type 'string'
2022-11-23 08:02:58 Feature 'deployment_preference_condition_fix' not available in Feature Toggle Config.
2022-11-23 08:02:58,760 | Translated template is:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'demo-app

  Sample SAM Template for demo-app

  '
Resources:
  HelloWorldApiStageDev:
    Type: AWS::ApiGateway::Stage
    Properties:
      RestApiId:
        Ref: HelloWorldApi
      DeploymentId:
        Ref: DeploymentDev
  DeploymentDev:
    Type: AWS::ApiGateway::Deployment
    Properties:
      RestApiId:
        Ref: HelloWorldApi
      Description: Deployment to Dev Stage
      StageName: dev
  DeploymentProd:
    Type: AWS::ApiGateway::Deployment
    Properties:
      RestApiId:
        Ref: HelloWorldApi
      Description: Deployment to Prod Stage
      StageName: Prod
  AliasErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
      - Name: Resource
        Value:
          Fn::Sub: ${HelloWorldFunction}:live
      - Name: FunctionName
        Value:
          Ref: HelloWorldFunction
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0
  LatestVersionErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
      - Name: Resource
        Value:
          Fn::Sub: ${HelloWorldFunction}:live
      - Name: FunctionName
        Value:
          Ref: HelloWorldFunction
      - Name: ExecutedVersion
        Value:
          Fn::GetAtt:
          - HelloWorldFunctionVersion8ba114db48
          - Version
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0
  ApiGatewayAccountConfig:
    Type: AWS::ApiGateway::Account
    Properties:
      CloudWatchRoleArn:
        Fn::GetAtt:
        - ApiGatewayLoggingRole
        - Arn
  ApiGatewayLoggingRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - apigateway.amazonaws.com
          Action: sts:AssumeRole
      Path: /
      ManagedPolicyArns:
      - Fn::Sub: arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
  HelloWorldFunction:
    Properties:
      Architectures:
      - x86_64
      Code:
        S3Bucket: bucket
        S3Key: value
      Description: A very simple hello world Lambda function
      Environment:
        Variables:
          JAVA_TOOL_OPTIONS: -XX:+TieredCompilation -XX:TieredStopAtLevel=1
      FunctionName: my-lambda-hello-world
      Handler: helloworld.App::handleRequest
      MemorySize: 512
      Role:
        Fn::GetAtt:
        - HelloWorldFunctionRole
        - Arn
      Runtime: java11
      Tags:
      - Key: lambda:createdBy
        Value: SAM
      Timeout: 60
      TracingConfig:
        Mode: Active
    Type: AWS::Lambda::Function
  HelloWorldFunctionVersion8ba114db48:
    DeletionPolicy: Retain
    Properties:
      FunctionName:
        Ref: HelloWorldFunction
    Type: AWS::Lambda::Version
  HelloWorldFunctionAliaslive:
    Properties:
      FunctionName:
        Ref: HelloWorldFunction
      FunctionVersion:
        Fn::GetAtt:
        - HelloWorldFunctionVersion8ba114db48
        - Version
      Name: live
    Type: AWS::Lambda::Alias
    UpdatePolicy:
      CodeDeployLambdaAliasUpdate:
        ApplicationName:
          Ref: ServerlessDeploymentApplication
        DeploymentGroupName:
          Ref: HelloWorldFunctionDeploymentGroup
  HelloWorldFunctionRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
        Version: '2012-10-17'
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      - arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess
      Tags:
      - Key: lambda:createdBy
        Value: SAM
    Type: AWS::IAM::Role
  HelloWorldFunctionHelloWorldPermissionStage:
    Properties:
      Action: lambda:InvokeFunction
      FunctionName:
        Ref: HelloWorldFunctionAliaslive
      Principal: apigateway.amazonaws.com
      SourceArn:
        Fn::Sub:
        - arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${__ApiId__}/${__Stage__}/GET/hello
        - __ApiId__:
            Ref: HelloWorldApi
          __Stage__: '*'
    Type: AWS::Lambda::Permission
  HelloWorldApi:
    Properties:
      Body:
        info:
          version: '1.0'
          title:
            Ref: AWS::StackName
        paths:
          /hello:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri:
                  Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HelloWorldFunctionAliaslive}/invocations
              responses: {}
        openapi: '3.0'
      Description: A very simple demo api
      Name: my-demo-api
    Type: AWS::ApiGateway::RestApi
  HelloWorldApiDeploymente040ec5eb0:
    Properties:
      Description: 'RestApi deployment id: e040ec5eb09496520ccee5390ea638f268fbda58'
      RestApiId:
        Ref: HelloWorldApi
    Type: AWS::ApiGateway::Deployment
  HelloWorldApiStage:
    Properties:
      DeploymentId:
        Ref: HelloWorldApiDeploymente040ec5eb0
      RestApiId:
        Ref: HelloWorldApi
      StageName:
        Ref: HelloWorldApiStageDev
      TracingEnabled: true
    Type: AWS::ApiGateway::Stage
  ServerlessDeploymentApplication:
    Properties:
      ComputePlatform: Lambda
    Type: AWS::CodeDeploy::Application
  CodeDeployServiceRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - codedeploy.amazonaws.com
        Version: '2012-10-17'
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSCodeDeployRoleForLambda
    Type: AWS::IAM::Role
  HelloWorldFunctionDeploymentGroup:
    Properties:
      AlarmConfiguration:
        Alarms:
        - Name:
            Ref: AliasErrorMetricGreaterThanZeroAlarm
        - Name:
            Ref: LatestVersionErrorMetricGreaterThanZeroAlarm
        Enabled: true
      ApplicationName:
        Ref: ServerlessDeploymentApplication
      AutoRollbackConfiguration:
        Enabled: true
        Events:
        - DEPLOYMENT_FAILURE
        - DEPLOYMENT_STOP_ON_ALARM
        - DEPLOYMENT_STOP_ON_REQUEST
      DeploymentConfigName:
        Fn::Sub:
        - CodeDeployDefault.Lambda${ConfigName}
        - ConfigName: Canary10Percent10Minutes
      DeploymentStyle:
        DeploymentOption: WITH_TRAFFIC_CONTROL
        DeploymentType: BLUE_GREEN
      ServiceRoleArn:
        Fn::GetAtt:
        - CodeDeployServiceRole
        - Arn
    Type: AWS::CodeDeploy::DeploymentGroup
Outputs:
  HelloWorldApiProd:
    Description: API Gateway endpoint URL for Prod stage for Hello World function
    Value:
      Fn::Sub: https://${HelloWorldApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/
  HelloWorldApiDev:
    Description: API Gateway endpoint URL for Dev stage for Hello World function
    Value:
      Fn::Sub: https://${HelloWorldApi}.execute-api.${AWS::Region}.amazonaws.com/Dev/hello/
  HelloWorldFunction:
    Description: Hello World Lambda Function ARN
    Value:
      Fn::GetAtt:
      - HelloWorldFunction
      - Arn
  HelloWorldFunctionIamRole:
    Description: Implicit IAM Role created for Hello World function
    Value:
      Fn::GetAtt:
      - HelloWorldFunctionRole
      - Arn

C:\dev\poc\aws\sam\demo-app\template.yaml is a valid SAM Template

APPENDIX B

  HelloWorldMethod:
    Type: AWS::ApiGateway::Method
    DependsOn:
      - HelloWorldFunction
    Properties:
      RestApiId: !Ref HelloWorldApi
      ResourceId: !Ref HelloWorldResource
      HttpMethod: GET
      AuthorizationType: NONE
      Integration:
        Type: AWS_PROXY
        IntegrationHttpMethod: GET
        Uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HelloWorldFunction.Arn}${!stageVariables.Alias}/invocations"

[aahung]

Thanks for providing the info, we will take a look on how it could be implemented and if not possible by AWM SAM we will convert this into a feature request