Bug: Created DynamoDB autoscaling policy fails if there is the Cloudwatch Api is throttled (eg rate limited)

[frankfarrell]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Terraform v0.11.7
provider.aws v1.14.1

Affected Resource(s)

• aws_appautoscaling_policy

Terraform Configuration Files

variable "max_capacity" {}
variable "min_capacity" {
  default = 1.0
}
variable "target_utilization" {
  default = 70.0
}
variable table_name {}
variable account_id {}

resource "aws_appautoscaling_target" "dynamodb_table_target_read" {
  max_capacity       = "${var.max_capacity}"
  min_capacity       = "${var.min_capacity}"
  resource_id        = "table/${var.table_name}"
  role_arn           = "arn:aws:iam::${var.account_id}:role/aws-service-role/dynamodb.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_DynamoDBTable"
  scalable_dimension = "dynamodb:table:ReadCapacityUnits"
  service_namespace  = "dynamodb"
}


resource "aws_appautoscaling_policy" "dynamodb_table_policy_read" {
  name               = "DynamoDBReadCapacityUtilization:${aws_appautoscaling_target.dynamodb_table_target_read.resource_id}"
  policy_type        = "TargetTrackingScaling"
  resource_id        = "${aws_appautoscaling_target.dynamodb_table_target_read.resource_id}"
  scalable_dimension = "${aws_appautoscaling_target.dynamodb_table_target_read.scalable_dimension}"
  service_namespace  = "${aws_appautoscaling_target.dynamodb_table_target_read.service_namespace}"

  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "DynamoDBReadCapacityUtilization"
    }

    target_value = "${1.0 * var.target_utilization}"
  }
}

Debug Output

N/A

Panic Output

n/a

Expected Behavior

It should create the read policy correctly.

Actual Behavior

It failed because of a cloudwatch throttling error

aws_appautoscaling_policy.dynamodb_table_policy_read: Failed to create scaling policy: FailedResourceAccessException: Unable to create alarms for scaling policy DynamoDBReadCapacityUtilization:table/`tableName due to reason: Rate exceeded (Service: AmazonCloudWatch; Status Code: 400; Error Code: Throttling; Request ID: <xxxxxxxxxxxxxxxxxxxx>)
    status code: 400, request id: <yyyyyyyyyyyyyyyyyyyyyyyyy>

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply
2. Somehow cause a Cloudwatch throttling exception!

Important Factoids

No

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• Make aws_iam_policy_attachment warning less opaque #1234

[bflad]

This has been released in version 1.20.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!