Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to change DynamoDB billing_mode from PAY_PER_REQUEST to PROVISIONED when using GSIs #7447

Closed
plektra opened this issue Feb 5, 2019 · 6 comments · Fixed by #7453
Closed
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Milestone

Comments

@plektra
Copy link

plektra commented Feb 5, 2019

When updating a DynamoDB table with one or more global secondary indexes, changing billing_mode from PAY_PER_REQUEST to PROVISIONED, the update fails to apply. Terraform tries to create a new GSI with given read & write capacity, while the existing GSI is left with "0" capacity which makes it fail due to invalid parameter (ProvisionedThroughput must be specified when BillingMode is PROVISIONED).

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.10
provider.aws v1.57.0

Affected Resource(s)

  • aws_dynamodb_table

Terraform Configuration Files

resource "aws_dynamodb_table" "users" {
  name           = "users"
  billing_mode   = "PROVISIONED"
  read_capacity  = "10"
  write_capacity = "10"
  hash_key       = "uuid"

  attribute {
    name = "uuid"
    type = "S"
  }

  attribute {
    name = "nick_lower"
    type = "S"
  }

  attribute {
    name = "discriminator"
    type = "S"
  }

  global_secondary_index {
    name               = "username"
    hash_key           = "nick_lower"
    range_key          = "discriminator"
    write_capacity     = "10"
    read_capacity      = "10"
    projection_type    = "INCLUDE"
    non_key_attributes = [ "nick" ]
  }
}

Expected Behavior

Billing mode should have changed along with read & write capacities for the table and global secondary index(es).

Actual Behavior

terraform apply exits with an error.

Output:

aws_dynamodb_table.users: Modifying... (ID: users)
  billing_mode:                                           "PAY_PER_REQUEST" => "PROVISIONED"
  global_secondary_index.3667214579.hash_key:             "nick_lower" => ""
  global_secondary_index.3667214579.name:                 "username" => ""
  global_secondary_index.3667214579.non_key_attributes.#: "1" => "0"
  global_secondary_index.3667214579.non_key_attributes.0: "nick" => ""
  global_secondary_index.3667214579.projection_type:      "INCLUDE" => ""
  global_secondary_index.3667214579.range_key:            "discriminator" => ""
  global_secondary_index.3667214579.read_capacity:        "0" => "0"
  global_secondary_index.3667214579.write_capacity:       "0" => "0"
  global_secondary_index.3853577238.hash_key:             "" => "nick_lower"
  global_secondary_index.3853577238.name:                 "" => "username"
  global_secondary_index.3853577238.non_key_attributes.#: "0" => "1"
  global_secondary_index.3853577238.non_key_attributes.0: "" => "nick"
  global_secondary_index.3853577238.projection_type:      "" => "INCLUDE"
  global_secondary_index.3853577238.range_key:            "" => "discriminator"
  global_secondary_index.3853577238.read_capacity:        "" => "10"
  global_secondary_index.3853577238.write_capacity:       "" => "10"
  read_capacity:                                          "0" => "10"
  write_capacity:                                         "0" => "10"
Releasing state lock. This may take a few moments...

Error: Error applying plan:

1 error(s) occurred:

* aws_dynamodb_table.users: 1 error(s) occurred:

* aws_dynamodb_table.users: Error updating DynamoDB Table (users) billing mode: ValidationException: One or more parameter values were invalid: ProvisionedThroughput must be specified when BillingMode is PROVISIONED
	status code: 400, request id: E1T2ARB74U6K2FFET5M2U878MJVV4KQNSO5AEMVJF66Q9ASUAAJG

Steps to Reproduce

  1. Have a TF configuration with DynamoDB table applied, including one or more global secondary indexes and PAY_PER_REQUEST as billing_mode
  2. Change billing_mode in aws_dynamodb_table configuration from PAY_PER_REQUEST to PROVISIONED
  3. Run terraform apply
@bflad bflad added bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service. labels Feb 5, 2019
@bflad
Copy link
Contributor

bflad commented Feb 5, 2019

I am double checking this against the currently unreleased fix we merged yesterday (#7363) via additional acceptance testing and will report back shortly.

@bflad
Copy link
Contributor

bflad commented Feb 5, 2019

It looks like GSIs still have a similar issue when going PAY_BY_REQUEST to PROVISIONED:

--- FAIL: TestAccAWSDynamoDbTable_BillingModeUpdateWithGSI (141.28s)
    testing.go:538: Step 1 error: Error applying: 1 error occurred:
        	* aws_dynamodb_table.basic-dynamodb-table: 1 error occurred:
        	* aws_dynamodb_table.basic-dynamodb-table: Error updating DynamoDB Table (TerraformTestTable--1308651219154584046) billing mode: ValidationException: One or more parameter values were invalid: ProvisionedThroughput must be specified for index: TestTableGSI

I will submit a fix later today.

bflad added a commit that referenced this issue Feb 5, 2019
References:

* #7447
* #5471

Changes:

* tests/resource/aws_dynamodb_table: Ensure acceptance testing covers all scenarios involving billing_mode updates with and without global secondary indexes
* tests/resource/aws_dynamodb_table: Update testAccCheckDynamoDbTableHasBilling_Provisioned to handle missing BillingModeSummary
* resource/aws_dynamodb_table: Consolidate UpdateTable calls to prevent error when switching billing_mode from PAY_PER_REQUEST to PROVISIONED when global secondary indexes are present and reduce API traffic
* resource/aws_dynamodb_table: Utilize customizable update timeout in global secondary index update operations
* resource/aws_dynamodb_table: Increase default update timeout from 10 minutes to 60 minutes since even 30 minutes can trigger timeouts with no table data

Previous output from acceptance testing:

```
--- FAIL: TestAccAWSDynamoDbTable_BillingMode_GSI_PayPerRequestToProvisioned (142.20s)
    testing.go:538: Step 1 error: Error applying: 1 error occurred:
        	* aws_dynamodb_table.basic-dynamodb-table: 1 error occurred:
        	* aws_dynamodb_table.basic-dynamodb-table: Error updating DynamoDB Table (TerraformTestTable--132444596974972667) billing mode: ValidationException: One or more parameter values were invalid: ProvisionedThroughput must be specified for index: TestTableGSI
```

Output from acceptance testing:

```
--- PASS: TestAccAWSDynamoDbTable_attributeUpdate (441.98s)
--- PASS: TestAccAWSDynamoDbTable_attributeUpdateValidation (3.00s)
--- PASS: TestAccAWSDynamoDbTable_basic (108.09s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_PayPerRequestToProvisioned (158.73s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_ProvisionedToPayPerRequest (1027.81s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_PayPerRequestToProvisioned (127.56s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_ProvisionedToPayPerRequest (917.15s)
--- PASS: TestAccAWSDynamoDbTable_disappears (17.37s)
--- PASS: TestAccAWSDynamoDbTable_disappears_PayPerRequestWithGSI (181.60s)
--- PASS: TestAccAWSDynamoDbTable_enablePitr (286.99s)
--- PASS: TestAccAWSDynamoDbTable_encryption (222.91s)
--- PASS: TestAccAWSDynamoDbTable_extended (286.03s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateCapacity (178.27s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateNonKeyAttributes (416.57s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateOtherAttributes (661.25s)
--- PASS: TestAccAWSDynamoDbTable_importBasic (137.95s)
--- PASS: TestAccAWSDynamoDbTable_importTags (170.40s)
--- PASS: TestAccAWSDynamoDbTable_importTimeToLive (138.59s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecification (127.55s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecificationValidation (1.05s)
--- PASS: TestAccAWSDynamoDbTable_tags (144.68s)
--- PASS: TestAccAWSDynamoDbTable_ttl (139.86s)
```
@bflad
Copy link
Contributor

bflad commented Feb 5, 2019

Fix submitted: #7453

@bflad bflad added this to the v1.59.0 milestone Feb 11, 2019
@bflad
Copy link
Contributor

bflad commented Feb 11, 2019

The fix for this has been merged and will release with version 1.59.0 of the Terraform AWS Provider, likely middle of this week. 👍

@bflad
Copy link
Contributor

bflad commented Feb 14, 2019

This has been released in version 1.59.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ghost
Copy link

ghost commented Mar 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Mar 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants