Changes to S3 Bucket Lifecycle policies don't converge (>v2.64.0)

[dicconb]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

terraform -v
Terraform v0.12.28
+ provider.aws v2.70.0

Affected Resource(s)

• aws_s3_bucket

Terraform Configuration Files

provider "aws" {
  region = "us-east-1"
}

resource aws_s3_bucket "test" {
  lifecycle_rule {
    id                                     = "Test"
    enabled                                = true
    prefix                                 = ""
    abort_incomplete_multipart_upload_days = 7
  }
}

Debug Output

https://gist.github.com/dicconb/53fd9638f4c68923a2cb37b185187bd6

Expected Behavior

• First apply should create the lifecycle policy
• Second apply (and subsequent applies) should show no changes

Actual Behavior

• First apply creates the lifecycle policy, with an extraneous expiration block that isn't visible in the plan
  Plan:

# aws_s3_bucket.test will be created
  + resource "aws_s3_bucket" "test" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = (known after apply)
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + lifecycle_rule {
          + abort_incomplete_multipart_upload_days = 7
          + enabled                                = true
          + id                                     = "Test"
        }

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

Lifecycle rule:

aws s3api get-bucket-lifecycle --bucket terraform-20200721180953452800000001
{
    "Rules": [
        {
            "Expiration": {
                "ExpiredObjectDeleteMarker": false
            },
            "ID": "Test",
            "Status": "Enabled",
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
            }
        }
    ]
}

• Second apply attempts to remove the extraneous expiration block, and appears to succeed

      ~ lifecycle_rule {
            abort_incomplete_multipart_upload_days = 7
            enabled                                = true
            id                                     = "Test"
            tags                                   = {}

          - expiration {
              - days                         = 0 -> null
              - expired_object_delete_marker = false -> null
            }
        }
[...]
aws_s3_bucket.test: Modifying... [id=terraform-20200721165752214400000001]
aws_s3_bucket.test: Still modifying... [id=terraform-20200721165752214400000001, 10s elapsed]
aws_s3_bucket.test: Modifications complete after 19s [id=terraform-20200721165752214400000001]

However the lifecycle rule is still present

• Third apply is identical to second apply

Steps to Reproduce

1. terraform apply
2. terraform apply
3. terraform apply

Important Factoids

• This affects S3 lifecycle rules that only have an AbortIncompleteMultipartUpload action (ie they don't have Expiration or Transition actions)
  https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html#mpu-abort-incomplete-mpu-lifecycle-config
• The bug occurs if any lifecycle policy rules on the bucket are updated (not just the one with the AbortIncompleteMultipartUpload action)

Workaround:

The issue can be worked around by pinning to provider version v2.64.0 or lower for the terraform apply which updates the lifecycle policy, or a subsequent terraform apply. The latest provider version can be used for any terraform runs that don't attempt to update S3 Lifecycle rules.

References

• service/s3: update to using typelist #13425 - Problem appears to have been introduced here
  • https://github.com/terraform-providers/terraform-provider-aws/blame/d9f833b616af0ea41681e51a25e4d13f9af537ad/aws/resource_aws_s3_bucket.go#L2259-L2265

[dicconb]

Acceptance test to catch this issue: #14283

[dicconb]

Proposed fix: #14284

[anGie44]

Hi @dicconb , thank you for raising this issue and apologies it did not get linked sooner to the fix in v3.10.0 of the AWS provider. I'm going to close this issue as the fix (#15263) was released (and will follow-up on #14284).

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.