[Bug]: updating s3_object with aws provider 4.67.0 fails with putting S3 object ACL: AccessControlListNotSupported: The bucket does not allow ACLs

[thoeng-nuuday]

Terraform Core Version

1.4.6

AWS Provider Version

4.67.0,

Affected Resource(s)

• aws_s3_object
  fails with Error: putting S3 object ACL: AccessControlListNotSupported: The bucket does not allow ACLs

The aws_s3_object resource and aws_s3_bucket

resource "aws_s3_object" "nat_script" {
  bucket         = aws_s3_bucket.files_bucket.id
  key            = "common/opt/nodes/nat.sh"
  content_base64 = base64encode(templatefile("${path.module}/templates/nat.sh", { nat_routes = local.nat_routes }))
  etag           = md5(templatefile("${path.module}/templates/nat.sh", { nat_routes = local.nat_routes }))
}
resource "aws_s3_bucket" "files_bucket" {
  bucket        = "${local.resource_name_prefix}-files-${module.aws_parameters.region}"
  force_destroy = true
  tags          = local.tags
}

resource "aws_s3_bucket_ownership_controls" "files_bucket" {
  bucket = aws_s3_bucket.files_bucket.id
  rule {
    object_ownership = "BucketOwnerEnforced"
  }
}

resource "aws_s3_bucket_public_access_block" "files_bucket_public_access_block" {
  bucket = aws_s3_bucket.files_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = false

  depends_on = [
    aws_s3_bucket.files_bucket,
  ]
}

Expected Behavior

The object is updated/created in the s3 bucket

Actual Behavior

The apply fails

Relevant Error/Panic Output Snippet

Error: putting S3 object ACL: AccessControlListNotSupported: The bucket does not allow ACLs
	status code: 400, request id: XXXX, host id: XXXxx

  with module.cluster.module.nodes["telemetry"].aws_s3_object.nat_script,
  on .terraform/modules/cluster.nodes/files.tf line 25, in resource "aws_s3_object" "nat_script":
  25: resource "aws_s3_object" "nat_script" {

Terraform Configuration Files

resource "aws_s3_object" "nat_script" {
  bucket         = aws_s3_bucket.files_bucket.id
  key            = "a_file"
  content_base64 = base64encode(templatefile("${path.module}/a_file", { }))
  etag           = md5(templatefile("${path.module}/a_file", {  }))
}
resource "aws_s3_bucket" "files_bucket" {
  bucket        = "abucket_345345089904235345"
  force_destroy = true
}

resource "aws_s3_bucket_ownership_controls" "files_bucket" {
  bucket = aws_s3_bucket.files_bucket.id
  rule {
    object_ownership = "BucketOwnerEnforced"
  }
}

resource "aws_s3_bucket_public_access_block" "files_bucket_public_access_block" {
  bucket = aws_s3_bucket.files_bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = false

  depends_on = [
    aws_s3_bucket.files_bucket,
  ]
}

Steps to Reproduce

with aws provider 4.67.0
and a bucket with object_ownership = "BucketOwnerEnforced"
terraform apply

• it then fails

Debug Output

No response

Panic Output

No response

Important Factoids

It works when downgrading the aws provider version to 4.62.0

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[trevorrea]

Hi,

The Terraform code is not formatted so I'm having difficulty reading it but as far as I can see you have object_ownership = "BucketOwnerEnforced" and according to https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html this does not allow ACLs.

On the s3_object resource prior to 5.0.0 the acl argument defaulted to a value of private. See #27197 and compare the docs at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object#acl (5.1.0) and https://registry.terraform.io/providers/hashicorp/aws/4.67.0/docs/resources/s3_bucket_object#acl (4.67.0)

I think the solution to your problem is to set acl = null on the aws_s3_object resource for a short term fix or for a long term fix upgrade to 5.1.0+.

Hope that helps,
Trevor

[thoeng-nuuday]

My short term fix was pinning the aws provider to 4.62.0 instead of using 4.67.0 which broke the existing code.

And when using AWS provider 5.10.0, the following errors occurs - which I believe is covered by #31633

 Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
	status code: 400, request id: XXXXX, host id: XXXXX

  with module.cluster.module.nodes["telemetry"].aws_s3_object.cloudwatch_agent_configuration,
  on .terraform/modules/cluster.nodes/files.tf line 1, in resource "aws_s3_object" "cloudwatch_agent_configuration":
   1: resource "aws_s3_object" "cloudwatch_agent_configuration" {

[trevorrea]

I have seen the error in #31633 but not consistently. Some projects apply fine the first time and other require a second apply. Not ideal but a double apply does work if required.

[ZsoltPath]

My short term fix was pinning the aws provider to 4.62.0 instead of using 4.67.0 which broke the existing code.

And when using AWS provider 5.10.0, the following errors occurs - which I believe is covered by #31633

 Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
	status code: 400, request id: XXXXX, host id: XXXXX

  with module.cluster.module.nodes["telemetry"].aws_s3_object.cloudwatch_agent_configuration,
  on .terraform/modules/cluster.nodes/files.tf line 1, in resource "aws_s3_object" "cloudwatch_agent_configuration":
   1: resource "aws_s3_object" "cloudwatch_agent_configuration" {

Same here. But interestingly at next run there was nothing to update.

[robinpecha]

Running terraform apply twice fixed this issue for me.