Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/aws_s3_bucket_object: Fix object deletion for non-versioned objects #10352

Merged
merged 2 commits into from
Oct 3, 2019

Conversation

nywilken
Copy link
Contributor

@nywilken nywilken commented Oct 2, 2019

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Closes #10191

Release note for CHANGELOG:

* resource/aws_s3_bucket_object: Fix object deletion for non-versioned objects

Acceptance test before change

--- FAIL: TestAccAWSS3BucketObject_NonVersioned (32.38s)
   testing.go:630: Error destroying resource! WARNING: Dangling
   resources may exist. The full state and error is shown below.

    Error: errors during apply: Failed listing S3 object versions:
AccessDenied: Access Denied

Acceptance test after change

--- PASS: TestAccAWSS3BucketObject_NonVersioned (32.16s)

The test case for S3 is only reproducible for IAM users that have restricted permissions on bucket objects. This change introduces a simple test Provider configuration that can be used for assuming a restricted policy at runtime. This change introduces a testAccAssumeRoleARNPreCheck function that can be used for validating that a TF_ACC_ASSUME_ROLE_ARN environment variable is set for any tests using the testAccProviderConfigAssumeRolePolicy provider configuration block.

testAccAssumeRoleARNPreCheck unset

> make testacc TEST=./aws
TESTARGS='-run=TestAccAWSS3BucketObject_NonVersioned'
--- SKIP: TestAccAWSS3BucketObject_NonVersioned (1.56s)
   provider_test.go:756: skipping tests; TF_ACC_ASSUME_ROLE_ARN must be

testAccAssumeRoleARNPreCheck set

TESTARGS='-run=TestAccAWSS3BucketObject_NonVersioned'
--- PASS: TestAccAWSS3BucketObject_NonVersioned (32.47s)

Acceptance tests after change

--- SKIP: TestAccAWSS3BucketObject_NonVersioned (2.03s)
   provider_test.go:756: skipping tests; TF_ACC_ASSUME_ROLE_ARN must be set
=== CONT  TestAccAWSS3BucketObject_tagsLeadingSlash
--- PASS: TestAccAWSS3BucketObject_noNameNoKey (3.53s)
--- PASS: TestAccAWSS3BucketObject_empty (32.55s)
--- PASS: TestAccAWSS3BucketObject_source (36.75s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (37.46s)
--- PASS: TestAccAWSS3BucketObject_content (37.75s)
--- PASS: TestAccAWSS3BucketObject_etagEncryption (37.78s)
--- PASS: TestAccAWSS3BucketObject_sse (37.78s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (37.90s)
--- PASS: TestAccAWSS3BucketObject_kms (61.81s)
--- PASS: TestAccAWSS3BucketObject_updates (61.91s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (62.52s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (62.80s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (63.18s)
--- PASS: TestAccAWSS3BucketObject_metadata (82.99s)
--- PASS: TestAccAWSS3BucketObject_acl (86.10s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (86.15s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (87.09s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (105.55s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (105.43s)
--- PASS: TestAccAWSS3BucketObject_tags (107.64s)
--- PASS: TestAccAWSS3BucketObject_storageClass (127.35s)

@nywilken nywilken requested a review from a team October 2, 2019 23:00
@ghost ghost added size/M Managed by automation to categorize the size of a PR. service/s3 Issues and PRs that pertain to the s3 service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 2, 2019
@nywilken nywilken force-pushed the b-aws_s3_bucket_object-version-id-regression branch from 8d5ff9e to e649169 Compare October 2, 2019 23:01
…objects

Acceptance test before change
```
--- FAIL: TestAccAWSS3BucketObject_NonVersioned (32.38s)
    testing.go:630: Error destroying resource! WARNING: Dangling
    resources may exist. The full state and error is shown below.

    Error: errors during apply: Failed listing S3 object versions: AccessDenied: Access Denied
```

Acceptance test after change
```
--- PASS: TestAccAWSS3BucketObject_NonVersioned (32.16s)
```
There is a test case for S3 that is only reproducible for IAM users that have restricted permissions on bucket objects.
This change introduces a simple test Provider configuration that can be used for assuming a restricted policy at runtime.
This change introduces a testAccAssumeRoleARNPreCheck function that can be used for validating that a TF_ACC_ASSUME_ROLE_ARN
environment variable is set for any tests using the testAccProviderConfigAssumeRolePolicy provider configuration block.

testAccAssumeRoleARNPreCheck unset
```
> make testacc TEST=./aws TESTARGS='-run=TestAccAWSS3BucketObject_NonVersioned'
--- SKIP: TestAccAWSS3BucketObject_NonVersioned (1.56s)
    provider_test.go:756: skipping tests; TF_ACC_ASSUME_ROLE_ARN must be
```

testAccAssumeRoleARNPreCheck set
```
TF_ACC_ASSUME_ROLE_ARN=...  make testacc TEST=./aws TESTARGS='-run=TestAccAWSS3BucketObject_NonVersioned'
--- PASS: TestAccAWSS3BucketObject_NonVersioned (32.47s)
```

Acceptance tests after change
```
--- SKIP: TestAccAWSS3BucketObject_NonVersioned (2.03s)
    provider_test.go:756: skipping tests; TF_ACC_ASSUME_ROLE_ARN must be set
=== CONT  TestAccAWSS3BucketObject_tagsLeadingSlash
--- PASS: TestAccAWSS3BucketObject_noNameNoKey (3.53s)
--- PASS: TestAccAWSS3BucketObject_empty (32.55s)
--- PASS: TestAccAWSS3BucketObject_source (36.75s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (37.46s)
--- PASS: TestAccAWSS3BucketObject_content (37.75s)
--- PASS: TestAccAWSS3BucketObject_etagEncryption (37.78s)
--- PASS: TestAccAWSS3BucketObject_sse (37.78s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (37.90s)
--- PASS: TestAccAWSS3BucketObject_kms (61.81s)
--- PASS: TestAccAWSS3BucketObject_updates (61.91s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (62.52s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (62.80s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (63.18s)
--- PASS: TestAccAWSS3BucketObject_metadata (82.99s)
--- PASS: TestAccAWSS3BucketObject_acl (86.10s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (86.15s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (87.09s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (105.55s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (105.43s)
--- PASS: TestAccAWSS3BucketObject_tags (107.64s)
--- PASS: TestAccAWSS3BucketObject_storageClass (127.35s)
```
@nywilken nywilken force-pushed the b-aws_s3_bucket_object-version-id-regression branch from e649169 to 030d001 Compare October 2, 2019 23:01
@nywilken nywilken added bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. labels Oct 2, 2019
@nywilken nywilken requested a review from bflad October 2, 2019 23:35
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a non-blocking nit and documentation request below. 😄 🚀

Output from acceptance testing:

--- PASS: TestAccAWSS3BucketObject_noNameNoKey (6.04s)
--- PASS: TestAccAWSS3BucketObject_empty (15.99s)
--- PASS: TestAccAWSS3BucketObject_etagEncryption (20.96s)
--- PASS: TestAccAWSS3BucketObject_contentBase64 (21.43s)
--- PASS: TestAccAWSS3BucketObject_source (21.75s)
--- PASS: TestAccAWSS3BucketObject_content (21.73s)
--- PASS: TestAccAWSS3BucketObject_sse (21.93s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (22.08s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithOn (28.94s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (30.45s)
--- PASS: TestAccAWSS3BucketObject_updateSameFile (30.61s)
--- PASS: TestAccAWSS3BucketObject_updates (32.19s)
--- PASS: TestAccAWSS3BucketObject_metadata (34.61s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockLegalHoldStartWithNone (36.61s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithNone (34.98s)
--- PASS: TestAccAWSS3BucketObject_acl (37.31s)
--- PASS: TestAccAWSS3BucketObject_kms (40.56s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (42.77s)
--- PASS: TestAccAWSS3BucketObject_tags (43.37s)
--- PASS: TestAccAWSS3BucketObject_ObjectLockRetentionStartWithSet (37.47s)
--- PASS: TestAccAWSS3BucketObject_storageClass (46.97s)

func testAccAssumeRoleARNPreCheck(t *testing.T) {
v := os.Getenv("TF_ACC_ASSUME_ROLE_ARN")
if v == "" {
t.Skip("skipping tests; TF_ACC_ASSUME_ROLE_ARN must be set")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: In the future, we may want to augment this skip message to include more details about what this is and how to create it, e.g.

Suggested change
t.Skip("skipping tests; TF_ACC_ASSUME_ROLE_ARN must be set")
t.Skip("skipping tests; TF_ACC_ASSUME_ROLE_ARN environment variable must be set.\n" +
"This acceptance test expects the ARN of an IAM Role with full permissions " +
"in the testing AWS account that can be assumed with a restrictive session " +
"policy to verify resource functionality with those restrictive permissions.")

}
}

func testAccProviderConfigAssumeRolePolicy(policy string) string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really neat setup. Do you have time to document this awesome new way of "acceptance testing resources with restricted IAM permissions" in the contributing guide? I'm sure there are other resources that could benefit from something like this. ❤️

@bflad bflad added this to the v2.31.0 milestone Oct 3, 2019
@bflad bflad merged commit 6f0da98 into master Oct 3, 2019
@bflad bflad deleted the b-aws_s3_bucket_object-version-id-regression branch October 3, 2019 01:58
bflad added a commit that referenced this pull request Oct 3, 2019
@ghost
Copy link

ghost commented Oct 3, 2019

This has been released in version 2.31.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Nov 2, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Nov 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. service/s3 Issues and PRs that pertain to the s3 service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws_s3_bucket_object update fails with Access Denied
2 participants