Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider/aws: Fixing IAM data source policy generation to prevent spurious diffs #6956

Conversation

vancluever
Copy link
Contributor

Just re-submitting this as a new PR, this was "submitted" in #6881.

As mentioned there, there are a number of things AWS likes to do with a policy (namely resource policies versus IAM ones), that cause spurious diffs in Terraform even though nothing has changed, ie: sorting arrays, inserting an empty Sid attribute, and converting single-element lists to a string value.

Having the data source affords us the opportunity to ensure that these problems do not persist as we can now tweak the policy generation in one place.

Again, an example of the issue in action:

https://gist.github.com/vancluever/526e9c1d8153436e58244847b8fb20f1

The changes in this commit fixes things to allow a policy generated via the aws_iam_policy_document data source to properly pass a TF plan/apply without any changes when nothing has indeed changed.

References for this: I list a few here, however if you search in the issue backlog there are more:

S3: #6952
SNS: #6909
S3, discussing ordering: #5978
ElasticSearch: #5067
My original PR: #4278

@vancluever vancluever force-pushed the paybyphone_iam_datasource_spurious_diff_fix branch from cfe2d43 to 786db06 Compare June 14, 2016 22:12
@vancluever vancluever force-pushed the paybyphone_iam_datasource_spurious_diff_fix branch from 786db06 to 4d5e1a3 Compare July 8, 2016 18:29
@stack72
Copy link
Contributor

stack72 commented Aug 8, 2016

Hi @vancluever

Apologies for not getting back to this sooner, please can you rebase this from master and then we can get it tested and merged?

Thanks

Paul

@stack72 stack72 self-assigned this Aug 8, 2016
@stack72 stack72 added the waiting-response An issue/pull request is waiting for a response from the community label Aug 8, 2016
@vancluever
Copy link
Contributor Author

Woohoo! Will do @stack72!

 * Various string slices are sorted and truncated to strings if they
   only contain one element.
 * Sids are now included if they are empty.

This is to ensure what is sent to AWS matches what comes back, to
prevent recurring diffs even when the policy has changed.
@vancluever vancluever force-pushed the paybyphone_iam_datasource_spurious_diff_fix branch from 4d5e1a3 to 9872026 Compare August 9, 2016 23:56
@vancluever
Copy link
Contributor Author

@stack72 - rebase complete and ready for review!

@stack72
Copy link
Contributor

stack72 commented Aug 10, 2016

Hi @vancluever

This LGTM! Thanks :)

% make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSIAMPolicyDocument'
==> Checking that code complies with gofmt requirements...
/Users/stacko/Code/go/bin/stringer
go generate $(go list ./... | grep -v /terraform/vendor/)
2016/08/10 12:05:04 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSIAMPolicyDocument -timeout 120m
=== RUN   TestAccAWSIAMPolicyDocument
--- PASS: TestAccAWSIAMPolicyDocument (17.99s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    18.022s

P.

@stack72 stack72 merged commit 41c23b2 into hashicorp:master Aug 10, 2016
@vancluever
Copy link
Contributor Author

vancluever commented Aug 10, 2016

Yay! Thanks @stack72!

@dtolnay
Copy link
Contributor

dtolnay commented Aug 20, 2016

@stack72 @apparentlymart This change forces all code that deals with policy documents to copy this switch statement all over the place, as seen multiple times in the PR:

switch v := in.(type) {
case string:
    // handle string
case []string:
    // handle slice
default:
    // panic
 }

I think #7785 is a strictly better fix and I think this one should be reverted.

apparentlymart added a commit that referenced this pull request Aug 20, 2016
Some earlier work in #6956 implemented IAM policy normalization within
the aws_iam_policy_document data source. Some other (unmerged) work in
#7785 implemented normalization across many different IAM policy
attributes in the provider.

The two are in conflict due to a difference in approach. This is an
attempt to reconcile the two by generalizing the normalization already
implemented in #6956 and then applying it to the various places that
were addressed by #7785.
apparentlymart added a commit that referenced this pull request Aug 21, 2016
Earlier work in #6956 caused the IAM policy documents generated by the
aws_iam_policy_document data source to follow the normalization
conventions used by most AWS services.

However, use of this data source is optional and so hand-authored IAM
policy documents used with other resources can still suffer from
normalization issues.

By reorganizing the code a little we can make re-usable normalization and
validation functions, which we will be able to use across many different
resource implementations, pending changes in subsequent commits.

This is a continuation of initial work done by David Tolnay in issue
#7785.

This will cause some minor changes to the result of the
aws_iam_policy_document data source: string sets are now sorted in
forward lexographic order rather than reverse, and "Statements" and
"Sid" will now be omitted when empty, for consistency with all of
the other attributes.
apparentlymart added a commit that referenced this pull request Aug 21, 2016
Earlier work in #6956 caused the IAM policy documents generated by the
aws_iam_policy_document data source to follow the normalization
conventions used by most AWS services.

However, use of this data source is optional and so hand-authored IAM
policy documents used with other resources can still suffer from
normalization issues.

By reorganizing the code a little we can make re-usable normalization and
validation functions, which we will be able to use across many different
resource implementations, pending changes in subsequent commits.

This is a continuation of initial work done by David Tolnay in issue
#7785.

This will cause some minor changes to the result of the
aws_iam_policy_document data source: string sets are now sorted in
forward lexographic order rather than reverse, and "Statements" and
"Sid" will now be omitted when empty, for consistency with all of
the other attributes.
apparentlymart added a commit that referenced this pull request Aug 21, 2016
Earlier work in #6956 caused the IAM policy documents generated by the
aws_iam_policy_document data source to follow the normalization
conventions used by most AWS services.

However, use of this data source is optional and so hand-authored IAM
policy documents used with other resources can still suffer from
normalization issues.

By reorganizing the code a little we can make re-usable normalization and
validation functions, which we will be able to use across many different
resource implementations, pending changes in subsequent commits.

This is a continuation of initial work done by David Tolnay in issue
#7785.

This will cause some minor changes to the result of the
aws_iam_policy_document data source: string sets are now sorted in
forward lexographic order rather than reverse, and "Statements" and
"Sid" will now be omitted when empty, for consistency with all of
the other attributes.
apparentlymart added a commit that referenced this pull request Aug 21, 2016
Earlier work in #6956 caused the IAM policy documents generated by the
aws_iam_policy_document data source to follow the normalization
conventions used by most AWS services.

However, use of this data source is optional and so hand-authored IAM
policy documents used with other resources can still suffer from
normalization issues.

By reorganizing the code a little we can make re-usable normalization and
validation functions, which we will be able to use across many different
resource implementations, pending changes in subsequent commits.

This is a continuation of initial work done by David Tolnay in issue
#7785.

This will cause some minor changes to the result of the
aws_iam_policy_document data source: string sets are now sorted in
forward lexographic order rather than reverse, and "Statements" and
"Sid" will now be omitted when empty, for consistency with all of
the other attributes.
apparentlymart added a commit that referenced this pull request Aug 21, 2016
Earlier work in #6956 caused the IAM policy documents generated by the
aws_iam_policy_document data source to follow the normalization
conventions used by most AWS services.

However, use of this data source is optional and so hand-authored IAM
policy documents used with other resources can still suffer from
normalization issues.

By reorganizing the code a little we can make re-usable normalization and
validation functions, which we will be able to use across many different
resource implementations, pending changes in subsequent commits.

This is a continuation of initial work done by David Tolnay in issue
#7785.

This will cause some minor changes to the result of the
aws_iam_policy_document data source: string sets are now sorted in
forward lexographic order rather than reverse, and "Statements" and
"Sid" will now be omitted when empty, for consistency with all of
the other attributes.
@ghost
Copy link

ghost commented Apr 23, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement provider/aws waiting-response An issue/pull request is waiting for a response from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants