Whitelist "internal" annotations/labels that the user is managing in the resource

[mbarrien]

This PR is an alternative to #244 that closes #199 and closes #200. It is mostly a cherry picking of the strategy used in the fork of @sl1pm4t with a few small updates from me to catch up with the recent changes. I'm hoping it is okay to submit his code, and given that it follows the same licensing, should be mergeable.

At a high level, instead of having the user actively manage a whitelist as #244 does, this instead "automatically whitelists" any annotations/labels that the user is actively stating that they want to manage by its presence in the annotations or labels metadata. To do this, flattenMetadata is passed the resource spec, from which it extracts the list of annotations that the user has specified.

Any annotations/labels added by the Kubernetes system itself with the .kubernetes.io domain are still ignored the same way as before and will not come up as false changes in the plan. If the user does try to modify the annotation/label that is actually internal and system managed, they will get the issues of either continually being asked to update the resource or Kubernetes rejecting the change; the user will learn quickly not to try to manage the annotation. With this, the user no longer has to manage the whitelist at the provider level, reducing confusion.

[rpatrick00]

This is a much better solution than PR #244.

[dh-harald]

I agree, this solution is much better

[jhoblitt]

@alexsomesan Any thoughts on this approach? I suspect there is a very captive audience for this feature.

[ereOn]

I can confirm it works perfectly well and unblocked an otherwise unresolvable situation. I can't comment on the implementation level, but at the user level, this has the behavior I expected to begin with.

[rpatrick00]

#Hashicorp has a conflict of interest in their Kubernetes competitive product and basically don’t seem to care if the Kubernetes provider works or not. They should let the community maintain this provider.

[legal90]

@alexsomesan Could you please take a look at this PR?

[alexsomesan]

I agree this is an nicer approach. I'm looking over and testing this over the next day or two.

[pdecat]

I do like that way more than the provider level manual whitelisting option I proposed in 2017 😄

[sl1pm4t]

Thanks for pushing this across to the official provider @mbarrien !

[alexsomesan]

I've tested this change manually with a couple of scenarios under EKS, AKS and GKE. Looks great.

I added some mentions of the new behaviour around the docs.
I've also added another acceptance test that uses internal annotations (for load balancers).
Finally, I rebased this on current master to include the Ingress resource and enable it's annotations test.

Running the whole thing in CI now.

[alexsomesan]

CI is green.
All looks good. Merging.

[tdmalone]

This has now been released in 1.7.0: https://github.com/terraform-providers/terraform-provider-kubernetes/blob/master/CHANGELOG.md#170-may-22-2019
🎉 🚀

[pdecat]

this instead "automatically whitelists" any annotations/labels that the user is actively stating that they want to manage by its presence in the annotations or labels metadata.

FTR, the behavior with this implementation when importing existing ingresses is that:

• configured annotations are not set on the state,
• a following plan will reveal a diff for those missing annotations despite the fact they are actually set server side,
• applying will update the ingress' annotations and remove the non-terraform managed annotations (e.g. on GCP: ingress.kubernetes.io/backends, ingress.kubernetes.io/forwarding-rule, etc.)
• after a few seconds, the ingress controller re-applies the annotations it manages by itself.
• no diff remains during following plans.