AWSClient.partition and AWSClient.accountid not set on clients using AdRoll's hologram

[joelthompson]

Summary

Just a quick summary (though it's not in your template). When running with EC2 credentials, GetCredentials is called, which recognizes the fact that I have something that looks like EC2 instance metadata creds available and injects the EC2RoleProvider into the credential chain. Then, GetAccountInfo is called. It sees that the credential provider is the EC2RoleProvider and tries to get the instance profile ARN out of the metadata service by calling metadataClient.IAMInfo() out of the AWS Go SDK. That method merely tries calling the iam/info endpoint, which is an endpoint that Hologram doesn't implement. The result is that metadataClient.IAMInfo returns an error to GetAccountInfo, which then returns the error to Client, which then silently eats the error and skips setting the accountid and partition. This can be seen in the debug logs below where the aws_partition data source is getting set to the empty string and the aws_caller_identity data source raises an error as neither are getting set.

The net result of this is that anybody using AdRoll's hologram to get instance credentials locally won't be able to use the aws_caller_identity or aws_partition data sources. We could probably deal with that, but then #11359 changed up a number of resources to depend on the *AWSClient.partition being set to dynamically generate the ARN. The one we're running in to is sns_topic_policy -- trying to destroy an SNS topic policy goes down a code path which dynamically injects *AWSClient.partition into the ARN regex, and when the partition is empty, that regex match fails, which causes the delete to fail. In other words, for anbody who's using Hologram, they'll be able to create an SNS topic policy with Terraform, but not destroy it.

There's an additional subtlety/regression here. The call to GetAccountInfo is guarded by a check to see if SkipRequestingAccountId is set. If so, then the call to GetAccountInfo is skipped and *AWSClient.partition isn't set. This means that if anybody has SkipRequestingAccountId set, they also will see the same behavior -- they'll be able to create SNS topic policies but not destroy them, whereas prior to #11359 being merged, they would be able to do the same. I'd also guess that the whole point of the call to getAccountIdFromSnsTopicArn in the SNS topic provider is precisely to support the use case where SkipRequestingAccountId is set. I suspect that similar regressions will occur with the other resources touched in #11359, but sns_topic_policy is the only one I've examined in detail.

Not sure what the right path is here -- I feel like if the call to metadataClient.IAMInfo errors out, the code should just fall through to sts:GetCallerIdentity. However, that addresses our use case, but not the regression where setting SkipRequestingAccountId would still cause this to not be set. I believe the partition can be inferred from the region, so maybe try to infer it from the region?

Terraform Version

$ terraform -v
Terraform v0.8.8

Affected Resource(s)

• sns_topic_policy
• data_aws_caller_identity
• data_aws_partition

among others.

Terraform Configuration Files

provider "aws" {
  region = "us-east-1"
}
data "aws_partition" "current" { }
data "aws_caller_identity" "current" {}

Debug Output

$ TF_LOG=TRACE TF_LOG_PATH=tf.log terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.

data.aws_partition.current: Refreshing state...
data.aws_caller_identity.current: Refreshing state...
Error refreshing state: 1 error(s) occurred:

* data.aws_caller_identity.current: No AWS Account ID is available to the provider. Please ensure that
skip_requesting_account_id is not set on the AWS provider.

Can provide a full log if you want, doesn't seem necessary here. The relevant bits:

2017/03/14 16:12:09 [DEBUG] plugin: terraform: aws-provider (internal) 2017/03/14 16:12:09 [DEBUG] Reading Caller Identity.
2017/03/14 16:12:09 [DEBUG] plugin: terraform: aws-provider (internal) 2017/03/14 16:12:09 [DEBUG] No Account ID available, failing
2017/03/14 16:12:09 [DEBUG] plugin: terraform: aws-provider (internal) 2017/03/14 16:12:09 [DEBUG] Reading Partition.
2017/03/14 16:12:09 [DEBUG] plugin: terraform: aws-provider (internal) 2017/03/14 16:12:09 [DEBUG] Setting AWS Partition to .
2017/03/14 16:12:09 [ERROR] root: eval: *terraform.EvalReadDataApply, err: data.aws_caller_identity.current: No AWS Account ID is available to the provider. Please ensure that
skip_requesting_account_id is not set on the AWS provider.
2017/03/14 16:12:09 [ERROR] root: eval: *terraform.EvalSequence, err: data.aws_caller_identity.current: No AWS Account ID is available to the provider. Please ensure that
skip_requesting_account_id is not set on the AWS provider.
2017/03/14 16:12:09 [ERROR] root: eval: *terraform.EvalOpFilter, err: data.aws_caller_identity.current: No AWS Account ID is available to the provider. Please ensure that
skip_requesting_account_id is not set on the AWS provider.
2017/03/14 16:12:09 [ERROR] root: eval: *terraform.EvalSequence, err: data.aws_caller_identity.current: No AWS Account ID is available to the provider. Please ensure that
skip_requesting_account_id is not set on the AWS provider.
2017/03/14 16:12:09 [TRACE] [walkRefresh] Exiting eval tree: data.aws_caller_identity.current

Expected Behavior

In this case, nothing, but the account ID and partition should have been available to me, and TF shouldn't have errored out.

Actual Behavior

TF errored out

Steps to Reproduce

1. Run terraform apply with the above TF on a machine running AdRoll's Hologram

Important Factoids

Using AdRoll's hologram for credentials.

References

• provider/aws: Allow ARN identifier to be set #11359

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.