Cross-account SNS subscription fails with provider bug message

[ysgard]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

❯ terraform -v              Terraform v0.12.9
+ provider.aws v2.28.1
+ provider.kubernetes v1.9.0

Affected Resource(s)

• aws_sns_topic_subscription

Terraform Configuration Files

resource "aws_sns_topic_subscription" "share_petition_sqs_target" {
  topic_arn            = data.terraform_remote_state.fe.outputs.sns-topic-share_petition.arn
  protocol             = "sqs"
  endpoint             = aws_sqs_queue.message_bus_events.arn
  raw_message_delivery = true
  provider             = aws.root
}

Which yields the following with a plan:

 # aws_sns_topic_subscription.share_petition_sqs_target will be created
  + resource "aws_sns_topic_subscription" "share_petition_sqs_target" {
      + arn                             = (known after apply)
      + confirmation_timeout_in_minutes = 1
      + endpoint                        = "arn:aws:sqs:us-west-2:8438********:demo-activity_metrics_svc-message_bus_events"
      + endpoint_auto_confirms          = false
      + id                              = (known after apply)
      + protocol                        = "sqs"
      + raw_message_delivery            = true
      + topic_arn                       = "arn:aws:sns:us-west-2:4697********:message-bus-demo-fe-share-petition"
    }

Expected Behavior

A subscription to the given topic for the given endpoint should have occurred.

Actual Behavior

aws_sns_topic_subscription.share_petition_sqs_target: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to aws_sns_topic_subscription.share_petition_sqs_target,
provider "aws" produced an unexpected new value for was present, but now
absent.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Looking, I can see that the subscription was created but not confirmed.

Steps to Reproduce

Create a cross-account subscription request (see configuration above).

1. terraform apply

Important Factoids

There are two separate accounts - a root account in which the topic resides, and a demo account in which the sqs endpoint resides. The provider = aws.root gives admin access to the root account, while access to the subaccounts is provided via role switching, which is possibly the problem here. The providers are set up like so:

provider "aws" {
  alias  = "root"
  region = "us-west-2"
}

provider "aws" {
  profile = "demo"
  region  = "us-west-2"
}

The default 'root' profile has the keys, while the 'demo' profile provides the role definition necessary to access the demo (which has admin privileges in demo). So I can see it using the root account to create the subscription, then failing to obtain the confirmation URL from the SQS queue because it can't figure out how to access the demo account (because of provider=aws.root). Maybe a provider_endpoint that specifies permissions to obtain the endpoint subscription confirmation?

[kelseymok]

We are also having a similar issue with creating an SNS topic subscription in one account (zebra, same as SNS topic) but writing the state to another account (tiger):

resource "aws_sns_topic_subscription" "target" {
  topic_arn = "${aws_sns_topic.this.arn}"
  protocol  = "HTTPS"
  endpoint  = "https://some-awesome-url"
  endpoint_auto_confirms = true
  provider = aws.zebra
}

resource "aws_sns_topic" "this" {
  name = "my-cool-topic"
  provider = aws.zebra
}

provider "aws" {
  version = "~> 2.31"
  region  = var.region
  profile = "zebra"
  alias = "zebra"
}

provider "aws" {
  version = "~> 2.31"
  region  = var.region
  profile = "tiger"
  alias = "tiger"
}

Result

Resource created in Console but Terraform returns the following:

Error: Provider produced inconsistent result after apply

When applying changes to aws_sns_topic_subscription.target, provider
"aws" produced an unexpected new value for was present, but now absent.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Terraform Version

0.12.9

[chrisdpa]

@kelseymok We noticed that the unit tests were only using lowercase for protocol = "HTTPS" so we changed our protocol to protocol = "https" and it worked. See the test in https://github.com/terraform-providers/terraform-provider-aws/blob/98b8b848ca94031b20c3e626c9d40484e3af80de/aws/resource_aws_sns_topic_subscription_test.go#L588

[shinglyu]

Should we also allow uppercase HTTPS, or should we add a validation that check for only lowercase https? If we can have a consensus I might be able to contribute.

[chrisdpa]

My thought would be to enforce uppercase as that's what is used in ELBs. But either way the validator must only allow the correct case to pass, unlike at the moment.

[ghost]

This has been released in version 3.35.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!