aws_sns_topic_subscription doesn't support unconfirmed endpoints

[lxop]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.11

• provider.aws v1.60.0

Affected Resource(s)

• aws_sns_topic_subscription

Expected Behavior

A subscription is created with an email endpoint (or non-auto-confirming HTTP endpoint). If the subscription is not confirmed, then the endpoint simply doesn't receive any messages.

Actual Behavior

The resource doesn't allow these subscriptions. The resource docs note that

These are unsupported because the endpoint needs to be authorized and does not generate an ARN until the target email address has been validated. This breaks the Terraform model and as a result are not currently supported.

However the SNS API includes a parameter ReturnSubscriptionArn which can be set to true to request that the ARN is returned even if the subscription isn't confirmed. As such, an ARN is generated before the target email address has been validated, and this should then fit into Terraform's model.

[lorengordon]

Yes, please! 😁

[lorengordon]

FWIW, looks like this is a new-ish feature of the service, became available in the SDK in June 2018.

• https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md#release-v11360-2018-06-01

[lxop]

Starting to look at this

[lxop]

It's not looking pretty - creation appears to work fine, but it all falls over in the destruction department. It is not possible (by any means, API or console) to delete a subscription before it is confirmed. This seems like a fundamentally pointless restriction and I have asked AWS why it is the case, but it seems to have been this way for a long time so I'm not expecting any changes any time soon.

It would be possible to simply accept this restriction and report the error to the user during destruction, but this would then mean that an infrastructure deployment would be impossible to destroy if it includes an unconfirmed subscription. Making the failure non-fatal would mean that destroying a subscription does not necessarily destroy that subscription (if it were unconfirmed when the destroy command was performed, then the recipient subsequently confirmed the subscription, then it would be active but untracked by terraform).

So I'm not entirely sure what to do about this now. The HTTP/S subscription presumably suffers from this same problem if I specify auto-accepts on a subscription that doesn't actually auto-accept. I might have to look into how that is handled.

[bryanlalexander]

@lxop any additional updates on this? It seems these are automatically deleted after 72 hours if not yet confirmed. This edge-case may be acceptable or perhaps a hard-fail option to hard-fail if cleanup is not possible.

resource "aws_sns_topic_subscription" "topic-subscription" {
  topic_arn       = "${aws_sns_topic.topic.arn}"
  protocol        = "email"
  endpoint        = "${var.email_address}"
  require_cleanup = true
}

[alex-aire]

@bryanlalexander no update yet - I haven't had much time to work on it. I was/am considering this approach, but I'm not sure about how good it is to introduce a failure-mode to the destroy operation. It would need to also have a way to change the requirement after removing the resource block, otherwise a user could find themself stuck unable to destroy their infrastructure because the resource block that is failing doesn't exist anymore.
In saying that, I suppose the user could simply re-add a resource block with the appropriate name, and update the requirement there.

[bryanlalexander]

@alex-aire I'm happy to invest the time, if we can come to an agreeable solution. @bflad do you have a preference for a path forward on this item?

[jonjohnston]

Any progress on this? I would love to have this included.

[crobo1337]

Not sure if I'm having the same issue, or a slightly different one with the same potential solution. I'm registering a subscription with the pagerduty HTTPS API, the subscription auto confirms, and I can confirm that in the console, but every subsequent build fails with "Error: Protocol http/https is only supported for endpoints which auto confirms!"

[sagagliardo]

I'm also having similar issues to @crobo1337 with similar messages, setting up pager duty subs.

[devonbleak]

@crobo1337 @sagagliardo Re the PagerDuty issues, there are 2 different URLs in the docs: https://events.pagerduty.com/x-ere/{INTEGRATION_KEY} in the text and https://events.pagerduty.com/integration/{INTEGRATION_KEY}/enqueue in the screenshots. Only the second one (/integration/{INTEGRATION_KEY}/enqueue) would work for me. Hope that helps.

[scastria]

What is the status of this? It is very annoying not to be able to subscribe via email within TF. The fix is trivial. How many people need to upvote to get this merged in?

[breathingdust]

Hi all 👋 Just letting you know that this is issue is featured on this quarters roadmap. If a PR exists to close the issue a maintainer will review and either make changes directly, or work with the original author to get the contribution merged. If you have written a PR to resolve the issue please ensure the "Allow edits from maintainers" box is checked. Thanks for your patience and we are looking forward to getting this merged soon!

[scastria]

I thought this was fixed a long time ago. I have been using it probably for a year now. Well, at least the email endpoint. Maybe there are more endpoint types that are still not supported.

[gdavison]

Support was added in v3.30.0 of the AWS Provider, released 25 February 2021. One caveat is #25059, since some API calls still do not return an actual ARN while a subscription is pending.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.