Cannot use SQS redrive_allow_policy correctly without creating a cycle

[davecardwell]

The Example Usage for aws_sqs_queue shows a “source queue” with both a redrive_policy and a redrive_allow_policy set, however as per the AWS docs the “byQueue” redrive_allow_policy belongs on the dead letter queue itself:

The redrive allow policy specifies which source queues can access the dead-letter queue. This policy applies to a potential dead-letter queue.

Since you set a redrive_policy on the source queue which points to a dead letter queue, and a redrive_allow_policy on the dead letter queue restricting which source queues can redrive to it, it is not possible to represent this relationship in Terraform without creating a cycle.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

$ terraform -v
Terraform v1.1.2
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v3.71.0

Affected Resource(s)

• aws_sqs_queue

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_sqs_queue" "source_queue" {
    name = "source_queue"

    redrive_policy = jsonencode({
        deadLetterTargetArn = aws_sqs_queue.dead_letter_queue.arn
    })
}

resource "aws_sqs_queue" "dead_letter_queue" {
    name = "dead_letter_queue"

    redrive_allow_policy = jsonencode({
        redrivePermission = "byQueue"
        sourceQueueArns = ["${aws_sqs_queue.source_queue.arn}"]
    })
}

Debug Output

https://gist.github.com/davecardwell/236f84642a90536d442cbfc4434f8da1

Panic Output

N/A

Expected Behavior

Success. It should be possible to configure a RedriveAllowPolicy without creating a circular dependency.

Actual Behavior

Error: Cycle: aws_sqs_queue.dead_letter_queue, aws_sqs_queue.source_queue

Steps to Reproduce

1. terraform validate

Important Factoids

N/A

References

• AWS SQS dead letter queue docs

[sgabenov]

I think it would be good to move DLQ redrive to a separate resource, so it can be dependant on normal queue and DLQ and do not create a cycle loop.
Also, example in docs of terraform resource would be very helpful

[nandorholozsnyak]

Hey, we are also facing this issue, any news on it?

[rbresson-edfx]

Facing the same issue, I had to put redrivePermission = "allowAll" in order to make it work, but not satisfaisant

[hajdukd]

Same issue, tho gonna live with a workaround for now.

[ramooncamacho]

I tested it in version 1.1.7 because I was facing the same problem, and realized that the dead letter sqs queue needs to be implemented first.

You have to create arn string "arn:aws:sqs:region:account_number:mysqsqueue" in the redrive_allow_policy of the sqs dead letter queue. And at the main sqs queue you can point to the dead letter queue using arn attribute. This way the main sqs queue will depend on the sqs dead letter queue (you could also use depends_on, but it is not required in this case).

[taylor-knapp]

It would be nice if you could directly reference the name from the main queues, as TF can know that before applying.

resource "aws_sqs_queue" "main_queue" {
  name                       = "main-queue"
  redrive_policy             = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dead_letter.arn
    maxReceiveCount     = 10
  })
}

resource "aws_sqs_queue" "dead_letter" {
  name                       = "dead-letter-queue"
  redrive_allow_policy       = jsonencode({
    redrivePermission = "byQueue",
    sourceQueueArns   = [
      # Required to either hard code or use a local for the name variable to avoid the cycle error.
      "arn:aws:sqs:${var.region}:${var.account_id}:main-queue",
      # Would be nice to just reference the already known name.
      "arn:aws:sqs:${var.region}:${var.account_id}:${aws_sqs_queue.main_queue.name}"
    ]
  })
}

[dynamicnet]

same as @taylor-knapp , with the last AWS provider to date (4.24.0), this code led to cycle error

resource "aws_sqs_queue" "queue" {
  name                       = "queue"

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq_queue.arn
    maxReceiveCount     = 36
  })
}

resource "aws_sqs_queue" "dlq_queue" {
  name                       = "dlq_queue"

  redrive_allow_policy = jsonencode({
    redrivePermission = "byQueue",
    sourceQueueArns   = [aws_sqs_queue.queue.arn]
  })
}

[techotron]

Same workaround as #22577 (comment) I had to construct the sourceQueueArns manually:

  redrive_allow_policy  = jsonencode({
    redrivePermission = "byQueue",
    sourceQueueArns   = ["arn:aws:sqs:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:my-queue"]
  })  

[github-actions]

This functionality has been released in v4.31.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[stepintooracledba]

Hi,
I've tried the same redrive_policy using Provider(v4.32.0) still get same cycle error.

[timothyclarke]

@ewbankkit Please re-open this one as it is currently unsolved.
Following the example usage currently version 4.36.1 still generates a circular dependency error per the original post at the start of this thread

[chuong-tran]

The solution to prevent circle is to create separate redrive_allow_policy resource as the example in the URL below
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_redrive_allow_policy

[timothyclarke]

@chuong-tran The docs for the aws_sqs_queue resource still need to be updated as they are giving an invalid example

[chuong-tran]

@timothyclarke I see. Thanks!

[hsteidel]

The docs for the aws_sqs_queue resource still need to be updated as they are giving an invalid example https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_redrive_allow_policy

I was facing an issue with the sample provided at
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_redrive_allow_policy

Where I went wrong was by deviating. I had...

resource "aws_sqs_queue" "inbound_message_dlq" {
  name = "inbound-dlq"
}
resource "aws_sqs_queue_redrive_allow_policy" "inbound_message_dlq_redrive" {

And that gave me an AWS 400 Resource Error on the source queue InvalidParameterValue: Value RedrivePolicy

However, once I named both the DLQ and the RedriveAllowPolicy the same, it worked.

resource "aws_sqs_queue" "inbound_message_dlq" {
  name = "inbound-dlq"
}

resource "aws_sqs_queue_redrive_allow_policy" "inbound_message_dlq" {

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.