Skip to content

Commit

Permalink
fix: Add opt-in config to create order independent log filters (#78)
Browse files Browse the repository at this point in the history
## Which problem is this PR solving?
The cloudwatch-logs module accepts a list of log group names. It uses
terraform's count meta argument to create a list of subscription filter
resources, which makes the resources dependent on the order of the log
group names input.

Adding and removing log group names can cause terraform to create and
destroy resources for all log groups after the changed elements in the
list. This is troublesome if you dynamically generate the list of log
groups (e.g. by using `aws_cloudwatch_log_groups` data source), because
your plan can have quite a lot of changes, even if you're only adding or
removing a small number of log groups.

## Short description of the changes
Added an opt-in configuration option that uses `for_each` using the log
filter names to create an order independent list of filters. When the
list changes and the filter name is used, only differences in the
existing and updated list of filters are modified.

## How to verify that this has the expected result
I ran plan using this branch on my terraform project, and it resulted in
the following:

```terraform
  # module.honeycomb-cloudwatch-logs.aws_cloudwatch_log_subscription_filter.this[0] will be destroyed
  - resource "aws_cloudwatch_log_subscription_filter" "this" {
      - destination_arn = "arn:aws:firehose:us-east-1:123456:deliverystream/honeycomb-cloudwatch-logs" -> null
      - distribution    = "ByLogStream" -> null
      - id              = "cwlsf-123456" -> null
      - log_group_name  = "/my-log-group" -> null
      - name            = "/my-log-group-logs_subscription_filter" -> null
      - role_arn        = "arn:aws:iam::123456:role/honeycomb-cloudwatch-logs" -> null
        # (1 unchanged attribute hidden)
    }

  # honeycomb-cloudwatch-logs.aws_cloudwatch_log_subscription_filter.this["/my-log-group"] will be created
  + resource "aws_cloudwatch_log_subscription_filter" "this" {
      + destination_arn = "arn:aws:firehose:us-east-1:123456:deliverystream/honeycomb-cloudwatch-logs"
      + distribution    = "ByLogStream"
      + id              = (known after apply)
      + log_group_name  = "/my-log-group"
      + name            = "/my-log-group-logs_subscription_filter"
      + role_arn        = "arn:aws:iam::123456:role/honeycomb-cloudwatch-logs"
        # (1 unchanged attribute hidden)
    }
```
  • Loading branch information
aburgel authored Dec 17, 2024
1 parent b05a1de commit c9b7c91
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 2 deletions.
13 changes: 12 additions & 1 deletion modules/cloudwatch-logs/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,21 @@ module "kfh" {
}

resource "aws_cloudwatch_log_subscription_filter" "this" {
count = length(var.cloudwatch_log_groups)
count = !var.use_order_independent_filter_resource_naming ? length(var.cloudwatch_log_groups) : 0

name = "${var.cloudwatch_log_groups[count.index]}-logs_subscription_filter"
role_arn = aws_iam_role.this.arn
log_group_name = var.cloudwatch_log_groups[count.index]
filter_pattern = var.log_subscription_filter_pattern
destination_arn = module.kfh.kinesis_firehose_delivery_stream_arn
}

resource "aws_cloudwatch_log_subscription_filter" "filters" {
for_each = var.use_order_independent_filter_resource_naming ? toset(var.cloudwatch_log_groups) : []

name = "${each.key}-logs_subscription_filter"
role_arn = aws_iam_role.this.arn
log_group_name = each.key
filter_pattern = var.log_subscription_filter_pattern
destination_arn = module.kfh.kinesis_firehose_delivery_stream_arn
}
2 changes: 1 addition & 1 deletion modules/cloudwatch-logs/outputs.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
output "cloudwatch_log_subscription_filters" {
value = aws_cloudwatch_log_subscription_filter.this[*].name
value = var.use_order_independent_filter_resource_naming ? [for filter in aws_cloudwatch_log_subscription_filter.filters : filter.name] : aws_cloudwatch_log_subscription_filter.this[*].name
}
6 changes: 6 additions & 0 deletions modules/cloudwatch-logs/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,9 @@ variable "tags" {
default = {}
description = "A map of tags to apply to resources created by this module."
}

variable "use_order_independent_filter_resource_naming" {
type = bool
description = "Use order-independent naming for log group subscription filter resources."
default = false
}

0 comments on commit c9b7c91

Please sign in to comment.