aws-auth configmap changes after adding a node pool removing existing roles

[gabricc]

Description

Adding a new nodeGroup to an existing EKS cluster changes the aws-auth in an undesired way, it removed existing roles for the configmap:

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 19.21.0

• Terraform version: 1.6.6

• Provider version(s):

terraform providers -version
Terraform v1.6.6
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v5.32.1
+ provider registry.terraform.io/hashicorp/cloudinit v2.3.3
+ provider registry.terraform.io/hashicorp/kubernetes v2.15.0
+ provider registry.terraform.io/hashicorp/random v3.6.0
+ provider registry.terraform.io/hashicorp/time v0.10.0
+ provider registry.terraform.io/hashicorp/tls v4.0.5

Reproduction Code

data "aws_subnets" "subnets" {
  filter {
    name   = "vpc-id"
    values = [module.network.vpc_id]
  }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.21.0"

  cluster_name    = "slang-eks-${local.environment}"
  cluster_version = "1.28"

  cluster_endpoint_public_access = true

  cluster_enabled_log_types              = []
  cloudwatch_log_group_retention_in_days = 1

  cluster_addons = {
    coredns = {
      preserve = true

      timeouts = {
        create = "25m"
        delete = "10m"
      }
    }
    kube-proxy = {
    }
    vpc-cni = {
      preserve = true
    }
  }

  vpc_id     = module.network.vpc_id
  subnet_ids = data.aws_subnets.subnets.ids

  node_security_group_enable_recommended_rules = false

  # Enable node to node communication
  node_security_group_additional_rules = {
    ingress_self_all = {
      description = "Node to node all ports/protocols"
      protocol    = "-1"
      from_port   = 0
      to_port     = 0
      type        = "ingress"
      self        = true
    }
    egress_all = {
      description      = "Node all egress"
      protocol         = "-1"
      from_port        = 0
      to_port          = 0
      type             = "egress"
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
    }
    # Control plane to nodes
    ingress_cluster_to_node_all_traffic = {
      description                   = "Cluster API to Nodegroup all traffic"
      protocol                      = "-1"
      from_port                     = 0
      to_port                       = 0
      type                          = "ingress"
      source_cluster_security_group = true
    }
  }

  # EKS Managed Node Group(s)
  eks_managed_node_group_defaults = {
    disk_size      = 50
    instance_types = ["m6i.large", "m5.large", "m5n.large", "m5zn.large"]
  }

  eks_managed_node_groups = {

    spot_ng_1 = {
      create_security_group = false

      subnet_ids = module.network.public_subnets

      min_size                = 0
      max_size                = 20
      desired_size            = 0

      instance_types = ["t3.xlarge", "t2.xlarge"]
      capacity_type  = "SPOT"

      enable_monitoring = false

      taints = {
        dedicated = {
          key    = "node-type"
          value  = "spot"
          effect = "NO_SCHEDULE"
        }
      }
    },

    spot_ng_2 = {
      create_security_group = false

      subnet_ids = module.network.public_subnets

      min_size                = 0
      max_size                = 20
      desired_size            = 1

      instance_types = ["t3.xlarge", "t3.large", "t3a.xlarge"]
      capacity_type  = "SPOT"

      enable_monitoring = false

      taints = {
        dedicated = {
          key    = "node-type"
          value  = "spot"
          effect = "NO_SCHEDULE"
        }
      }
    }
  }

  # Fargate Profile(s)
  fargate_profiles = {
    default = {
      name = "default"
      selectors = [
        {
          namespace = "default"
        }
      ]
      # Using specific subnets instead of the subnets supplied for the cluster itself
      subnet_ids = module.network.private_subnets
    }
  }

  # aws-auth configmap
  manage_aws_auth_configmap = true

  aws_auth_roles = [
    {
      rolearn  = "arn:aws:iam::****:role/eks-engineers-role"
      username = "eks-engineers-role"
      groups   = ["readonly"]
    },
  ]

  aws_auth_users = [
    {
      userarn  = "arn:aws:iam::***:user/g.carvalho"
      username = "user"
      groups   = ["system:masters", "system:masters", "system:bootstrappers", "system:nodes", "eks-console-dashboard-full-access-group"]
    },
  ]

  aws_auth_accounts = [
    "***"
  ]

  tags = merge(
    local.base_common_tags,
    local.tags,
    {
      component = "slang-${local.environment}-eks"
    }
  )
}

Steps to reproduce the behavior:
Create a new nodeGroup inside the eks module, run terraform plan.
I'm using terraform workspaces.

Expected behavior

A new nodeGroup should be created and the aws-auth should be changed to add the required roles for the new nodeGroup.

Actual behavior

aws-auth is changed in an undesired way, it removes existing roles from the configmap.

Terminal Output Screenshot(s)

Attached above ☝️

[gaspo53]

This exact thing is happening here all the time, if any day I execute a plan, I'll get tons of resources being recreated/updated.

[bryantbiggs]

FYI: It will be nearly impossible to help troubleshoot without a reproduction

I'll get tons of resources being recreated/updated."

Sounds like you are using an explicit depends_on somewhere which is a no no

[gaspo53]

Hey, Bryan. No, I'm not, all policies are being recreated, etc. This is my spec:

module "eks_cluster" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19"

  cluster_name             = local.eks_cluster_name
  cluster_version          = var.cluster_version
  cluster_ip_family        = "ipv4"
  iam_role_use_name_prefix = true
  vpc_id                   = module.vpc.vpc_id
  subnet_ids               = module.vpc.private_subnets

  enable_irsa               = true
  create_aws_auth_configmap = false
  manage_aws_auth_configmap = true

  cluster_endpoint_private_access = true
  cluster_endpoint_public_access  = true
  create_cni_ipv6_iam_policy      = false

  create_kms_key            = false
  cluster_encryption_config = {}

  node_security_group_enable_recommended_rules = true

  eks_managed_node_group_defaults = {
    ami_type       = "AL2_x86_64"
    instance_types = ["m6i.large", "m5.large", "m5n.large", "m5zn.large"]
    disk_size      = 50
    # We are using the IRSA created below for permissions
    # However, we have to deploy with the policy attached FIRST (when creating a fresh cluster)
    # and then turn this off after the cluster/node group is created. Without this initial policy,
    # the VPC CNI fails to assign IPs and nodes cannot join the cluster
    # See https://github.com/aws/containers-roadmap/issues/1666 for more context
    iam_role_attach_cni_policy = true
  }

  eks_managed_node_groups = {
    default_node_group = {
      name                   = "eks-staging-default"
      desired_size           = 1
      min_size               = 1
      max_size               = 10
      instance_types         = ["m5a.large"]
      # capacity_type          = "ON_DEMAND"
      capacity_type  = "SPOT"
      create_launch_template = true
      disk_size              = 50
      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size           = 50
            volume_type           = "gp3"
            iops                  = 3000
            throughput            = 125
            encrypted             = true
            delete_on_termination = true
          }
        }
      }
      labels = {
        GithubRepo = "infra"
      }
      tags = {
        Name = "eks-staging-default-node"
      }
      taints = []
      update_config = {
        max_unavailable_percentage = 50 # or set `max_unavailable`
      }

      iam_role_additional_policies = {
        AmazonSSMManagedInstanceCore       = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
        AmazonEC2ContainerRegistryReadOnly = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
      }
    }

  }

  node_security_group_additional_rules = {
    efs_ingress = {
      description = "Allow EFS access"
      protocol    = "tcp"
      from_port   = 2049
      to_port     = 2049
      type        = "ingress"
      cidr_blocks = ["172.16.1.0/24", "172.16.2.0/24", "172.16.3.0/24"]
    }

    vault_ingress = {
      description = "Allow Vault access"
      protocol    = "tcp"
      from_port   = 8080
      to_port     = 8080
      type        = "ingress"
      cidr_blocks = ["172.16.1.0/24", "172.16.2.0/24", "172.16.3.0/24"]
    }
  }

  aws_auth_users = concat(
    [for user in data.aws_iam_group.infrastructure.users : {
      userarn  = user.arn
      username = user.user_name
      groups   = ["system:masters"]
      }
    ],
    [for user in data.aws_iam_group.developers.users : {
      userarn  = user.arn
      username = user.user_name
      groups   = ["system:bootstrappers", "system:nodes", "developer-access-group"]
      }
    ],
    [for user in local.iam_service_accounts : {
      userarn  = user.arn
      username = user.user_name
      groups   = ["system:bootstrappers", "system:nodes", "eks-console-dashboard-full-access-group"]
      }
    ]
  )

  aws_auth_roles = concat(
    [for role in local.sso_admin_roles : {
      rolearn  = role.arn
      username = role.id
      groups   = ["system:masters"]
      }
    ],
    [for role in local.sso_dev_roles : {
      rolearn  = role.arn
      username = role.id
      groups   = ["system:bootstrappers", "system:nodes", "developer-access-group"]
      }
    ]
  )

  cluster_addons = {
    coredns = {
      preserve    = true
      most_recent = true

      timeouts = {
        create = "25m"
        delete = "10m"
      }
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
  }

  tags = {
    environment = local.environment
  }
}

Do you see anything wrong?

[bryantbiggs]

I don't see anything that stands out, but I can't deploy what you have provided and I don't know what you are seeing in a the plan diff so its hard to say 🤷🏽‍♂️

[gaspo53]

I'll give you an example, later today or tomorrow.

[gabricc]

hello @bryantbiggs ! Does the screenshot attached + the code I provided when opened the issue helps in the troubleshooting or you need more info?

[bryantbiggs]

no because the code is not deployable and the screenshot doesn't show the full diff

adding or removing a nodegroup doesn't remove the configmap, nor does it erase the contents of the configmap. what I suspect you are seeing is just merely the computed value diff that isn't fully rendered since the values aren't known until the change has been applied - as can be seen by the -> known after apply note

[gaspo53]

@bryantbiggs here you go: I've added a clusterrole, and then an eks-addon has been updated. Look at the plan:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # module.eks_cluster.aws_eks_addon.this["coredns"] will be updated in-place
  ~ resource "aws_eks_addon" "this" {
      ~ addon_version     = "v1.10.1-eksbuild.6" -> "v1.10.1-eksbuild.7"
        id                = "songfinch-production-us-east-1:coredns"
        tags              = {
            "environment" = "production"
        }
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.eks_cluster_rbacs[0].data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].kubernetes_manifest.this["namespace-manager"] will be created
  + resource "kubernetes_manifest" "this" {
      + manifest = {
          + apiVersion = "rbac.authorization.k8s.io/v1"
          + kind       = "ClusterRole"
          + metadata   = {
              + name = "namespace-manager-clusterrole"
            }
          + rules      = [
              + {
                  + apiGroups = [
                      + "",
                    ]
                  + resources = [
                      + "serviceaccounts",
                    ]
                  + verbs     = [
                      + "get",
                      + "list",
                      + "patch",
                      + "create",
                      + "delete",
                    ]
                },
              + {
                  + apiGroups = [
                      + "",
                    ]
                  + resources = [
                      + "namespaces",
                    ]
                  + verbs     = [
                      + "get",
                      + "patch",
                    ]
                },
            ]
        }
      + object   = {
          + aggregationRule = {
              + clusterRoleSelectors = (known after apply)
            }
          + apiVersion      = "rbac.authorization.k8s.io/v1"
          + kind            = "ClusterRole"
          + metadata        = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "namespace-manager-clusterrole"
              + namespace                  = (known after apply)
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + rules           = [
              + {
                  + apiGroups       = [
                      + "",
                    ]
                  + nonResourceURLs = (known after apply)
                  + resourceNames   = (known after apply)
                  + resources       = [
                      + "serviceaccounts",
                    ]
                  + verbs           = [
                      + "get",
                      + "list",
                      + "patch",
                      + "create",
                      + "delete",
                    ]
                },
              + {
                  + apiGroups       = [
                      + "",
                    ]
                  + nonResourceURLs = (known after apply)
                  + resourceNames   = (known after apply)
                  + resources       = [
                      + "namespaces",
                    ]
                  + verbs           = [
                      + "get",
                      + "patch",
                    ]
                },
            ]
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_manifest.this_binding["namespace-manager"] will be created
  + resource "kubernetes_manifest" "this_binding" {
      + manifest = {
          + apiVersion = "rbac.authorization.k8s.io/v1"
          + kind       = "ClusterRoleBinding"
          + metadata   = {
              + name = "namespace-manager-binding"
            }
          + roleRef    = {
              + apiGroup = "rbac.authorization.k8s.io"
              + kind     = "ClusterRole"
              + name     = "namespace-manager-clusterrole"
            }
          + subjects   = [
              + {
                  + kind      = "User"
                  + name      = "namespace-manager"
                  + namespace = "devops"
                },
            ]
        }
      + object   = {
          + apiVersion = "rbac.authorization.k8s.io/v1"
          + kind       = "ClusterRoleBinding"
          + metadata   = {
              + annotations                = (known after apply)
              + creationTimestamp          = (known after apply)
              + deletionGracePeriodSeconds = (known after apply)
              + deletionTimestamp          = (known after apply)
              + finalizers                 = (known after apply)
              + generateName               = (known after apply)
              + generation                 = (known after apply)
              + labels                     = (known after apply)
              + managedFields              = (known after apply)
              + name                       = "namespace-manager-binding"
              + namespace                  = (known after apply)
              + ownerReferences            = (known after apply)
              + resourceVersion            = (known after apply)
              + selfLink                   = (known after apply)
              + uid                        = (known after apply)
            }
          + roleRef    = {
              + apiGroup = "rbac.authorization.k8s.io"
              + kind     = "ClusterRole"
              + name     = "namespace-manager-clusterrole"
            }
          + subjects   = [
              + {
                  + apiGroup  = (known after apply)
                  + kind      = "User"
                  + name      = "namespace-manager"
                  + namespace = "devops"
                },
            ]
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_service_account.aws_cluster_autoscaler_role[0] will be updated in-place
  ~ resource "kubernetes_service_account" "aws_cluster_autoscaler_role" {
        id                              = "kube-system/cluster-autoscaler"
        # (1 unchanged attribute hidden)

      ~ metadata {
          ~ annotations      = {
              - "eks.amazonaws.com/role-arn" = "arn:aws:iam::158566655292:role/AmazonEKSClusterAutoscaleRole-us-east-1"
            } -> (known after apply)
            name             = "cluster-autoscaler"
            # (5 unchanged attributes hidden)
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_service_account.aws_external_dns_role[0] will be updated in-place
  ~ resource "kubernetes_service_account" "aws_external_dns_role" {
        id                              = "external-dns/aws-external-dns"
        # (1 unchanged attribute hidden)

      ~ metadata {
          ~ annotations      = {
              - "eks.amazonaws.com/role-arn" = "arn:aws:iam::158566655292:role/AmazonEKSExternalDnsRole-us-east-1"
            } -> (known after apply)
            name             = "aws-external-dns"
            # (5 unchanged attributes hidden)
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_service_account.aws_load_balancer_controller will be updated in-place
  ~ resource "kubernetes_service_account" "aws_load_balancer_controller" {
        id                              = "kube-system/aws-load-balancer-controller"
        # (1 unchanged attribute hidden)

      ~ metadata {
          ~ annotations      = {
              - "eks.amazonaws.com/role-arn" = "arn:aws:iam::158566655292:role/AmazonEKSLoadBalancerControllerRole-us-east-1"
            } -> (known after apply)
            name             = "aws-load-balancer-controller"
            # (5 unchanged attributes hidden)
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_service_account.ebs_csi_controller_sa will be updated in-place
  ~ resource "kubernetes_service_account" "ebs_csi_controller_sa" {
        id                              = "kube-system/ebs-csi-controller-sa"
        # (1 unchanged attribute hidden)

      ~ metadata {
          ~ annotations      = {
              - "eks.amazonaws.com/role-arn" = "arn:aws:iam::158566655292:role/eks-ebs-us-east-1"
            } -> (known after apply)
            name             = "ebs-csi-controller-sa"
            # (5 unchanged attributes hidden)
        }
    }

  # module.eks_cluster_rbacs[0].kubernetes_service_account.efs_csi_controller_sa will be updated in-place
  ~ resource "kubernetes_service_account" "efs_csi_controller_sa" {
        id                              = "kube-system/efs-csi-controller-sa"
        # (1 unchanged attribute hidden)

      ~ metadata {
          ~ annotations      = {
              - "eks.amazonaws.com/role-arn" = "arn:aws:iam::158566655292:role/eks-efs-us-east-1"
            } -> (known after apply)
            name             = "efs-csi-controller-sa"
            # (5 unchanged attributes hidden)
        }
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.data.aws_iam_policy_document.cluster_autoscaler[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "cluster_autoscaler" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "autoscaling:DescribeAutoScalingGroups",
              + "autoscaling:DescribeAutoScalingInstances",
              + "autoscaling:DescribeLaunchConfigurations",
              + "autoscaling:DescribeScalingActivities",
              + "autoscaling:DescribeTags",
              + "ec2:DescribeImages",
              + "ec2:DescribeInstanceTypes",
              + "ec2:DescribeLaunchTemplateVersions",
              + "ec2:GetInstanceTypesFromInstanceRequirements",
              + "eks:DescribeNodegroup",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "autoscaling:SetDesiredCapacity",
              + "autoscaling:TerminateInstanceInAutoScalingGroup",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "owned",
                ]
              + variable = "autoscaling:ResourceTag/kubernetes.io/cluster/songfinch-production-us-east-1"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:kube-system:cluster-autoscaler",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.aws_iam_policy.cluster_autoscaler[0] will be updated in-place
  ~ resource "aws_iam_policy" "cluster_autoscaler" {
        id          = "arn:aws:iam::158566655292:policy/AmazonEKS_Cluster_Autoscaler_Policy-20230412133101254000000001"
        name        = "AmazonEKS_Cluster_Autoscaler_Policy-20230412133101254000000001"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "eks:DescribeNodegroup",
                          - "ec2:GetInstanceTypesFromInstanceRequirements",
                          - "ec2:DescribeLaunchTemplateVersions",
                          - "ec2:DescribeInstanceTypes",
                          - "ec2:DescribeImages",
                          - "autoscaling:DescribeTags",
                          - "autoscaling:DescribeScalingActivities",
                          - "autoscaling:DescribeLaunchConfigurations",
                          - "autoscaling:DescribeAutoScalingInstances",
                          - "autoscaling:DescribeAutoScalingGroups",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action    = [
                          - "autoscaling:TerminateInstanceInAutoScalingGroup",
                          - "autoscaling:SetDesiredCapacity",
                        ]
                      - Condition = {
                          - StringEquals = {
                              - "autoscaling:ResourceTag/kubernetes.io/cluster/songfinch-production-us-east-1" = "owned"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags        = {}
        # (6 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.aws_iam_role.this[0] must be replaced
-/+ resource "aws_iam_role" "this" {
      ~ arn                   = "arn:aws:iam::158566655292:role/AmazonEKSClusterAutoscaleRole-us-east-1" -> (known after apply)
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub" = "system:serviceaccount:kube-system:cluster-autoscaler"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
      ~ create_date           = "2024-01-11T13:19:10Z" -> (known after apply)
      ~ id                    = "AmazonEKSClusterAutoscaleRole-us-east-1" -> (known after apply)
      ~ managed_policy_arns   = [
          - "arn:aws:iam::158566655292:policy/AmazonEKS_Cluster_Autoscaler_Policy-20230412133101254000000001",
        ] -> (known after apply)
      ~ name                  = "AmazonEKSClusterAutoscaleRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
      + name_prefix           = (known after apply)
      - tags                  = {} -> null
      ~ tags_all              = {} -> (known after apply)
      ~ unique_id             = "AROASJ22PEU6H6ZYRAEM4" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.cluster_autoscaler_role.aws_iam_role_policy_attachment.cluster_autoscaler[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "cluster_autoscaler" {
      ~ id         = "AmazonEKSClusterAutoscaleRole-us-east-1-20240111131910965000000004" -> (known after apply)
      ~ role       = "AmazonEKSClusterAutoscaleRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.data.aws_iam_policy_document.ebs_csi[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "ebs_csi" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "ec2:AttachVolume",
              + "ec2:CreateSnapshot",
              + "ec2:DescribeAvailabilityZones",
              + "ec2:DescribeInstances",
              + "ec2:DescribeSnapshots",
              + "ec2:DescribeTags",
              + "ec2:DescribeVolumes",
              + "ec2:DescribeVolumesModifications",
              + "ec2:DetachVolume",
              + "ec2:ModifyVolume",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "ec2:CreateTags",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
            ]

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "CreateVolume",
                  + "CreateSnapshot",
                ]
              + variable = "ec2:CreateAction"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteTags",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
            ]
        }
      + statement {
          + actions   = [
              + "ec2:CreateVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "true",
                ]
              + variable = "aws:RequestTag/ebs.csi.aws.com/cluster"
            }
        }
      + statement {
          + actions   = [
              + "ec2:CreateVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "*",
                ]
              + variable = "aws:RequestTag/CSIVolumeName"
            }
        }
      + statement {
          + actions   = [
              + "ec2:CreateVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "owned",
                ]
              + variable = "aws:RequestTag/kubernetes.io/cluster/*"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "true",
                ]
              + variable = "ec2:ResourceTag/ebs.csi.aws.com/cluster"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "*",
                ]
              + variable = "ec2:ResourceTag/CSIVolumeName"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "owned",
                ]
              + variable = "ec2:ResourceTag/kubernetes.io/cluster/*"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteVolume",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "*",
                ]
              + variable = "ec2:ResourceTag/kubernetes.io/created-for/pvc/name"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteSnapshot",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "*",
                ]
              + variable = "ec2:ResourceTag/CSIVolumeSnapshotName"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DeleteSnapshot",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "true",
                ]
              + variable = "ec2:ResourceTag/ebs.csi.aws.com/cluster"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:kube-system:ebs-csi-controller-sa",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.aws_iam_policy.ebs_csi[0] will be updated in-place
  ~ resource "aws_iam_policy" "ebs_csi" {
        id          = "arn:aws:iam::158566655292:policy/AmazonEKS_EBS_CSI_Policy-20230823132028815300000002"
        name        = "AmazonEKS_EBS_CSI_Policy-20230823132028815300000002"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ec2:ModifyVolume",
                          - "ec2:DetachVolume",
                          - "ec2:DescribeVolumesModifications",
                          - "ec2:DescribeVolumes",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeSnapshots",
                          - "ec2:DescribeInstances",
                          - "ec2:DescribeAvailabilityZones",
                          - "ec2:CreateSnapshot",
                          - "ec2:AttachVolume",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action    = "ec2:CreateTags"
                      - Condition = {
                          - StringEquals = {
                              - "ec2:CreateAction" = [
                                  - "CreateVolume",
                                  - "CreateSnapshot",
                                ]
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = [
                          - "arn:aws:ec2:*:*:volume/*",
                          - "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  - {
                      - Action   = "ec2:DeleteTags"
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:ec2:*:*:volume/*",
                          - "arn:aws:ec2:*:*:snapshot/*",
                        ]
                    },
                  - {
                      - Action    = "ec2:CreateVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:CreateVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/CSIVolumeName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:CreateVolume"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/kubernetes.io/cluster/*" = "owned"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/CSIVolumeName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/kubernetes.io/cluster/*" = "owned"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteVolume"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/kubernetes.io/created-for/pvc/name" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/CSIVolumeSnapshotName" = "*"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "ec2:DeleteSnapshot"
                      - Condition = {
                          - StringLike = {
                              - "ec2:ResourceTag/ebs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags        = {}
        # (6 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.aws_iam_role.this[0] must be replaced
-/+ resource "aws_iam_role" "this" {
      ~ arn                   = "arn:aws:iam::158566655292:role/eks-ebs-us-east-1" -> (known after apply)
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub" = "system:serviceaccount:kube-system:ebs-csi-controller-sa"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
      ~ create_date           = "2024-01-11T13:19:10Z" -> (known after apply)
      ~ id                    = "eks-ebs-us-east-1" -> (known after apply)
      ~ managed_policy_arns   = [
          - "arn:aws:iam::158566655292:policy/AmazonEKS_EBS_CSI_Policy-20230823132028815300000002",
        ] -> (known after apply)
      ~ name                  = "eks-ebs-us-east-1" # forces replacement -> (known after apply) # forces replacement
      + name_prefix           = (known after apply)
      - tags                  = {} -> null
      ~ tags_all              = {} -> (known after apply)
      ~ unique_id             = "AROASJ22PEU6MRB5N4QMS" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.ebs_csi_role.aws_iam_role_policy_attachment.ebs_csi[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "ebs_csi" {
      ~ id         = "eks-ebs-us-east-1-20240111131910841100000003" -> (known after apply)
      ~ role       = "eks-ebs-us-east-1" # forces replacement -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.data.aws_iam_policy_document.efs_csi[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "efs_csi" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "ec2:DescribeAvailabilityZones",
              + "elasticfilesystem:DescribeAccessPoints",
              + "elasticfilesystem:DescribeFileSystems",
              + "elasticfilesystem:DescribeMountTargets",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "elasticfilesystem:CreateAccessPoint",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "true",
                ]
              + variable = "aws:RequestTag/efs.csi.aws.com/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticfilesystem:TagResource",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringLike"
              + values   = [
                  + "true",
                ]
              + variable = "aws:RequestTag/efs.csi.aws.com/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticfilesystem:DeleteAccessPoint",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "true",
                ]
              + variable = "aws:ResourceTag/efs.csi.aws.com/cluster"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:kube-system:efs-csi-controller-sa",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.aws_iam_policy.efs_csi[0] will be updated in-place
  ~ resource "aws_iam_policy" "efs_csi" {
        id          = "arn:aws:iam::158566655292:policy/AmazonEKS_EFS_CSI_Policy-20230405192355651300000001"
        name        = "AmazonEKS_EFS_CSI_Policy-20230405192355651300000001"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "elasticfilesystem:DescribeMountTargets",
                          - "elasticfilesystem:DescribeFileSystems",
                          - "elasticfilesystem:DescribeAccessPoints",
                          - "ec2:DescribeAvailabilityZones",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action    = "elasticfilesystem:CreateAccessPoint"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/efs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "elasticfilesystem:TagResource"
                      - Condition = {
                          - StringLike = {
                              - "aws:RequestTag/efs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "elasticfilesystem:DeleteAccessPoint"
                      - Condition = {
                          - StringEquals = {
                              - "aws:ResourceTag/efs.csi.aws.com/cluster" = "true"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags        = {}
        # (6 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.aws_iam_role.this[0] must be replaced
-/+ resource "aws_iam_role" "this" {
      ~ arn                   = "arn:aws:iam::158566655292:role/eks-efs-us-east-1" -> (known after apply)
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub" = "system:serviceaccount:kube-system:efs-csi-controller-sa"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
      ~ create_date           = "2024-01-11T13:19:10Z" -> (known after apply)
      ~ id                    = "eks-efs-us-east-1" -> (known after apply)
      ~ managed_policy_arns   = [
          - "arn:aws:iam::158566655292:policy/AmazonEKS_EFS_CSI_Policy-20230405192355651300000001",
        ] -> (known after apply)
      ~ name                  = "eks-efs-us-east-1" # forces replacement -> (known after apply) # forces replacement
      + name_prefix           = (known after apply)
      - tags                  = {} -> null
      ~ tags_all              = {} -> (known after apply)
      ~ unique_id             = "AROASJ22PEU6K67SPRR2B" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.efs_csi_role.aws_iam_role_policy_attachment.efs_csi[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "efs_csi" {
      ~ id         = "eks-efs-us-east-1-20240111131911367200000007" -> (known after apply)
      ~ role       = "eks-efs-us-east-1" # forces replacement -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].data.aws_iam_policy_document.external_dns[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "external_dns" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "route53:ChangeResourceRecordSets",
            ]
          + resources = [
              + "arn:aws:route53:::hostedzone/Z024960631A9EK08P92G7",
            ]
        }
      + statement {
          + actions   = [
              + "route53:ListHostedZones",
              + "route53:ListResourceRecordSets",
              + "route53:ListTagsForResource",
            ]
          + resources = [
              + "*",
            ]
        }
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:external-dns:aws-external-dns",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].aws_iam_policy.external_dns[0] will be updated in-place
  ~ resource "aws_iam_policy" "external_dns" {
        id          = "arn:aws:iam::158566655292:policy/AmazonEKS_External_DNS_Policy-20230405192356889000000003"
        name        = "AmazonEKS_External_DNS_Policy-20230405192356889000000003"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = "route53:ChangeResourceRecordSets"
                      - Effect   = "Allow"
                      - Resource = "arn:aws:route53:::hostedzone/Z024960631A9EK08P92G7"
                    },
                  - {
                      - Action   = [
                          - "route53:ListTagsForResource",
                          - "route53:ListResourceRecordSets",
                          - "route53:ListHostedZones",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags        = {}
        # (6 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].aws_iam_role.this[0] must be replaced
-/+ resource "aws_iam_role" "this" {
      ~ arn                   = "arn:aws:iam::158566655292:role/AmazonEKSExternalDnsRole-us-east-1" -> (known after apply)
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub" = "system:serviceaccount:external-dns:aws-external-dns"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
      ~ create_date           = "2024-01-11T13:19:10Z" -> (known after apply)
      ~ id                    = "AmazonEKSExternalDnsRole-us-east-1" -> (known after apply)
      ~ managed_policy_arns   = [
          - "arn:aws:iam::158566655292:policy/AmazonEKS_External_DNS_Policy-20230405192356889000000003",
        ] -> (known after apply)
      ~ name                  = "AmazonEKSExternalDnsRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
      + name_prefix           = (known after apply)
      - tags                  = {} -> null
      ~ tags_all              = {} -> (known after apply)
      ~ unique_id             = "AROASJ22PEU6M53ZYRFZB" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.external_dns_role[0].aws_iam_role_policy_attachment.external_dns[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "external_dns" {
      ~ id         = "AmazonEKSExternalDnsRole-us-east-1-20240111131911367200000006" -> (known after apply)
      ~ role       = "AmazonEKSExternalDnsRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.data.aws_iam_policy_document.load_balancer_controller[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "load_balancer_controller" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions   = [
              + "iam:CreateServiceLinkedRole",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + (known after apply),
                ]
              + variable = "iam:AWSServiceName"
            }
        }
      + statement {
          + actions   = [
              + "ec2:DescribeAccountAttributes",
              + "ec2:DescribeAddresses",
              + "ec2:DescribeAvailabilityZones",
              + "ec2:DescribeCoipPools",
              + "ec2:DescribeInstances",
              + "ec2:DescribeInternetGateways",
              + "ec2:DescribeNetworkInterfaces",
              + "ec2:DescribeSecurityGroups",
              + "ec2:DescribeSubnets",
              + "ec2:DescribeTags",
              + "ec2:DescribeVpcPeeringConnections",
              + "ec2:DescribeVpcs",
              + "ec2:GetCoipPoolUsage",
              + "elasticloadbalancing:DescribeListenerCertificates",
              + "elasticloadbalancing:DescribeListeners",
              + "elasticloadbalancing:DescribeLoadBalancerAttributes",
              + "elasticloadbalancing:DescribeLoadBalancers",
              + "elasticloadbalancing:DescribeRules",
              + "elasticloadbalancing:DescribeSSLPolicies",
              + "elasticloadbalancing:DescribeTags",
              + "elasticloadbalancing:DescribeTargetGroupAttributes",
              + "elasticloadbalancing:DescribeTargetGroups",
              + "elasticloadbalancing:DescribeTargetHealth",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "acm:DescribeCertificate",
              + "acm:ListCertificates",
              + "cognito-idp:DescribeUserPoolClient",
              + "iam:GetServerCertificate",
              + "iam:ListServerCertificates",
              + "shield:CreateProtection",
              + "shield:DeleteProtection",
              + "shield:DescribeProtection",
              + "shield:GetSubscriptionState",
              + "waf-regional:AssociateWebACL",
              + "waf-regional:DisassociateWebACL",
              + "waf-regional:GetWebACL",
              + "waf-regional:GetWebACLForResource",
              + "wafv2:AssociateWebACL",
              + "wafv2:DisassociateWebACL",
              + "wafv2:GetWebACL",
              + "wafv2:GetWebACLForResource",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "ec2:AuthorizeSecurityGroupIngress",
              + "ec2:CreateSecurityGroup",
              + "ec2:RevokeSecurityGroupIngress",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "ec2:CreateTags",
            ]
          + resources = [
              + (known after apply),
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:RequestTag/elbv2.k8s.aws/cluster"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "CreateSecurityGroup",
                ]
              + variable = "ec2:CreateAction"
            }
        }
      + statement {
          + actions   = [
              + "ec2:CreateTags",
              + "ec2:DeleteTags",
            ]
          + resources = [
              + (known after apply),
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"
            }
          + condition {
              + test     = "Null"
              + values   = [
                  + "true",
                ]
              + variable = "aws:RequestTag/elbv2.k8s.aws/cluster"
            }
        }
      + statement {
          + actions   = [
              + "ec2:AuthorizeSecurityGroupIngress",
              + "ec2:DeleteSecurityGroup",
              + "ec2:RevokeSecurityGroupIngress",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddTags",
              + "elasticloadbalancing:CreateLoadBalancer",
              + "elasticloadbalancing:CreateTargetGroup",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:RequestTag/elbv2.k8s.aws/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddTags",
              + "elasticloadbalancing:CreateListener",
              + "elasticloadbalancing:CreateRule",
              + "elasticloadbalancing:DeleteListener",
              + "elasticloadbalancing:DeleteRule",
            ]
          + resources = [
              + "*",
            ]
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddTags",
              + "elasticloadbalancing:RemoveTags",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
              + (known after apply),
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"
            }
          + condition {
              + test     = "Null"
              + values   = [
                  + "true",
                ]
              + variable = "aws:RequestTag/elbv2.k8s.aws/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddTags",
              + "elasticloadbalancing:RemoveTags",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
              + (known after apply),
              + (known after apply),
            ]
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:DeleteLoadBalancer",
              + "elasticloadbalancing:DeleteTargetGroup",
              + "elasticloadbalancing:ModifyLoadBalancerAttributes",
              + "elasticloadbalancing:ModifyTargetGroup",
              + "elasticloadbalancing:ModifyTargetGroupAttributes",
              + "elasticloadbalancing:SetIpAddressType",
              + "elasticloadbalancing:SetSecurityGroups",
              + "elasticloadbalancing:SetSubnets",
            ]
          + resources = [
              + "*",
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:ResourceTag/elbv2.k8s.aws/cluster"
            }
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddTags",
            ]
          + resources = [
              + (known after apply),
              + (known after apply),
              + (known after apply),
            ]

          + condition {
              + test     = "Null"
              + values   = [
                  + "false",
                ]
              + variable = "aws:RequestTag/elbv2.k8s.aws/cluster"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "CreateTargetGroup",
                  + "CreateLoadBalancer",
                ]
              + variable = "elasticloadbalancing:CreateAction"
            }
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:DeregisterTargets",
              + "elasticloadbalancing:RegisterTargets",
            ]
          + resources = [
              + (known after apply),
            ]
        }
      + statement {
          + actions   = [
              + "elasticloadbalancing:AddListenerCertificates",
              + "elasticloadbalancing:ModifyListener",
              + "elasticloadbalancing:ModifyRule",
              + "elasticloadbalancing:RemoveListenerCertificates",
              + "elasticloadbalancing:SetWebAcl",
            ]
          + resources = [
              + "*",
            ]
        }
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:kube-system:aws-load-balancer-controller",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.aws_iam_policy.load_balancer_controller[0] will be updated in-place
  ~ resource "aws_iam_policy" "load_balancer_controller" {
        id          = "arn:aws:iam::158566655292:policy/AmazonEKS_AWS_Load_Balancer_Controller-20230405192357291800000004"
        name        = "AmazonEKS_AWS_Load_Balancer_Controller-20230405192357291800000004"
      ~ policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "iam:CreateServiceLinkedRole"
                      - Condition = {
                          - StringEquals = {
                              - "iam:AWSServiceName" = "elasticloadbalancing.amazonaws.com"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action   = [
                          - "elasticloadbalancing:DescribeTargetHealth",
                          - "elasticloadbalancing:DescribeTargetGroups",
                          - "elasticloadbalancing:DescribeTargetGroupAttributes",
                          - "elasticloadbalancing:DescribeTags",
                          - "elasticloadbalancing:DescribeSSLPolicies",
                          - "elasticloadbalancing:DescribeRules",
                          - "elasticloadbalancing:DescribeLoadBalancers",
                          - "elasticloadbalancing:DescribeLoadBalancerAttributes",
                          - "elasticloadbalancing:DescribeListeners",
                          - "elasticloadbalancing:DescribeListenerCertificates",
                          - "ec2:GetCoipPoolUsage",
                          - "ec2:DescribeVpcs",
                          - "ec2:DescribeVpcPeeringConnections",
                          - "ec2:DescribeTags",
                          - "ec2:DescribeSubnets",
                          - "ec2:DescribeSecurityGroups",
                          - "ec2:DescribeNetworkInterfaces",
                          - "ec2:DescribeInternetGateways",
                          - "ec2:DescribeInstances",
                          - "ec2:DescribeCoipPools",
                          - "ec2:DescribeAvailabilityZones",
                          - "ec2:DescribeAddresses",
                          - "ec2:DescribeAccountAttributes",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "wafv2:GetWebACLForResource",
                          - "wafv2:GetWebACL",
                          - "wafv2:DisassociateWebACL",
                          - "wafv2:AssociateWebACL",
                          - "waf-regional:GetWebACLForResource",
                          - "waf-regional:GetWebACL",
                          - "waf-regional:DisassociateWebACL",
                          - "waf-regional:AssociateWebACL",
                          - "shield:GetSubscriptionState",
                          - "shield:DescribeProtection",
                          - "shield:DeleteProtection",
                          - "shield:CreateProtection",
                          - "iam:ListServerCertificates",
                          - "iam:GetServerCertificate",
                          - "cognito-idp:DescribeUserPoolClient",
                          - "acm:ListCertificates",
                          - "acm:DescribeCertificate",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "ec2:RevokeSecurityGroupIngress",
                          - "ec2:CreateSecurityGroup",
                          - "ec2:AuthorizeSecurityGroupIngress",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action    = "ec2:CreateTags"
                      - Condition = {
                          - Null         = {
                              - "aws:RequestTag/elbv2.k8s.aws/cluster" = "false"
                            }
                          - StringEquals = {
                              - "ec2:CreateAction" = "CreateSecurityGroup"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:security-group/*"
                    },
                  - {
                      - Action    = [
                          - "ec2:DeleteTags",
                          - "ec2:CreateTags",
                        ]
                      - Condition = {
                          - Null = {
                              - "aws:RequestTag/elbv2.k8s.aws/cluster"  = "true"
                              - "aws:ResourceTag/elbv2.k8s.aws/cluster" = "false"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "arn:aws:ec2:*:*:security-group/*"
                    },
                  - {
                      - Action    = [
                          - "ec2:RevokeSecurityGroupIngress",
                          - "ec2:DeleteSecurityGroup",
                          - "ec2:AuthorizeSecurityGroupIngress",
                        ]
                      - Condition = {
                          - Null = {
                              - "aws:ResourceTag/elbv2.k8s.aws/cluster" = "false"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = [
                          - "elasticloadbalancing:CreateTargetGroup",
                          - "elasticloadbalancing:CreateLoadBalancer",
                          - "elasticloadbalancing:AddTags",
                        ]
                      - Condition = {
                          - Null = {
                              - "aws:RequestTag/elbv2.k8s.aws/cluster" = "false"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action   = [
                          - "elasticloadbalancing:DeleteRule",
                          - "elasticloadbalancing:DeleteListener",
                          - "elasticloadbalancing:CreateRule",
                          - "elasticloadbalancing:CreateListener",
                          - "elasticloadbalancing:AddTags",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action    = [
                          - "elasticloadbalancing:RemoveTags",
                          - "elasticloadbalancing:AddTags",
                        ]
                      - Condition = {
                          - Null = {
                              - "aws:RequestTag/elbv2.k8s.aws/cluster"  = "true"
                              - "aws:ResourceTag/elbv2.k8s.aws/cluster" = "false"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = [
                          - "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*",
                        ]
                    },
                  - {
                      - Action   = [
                          - "elasticloadbalancing:RemoveTags",
                          - "elasticloadbalancing:AddTags",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*",
                        ]
                    },
                  - {
                      - Action    = [
                          - "elasticloadbalancing:SetSubnets",
                          - "elasticloadbalancing:SetSecurityGroups",
                          - "elasticloadbalancing:SetIpAddressType",
                          - "elasticloadbalancing:ModifyTargetGroupAttributes",
                          - "elasticloadbalancing:ModifyTargetGroup",
                          - "elasticloadbalancing:ModifyLoadBalancerAttributes",
                          - "elasticloadbalancing:DeleteTargetGroup",
                          - "elasticloadbalancing:DeleteLoadBalancer",
                        ]
                      - Condition = {
                          - Null = {
                              - "aws:ResourceTag/elbv2.k8s.aws/cluster" = "false"
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = "*"
                    },
                  - {
                      - Action    = "elasticloadbalancing:AddTags"
                      - Condition = {
                          - Null         = {
                              - "aws:RequestTag/elbv2.k8s.aws/cluster" = "false"
                            }
                          - StringEquals = {
                              - "elasticloadbalancing:CreateAction" = [
                                  - "CreateTargetGroup",
                                  - "CreateLoadBalancer",
                                ]
                            }
                        }
                      - Effect    = "Allow"
                      - Resource  = [
                          - "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
                          - "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*",
                        ]
                    },
                  - {
                      - Action   = [
                          - "elasticloadbalancing:RegisterTargets",
                          - "elasticloadbalancing:DeregisterTargets",
                        ]
                      - Effect   = "Allow"
                      - Resource = "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*"
                    },
                  - {
                      - Action   = [
                          - "elasticloadbalancing:SetWebAcl",
                          - "elasticloadbalancing:RemoveListenerCertificates",
                          - "elasticloadbalancing:ModifyRule",
                          - "elasticloadbalancing:ModifyListener",
                          - "elasticloadbalancing:AddListenerCertificates",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags        = {}
        # (6 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.aws_iam_role.this[0] must be replaced
-/+ resource "aws_iam_role" "this" {
      ~ arn                   = "arn:aws:iam::158566655292:role/AmazonEKSLoadBalancerControllerRole-us-east-1" -> (known after apply)
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8:sub" = "system:serviceaccount:kube-system:aws-load-balancer-controller"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::158566655292:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/752C8EBC30A6FB21F4FFE84D9B59B1F8"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
      ~ create_date           = "2024-01-11T13:19:10Z" -> (known after apply)
      ~ id                    = "AmazonEKSLoadBalancerControllerRole-us-east-1" -> (known after apply)
      ~ managed_policy_arns   = [
          - "arn:aws:iam::158566655292:policy/AmazonEKS_AWS_Load_Balancer_Controller-20230405192357291800000004",
        ] -> (known after apply)
      ~ name                  = "AmazonEKSLoadBalancerControllerRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
      + name_prefix           = (known after apply)
      - tags                  = {} -> null
      ~ tags_all              = {} -> (known after apply)
      ~ unique_id             = "AROASJ22PEU6KLE4U74H2" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # module.eks_cluster_rbacs[0].module.load_balancer_controller_role.aws_iam_role_policy_attachment.load_balancer_controller[0] must be replaced
-/+ resource "aws_iam_role_policy_attachment" "load_balancer_controller" {
      ~ id         = "AmazonEKSLoadBalancerControllerRole-us-east-1-20240111131911225200000005" -> (known after apply)
      ~ role       = "AmazonEKSLoadBalancerControllerRole-us-east-1" # forces replacement -> (known after apply) # forces replacement
        # (1 unchanged attribute hidden)
    }

Plan: 12 to add, 11 to change, 10 to destroy.

[bryantbiggs]

99.99% of that has nothing to do with this module - I see one resource change related to the cluster, which looks like its just pulling the latest patch version:

  # module.eks_cluster.aws_eks_addon.this["coredns"] will be updated in-place
  ~ resource "aws_eks_addon" "this" {
      ~ addon_version     = "v1.10.1-eksbuild.6" -> "v1.10.1-eksbuild.7"
        id                = "songfinch-production-us-east-1:coredns"
        tags              = {
            "environment" = "production"
        }
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

[gaspo53]

OMG, my fault. Sorry for the noise

[bryantbiggs]

closing this for now - we're close to shipping v20.0 (#2858) which replaces the use of the aws-auth configmap with cluster access entries and improves this entire experience

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.