IAM Role missing kms:ListGrants permission

[caiohasouza]

Description

I noticed that we are getting a huge amount of CloudWatch alarms for AccessDenied, it seems there is a missing permission or perhaps a component is doing something it should not trying to perform kms:ListGrants.

Versions

• Terraform: Terraform v0.13.7
• Module: 17.7.0

Expected behavior

No more access denied errors

Actual behavior

Lot's of access denied errors

Additional context

{
  "eventVersion": "1.08",
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "GROAV5MWD4DMOKAMGZMAU:health-check-session",
    "arn": "arn:aws:sts::111111111111:assumed-role/cluster-production20210330204212700000000002/health-check-session",
    "accountId": "111111111111",
    "accessKeyId": "ASIAV5M2D4DMNQ111111",
    "sessionContext": {
      "sessionIssuer": {
        "type": "Role",
        "principalId": "GROAV5MWD4DMOKAMGZMAU",
        "arn": "arn:aws:iam::111111111111:role/cluster-production20210330204212700000000002",
        "accountId": "111111111111",
        "userName": "cluster-production20210330204212700000000002"
      },
      "webIdFederationData": {},
      "attributes": {
        "creationDate": "2022-03-02T11:43:47Z",
        "mfaAuthenticated": "false"
      }
    },
    "invokedBy": "eks.amazonaws.com"
  },
  "eventTime": "2022-03-02T11:43:47Z",
  "eventSource": "kms.amazonaws.com",
  "eventName": "ListGrants",
  "awsRegion": "sa-east-1",
  "sourceIPAddress": "eks.amazonaws.com",
  "userAgent": "eks.amazonaws.com",
  "errorCode": "AccessDenied",
  "errorMessage": "User: arn:aws:sts::111111111111:assumed-role/cluster-production20210330204212700000000002/health-check-session is not authorized to perform: kms:ListGrants on resource: arn:aws:kms:sa-east-1:111111111111:key/11111111-1111-1111-1111-111111111111 because no resource-based policy allows the kms:ListGrants action",
  "requestParameters": null,
  "responseElements": null,
  "requestID": "e69b0cde-45ec-4978-bcfd-4756da111111",
  "eventID": "b53be6f5-08b0-414e-97ee-7fdb9a111111",
  "readOnly": true,
  "eventType": "AwsApiCall",
  "managementEvent": true,
  "recipientAccountId": "111111111111",
  "eventCategory": "Management"
}

[bryantbiggs]

this looks like your KMS policy needs to add that permission - can you share your configuration

[caiohasouza]

Hi,

Hm, but the IAM Role/Policy is created by the module (https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/main.tf#L207), right?

Regards

[bryantbiggs]

No, this is your KMS key policy - see the error message above .. because no resource-based policy allows the kms:ListGrants action

[caiohasouza]

Right,

My KMS Key policy:
{ "Version": "2012-10-17", "Id": "key-default-1", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111111111111:root" }, "Action": "kms:*", "Resource": "*" } ] }

There are something wrong?

[bryantbiggs]

wow, this is not documented well at all by EKS. I was only able to find this https://systemweakness.com/iam-permissions-to-enable-secrets-encryption-on-aws-eks-fc2fbcad958b which looks just like the issue you are seeing so pushing up a PR now

[caiohasouza]

Hi @bryantbiggs

Yes! I found the same. Ok, i will wait, thank you!

Regards

[antonbabenko]

This issue has been resolved in version 18.8.0 🎉

[caiohasouza]

Perfect, thank you so much!!

[damienleger]

For people dropping by stuck with module in v17 (because of v18 breaking change). Quick manual how-to fix:

On your EKS console, open your your KMS key with (1), your cluster IAM role with (2).

Add this inline policy to IAM role (2)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ListGrants",
                "kms:DescribeKey"
            ],
            "Resource": "<replace with KMS arn from (1)>"
        }
    ]
}

And that's fixed! This doesn't break the terraform, you can re-apply plan and the manual added policy will stay 👌

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.