Failed to check aws_kms_key_invalid_policy rule: oversize unknown value refinement

[ericrichtert]

Summary

After upgrading tflint from 0.46.1 to 0.47.0, we get an "Failed to check aws_kms_key_invalid_policy rule: oversize unknown value refinement" based on the code below:

Command

tflint

Terraform Configuration

resource "aws_kms_key" "tf-kms-key-alerting" {
  description              = "KMS key for encrypting alerting data via SNS"
  deletion_window_in_days  = 7
  key_usage                = "ENCRYPT_DECRYPT"
  customer_master_key_spec = "SYMMETRIC_DEFAULT"
  enable_key_rotation      = true
  policy = templatefile(
    "${path.module}/policies/kms-policy.json.tmpl",
    {
      K8SACCOUNTID    = var.aws_alias_2_aws_id[var.aws_alias_2_k8s_alias[data.aws_iam_account_alias.current.account_alias]]
      HUIDIGACCOUNTID = data.aws_caller_identity.current.account_id
    }
  )
}

TFLint Configuration

config {
  format = "compact"
  plugin_dir = "~/.tflint.d/plugins"

  module = false
  force = false
  disabled_by_default = false
}

plugin "aws" {
    enabled = true
    version = "0.23.1"
    source  = "github.com/terraform-linters/tflint-ruleset-aws"
}

plugin "opa" {
  enabled = true
  version = "0.2.0"
  source  = "github.com/terraform-linters/tflint-ruleset-opa"
}

plugin "terraform" {
    enabled = true
    version = "0.4.0"
    source  = "github.com/terraform-linters/tflint-ruleset-terraform"
}

rule "terraform_required_providers" {
  enabled = false
}

rule "terraform_naming_convention" {
  enabled = false
}

rule "terraform_standard_module_structure" {
  enabled = false
}

Output

Failed to check `aws_kms_key_invalid_policy` rule: oversize unknown value refinement

The trace logs looks like:

14:51:35 [DEBUG] go-plugin@v1.4.10/client.go:1046: tflint-ruleset-aws: 14:51:35 [TRACE] models/aws_kms_key_invalid_policy.go:58: Check `%s` rule: EXTRA_VALUE_AT_END=aws_kms_key_invalid_policy
14:51:35 [TRACE] interceptor/logging.go:15: gRPC request: direction=plugin2host method=/proto.Runner/GetModuleContent req="schema:{blocks:{type:\"resource\" label_names:\"type\" label_names:\"name\" body:{attributes:{name:\"policy\"} Mode:SCHEMA_MODE_DEFAULT}} Mode:SCHEMA_MODE_DEFAULT} option:{module_ctx:MODULE_CTX_TYPE_SELF hint:{resource_type:\"aws_kms_key\"} expand_mode:EXPAND_MODE_EXPAND}"
14:51:35 [TRACE] interceptor/logging.go:15: gRPC request: direction=plugin2host method=/proto.Runner/GetFile req="name:\"kms.tf\""
14:51:35 [TRACE] interceptor/logging.go:15: gRPC request: direction=plugin2host method=/proto.Runner/EvaluateExpr req="option:{type:\"\\"string\\"\" module_ctx:MODULE_CTX_TYPE_SELF} expression:{bytes:\"templatefile(\r\n    \\"${path.module}/policies/kms-policy.json.tmpl\\",\r\n    {\r\n      K8SACCOUNTID    = var.aws_alias_2_aws_id[var.aws_alias_2_k8s_alias[data.aws_iam_account_alias.current.account_alias]]\r\n      HUIDIGACCOUNTID = data.aws_caller_identity.current.account_id\r\n    }\r\n  )\" range:{filename:\"kms.tf\" start:{line:7 column:12 byte:301} end:{line:13 column:4 byte:580}}} 1:\"templatefile(\r\n    \\"${path.module}/policies/kms-policy.json.tmpl\\",\r\n    {\r\n      K8SACCOUNTID    = var.aws_alias_2_aws_id[var.aws_alias_2_k8s_alias[data.aws_iam_account_alias.current.account_alias]]\r\n      HUIDIGACCOUNTID = data.aws_caller_identity.current.account_id\r\n    }\r\n  )\" 2:\"\n\x06kms.tf\x12\x07\x08\x07\x10\x0c\x18\xad\x02\x1a\x07\x08\r\x10\x04\x18\xc4\x04\""
14:51:35 [DEBUG] go-plugin@v1.4.10/client.go:1046: tflint-ruleset-aws: 14:51:35 [ERROR] interceptor/logging.go:18: failed to gRPC request: direction=host2plugin method=/proto.RuleSet/Check err="rpc error: code = Aborted desc = Failed to check `aws_kms_key_invalid_policy` rule: oversize unknown value refinement"

TFLint Version

TFLint version 0.47.0
+ ruleset.aws (0.23.1)
+ ruleset.opa (0.2.0)
+ ruleset.terraform (0.4.0)

Terraform Version

# terraform version
Terraform v1.4.6
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.67.0
+ provider registry.terraform.io/hashicorp/external v2.3.1

Operating System

• Linux
• macOS
• Windows

[bendrucker]

Please go ahead and re-format this to use the form properly, removing the duplicate config and version info from the summary and moving the logs into the output block. Single backticks are for inline code and it's very difficult to read without proper 3 backtick blocks.

[bendrucker]

Also, we need to be able to reproduce this issue, which means you need to further isolate your configuration so that it's actually runnable. If your template file isn't relevant to reproducing the issue, exclude it. If it is, we need some version of the template as well.

[ericrichtert]

I reformatted the issue and did some more research. This code is running fine:

resource "aws_kms_key" "tf-kms-key-alerting" {
  description              = "KMS key for encrypting alerting data via SNS"
  deletion_window_in_days  = 7
  key_usage                = "ENCRYPT_DECRYPT"
  customer_master_key_spec = "SYMMETRIC_DEFAULT"
  enable_key_rotation      = true
  policy = templatefile(
    "kms-policy.json.tmpl",
    {
      K8SACCOUNTID    = "012345678901"
      HUIDIGACCOUNTID = "012345678902"
    }
  )
}

As soon as I use the data.aws_iam_account_alias.current.aws_iam_account_alias statement, it fails

data "aws_iam_account_alias" "current" {}

resource "aws_kms_key" "tf-kms-key-alerting" {
  description              = "KMS key for encrypting alerting data via SNS"
  deletion_window_in_days  = 7
  key_usage                = "ENCRYPT_DECRYPT"
  customer_master_key_spec = "SYMMETRIC_DEFAULT"
  enable_key_rotation      = true
  policy = templatefile(
    "kms-policy.json.tmpl",
    {
      K8SACCOUNTID    = "012345678901"
      HUIDIGACCOUNTID = data.aws_iam_account_alias.current.aws_iam_account_alias
    }
  )
}

[wata727]

Thank you for reporting this. This error may have occurred here.
https://github.com/zclconf/go-cty/blob/v1.13.2/cty/msgpack/unknown.go#L170-L175

HCL v2.17.0, which is used in TFLint v0.47, introduces the concept of value refinements. However, refinements larger than 1kiB seem to have serialization issues when communicating with plugins.

There are several possible workarounds, but perhaps the simplest is to remove all refinements on serialization. Fortunately, I think there are no plugins yet that rely on unknown value refinements.

[wata727]

This issue has been fixed in the upstream, but for several reasons it is not included in TFLint. See hashicorp/hcl#617 (comment) for details.

The current workaround is to disable the rule that raises the error. This problem does not occur if no expression evaluation occurs. Fortunately, this problem shouldn't occur unless you're evaluating huge template files (e.g. JSON) with unknown values at the end. If there are other situations where this problem occurs, please comment on this issue.

If the impact of this issue is significant, temporarily removing the refinements can mitigate the impact of the issue. See hashicorp/hcl#616 (comment)