Support partial rendering of templatefile instead of marking the whole value "known after apply"

[jcogilvie]

Terraform Version

1.3

Use Cases

If I have a template file present in my plan somewhere, and one element of the template is unknown, it should render what parts of the template it can, rather than opting to render none of it.

Specifically: I use helm. Terraform typically renders my helm template during the plan, which is very useful. However, if I add a new infra object (like an SNS topic) to terraform and pass the corresponding ARN into the helm template, the whole template becomes (known after apply). This is now useless in telling me what my helm chart will look like at apply time.

If instead I am referring to a known ARN, the template renders successfully and I'm able to see in the plan how the template will render at apply time.

Attempted Solutions

If I run terraform in two stages, with the first pass targeting and applying the new infra that is created, and the second pass for everything else (including helm), the plan for the second pass provides me the whole helm template.

Proposal

I would like to see (known after apply) moved inside the rendered template, rather than replacing its value.

For instance:

template.yaml.tftpl

service:
  snsTopicArn: ${sns_topic}

locals {
  helm_values = templatefile("template.yaml.tftpl", {
    sns_topic = aws_sns_topic.my_topic.arn 
  })
}

resource "aws_sns_topic" "my_topic" {
   ...
}

data "helm_template" "my_helm_template {
  ...
  values = [local.helm_values]
}

Currently, the values of helm_template shows in the plan as (known after apply).

It would be more useful if it would show:

values = [
  service:
    snsTopicArn: (known after apply)
]

That is, push the string (known after apply) into the templatefile at plan time where it is feasible.

References

No response

[apparentlymart]

Hi @jcogilvie! Thanks for sharing this feature request.

The behavior you are describing actually belongs to the Terraform language's handling of unknown values rather than to templatefile in particular: this request is essentially asking for Terraform to be able to track a partially unknown string, where some parts of it are known and other parts are not.

Coincidentally I've been working on a prototype this last week which partially addresses that, by allowing Terraform to track a known prefix of a string in certain cases, although my motivation for doing that was to give partial information to downstream expressions so that e.g. value != "" can return true even if the string is only partially known, rather than to show this information in the UI. It's interesting to ponder what it might look like to show that information in the UI; I would imagine something similar to what you showed of placing the "known after apply" annotation inline, so perhaps like this:

  values = [
    <<-EOT
    service:
      snsTopicArn: ${ (...remainder known after apply) }
    EOT
  ]

In your case this happens to get something close to what you wanted just because the unknown value was at the end of the template. If this had been a more complicated template with multiple unknown values in it or where the unknown value were in the middle of the string then the current level of tracking I've prototyped would only be able to show the known part up to the first unknown value interpolation, because the template language immediately stops evaluating at the point where it first encounters an unknown value.

I think there's already another issue open that is talking about the possibility of partially-unknown strings in a different situation than template rendering, but I wasn't able to find it with a quick search, so for now we'll leave this one as a separate issue and maybe we'll consolidate the two later if we find that they are substantially representing the same problem. Thanks again!

[apparentlymart]

Terraform v1.6 now includes the foundational concept of "unknown string with a known prefix", and Terraform is now using it to retain some more information during evaluation. For example, "foo-${unknown_thing}" != "" can now return true rather than an unknown result, because Terraform knows that the result must at least begin with foo-.

In principle then, it would now be possible to use that same information to show the known prefixes of unknown strings in the plan UI. I think what's missing to get there is:

• The JSON representation of plans -- as would be produced from terraform show -json tfplan -- does not currently have any way to describe partially-known unknown values, and the UI is built in terms of that JSON output.
• Once that is solved, we'd need to figure out how best to thread that new information through to the relevant parts of the UI code and how best to present a string with a known prefix. (The example I gave in my previous comment -- rendering as if it's a template interpolation of a single unknown value -- is one possibility.)

It's also worth noting that since the feature we've implemented was primarily motivated by making fewer expressions return unknown values in the main language rather than for UI presentation, and so what we have is limited only to known prefixes. That happens to work for the motivating example in this issue, since the unknown value is interpolated at the end. It would also work for some shorter examples such as interpolating an id to the end of a constructed AWS ARN. It would not be so useful for larger multi-line strings with many different unknown values interpolated throughout them, so if we do implement the above then we'll need to decide if that constitutes solving this issue as much as we intend to or if a more complete modelling of "partially-unknown strings" is still needed.