Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local Values #15449

Merged
merged 5 commits into from
Aug 21, 2017
Merged

Local Values #15449

merged 5 commits into from
Aug 21, 2017

Conversation

apparentlymart
Copy link
Contributor

@apparentlymart apparentlymart commented Jul 1, 2017

Terraform currently has the concepts of variables and outputs, which are roughly analogous to function parameters and function return values.

Terraform currently lacks a comparable idea to local variables, allowing names to be given to results internally to a module, so that the results of complex expressions can be re-used in multiple locations without duplication.

This change adds a new concept called "local values" -- or just "locals" for short -- that fills this gap.

Consider the following (rather contrived) example:

variable "name" {
}

locals {
  defaulted_name = "${var.name != "" ? var.name : "assumed human"}"
  salutation     = "Hello"
}

output "greeting" {
  value = "${local.salutation}, ${local.defaulted_name}!"
}

It is in retrospect unfortunate that our existing concept of "variable" is squatting on this term which would arguably have been more appropriate here. but since it's already taken the term "local value" is used instead to suggest the idea that we're just assigning a name to a value and that this is just a temporary storage location used to do other work.

I diverged from the rather-more-structural syntax for variable and output blocks to make a flat container for simple key/value assignments, which is intended to make local values feel more "lightweight" and transient than variables and outputs. Whereas variables and outputs together define the interface for a module, local values are an implementation detail that can be freely changed at any time, and thus they don't warrant as much ceremony around type declarations, descriptions, etc.

A module can have any number of locals blocks, which each contribute to a flat namespace of named values. The locals containers, as well as making the syntax less ambiguous, allow users to group together related values in a single construct where that improves readability.

This is intended as a solution for #4084, which is a long-standing feature request for some way to factor out complex expressions so that they can be reused. Currently people work around this limitation using null_resource and null_data_source containers, which works with some caveats but does not lend itself well to producing readable, maintainable configurations.

This also addresses #8002.


Implementation-wise, locals live in another map inside the module state. Unlike other constructs in there, locals are never persisted to disk or remote state, since they are by definition temporary results that can be re-computed trivially when needed.

Due to known quirks in current HCL parsing, local values will initially inherit some of the weird parsing quirks already seen with variables when given as values nested data structures such as lists of maps. This should get resolved by later configuration language improvements. In the mean time, these quirks can be worked around using the list and map interpolation functions to construct structures within the interpolation language rather than in HCL itself.

@apparentlymart
Copy link
Contributor Author

Hi @cemo,

I'm going to respond to your questions here on the PR just because it's easier that way to see the discussion all in one place.

A local value can in principle be anything that's allowed to be assigned to a HCL attribute via the interpolation language, but as noted in the PR summary there's likely to be similar quirks as with using the list and map syntax with outputs, such as it confusing maps with lists of maps, since the same limitations of HCL apply here. (These will be fixed separately by a later change.)

Locals can be declared in any module, including the root module. They are visible only within the module they are defined, but as with all other values can be passed to parent and child modules via interpolation into the variables/outputs.

I'm not sure what you mean by "depend on other modules". If you can elaborate with an example I'm happy to answer!

@kwilczynski
Copy link
Contributor

@apparentlymart so finally, the abuse of null_data_source will end, hopefully.

@cemo
Copy link

cemo commented Jul 2, 2017

Thanks for clarification @apparentlymart.

I meant to use outputs from other modules in the root module. I has some usages as this:

resource "null_resource" "external" {
  triggers {
    name            = "external"
    certificate-arn = "arn"
    cdr             = "0.0.0.0/0"
    subnet-ids      = "${module.platform.subnet-ids-public}"
  }
}```

at here `subnet-ids` are output of some modules.  

@apparentlymart
Copy link
Contributor Author

apparentlymart commented Jul 2, 2017

I think you're asking whether module values can be used to populate locals, in which case yes, you could write what you wrote above as follows:

locals {
  name            = "external"
  certificate-arn = "arn"
  cdr             = "0.0.0.0/0"
  subnet-ids      = "${module.platform.subnet-ids-public}"
}

You could then use your subnet-ids value somewhere else in the same module where this locals block appears:

  anything = "${local.subnet-ids}"

The general idea is that any expression can be used as a local, though this excludes context-sensitive variable types such as self. and count. which only have meaning when used inside a particular resource block.

Whether it is useful to use such simple locals as the above I'm not sure; I'd mainly anticipated using this for complex expressions that would otherwise need to be duplicated at many points in the program, as is often the case today with "defaulted" values (as shown in my example above), concatenation of splat results, etc.

within a module.
---

# Variable Configuration
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This heading is incorrect

@apparentlymart
Copy link
Contributor Author

As context for anyone who finds this and wonders what's going on here: we weren't able to land this in time for the 0.10.0 feature freeze, so it's on hold for the moment but we'll look at it again for inclusion in a patch release after 0.10.0 final is out.

@josb
Copy link

josb commented Aug 10, 2017

As other keywords are singular, wouldn't local be better?

@apparentlymart
Copy link
Contributor Author

This block has a plural name because it defines potentially several local values at once. The interpolation variable prefix local. is singular, reflecting that it is only one.

Defining multiple things at once in a single block here is unusual, but I went this route because locals should feel like a more "lightweight" construct than variables (they don't have defaults, explicit type declarations, etc).

@JonCubed
Copy link

@apparentlymart can/will you be able load local values from a file?

The use case for this is I have environment specific settings that I don't really want a user to overwrite from the command line when running terraform, which is the case when using variables and variable files.

@apparentlymart
Copy link
Contributor Author

It would work to make a .tf file in the module that only contains a local block, or to use the file function to populate a single value from a file.

In future we plan to have a jsondecode function that could allow a complex value to be loaded into a value.

@JonCubed
Copy link

thanks @apparentlymart, I'll keep an eye out for jsondecode

@nodesocket
Copy link

nodesocket commented Aug 14, 2017

👍

I believe local variables would address my problem. Currently I have two output variables:

variable "_exec" {
    type = "list"
    default = [
        "rm -f /home/ubuntu/.ssh/authorized_keys",
        "sudo rm -f /var/log/startupscript.log"
    ]
    description = "A list of required commands to execute on each instance"
}

variable "exec" {
    type = "list"
    default = []
    description = "A list of additional commands to execute on each instance"
}

I am using _exec to internally signify that is a variable _exec should not be modified, even though users of the modules can still modify it.

provisioner "remote-exec" {
    inline = "${concat("${var._exec}", "${var.exec}")}"
    ...
}

Copy link
Member

@jbardin jbardin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job! Just one minor comment, but otherwise this looks great!

@@ -148,6 +148,42 @@ func outputsStr(os []*Output) string {
return strings.TrimSpace(result)
}

func localsStr(ls []*Local) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be in a test file, or is it intended to be used elsewhere eventually?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm honestly not sure why these are here vs. in a test-only file, but I was following the precedent set by the others above.

A local value is similar to an output in that it exists only within state
and just always evaluates its value as best it can with the current state.
Therefore it has a single graph node type for all walks, which will
deal with that evaluation operation.
We stash the locals in the module state in a map that is ignored for JSON
serialization. We don't include locals in the persisted state because they
can be trivially recomputed and this allows us to assume that they will
pass through verbatim, without any normalization or other transforms
caused by the JSON serialization.

From a user standpoint a local is just a named alias for an expression,
so it's desirable that the result passes through here in as raw a form
as possible, so it behaves as closely as possible to simply using the
given expression directly.
@apparentlymart
Copy link
Contributor Author

This is now merged, and should be included in the next release of Terraform.

For a feature of this size it's likely that there will be some quirks and misbehaviors in this first pass that we'll need to address with subsequent changes. If you've found such a thing, please open a new top-level issue to describe the problem since that way it's easier to keep track of potentially-several issues at once without them all collapsing into a single flat thread. Thanks!

@apparentlymart apparentlymart deleted the f-local-values branch August 21, 2017 22:37
@hashicorp hashicorp locked and limited conversation to collaborators Aug 21, 2017
@apparentlymart
Copy link
Contributor Author

This was included in 0.10.3, with some fixes in subsequent releases.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants