Encrypt State on save/load

[maludwig]

It would be very nice to add encryption to the state file, especially for local state files, or repo controlled state files. I'm happy to lend a hand to implement it myself, if you like.

Current Terraform Version

3.52.0

Use-cases

People who want to be secure, and not have sensitive info written down on their local disk or written to the S3 (or other) state backends. For those who repo-control their state, to protect it from nefarious actors (similarly to ansible-vault).

Attempted Solutions

I went here and found out it wasn't supported: https://www.terraform.io/docs/language/state/sensitive-data.html

Proposal

My vision for it was simply like, in the main.tf, have a line, something like:

terraform {
  encrypt_state = "yes"
}

Or some other place, and then, when you run terraform for the first time, it asks you "hey what password do you want?" Maybe it gives you some sass if your password is "password", maybe it asks you to repeat the password to make sure you typed it right.

Then, when saving the state, it simply encrypts it with a key derived from that password then saves that binary data as base64, so that it is friendly for transmission and doesn't confuse the heck out of text editors/git/terminals/etc. (similar to ansible-vault).

Then, when you run it a second time, it asks "what password did you use last time?" and uses that password to decrypt the state.

References

This one is similar, but my idea was more generic. So even if, for example, one was using the "encrypt at rest" functionality of S3, you could still do extra encryption on top of that, so that the data that S3 encrypts serverside is pre-encrypted, and any future backend that doesn't support encrypted data would also get encryption.

#28957

[maludwig]

An alternate solution (which I'm proposing here to make it visually distinct, would be to only encrypt "sensitive" things, where the user can select (or override) what they consider sensitive. It just seems much more complicated to implement.

For example:

main.tf

terraform {
  encrypt_state = "only_sensitive"
}

resource "random_string" "tf_rds_production_password" {
  sensitive = true
  length = 20
  special = false
}

resource "random_string" "tf_rds_dev_password" {
  sensitive = false
  length = 20
  special = false
}

terraform.tfstate result

{
  "version": 4,
  "terraform_version": "0.15.5",
  "serial": 3,
  "lineage": "cce287c2-ea5e-7e1b-bafe-ceeb30a5b877",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "random_string",
      "name": "tf_rds_dev_password",
      "provider": "provider[\"registry.terraform.io/hashicorp/random\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "id": "yT8zs8LvwzZRropBNhpv",
            "keepers": null,
            "length": 20,
            "lower": true,
            "min_lower": 0,
            "min_numeric": 0,
            "min_special": 0,
            "min_upper": 0,
            "number": true,
            "override_special": null,
            "result": "yT8zs8LvwzZRropBNhpv",
            "special": false,
            "upper": true
          },
          "sensitive_attributes": [],
          "private": "eyJzY2hlbWFfdmVyc2lvbiI6IjEifQ=="
        }
      ]
    },
    {
      "mode": "managed",
      "type": "random_string",
      "name": "tf_rds_production_password",
      "provider": "provider[\"registry.terraform.io/hashicorp/random\"]",
      "instances": "
ENC_BCRYPT_KFpOM3NDyk_BIk4dJ76WocgYdBkK9y9x4apga4VQWElowfVHPmlJQxx4lyr8tal0apBnpyGhiJSV+FkOQI/rF7p
cFu5ZjRUaJdanSwIdZSN8UNS+RjWkRhw/KO3AZsJYZ9iTfb88ILuDXt9Y62RxHQQpu9ryXzoiJay
AdvHafgohVgBUuWWxjSWowfDOAnznBzG/ywVV8RpdFThVzoXNodpl8Jbmt8K3usB3tlAtI8MpDAr
dPk1c2KMMRofolzbkxHgFtVeLiPn94c1l6Xp3S+lHxkTRA7KFpOM3NDykXQPQ2frK3Pt0/fIb/A0
0W5QYmc76519CEJZdP6gTZmMy0W9BjqAucmP/Ol9myHiMibLjBGFtJ0EqWSrXXjfP0/iVbcJLll3
GHPKgo8KNtdVz9o4WkT86Iun6Di0t5PTkIEgyPQzcoX+YYhBaC8Z9sFCEQ7mweLr1s/Z4+o1+QPX
VSOcSC93/63sTfpQrrGPH01jXELNHAmqjmsuBGXI8sReaPiC7NoKpNA7A1X7jgyf0FOLyaUz09C7
DL2UuTY2eWF8h0UAkY78IKfSaCv6BvqUaFl3AmL0AWa0oAggRDCCVHfPeSh+jk/mFzTRF4Sprn4p
gMRB30lUM4SMcf7lvBpmpxEBJ72DNZKPaoaBK8uKwPhM4jQKwk0rYB9PTk4kbR1eCiVxkDliTjPZ
OCOHrrUJ0++twll644ApqIL/QY8XgULrK3j68jnTs4vw7S278sfY7IRiiBPS0Zs7iLPqq/d/WhJk
csWiu1Tnlqtc6e+N9tGNkMtJGSt4Z9D91AXWCZcIkTXxyRBOKLUuUN0ioM+JgSdVxeZkyNmcDXHq
jOP83QU4mcxLH6uG85WkSk9K/QVYFNZy1FvsIyunh1PTSmJowG9dGaJQN8Alqo3QOANnTtxFjddF
kQ=="
    }
  ]
}

[ramondeklein]

Motivation

It's sometimes tough to limit access to a remote container, because administrators may already have full access to a cloud environment. If the state is stored inside the cloud, then there is no way to protect these administrators from fetching these passwords. Even when security is breached, then all state is out in the open. By protecting the state with a password that isn't stored in the same cloud, security is hardened.

Implementation suggestion

To keep things simple, you should just encrypt the entire state. I would implement this functionality in the back-end provider, because some backends may also use some additional encryption where keys can be stored inside a vault. This would also make the implementation much easier.

Let's consider the Azure back-end provider. You can specify secret value that is hashed to a 256-bit AES encryption key. This encryption key is used to decrypt the state in client.go:50 and used to encrypt the state in client.go:95. That would be all there is to it.

I would be happy to provide the implementation for the Azure provider. The back-end configuration would then look something like:

terraform {
  backend "azurerm" {
    resource_group_name  = "StorageAccount-ResourceGroup"
    storage_account_name = "abcd1234"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
    encryption_phrase    = "Add your encryption phrase here"
  }
}

Of course, it must be possible to fetch the encryption_key via an environment variable, so it isn't stored in the script itself.

[StephanHCB]

For remote state, there is an open pull request that works for all backends. See #9556

[ramondeklein]

@StephanHCB Let's close this one in favor of a PR that works for almost all storage backends. It's a shame that maintainers don't respond to the PR at all...

[maludwig]

@StephanHCB I love your PR, it's far better than what I could have come up with. I've Approved it, but I'm not sure if that's at all meaningful, since I'm not a maintainer.

For reference, here's the PR: #28603