Add "secrets" support for ECS Task Container Definitions

[starpebble]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Please add support for "secrets"

An AwsEcsContainerDefinition data source should support defining container secrets. So should a terraform "aws_ecs_task_definition" resource.

A ecs task container may define "secrets", docs for the Task Definition Parameters.

New or Affected Resource(s)

aws_ecs_task_definition resource and data source, the container_definitions attribute

Potential Terraform Configuration

Example: the ssm parameter db.password should be passed to the container definition. The container definition will have a "secrets" section that is non-null.

resource "aws_ecs_task_definition" "service-api" {
family = "service-api"
container_definitions = <<DEFINITION
[
{
"essential": true,
"image": "aaa.dkr.ecr.us-east-1.amazonaws.com/service:latest",
"memory": 3072,
"memoryReservation": 2048,
"name": "service-api",
"portMappings": [
{
"containerPort": 80,
"hostPort": 0
}
],
"dockerLabels" : {
"project": "terraform-security-improvements",
},
"environment" : [
{
"name": "AN_ENV_KEY",
"value": "some environment variable"
}
],
"secrets" : [
{
"name": "DB_PASSWORD",
"valueFrom": "dev.db.password"
}
]
}
]
DEFINITION
}

References

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html

• #0000

[gazoakley]

The container_definitions attribute just takes a bunch of JSON - I don't think it actually checks it matches the schema. You might find this works already 👍

[starpebble]

@gazoakley My hypothesis is that Terraform does not detect the key 'secrets' in the container definition json.

Evidence:

1. the dictionary for the key "secrets" in the json are not stored by aws when provided by terraform

2. The below container json, 'terraform apply' understands changes to "portMappings." 'terraform apply' modifies the resource when portMappings. Terraform does not modify resources when I tweak the secrets section. Terraform does modify the resource when I alter "environment" variables. Why would terraform detect 'environment' and not detect 'secrets'? Hence, my hypothesis.

[
  {
    "name": "microservice",
    "image": "service/latest",
    "cpu": 10,
    "memory": 512,
    "essential": true,
    "portMappings": [
      {
        "containerPort": 8080,
        "hostPort": 80
      }
    ],
    "environment" : [
      { "name" : "PASSWORD", "value" : "string" }
    ],
    "secrets": [
      {
        "name": "environment_variable_name",
        "valueFrom": "arn:aws:ssm:region:aws_account_id:parameter/parameter_name"
      }
    ]
  }
]

provider "aws" {
  region     = "us-east-1"
}

resource "aws_ecs_task_definition" "microservice" {
  family                = "microservice"
  container_definitions = "${file("task-definitions/microservice.json")}"
}

[gazoakley]

@starpebble: Apologies - it does unmarshal it. On the plus side, I think you'll find this works automagically in the next version of the provider 😃(that whole JSON structure is unmarshalled based on the structures defined in aws-sdk-go, and it's already updated to the version needed for this in #6484)

[starpebble]

@gazoakley - This is good news. I will double check. v1.46.0 will be available soon. I can verify whether the new provider supports ecs container secrets. I may build the provider myself and install it locally, too.

[bflad]

This has been released in version 1.46.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[starpebble]

My terraform managed ecs container will use secrets now, with aws provider v1.46. Great! thx u @gazoakley @bflad @mbfrahry

N.B. There is an aws problem with out of date ecs agents on ec2 hosts. My cluster reported errors about missing attributes when the cluster launched my tasks with container secrets:

unable to place a task because no container instance met all of its requirements. The closest matching container-instance ****** is missing an attribute required by your task

I updated my agent with sudo yum update -y ecs-init, sudo service docker restart && sudo start ecs, on an amz linux host with ssm:

aws ssm send-command --document-name "AWS-RunShellScript" --document-version "\$DEFAULT" --targets "Key=instanceids,Values=***************" --parameters '{"workingDirectory":[""],"executionTimeout":["3600"],"commands":["sudo yum update -y ecs-init","sudo service docker restart && sudo start ecs"]}' --timeout-seconds 600 --max-concurrency "50" --max-errors "0" --cloud-watch-output-config '{"CloudWatchOutputEnabled":true,"CloudWatchLogGroupName":"**********"}'

[gcoop]

This still isn't supported by v1.46.0. I get the following error using Terraform v0.11.8 and AWS provider v1.46.0.

ClientException: When you are specifying container secrets, you must also specify a value for 'executionRoleArn'. status code: 400

I think executionRoleArn needs to be added to the ecs_task_definition attributes (https://www.terraform.io/docs/providers/aws/d/ecs_task_definition.html) to support secrets within the containerDefinitions.

[gazoakley]

@gcoop: That's the documentation for the data source. The resource docs are here: https://www.terraform.io/docs/providers/aws/r/ecs_task_definition.html#execution_role_arn

[stephencoe]

@gcoop I have seen this throw the same error when "secrets": [] exists in the definition. It seems the key existing there is causing AWS SDK to complain

[rlopezro]

@gcoop are you still running into issues? Also, which region were you using?

[aairey]

I am still facing this issue in eu-west-1.
terraform 0.11.11, aws provider 1.58.0.

Adding secrets to the container definition gives me this error consistently:

ClientException: When you are specifying container secrets, you must also specify a value for 'executionRoleArn'.

I tried setting task_role_arn on the resource as this would be the most logical place to do it, but this did not help.
Even with setting executionRoleArn in the container definition itself does not work.

I tried a 2-step apply approach as well - adding the role then the secrets - for both methods, but to no avail.

edit: Realised later that task_role_arn is the role for the task, whereas executionRoleArn is what the ecs-agent uses, and thus needs to be able to pass the secrets as ENV variables to the docker daemon.

edit (solved): I had to specify my executionRoleArn outside the container_definition argument of course ...
So my resource definition now looks like this:

resource "aws_ecs_task_definition" "microservice" {
  family = "microservice-${var.environment}"
  execution_role_arn = "${data.aws_iam_role.ecs_taskexecution_role}"
  container_definitions = "${data.template_file.task_definition.rendered}"
}

[gcoop]

No longer facing the issue my end @rlopezro .

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!