Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support FIPS endpoint #692

Merged
merged 6 commits into from
Jan 16, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 32 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ Line Interface](http://aws.amazon.com/cli/) product detail page.
- [Using Route53 Service Discovery](#using-route53-service-discovery)
- [Viewing Running Tasks](#viewing-running-tasks)
- [Viewing Container Logs](#viewing-container-logs)
- [Using Private Registry Authentication](#using-private-registry-authentication)
- [Using FIPS Endpoints](#using-fips-endpoints)
- [Using Private Registry Authentication](#using-private-registry-authentication)
- [Amazon ECS CLI Commands](#amazon-ecs-cli-commands)
- [Contributing to the CLI](#contributing-to-the-cli)
- [License](#license)
Expand Down Expand Up @@ -338,17 +339,17 @@ For the EC2 launch type, the ECS CLI always creates EC2 instances that include t
echo ECS_CLUSTER={ clusterName } >> /etc/ecs/ecs.config
```

This user data directs the EC2 instance to join your ECS Cluster. You can optionally include extra user data with `--extra-user-data`; this flag takes a file name as its argument.
This user data directs the EC2 instance to join your ECS Cluster. You can optionally include extra user data with `--extra-user-data`; this flag takes a file name as its argument.
The flag can be used multiple times to specify multiple files. Extra user data can be shell scripts or cloud-init directives- see the [EC2 documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) for more information.
The ECS CLI takes all the User Data, and packs it into a MIME Multipart archive which can be used by cloud-init on the EC2 instance. The ECS CLI even allows existing MIME Multipart archives to be passed in with `--extra-user-data`.
The CLI will unpack the existing archive, and then repack it into the final archive (preserving all header and content type information). Here is an example of specifying extra user data:

```
ecs-cli up \
--capability-iam \
--extra-user-data my-shellscript \
--extra-user-data my-cloud-boot-hook \
--extra-user-data my-mime-multipart-archive \
ecs-cli up \
--capability-iam \
--extra-user-data my-shellscript \
--extra-user-data my-cloud-boot-hook \
--extra-user-data my-mime-multipart-archive \
--launch-type EC2
```

Expand Down Expand Up @@ -850,7 +851,26 @@ OPTIONS:
--timestamps, -t [Optional] Shows timestamps on each line in the log output.
```

## Using Private Registry Authentication
### Using FIPS Endpoints
The ECS-CLI supports using [FIPS endpoints](https://aws.amazon.com/compliance/fips/) for calls to ECR. To ensure you are accessing ECR using FIPS endpoints, use the `--use-fips` flag on the `push`, `pull`, or `images` command. FIPS endpoints are currently available in us-west-1, us-west-2, us-east-1, us-east-2, and in the [GovCloud partition](https://docs.aws.amazon.com/govcloud-us/latest/ug-west/using-govcloud-endpoints.html).

```
$ ecs-cli push myRepository:latest --use-fips --debug
DEBU[0000] Using FIPS endpoint: https://ecr-fips.us-west-2.amazonaws.com
INFO[0000] Getting AWS account ID...
DEBU[0000] Getting authorization token...
DEBU[0000] Checking file cache registry=xxxxxxxxxx123
DEBU[0000] Calling ECR.GetAuthorizationToken registry=xxxxxxxxxx123
DEBU[0000] Saving credentials to file cache registry=xxxxxxxxxx123
DEBU[0000] Retrieved authorization token via endpoint: https://xxxxxxxxxxx123.dkr.ecr-fips.us-west-2.amazonaws.com
INFO[0000] Tagging image image=myRepository repository=xxxxxxxxxxx123.dkr.ecr-fips.us-west-2.amazonaws.com/myRepository tag=latest
INFO[0000] Image tagged
DEBU[0000] Check if repository exists repository=myRepository
INFO[0000] Pushing image repository=xxxxxxxxxxx123.dkr.ecr-fips.us-west-2.amazonaws.com/myRepository tag=latest
INFO[0002] Image pushed
```

### Using Private Registry Authentication

If you want to use privately hosted container images with ECS, the ECS CLI can store your private registry credentials in AWS Secrets Manager and create an IAM role which ECS can use to access the credentials and private images. This allows you to:

Expand All @@ -863,7 +883,7 @@ Using privately hosted images with the ECS CLI is done in two parts:
1) Create new AWS Secrets Manager secrets and an IAM Task Execution Role with `ecs-cli registry-creds up`
2) Run `ecs-cli compose` commands to create and run a task definition that includes the new resources

### Storing private registry credentials with `ecs-cli registry-creds up`
#### Storing private registry credentials with `ecs-cli registry-creds up`

To get started, first create an input file that contains the name of your registry and the credentials needed to access it:

Expand All @@ -876,7 +896,7 @@ registry_credentials:
my-registry.example.com:
secrets_manager_arn: # required when using (with no modification) or updating an existing secret
username: myUserName # required when creating or updating a new secret
password: ${MY_PASSWORD} # required when creating or updating a new secret
password: ${MY_PASSWORD} # required when creating or updating a new secret
kms_key_id: # optional custom KMS Key ID to use to encrypt new secret
container_names: # required to match credential resources with docker-compose services
- web
Expand All @@ -891,7 +911,7 @@ Other options:
* If you want to encrypt the AWS Secrets Manager secret for your registry with a custom KMS Key, then add the ARN, ID or Alias of the Key in the `kms_key_id` field. Otherwise, AWS Secrets Manager will use the default key in your account.
* If you don't want to create or update an IAM Task Execution Role for these secrets, use the `--no-role` flag instead of specifying a role name.
* If you don't want to generate an output file for use with `compose` or for records purposes, use the `--no-output-file` flag.
* If you want the output file to be created in a specific directory on your machine, you can specify it with the `--output-dir <value>` flag. Otherwise, the file will be created in your working directory.
* If you want the output file to be created in a specific directory on your machine, you can specify it with the `--output-dir <value>` flag. Otherwise, the file will be created in your working directory.

After creating the input file, run the `registry-creds up` command on the file and pass in the name of the new or existing Task Execution Role you want to use for the secrets:

Expand Down Expand Up @@ -933,7 +953,7 @@ This file contains:

We can now use this file with `ecs-cli compose` commands to start a task with images in our private registry.

### Using private registry credentials when launching tasks or services
#### Using private registry credentials when launching tasks or services

Now that we have an output file that identifies which resources we need to use our private registry, the ECS CLI will incorporate them into our Docker Compose project when we run `ecs-cli compose`.

Expand Down Expand Up @@ -984,7 +1004,6 @@ INFO[0018] Started container... container=bf35a813-dd76-4fe0-b5a2-c1334c2331f4/l

For more information about using private registries with ECS, see [Private Registry Authentication for Tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html).


## Amazon ECS CLI Commands

For a complete list of commands, see the
Expand Down
59 changes: 38 additions & 21 deletions ecs-cli/modules/cli/image/image_app.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ func ImagePush(c *cli.Context) {
if err != nil {
logrus.Fatal("Error executing 'push': ", err)
}
ecrClient := ecrclient.NewClient(commandConfig)

ecrClient := getECRClient(c, commandConfig)
stsClient := stsclient.NewClient(commandConfig)

if err := pushImage(c, rdwr, dockerClient, ecrClient, stsClient); err != nil {
Expand All @@ -89,7 +90,8 @@ func ImagePull(c *cli.Context) {
if err != nil {
logrus.Fatal("Error executing 'pull': ", err)
}
ecrClient := ecrclient.NewClient(commandConfig)

ecrClient := getECRClient(c, commandConfig)
stsClient := stsclient.NewClient(commandConfig)

if err := pullImage(c, rdwr, dockerClient, ecrClient, stsClient); err != nil {
Expand All @@ -109,15 +111,32 @@ func ImageList(c *cli.Context) {
logrus.Fatal("Error executing 'images': ", err)
}

ecrClient := ecrclient.NewClient(commandConfig)
ecrClient := getECRClient(c, commandConfig)

if err := getImages(c, rdwr, ecrClient); err != nil {
logrus.Fatal("Error executing 'images': ", err)
return
}
}

func getECRClient(c *cli.Context, commandConfig *config.CommandConfig) ecrclient.Client {
ecrClient := ecrclient.NewClient(commandConfig)

useFips := c.Bool(flags.UseFIPSFlag)

if useFips {
fipsClient, err := ecrclient.NewFipsClient(commandConfig)
if err != nil {
logrus.Fatal("Error creating FIPS client: ", err)
}
ecrClient = fipsClient
}
return ecrClient
}

func pushImage(c *cli.Context, rdwr config.ReadWriter, dockerClient dockerclient.Client, ecrClient ecrclient.Client, stsClient stsclient.Client) error {
registryID := c.String(flags.RegistryIdFlag)

args := c.Args()

if len(args) != 1 {
Expand Down Expand Up @@ -253,12 +272,12 @@ func listImagesContent(w *tabwriter.Writer, info imageInfo, count int) {

func printImageRow(w io.Writer, info imageInfo) {
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t\n",
info.RepositoryName,
info.Tag,
info.ImageDigest,
info.PushedAt,
info.Size,
)
info.RepositoryName,
info.Tag,
info.ImageDigest,
info.PushedAt,
info.Size,
)
}

func getTagStatus(c *cli.Context) string {
Expand Down Expand Up @@ -297,17 +316,15 @@ func getECRAuth(registryURI string, registryID string,
return ecrClient.GetAuthorizationToken(registryURI)
}

func splitImageName(image string, seperatorRegExp string,
format string) (registry string, repository string, tag string, err error) {

func splitImageName(image string, seperatorRegExp string, format string) (registry string, repository string, tag string, err error) {
re := regexp.MustCompile(
`^(?:((?:[a-zA-Z0-9][a-zA-Z0-9-_]*)\.dkr\.ecr\.[a-zA-Z0-9\-_]+\.amazonaws\.com(?:\.cn)?)/)?` + // repository uri (Optional)
`([0-9a-z\-_/]+)` + // repository
`(?:` + seperatorRegExp + `([0-9A-Za-z_.\-:]+))?$`) // tag (Optional)
matches := re.FindStringSubmatch(image)
if len(matches) == 0 {
return "", "", "", fmt.Errorf("Please specify the image name in the correct format [%s]", format)
}
`^(?:((?:[a-zA-Z0-9][a-zA-Z0-9-_]*)\.dkr\.ecr(\-fips)?\.[a-zA-Z0-9\-_]+\.amazonaws\.com(?:\.cn)?)/)?` + // registry uri (Optional)
`([0-9a-z\-_/]+)` + // repository
`(?:` + seperatorRegExp + `([0-9A-Za-z_.\-:]+))?$`) // tag or sha (Optional)
matches := re.FindStringSubmatch(image)
if len(matches) == 0 {
return "", "", "", fmt.Errorf("Please specify the image name in the correct format [%s]", format)
}

return matches[1], matches[2], matches[3], nil
}
return matches[1], matches[3], matches[4], nil
}
Loading