Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS::ECS::Cluster-CapacityProvider #301

Closed
alexhall opened this issue Dec 11, 2019 · 17 comments
Closed

AWS::ECS::Cluster-CapacityProvider #301

alexhall opened this issue Dec 11, 2019 · 17 comments
Labels
compute EC2, ECR, ECS, EKS, Lambda, Batch, Elastic Beanstalk, Serverless Application Repository
Milestone

Comments

@alexhall
Copy link

1. Title

AWS::ECS::Cluster-CapacityProvider

2. Scope of request

AWS recently announced the availability of Capacity Providers for ECS clusters to manage scaling on EC2 autoscaling groups. We would like to use this new feature in our CloudFormation templates.

3. Expected behavior

When creating or updating an AWS::ECS::Cluster resource, it should be possible to specify a CapacityProvider property that, if set, will be used to manage scale-up/scale-down of an EC2 autoscaling group. Removing CapacityProvider from an existing Cluster should disable managed scaling of the EC2 autoscaling group.

I don't have strong opinions on whether the CapacityProvider property should be specified as a reference to a new AWS::ECS::CapacityProvider resource, or inline as part of the Cluster resource.

4. Suggest specific test cases

5. Helpful Links to speed up research and evaluation

Blog post: https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-ecs-cluster-auto-scaling-now-available/
User guide: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html
API docs: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CapacityProvider.html

6. Category

Compute

7. Any additional context (optional)

@luiseduardocolon luiseduardocolon added the compute EC2, ECR, ECS, EKS, Lambda, Batch, Elastic Beanstalk, Serverless Application Repository label Dec 12, 2019
@rclark
Copy link

rclark commented Dec 14, 2019

I do feel strongly that this should be implemented as a new resource type for AWS::ECS::CapacityProvider. We should be able to reference those resources in a CapacityProviders property of an AWS::ECS::Cluster.

@softprops
Copy link

I'm very interested in using the new fargate spot capacity provider for reducing our aws bill. We manage all of our our ECS infra through cloudformation so not having this feature is a blocker for us

@dhoeric
Copy link

dhoeric commented Feb 14, 2020

aws-cli already support on specifying FARGATE_SPOT & FARGATE on --capacity-providers.
That might be helpful on integrating with cloudformation :)

ref: https://docs.aws.amazon.com/cli/latest/reference/ecs/create-cluster.html

@yeslayla
Copy link

I feel like this is currently blocked internally for the CFN team because you are unable to delete capacity providers. See aws/containers-roadmap#632

@dhoeric If you look at the CLI commands for capacity providers, you'll find:

  • create-capacity-provider
  • describe-capacity-providers
  • put-cluster-capacity-providers

but no way to detach or delete the capacity provider which is a roadblock even for creating a custom resource.

@wmcewan
Copy link

wmcewan commented Apr 22, 2020

upvote

@JohnPreston
Copy link

JohnPreston commented May 1, 2020

aws-cli already support on specifying FARGATE_SPOT & FARGATE on --capacity-providers.
That might be helpful on integrating with cloudformation :)

ref: https://docs.aws.amazon.com/cli/latest/reference/ecs/create-cluster.html

Just opened an issue for this
#459

@sean-kang
Copy link

Do you have ETA for this?

@tschutte
Copy link

I feel like this is currently blocked internally for the CFN team because you are unable to delete capacity providers. See aws/containers-roadmap#632

@dhoeric If you look at the CLI commands for capacity providers, you'll find:

* create-capacity-provider

* describe-capacity-providers

* put-cluster-capacity-providers

but no way to detach or delete the capacity provider which is a roadblock even for creating a custom resource.

The same is true of task definitions (cannot delete) and that hasn't stopped CFN from supporting it. So hopefully the team is just slow to implement.

@jsherlockppb
Copy link

Issue #239 is also quite relevant for this. Currently, we can't configure instance protection on an Auto Scaling Group in Cloudformation, which is a requirement for enabling Managed Termination Protection in CAS. Point 10 in the Capacity Provider developer guide states:

When managed termination protection is enabled, Amazon ECS prevents Amazon EC2 instances that contain tasks and that are in an Auto Scaling group from being terminated during a scale-in action. Managed termination protection can only be enabled if the Auto Scaling group also has instance protection from scale in enabled and if managed scaling is enabled.

It is only possible to enable managed termination protection if the underlying auto scaling group is already configured to enable termination protection, so even if we have the ability to provision a Capacity Provider via Cloudformation, we won't be able to enable Managed Termination Protection if the underlying auto scaling group is also provisioned via Cloudformation (ego, without termination protection), until issue 239 is resolved.

@luiseduardocolon
Copy link
Contributor

Right now I am expecting to see this in the next 6 weeks. Stay tuned. :)

@JohnPreston
Copy link

@luiseduardocolon Great news on #301.
How is that going to impact #459
It would be nice in the current circumstances to be able to define the default strategy already given FARGATE and FARGATE_SPOT are here by default.

@mwarkentin
Copy link

Looks like this is just about here (not actually seeing it in the docs yet though): https://twitter.com/aws_doc/status/1273943424849383424

@mwarkentin
Copy link

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-capacityprovider.html

@rhlarora84
Copy link

I do not see any update to Autoscaling Resource yet. Unless that is updated to support instance protection scale in, we can't use the termination protection on capacity provider. I hope that is coming soon.

@JohnPreston
Copy link

How does one get the services in the cluster to use the cluster capacity provider strategy by default?
Seems like there is a specific API call for services (boto ref)
I had opened #459 so, are we hoping to see this done too given the successful release of #301 ?
Many thanks! You are AWSome!

@manokaran3529
Copy link

How does one get the services in the cluster to use the cluster capacity provider strategy by default?
Seems like there is a specific API call for services (boto ref)
I had opened #459 so, are we hoping to see this done too given the successful release of #301 ?
Many thanks! You are AWSome!

Please remove the 'LaunchType' property from your ecs service resource block of your cfn template or add a condition to pass 'AWS::NoValue' then the service would pick the default capacity provider strategy

@JohnPreston
Copy link

@manokaran3529 Thanks for the quick response. Happy to find where in the docs make a PR to let people know about it (to update this)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compute EC2, ECR, ECS, EKS, Lambda, Batch, Elastic Beanstalk, Serverless Application Repository
Projects
None yet
Development

No branches or pull requests