aws-ecs: support capacity provider

[pahud]

https://aws.amazon.com/tw/about-aws/whats-new/2019/12/amazon-ecs-capacity-providers-now-available/

Use Case

Proposed Solution

Other

dependency aws/containers-roadmap#631

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[ali-habibzadeh]

What is the likelihood of this being added to cfn?

[mb-dev]

Capacity Providers were just added to Cfn: 4ce27f4#diff-4a4e15a081904ee16b4d84e2f5cf5aee

[mb-dev]

FYI: @SoManyHs

[instigardo]

 const ondemandCapacity = cluster.addCapacity("OnDemandCapacity",{
          instanceType: new InstanceType(instanceType),
          minCapacity: 1,
          maxCapacity: 2,
          vpcSubnets: {
            subnets: vpc.privateSubnets
          }
        })
        const ondemandCapacityProvider = new CfnCapacityProvider(this, "OnDemandCapacityProvider", {
          name: "ondemandCapacityProvider",
          autoScalingGroupProvider: {
            autoScalingGroupArn: ondemandCapacity.autoScalingGroupArn,
            managedScaling: {
              maximumScalingStepSize: 10,
              minimumScalingStepSize: 1,
              status: "ENABLED",
              targetCapacity: 85
            }
          }
        })

This keeps on giving error
Invalid request provided: CreateCapacityProvider error: The specified Auto Scaling group ARN is invalid. Specify a valid ARN and try again.

[mb-dev]

See here about some issues with the current implementation:
aws/containers-roadmap#631

These are items that still planned to be implemented:

Ability to reference the ASG name in the AWS::ECS::CapacityProvider resource
Ability to specify a custom capacity provider strategy in the AWS::ECS::Service resource
Ability to enable scale-in protection in the AWS::AutoScaling::AutoScalingGroup resource

[gandroz]

 const ondemandCapacity = cluster.addCapacity("OnDemandCapacity",{
          instanceType: new InstanceType(instanceType),
          minCapacity: 1,
          maxCapacity: 2,
          vpcSubnets: {
            subnets: vpc.privateSubnets
          }
        })
        const ondemandCapacityProvider = new CfnCapacityProvider(this, "OnDemandCapacityProvider", {
          name: "ondemandCapacityProvider",
          autoScalingGroupProvider: {
            autoScalingGroupArn: ondemandCapacity.autoScalingGroupArn,
            managedScaling: {
              maximumScalingStepSize: 10,
              minimumScalingStepSize: 1,
              status: "ENABLED",
              targetCapacity: 85
            }
          }
        })

This keeps on giving error
Invalid request provided: CreateCapacityProvider error: The specified Auto Scaling group ARN is invalid. Specify a valid ARN and try again.

got the same issue in v1.53 :(

[mb-dev]

@gandroz did you try putting asg name in the autoScalingGroupArn field? they added this option recently.

[gandroz]

@gandroz did you try putting asg name in the autoScalingGroupArn field? they added this option recently.

hmmm good catch @mb-dev, using asg.autoScalingGroupName as the autoScalinGroupArn option does not raise an error, however I can see the capacity provider in the CloudFormation template but not in the ECS console....

How to link the capacity provider to the cluster ?

[pahud]

Hi folks,

I am working on the PR today and seems working fine in my integ test.

See: https://twitter.com/pahudnet/status/1285440342045515777

Will create a PR draft shortly.

[iamhopaul123]

We'll hold this issue because there are some feature gaps for CFN Capacity Provider to be narrowed. For example, the cyclic dependency that is already discussed here. Even if we try to break the cyclic dependency by using the hard code cluster name, we'll still have problems when deleting resources. Hence, we decided to hold this issue and wait for CFN update on Capacity Provider before we support it in CDK.

[andreialecu]

@iamhopaul123 I believe this may have been unblocked recently?

https://aws.amazon.com/about-aws/whats-new/2020/10/aws-fargate-spot-for-amazon-ecs-now-supported-in-aws-cloudformation/

I'm not sure if it only applies to Fargate and whether it can work with EC2 as well.

[iamhopaul123]

Hi @andreialecu thank you so much for bringing this great news to us! We'll take a look at it to see if it could fit in fixing this issue.

[iamhopaul123]

Hello @andreialecu, looks like number 4 and 5 of aws/containers-roadmap#631 (comment) are still waiting to be fixed. Unfortunately though CFN support could simplify the CDK support for quite a lot, without fixing the circular dependency problem there is not much we could do. We'll keep waiting until there are more updates on this.

[pv93]

Hello everyone. Is there any news on this issue?

[muraleee]

Is this coming soon?

[iamhopaul123]

Hello, we don't have any update for now. We'll update this issue we there's any news.

[rsmogura]

We are interest in this too, mainly due possibilities to use launch templates.

I think I’ve got idea how to solve it and I did preliminary tests.

We could create custom cloud formation resource “CapacityProviderAttachment”, which will have two inputs cluster & capacity providers - this is simple lambda. Custom resource could be part of “bootstrap” or some kind of inner stack (I prefer later as CF templates are commonly used).

The resource would call put-capacity-providers on creation and remove from cluster on deletion.

I would have to validate deletion only, to have final results.

[otterley]

Supporting EC2 Capacity Providers may now be possible, now that the AWS::ECS::ClusterCapacityProviderAssociations resource is available.

[rsmogura]

I missed this one. So this ticket and bunch of other leading to launch templates can be shortly solved.``

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.