Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(batch): grantSubmitJob method #26729

Merged
merged 6 commits into from
Aug 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions packages/@aws-cdk/aws-batch-alpha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -658,3 +658,35 @@ B => 2 vCPU - WAITING
In this situation, Batch will allocate **Job A** to compute resource #1 because it is the most cost efficient resource that matches the vCPU requirement. However, with this `BEST_FIT` strategy, **Job B** will not be allocated to our other available compute resource even though it is strong enough to handle it. Instead, it will wait until the first job is finished processing or wait a similar `m5.xlarge` resource to be provisioned.

The alternative would be to use the `BEST_FIT_PROGRESSIVE` strategy in order for the remaining job to be handled in larger containers regardless of vCPU requirement and costs.

### Permissions

You can grant any Principal the `batch:submitJob` permission on both a job definition and a job queue like this:

```ts
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';

declare const vpc: ec2.IVpc;

const ecsJob = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
}),
});

const queue = new batch.JobQueue(this, 'JobQueue', {
computeEnvironments: [{
computeEnvironment: new batch.ManagedEc2EcsComputeEnvironment(this, 'managedEc2CE', {
vpc,
}),
order: 1,
}],
priority: 10,
});

const user = new iam.User(this, 'MyUser');
ecsJob.grantSubmitJob(user, queue);
```
13 changes: 13 additions & 0 deletions packages/@aws-cdk/aws-batch-alpha/lib/ecs-job-definition.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import { Construct } from 'constructs';
import { CfnJobDefinition } from 'aws-cdk-lib/aws-batch';
import { EcsEc2ContainerDefinition, IEcsContainerDefinition } from './ecs-container-definition';
import { baseJobDefinitionProperties, IJobDefinition, JobDefinitionBase, JobDefinitionProps } from './job-definition-base';
import * as iam from 'aws-cdk-lib/aws-iam';
import { IJobQueue } from './job-queue';

/**
* A JobDefinition that uses ECS orchestration
Expand Down Expand Up @@ -102,6 +104,17 @@ export class EcsJobDefinition extends JobDefinitionBase implements IEcsJobDefini
this.jobDefinitionName = EcsJobDefinition.getJobDefinitionName(scope, this.jobDefinitionArn);
}

/**
* Grants the `batch:submitJob` permission to the identity on both this job definition and the `queue`
*/
public grantSubmitJob(identity: iam.IGrantable, queue: IJobQueue) {
iam.Grant.addToPrincipal({
actions: ['batch:SubmitJob'],
grantee: identity,
resourceArns: [this.jobDefinitionArn, queue.jobQueueArn],
});
}

private renderPlatformCapabilities() {
if (this.container instanceof EcsEc2ContainerDefinition) {
return [Compatibility.EC2];
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import { Template } from 'aws-cdk-lib/assertions';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import { DefaultTokenResolver, Size, StringConcat, Stack, Tokenization } from 'aws-cdk-lib';
import { Compatibility, EcsEc2ContainerDefinition, EcsFargateContainerDefinition, EcsJobDefinition } from '../lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import { Compatibility, EcsEc2ContainerDefinition, EcsFargateContainerDefinition, EcsJobDefinition, JobQueue, ManagedEc2EcsComputeEnvironment } from '../lib';
import { Vpc } from 'aws-cdk-lib/aws-ec2';

test('EcsJobDefinition respects propagateTags', () => {
// GIVEN
Expand Down Expand Up @@ -127,3 +129,44 @@ test('JobDefinitionName is parsed from arn in imported job', () => {
// THEN
expect(importedJob.jobDefinitionName).toEqual('job-def-name');
});

test('grantSubmitJob() grants the job role the correct actions', () => {
// GIVEN
const stack = new Stack();
const ecsJob = new EcsJobDefinition(stack, 'ECSJob', {
container: new EcsFargateContainerDefinition(stack, 'EcsContainer', {
cpu: 256,
memory: Size.mebibytes(2048),
image: ecs.ContainerImage.fromRegistry('foorepo/fooimage'),
}),
});
const queue = new JobQueue(stack, 'queue');

queue.addComputeEnvironment(
new ManagedEc2EcsComputeEnvironment(stack, 'env', {
vpc: new Vpc(stack, 'VPC'),
}),
1,
);

const user = new iam.User(stack, 'MyUser');

// WHEN
ecsJob.grantSubmitJob(user, queue);

// THEN
Template.fromStack(stack).hasResourceProperties('AWS::IAM::Policy', {
PolicyDocument: {
Statement: [{
Action: 'batch:SubmitJob',
Effect: 'Allow',
Resource: [
{ Ref: 'ECSJobFFFEA569' },
{ 'Fn::GetAtt': ['queue276F7297', 'JobQueueArn'] },
],
}],
Version: '2012-10-17',
},
PolicyName: 'MyUserDefaultPolicy7B897426',
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"version": "33.0.0",
"files": {
"21fbb51d7b23f6a6c262b46a9caee79d744a3ac019fd45422d988b96d44b2a22": {
"source": {
"path": "BatchEcsJobDefinitionTestDefaultTestDeployAssertE5BAAC9B.template.json",
"packaging": "file"
},
"destinations": {
"current_account-current_region": {
"bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}",
"objectKey": "21fbb51d7b23f6a6c262b46a9caee79d744a3ac019fd45422d988b96d44b2a22.json",
"assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-${AWS::Region}"
}
}
}
},
"dockerImages": {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"Parameters": {
"BootstrapVersion": {
"Type": "AWS::SSM::Parameter::Value<String>",
"Default": "/cdk-bootstrap/hnb659fds/version",
"Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
}
},
"Rules": {
"CheckBootstrapVersion": {
"Assertions": [
{
"Assert": {
"Fn::Not": [
{
"Fn::Contains": [
[
"1",
"2",
"3",
"4",
"5"
],
{
"Ref": "BootstrapVersion"
}
]
}
]
},
"AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
}
]
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"version":"33.0.0"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"version": "33.0.0",
"testCases": {
"BatchEcsJobDefinitionTest/DefaultTest": {
"stacks": [
"stack"
],
"assertionStack": "BatchEcsJobDefinitionTest/DefaultTest/DeployAssert",
"assertionStackName": "BatchEcsJobDefinitionTestDefaultTestDeployAssertE5BAAC9B"
}
}
}
Loading