Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): add executionRoleArn as EmrContainersStartJobRun prop #26926

Closed

Conversation

msambol
Copy link
Contributor

@msambol msambol commented Aug 29, 2023

Adding an optional construct prop executionRoleArn for EmrContainersStartJobRun.
This allows the role ARN to be supplied as a string and thus taken from JSON input path.

One doubt: Should I throw an error if both executionRole and executionRoleArn are supplied?
Or leave as-is and default to executionRole?

The integration test successfully deploys the stack but I had the damnedest time starting the state machine
because of EKS/k8s issues. Let me know if this is part of success criteria and I will spend more time on it.

Closes #21319.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team August 29, 2023 16:36
@github-actions github-actions bot added repeat-contributor [Pilot] contributed between 3-5 PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 labels Aug 29, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@msambol msambol changed the title feat(aws-stepfunctions-tasks): add executionRoleArn as EmrContainersStartJobRun prop feat(stepfunctions-tasks): add executionRoleArn as EmrContainersStartJobRun prop Aug 29, 2023
@msambol msambol force-pushed the aws-stepfunctions-tasks-execution-role-arn branch from 184dffc to 51a9499 Compare August 29, 2023 20:21
@aws-cdk-automation aws-cdk-automation dismissed their stale review August 29, 2023 22:16

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: e8965d9
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 29, 2023
* The execution role arn for the job run.
*
* Used when both `virtualClusterId` and `executionRoleArn` need to be taken from a JSON input path.
* Conflicts with `executionRole` when both provided. `executionRole` takes precedence over `executionRoleArn`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If both executionRole and executionRoleArn are set, please throw an error. This will help developers finding errors when both properties are set and the logic doesn't work as expected.

Comment on lines +310 to +315
private jobExecutionRoleFromArn(exectionRoleArn?: string): iam.IRole | undefined {
if (exectionRoleArn === undefined) {
return undefined;
}
return iam.Role.fromRoleArn(this, 'Job-Execution-Role-From-Arn', exectionRoleArn);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand it, this PR is implemented to solve a problem with executionRole, because iam.Role can't be used here. In this code snippet, executionRoleArn is used to create an iam.Role again. Isn't it enough to note in the executionRole property that e.g. importedValues cannot be used? Do we really need this new property?

virtualCluster: tasks.VirtualClusterInput.fromTaskInput(sfn.TaskInput.fromJsonPathAt('$.VirtualClusterId')),
releaseLabel: tasks.ReleaseLabel.EMR_6_2_0,
jobName: 'EMR-Containers-Job',
executionRoleArn: sfn.TaskInput.fromJsonPathAt('$.ExecutionRoleArn').value,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a mouthful, but wouldn't

executionRole: iam.Role.fromRoleArn(this, 'ExecRole', sfn.TaskInput.fromJsonPathAt('$.ExecutionRoleArn').value)

Work here?

@msambol
Copy link
Contributor Author

msambol commented Aug 30, 2023

@rix0rrr / @jumic Upon further investigation, I don't think this is technically doable (at least using input from SFN) and I am inclined to close the related issue. How can we do the below for example when the role ARN isn't known until the start machine is executed?

https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/emrcontainers/start-job-run.ts#L300-L302

@msambol
Copy link
Contributor Author

msambol commented Aug 31, 2023

To elaborate, this PR works fine if you provide the actual ARN as a string for executionRoleArn. But when you specify the input from the step function, it fails with a template error.

Code:

executionRoleArn: sfn.TaskInput.fromJsonPathAt('$.ExecutionRoleArn').value

Result:

 ❌  aws-stepfunctions-tasks-emr-containers-all-services-test failed: Error [ValidationError]: Template error: Fn::Select  cannot select nonexistent value at index 5

The same thing happens when trying this:

executionRole: iam.Role.fromRoleArn(this, 'ExecRole', sfn.TaskInput.fromJsonPathAt('$.ExecutionRoleArn').value)

@mrgrain
Copy link
Contributor

mrgrain commented Sep 19, 2023

Thanks @msambol ! This is unfortunate. 😕 Closing this PR for now. If you can take your questions back to the issue (or a new issue if it's sufficiently different) our triage team can have a look and maybe find a workaround with you.

@mrgrain mrgrain closed this Sep 19, 2023
@msambol msambol deleted the aws-stepfunctions-tasks-execution-role-arn branch October 7, 2023 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. repeat-contributor [Pilot] contributed between 3-5 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws-stepfunctions-tasks: EmrContainersStartJobRun could not use Role from SF input
5 participants