Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameter to pass role ARN to GlueJobOperator #33408

Merged
merged 10 commits into from
Aug 15, 2023
Merged

Conversation

erdos2n
Copy link
Contributor

@erdos2n erdos2n commented Aug 15, 2023

GlueOperator currently requires iam_role_name and endpoint_url to fetch the ARN from AWS. This ARN eventually gets passed into the config for Glue Job.

Resolves #33248

@boring-cyborg boring-cyborg bot added area:providers provider:amazon-aws AWS/Amazon - related issues labels Aug 15, 2023
@erdos2n
Copy link
Contributor Author

erdos2n commented Aug 15, 2023

@eladkal , @o-nikolas I added test cases for the role_arn, can you let me know if the tests were added correctly?
Thank you

airflow/providers/amazon/aws/hooks/glue.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/hooks/glue.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/hooks/glue.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/operators/glue.py Outdated Show resolved Hide resolved
tests/providers/amazon/aws/hooks/test_glue.py Show resolved Hide resolved
Copy link
Contributor

@vincbeck vincbeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vincbeck
Copy link
Contributor

You have some static checks failure, please run them to fix these failures.

@erdos2n erdos2n changed the title [WIP] Issue ##33248 - GlueOperator IAM ARN Issue ##33248 - GlueOperator IAM ARN Aug 15, 2023
@vincbeck vincbeck changed the title Issue ##33248 - GlueOperator IAM ARN Add parameter to pass role ARN to GlueJobOperator Aug 15, 2023
@vincbeck vincbeck merged commit 60df705 into apache:main Aug 15, 2023
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:amazon-aws AWS/Amazon - related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GlueOperator: iam_role_arn as a parameter
2 participants