Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added databricks_conn_id as templated field #24945

Merged
merged 2 commits into from
Jul 11, 2022
Merged

Added databricks_conn_id as templated field #24945

merged 2 commits into from
Jul 11, 2022

Conversation

skadyan
Copy link
Contributor

@skadyan skadyan commented Jul 10, 2022

Closes: #24938

Copy link
Contributor

@josh-fell josh-fell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@josh-fell
Copy link
Contributor

Maybe all of the Databricks operators would benefit from this templating across the board? Just thinking others might have a similar use case to yours but use different operators. Not required for this PR of course (but feel free to add if you'd like), just tossing the idea out there.

@potiuk
Copy link
Member

potiuk commented Jul 11, 2022

Maybe all of the Databricks operators would benefit from this templating across the board? Just thinking others might have a similar use case to yours but use different operators. Not required for this PR of course (but feel free to add if you'd like), just tossing the idea out there.

Or maybe (just a thought) we could automatically make ALL conn_id fields "templated" :) across all operators in all providers.

That would require likely some trickery.

@potiuk potiuk merged commit 8dfe7bf into apache:main Jul 11, 2022
potiuk pushed a commit to alexott/airflow that referenced this pull request Jul 26, 2022
* make `databricks_conn_id` templated in all operators (follow up from apache#24945)
* rename `python_named_parameters` to `python_named_params` in `DatabricksRunNowOperator`
  to match changes in the Databricks jobs API (it shouldn't break existing DAGs as noone
  complained yet after the changes were made)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for dynamic databricks connection id
3 participants