-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix typos in DatabricksSubmitRunOperator #36248
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind adding tests for the DatabricksHook to cover this for the future please?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this bug was not detected in the tests, IMHO, we need to update them to avoid reintroducing it in the future. Could you please add a new test or update an existing one to cover this call?
@@ -521,7 +521,7 @@ def execute(self, context: Context): | |||
): | |||
# If pipeline_id is not provided, we need to fetch it from the pipeline_name | |||
pipeline_name = self.json["pipeline_task"]["pipeline_name"] | |||
self.json["pipeline_task"]["pipeline_id"] = self._hook.get_pipeline_id(pipeline_name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a mistery . MyPy uses a number of heuristics to find types of particular objects and sometime it gives up or sometimes drop types if they are too complex to analyse. Generally MyPy's behaveriour is often non-deterministic/difficult to reason about.
I can try, but it'll take me a bit to understand how the testing code works. Might need some pointers |
Best to look at other, neighbouring tests and do it in s similar way. TESTING.rst contains detailed description of different kinds of tests we have and how to run them locally |
Alright, took a bit but I got some good tests in there. Found a few more bugs along the way. |
Finally got around to trying this, realized I had a couple major typos in my original PR. Works to execute by pipeline name after these two changes.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.