-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Review an improvement plan for Bigquery operators #231
Comments
Sensors:
Operators:Helper classes:
Helper classes for Check operators:
They seem to follow naming convention. Query operators:
Only Proposition:
Hooks:PEP 249
Currently Proposition:
Other:
General questions:
General thoughts:
|
I think BQ belongs to most used operators and we should preserve backward compatibility. There are two ways to approach this: build object from kwargs or keep key words arguments for time being and rise an error when "object" was not passed. Personally I prefer the second option as it's easier to remove something in future than add a "builder" now (that will be remove - time wasting). I would suggest also adding "create_my_object" method for easier transfer between old and new approach. Regarding Cursor - I think we should have two hook / operators. One for average users (implementing what python BQ clients has) and one for those who know what a cursor is. This should simplify the refactor. |
I agree that backwards compatibility is really important for BQ. My proposal is to only leave the QueryJobConfig object as parameter and if it is missing, use builder that can be applied to kwargs (raising deprecation warning). We should be able to pass both the We could also think about implementing template handling for QueryJobConfig-like object (if it does not work like that for now). We can vote on those approaches when everyone chimes in :). I can be convinced otherwise. For the cursor - I think indeed it should be separated out. |
Is it done? @TobKed |
@nuclearpinguin, not yet, to do this I am working on #126 with with @mik-laj to be sure that nothing is broken when any changes will be introduced. |
No description provided.
The text was updated successfully, but these errors were encountered: