-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove derive(Copy)
from Operator
#11132
Conversation
derive(Copy) from
Operator`derive(Copy)
from Operator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @samuelcolvin -- I think this change makes sense to me
Another option which might be less impactful and likely be more performance could be:
- Leave
#[derive(Copy)]
forOperator
- Model custom operators using an
Arc
(soCopy
would be relatively cheap)
So like
enum Operator {
...
Custom(Arc<str>)
}
While making Arc
Copy
is not typically done, it might be ok in this case to reduce API churn 🤔
What do you think?
cc @jayzhan211
This suggests it's not safe to implement Perhaps I'm missing something? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me - I don't see any other way around this change
Thanks @samuelcolvin
d544a63
to
b86655f
Compare
🚀 |
Thanks @samuelcolvin |
This reverts commit b468ba7.
* remove derive Copy from Operator * fix formatting and more cloning * fix remaining case * fix docs case * fix operator borrows
Rationale for this change
This is the first step towards solving #10981 (comment) - providing a general way to support more operators without having them all added to datafusion core.
The idea is that this would allow
Operator
to include another member of the union that in turn could includingCustom(String)
, but also potentially a function that defines extra operators' signatures, without having to hard code them intodatafusion/expr/src/type_coercion/binary.rs::signature
.I'm creating this as a standalone PR, so:
To see if there's willingness to accept this change before I go forwardThe rest of the work is ready for review - Custom operator support #11137What changes are included in this PR?
Copy
trait is no longer derived forOperator
(this meansOperator
better matches most other related logical plan structs)Operator
are updated to minimise changes while avoid clones where possibleAre these changes tested?
Yes, existing tests (should) pass.
Are there any user-facing changes?
Some public function signatures might have changed, if you're willing to consider this change, let me know if you want me to revert some of those public signature changes.