You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In the "simplify expressions" rule, we evaluate and fold expressions that result in scalar values and we depend on the physical expression crate to do this, which is fine, but we also depend on the core crate for create_physical_expr and we can't have the new datafusion-optimizer crate depend on core.
Describe the solution you'd like
It looks like we should be able to move create_physical_expr and associated functions to the datafusion-physical-expr crate.
andygrove
changed the title
Logical optimizer rule "simplify expressions" should not depend on physical expressions
Logical optimizer rule "simplify expressions" should not depend on physical planner and expressions
May 23, 2022
I think it is important for expression simplification that expressions can be partially evaluated at optimization time.
In terms of evaluating on logical plan Exprs directly, I agree there is no theoretical reason this can not be done, but if we did so it will result in dual expression evaluation systems that would have be kept in sync. This might be especially hard if someone wanted to use a different physical expression evaluation system
I understand the desire to remove the direct code dependence on the execution engine from the optimizer -- perhaps we can do so via a trait -- for example, we could pass some sort of dyn ExprEvaluator to the optimizer for use in simplify expressions and elsewhere.
By default, DataFusion could provide the evaluator based on PhysicalExpr -- and for those users of the optimizer itself they could decide the most appropriate way to provide expression evaluation to the optimizer
@alamb Now that I have started looking at this more closely it turns out to be less of an issue than I first thought. We just need to move create_physical_expr from core to the physical-expr crate. I have updated the issue description.
andygrove
changed the title
Logical optimizer rule "simplify expressions" should not depend on physical planner and expressions
Logical optimizer rule "simplify expressions" should not depend on the core datafusion crate
Jun 2, 2022
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In the "simplify expressions" rule, we evaluate and fold expressions that result in scalar values and we depend on the physical expression crate to do this, which is fine, but we also depend on the core crate for
create_physical_expr
and we can't have the newdatafusion-optimizer
crate depend on core.Describe the solution you'd like
It looks like we should be able to move
create_physical_expr
and associated functions to thedatafusion-physical-expr
crate.Describe alternatives you've considered
None
Additional context
This is related to #2345
The text was updated successfully, but these errors were encountered: