Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Minor: avoid clone while calculating union equivalence properties #12722

Merged
merged 4 commits into from
Oct 4, 2024
Merged

Minor: avoid clone while calculating union equivalence properties #12722

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
988d23a
Minor: avoid clone while calculating union equivalence properties
alamb Oct 2, 2024
a16c7e7
Update datafusion/physical-expr/src/equivalence/properties.rs
alamb Oct 3, 2024
ce400c6
Merge remote-tracking branch 'apache/main' into alamb/less_clone
alamb Oct 3, 2024
8d5940e
fmt
alamb Oct 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions datafusion/physical-expr/src/equivalence/properties.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ use crate::{

use arrow_schema::{SchemaRef, SortOptions};
use datafusion_common::tree_node::{Transformed, TransformedResult, TreeNode};
use datafusion_common::{plan_err, JoinSide, JoinType, Result};
use datafusion_common::{internal_err, plan_err, JoinSide, JoinType, Result};
use datafusion_expr::interval_arithmetic::Interval;
use datafusion_expr::sort_properties::{ExprProperties, SortProperties};
use datafusion_physical_expr_common::utils::ExprPropertiesNode;
Expand Down Expand Up @@ -1677,14 +1677,22 @@ pub fn calculate_union(
) -> Result<EquivalenceProperties> {
// TODO: In some cases, we should be able to preserve some equivalence
// classes. Add support for such cases.
let mut init = eqps[0].clone();
let mut iter = eqps.into_iter();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new version also will return an error rather than panic if no equivalence properties are returned

let Some(mut acc) = iter.next() else {
return internal_err!(
"Cannot calculate EquivalenceProperties for a union with no inputs"
);
};

// Harmonize the schema of the init with the schema of the union:
if !init.schema.eq(&schema) {
init = init.with_new_schema(schema)?;
if !acc.schema.eq(&schema) {
acc = acc.with_new_schema(schema)?;
}
// Fold in the rest of the EquivalenceProperties:
for props in iter {
acc = calculate_union_binary(acc, props)?;
}
eqps.into_iter()
.skip(1)
.try_fold(init, calculate_union_binary)
Ok(acc)
}

#[cfg(test)]
Expand Down