Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: eliminate useless join | convert inner to outer when condition is true #6443

Merged
merged 2 commits into from
May 26, 2023

Conversation

jackwener
Copy link
Member

@jackwener jackwener commented May 25, 2023

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

  • eliminate useless join
  • convert inner to outer when condition is true
  • simplify code

Are these changes tested?

  • UT
  • sqllogicaltest

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels May 25, 2023
@jackwener jackwener force-pushed the eliminate branch 2 times, most recently from 42062e5 to 297ff14 Compare May 26, 2023 06:38
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me -- thank you @jackwener

I wonder if you actually hit any of these queries somehow or if this is a theoretical concern?

datafusion/expr/src/logical_plan/display.rs Show resolved Hide resolved
datafusion/optimizer/src/eliminate_join.rs Outdated Show resolved Hide resolved
@jackwener
Copy link
Member Author

jackwener commented May 26, 2023

I wonder if you actually hit any of these queries somehow or if this is a theoretical concern?

Yes, I hit this problem when I add optimization for Apache Doris. And It also is a theoretical concern.

we should do this optimization because inner join should exist real edge (equal/non-equal condition) in semantic, this rule convert fake inner join to be cross join.

If we don't handle it, some rule may ignore this corner case in some optimization, it may occur BUG. Such as Join Reorder....

Paper: On the Correct and Complete Enumeration of the Core Search Space describe this conditon is Degenerate Predicates

image

In the future, we should handle more Degenerate Predicates condition

@alamb alamb merged commit a9f0c7a into apache:main May 26, 2023
@jackwener jackwener deleted the eliminate branch May 26, 2023 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants