Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ [ClickBench] Remove redundant projections #1697

Closed
joocer opened this issue May 28, 2024 · 1 comment
Closed

✨ [ClickBench] Remove redundant projections #1697

joocer opened this issue May 28, 2024 · 1 comment
Labels
⚙️ Query Optimizer Component Label - Optimizer

Comments

@joocer
Copy link
Contributor

joocer commented May 28, 2024

We are diagnosing performance issues with the ClickBench benchmark and identified this inefficiency. It is unlikely to be material whilst on a single thread executor, but when multithreaded, the cost of the additional call will increase.

This plan has two redundant projections, after the scan we only have the URL column so the two projections aren't doing anything

└─ EXIT
   └─ PROJECT (URL)
      └─ SUBQUERY AS a
         └─ LIMIT (1000000)
            └─ PROJECT (URL)
               └─ FILTER (URL LIKE '%google%')
                  └─ SCAN (hits) [URL]
@joocer
Copy link
Contributor Author

joocer commented May 28, 2024

In trying to address this a bug was found in the projection pushdown strategy - it changes the type of the columns attribute in the Scan, SubQuery and Union operators to be lists of FlatColumns rather than LogicalColumns.

Fixing this is likely to have knock-on impacts to these operators and potentially to other optimization strategies.

@joocer joocer changed the title ✨ Remove redundant projections ✨ [ClickBench] Remove redundant projections May 29, 2024
joocer added a commit that referenced this issue May 29, 2024
joocer added a commit that referenced this issue May 29, 2024
@joocer joocer closed this as completed May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⚙️ Query Optimizer Component Label - Optimizer
Projects
None yet
Development

No branches or pull requests

1 participant