-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix filter pushdown for DELETE queries #13705
Conversation
mbasmanova
commented
Nov 14, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@mbasmanova i guess you need to rebase, unrelated build issue |
5d01ecb
to
57487b6
Compare
{ | ||
return true; | ||
if (!tableLayoutHandle.isPresent()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need a requireNonNull
check for the arguments here or are they handled in the caller?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I've seen in Presto codebase, requireNonNull
is used only in constructors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, the question was more geared towards do we need to null check using either requireNonNull
or explicit arg==null
.
@@ -1677,6 +1677,15 @@ public void testMetadataDelete() | |||
|
|||
assertQuery("SELECT * from test_metadata_delete", "SELECT orderkey, linenumber, linestatus FROM lineitem WHERE linestatus<>'F' or linenumber<>3"); | |||
|
|||
// TODO This use case can be supported |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please explain what this //TODO
means.
Is this a failing test that needs to be fixed later on or some functionality that needs to be added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ajaygeorge This is missing functionality. This query should work fine, but it doesn't. Since this is existing behavior, I'm keeping it like that for pushdown-query code path.
57487b6
to
c7929e7
Compare
As a side note, I have always wondered what is the reasoning behind presto's choice behind accepting
|
I don't have an answer to that. |
Do we even need an If it has been the standard that we have been following then it makes sense to keep it as is. LGTM as well ! |
Yes, this is how it is currently done in the codebase. |
c7929e7
to
ce14c62
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one minor comment
{ | ||
return true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have an enumeration of comments to explain the cases when metadata delete can happen and when it can't? By looking at the code, it seems that it's not an all or nothing call given the filter could be "simple" or very "complicated" but turning to be simple if we do constant folding or even evaluation. Of course, there is a limit we can push. We can only inference the delete if the filter is not "that complicated" .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@highker James, thank you review.
Strictly speaking we should be able to support metadata delete for any filter that doesn't touch non-partition columns. Today we only support it for range filters on partition columns though (regardless of whether filter pushdown is on or off). I'll add a comment and a TODO.
// Allow metadata delete for range filters on partition columns.
// TODO Add support for metadata delete for any filter on partition columns.
ce14c62
to
a0cf226
Compare