fix(rust, python): Clear window cache after evaluate predication expr #10505
Conversation
github-actions
bot
added
fix
| @@ -35,6 +35,9 @@ impl Executor for FilterExec { | |||
| state.insert_has_window_function_flag() | |||
| } | |||
| let s = self.predicate.evaluate(&df, state)?; | |||
| if state.cache_window() { | |||
There was a problem hiding this comment.
Might replace this by if has_window to reduce the overhead of atomic load? 🤔
There was a problem hiding this comment.
Yes an you put the cache_window and the clear_window_expr behind a if self.has_windows?
There was a problem hiding this comment.
Do you mean to replace it with
if self.has_window() && state.cache_window() {
state.clear_window_expr_cache();
}There was a problem hiding this comment.
This is what we do during projection.
if has_windows {
state.clear_window_expr_cache();
}I believe the && state.cache_window() is redundant as it should always be set if has_windows is true.
| @@ -35,6 +35,9 @@ impl Executor for FilterExec { | |||
| state.insert_has_window_function_flag() | |||
| } | |||
| let s = self.predicate.evaluate(&df, state)?; | |||
| if state.cache_window() { | |||
There was a problem hiding this comment.
Yes an you put the cache_window and the clear_window_expr behind a if self.has_windows?
|
Thanks @ritchie46 for the review. I have updated this, but I'm not sure if I totally understand all comments, so feel free to append new comments. |
|
The redundant |
|
Alright! 🙌 |
This fixes #10499.