Treat unbounded windows as truly non-finite. #8802
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Follow-up to rapidsai/cudf#13727. This change addresses the slowness in window aggregations for windows defined as `[UNBOUNDED PRECEDING, UNBOUNDED FOLLOWING]`. Before this change, unbounded row window bounds were interpreted as finite values, e.g. `[MAX_INT, MAX_INT]`. While this might be technically indistinguishable from a fully unbounded window, it causes the optimization in rapidsai/cudf/pull/13727 not to be triggered, because the window bounds are still finite. The change in this PR allows the plugin to detect unbounded windows, and mark them as such for `libcudf`. The `libcudf` window function primitives can then detect fully unbounded windows, and use a faster/optimized path for execution. Preliminary test results indicate that `[UNBOUNDED PRECEDING, UNBOUNDED FOLLOWING]` window function computations over 1B rows and thousands of groups are sped up by a factor of 10-14x over the previous/naive GPU implementation. Signed-off-by: MithunR <mythrocks@gmail.com>
|
Build |
|
rapidsai/cudf#13727 has been merged. We should be able to re-run tests after the dependency makes its way to |
revans2
approved these changes
Jul 26, 2023
|
Build |
|
Hmm. This error is concerning. Investigating now. |
|
It looks like Note: This test runs locally, with rapidsai/cudf#13727 installed. |
|
Build |
|
This is merged. Thank you for reviewing, @revans2. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on rapidsai/cudf#13727.
This change addresses the slowness in window aggregations for windows defined as
[UNBOUNDED PRECEDING, UNBOUNDED FOLLOWING]. Before this change, unbounded row window bounds were interpreted as finite values, e.g.[MAX_INT, MAX_INT]. While this might be technically indistinguishable from a fully unbounded window, it causes the optimization in rapidsai/cudf/pull/13727 not to be triggered, because the window bounds are still finite.The change in this PR allows the plugin to detect unbounded windows, and mark them as such for
libcudf. Thelibcudfwindow function primitives can then detect fully unbounded windows, and use a faster/optimized path for execution.Preliminary test results indicate that
[UNBOUNDED PRECEDING, UNBOUNDED FOLLOWING]window function computations over 1B rows and thousands of groups are sped up by a factor of 10-14x over the previous/naive GPU implementation.The prior tests cover the UNBOUNDED scenario already. An additional test was added.