-
-
Notifications
You must be signed in to change notification settings - Fork 404
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cuDFInterface to work with cuDF GPU dataframes and cupy support for XArrayInterface #3982
Conversation
2905948
to
baa7e65
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool! Yes, it will be very important to be able to test this in CI...
I think we could do this with import cupy
import cudf
cdf = cudf.from_pandas(...)
ca = cupy.array(cdf["col1"].to_gpu_array(), copy=False)
hist, bin_edges = cupy.histogram(ca) We'll probably also want to add a cupy interface for grid data eventually as well, so I'd be in favor of requiring that both |
00a896e
to
e46a44f
Compare
This PR now also supports cupy backed xarrays using a two line change. |
Okay, this PR is now almost ready. I've added tests, which won't run on Travis of course but can be run locally. I'll add some additional tests for datashader integration. There are one major class of tests that are failing which is groupby/aggregation tests. It seems like cuDF groupby(..., sort=False) results in reverse sorting which seems quite odd. I'll keep looking at that but in the worst case I'll disable those tests until the issue has been fixed upstream. |
fd21ebe
to
bd00e66
Compare
bd00e66
to
dd32306
Compare
Co-Authored-By: James A. Bednar <jbednar@users.noreply.github.com>
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
This PR adds a new data interface to allow HoloViews to work directly with cuDF GPU dataframes.
Datashader support:
Other things to do:
One major thing to figure out is how we will set up CI tests for the GPU.