Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Columnar (non-GPU) support for scala UDFs #306

Closed
revans2 opened this issue Jun 29, 2020 · 2 comments
Closed

[FEA] Columnar (non-GPU) support for scala UDFs #306

revans2 opened this issue Jun 29, 2020 · 2 comments
Labels
feature request New feature or request P2 Not required for release performance A performance related task/issue

Comments

@revans2
Copy link
Collaborator

revans2 commented Jun 29, 2020

Scala UDFs are implemented using the ScalaUDF expression class. It is unlikely that we are going to be able to support running these UDFs on the GPU in the near future. So until then we should look at what it would take to pull back to the CPU just the columns we need to process the UDF. Process it on the CPU in a tight columnar loop, and then send the result back to the GPU.

There is a lot of room for follow on work to look at things like.

We will need to do some performance tests to see if this is viable or if we should do something else. Are there rules about when this is good and when this is bad, like if there are too many columns or the columns are not fixed width does it make since to do this?

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jun 29, 2020
@sameerz sameerz added P2 Not required for release performance A performance related task/issue and removed ? - Needs Triage Need team to review and classify labels Aug 11, 2020
@jlowe
Copy link
Member

jlowe commented Jan 14, 2022

Has this been addressed by #3897?

@mattahrens
Copy link
Collaborator

Closing as work has been completed

tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
[auto-merge] bot-auto-merge-branch-22.06 to branch-22.08 [skip ci] [bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request P2 Not required for release performance A performance related task/issue
Projects
None yet
Development

No branches or pull requests

4 participants