support GpuSortArray #2616
support GpuSortArray #2616
Conversation
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/collectionOperations.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/collectionOperations.scala
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
Outdated
Show resolved
Hide resolved
| TypeSig.ARRAY.nested(TypeSig.all)), | ||
| ("ascendingOrder", TypeSig.BOOLEAN, TypeSig.BOOLEAN)), | ||
| (in, conf, p, r) => new BinaryExprMeta[SortArray](in, conf, p, r) { | ||
| override def convertToGpu(lhs: Expression, rhs: Expression): GpuExpression = { |
There was a problem hiding this comment.
If we don't support literal arrays we have to tag it here so we fall back to the CPU.
There was a problem hiding this comment.
I think we can support literal arrays. And I made up the corresponding test.
There was a problem hiding this comment.
It gets hard to test this because if both parts are literals spark will optimize it before we ever see it and replace the entire thing with a literal array that is already sorted. There is a kind of hidden config to disable this, but it would take some digging to find it.
There was a problem hiding this comment.
What you wrote is good enough for me. The test is likely not testing what we expect it to, but the code looks like it should work fine so I am OK with it.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/collectionOperations.scala
Outdated
Show resolved
Hide resolved
Co-authored-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: sperlingxx <lovedreamf@gmail.com>
|
build |
Closes #2557. This PR is to support sort_array function on GPU with cuDF method ColumnVector.listSortRows. Signed-off-by: sperlingxx <lovedreamf@gmail.com> Co-authored-by: Gera Shegalov <gshegalov@nvidia.com>
Closes #2557.
This PR is to support sort_array function on GPU with cuDF method
ColumnVector.listSortRows.