Advertise CPU sort order and partitioning expressions to Catalyst [databricks]

[jlowe]

Fixes #3713

Changes the GPU nodes in the Catalyst plan to advertise CPU SortOrder and Partitioning classes and child expressions. This helps the Spark code better recognize that portions of the plan are still compatible from a sorting and partitioning perspective even after the GPU has updated the plan. Spark has a number of places in the code where it is matching against specific Spark CPU partition classes or checking if CPU sort/partition expressions match, Returning the CPU expressions from the GPU nodes' outputPartitioning, outputOrdering, requiredChildDistribution and requiredChildOrderingmethods helps us pass these checks in Apache Spark code.

This is accomplished by passing both the GPU and CPU expressions to exec nodes that need to override their output ordering or partitioning. It uses the GPU expressions during computation but "advertises" the CPU expressions from the Catalyst methods to query the node's intentions.

[jlowe]

This depends on #3691. Marking as a draft until the dependency is merged.

[jlowe]

build

[jlowe]

build

[abellina]

Overall this makes sense, I had one comment on whether we wanted to fix or add more explicit asserts if the gpu order was complex (required an eval). I am not sure what the outcome of the failure is here => failed query, or less performant.

[tgravescs]

build

[jlowe]

build

[jlowe]

build

[jlowe]

Ran into a couple of issues with passing the CPU expressions as part of the regular arguments to the case classes:

• These arguments would appear in the plan, being mostly redundant with the GPU form of the expressions
• The assertOnGpu code would "find" these expressions and assert that they're not on the GPU

By passing them as a separate parameter list to the case classes, they no longer appear in the expressions list for these classes. That avoids the CPU expressions appearing in the plan explain output and being seen by generic tree traversal code.

[jlowe]

build

[tgravescs]

db 8.2 failures:
` pyspark.sql.utils.IllegalArgumentException: The expression knownfloatingpointnormalized(normalizenanandzero(a#133227)) AS a#133227 is not columnar class org.apache.spark.sql.catalyst.expressions.Alias

`

18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[Left-Float][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[Left-Double][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[LeftSemi-Float][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[LeftSemi-Double][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[LeftAnti-Float][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_right_broadcast_nested_loop_join_condition_missing[LeftAnti-Double][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_left_broadcast_nested_loop_join_condition_missing[Right-Float][IGNORE_ORDER({'local': True}), INCOMPAT]
18:34:16  FAILED ../../src/main/python/join_test.py::test_left_broadcast_nested_loop_join_condition_missing[Right-Double][IGNORE_ORDER({'local': True}), INCOMPAT]

[jlowe]

build

[jlowe]

build

[gerashegalov]

nits:

prefer Seq.empty to Nil.

could also just have cpuLeftKeys ++ cpuRightKeys ?

[gerashegalov]

nit: why :: Nil

[gerashegalov]

nit: Seq.empty