[FEA] Support Databricks 8.2 ML Runtime

[sameerz]

Is your feature request related to a problem? Please describe.
Databricks now has an 8.2 ML runtime that supports GPUs and runs Spark 3.1.1.

Describe the solution you'd like
The Spark plugin should support the Databricks 8.1 ML runtime

Describe alternatives you've considered
Databricks 8.0 ML does not appear to support GPUs.

Additional context
Ideally we will support an 8.x LTS version, but it appears to be too early in the 8.x line for a long term support version. Let's start with 8.2 ML, we may need to change the target supported version later.

[tgravescs]

the 8.2 ML driver version is the same as 8.1:
NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.0

[tgravescs]

Note it looks like for the flatMapGroupsInPandas we have to turn off adaptive otherwise we hit the stack overflow issues with the estimator.
spark.conf.set("spark.sql.adaptive.enabled", "false")

[tgravescs]

supported with #2381