No Spark Shim Provider found for 3.3.0

[j-buczak]

Describe the bug
I have build Spark for Velox and want to use it with s3 bucket. I have added gluten-path/cpp/build/releases to PATH and move gluten-path/package/velox/spark33/target/gluten-spark3.3_2.12-1.0.0-SNAPSHOT-jar-with-dependencies.jar to Spark jars folder.

When running TPC-H data generation code I had such a bug:

Exception in thread "main" java.lang.IllegalStateException: No Spark Shim Provider found for 3.3.0
        at io.glutenproject.sql.shims.SparkShimLoader$.loadSparkShimProvider(SparkShimLoader.scala:61)
        at io.glutenproject.sql.shims.SparkShimLoader$.getSparkShimProvider(SparkShimLoader.scala:76)
        at io.glutenproject.sql.shims.SparkShimLoader$.getSparkShims(SparkShimLoader.scala:32)
...

Adding path-to-gluten/shims/target/spark-sql-columnar-shims-1.0.0-SNAPSHOT-3.3.jar to Sprak jars folder didn't work.

Expected behavior
No bug or beter instructions how to include necessary dirs.

[zhouyuan]

Hi @j-buczak

Spark331 is released on Oct 25, 2022, with many bug fixes(https://spark.apache.org/releases/spark-release-3-3-1.html)

Can you please have a try with Spark331 release? it can be downloaded here:
https://spark.apache.org/downloads.html

Note:
Gluten dropped Spark-330 support as of this patch:
#534

-yuan

[j-buczak]

Thanks, it solves this issue!