[SPARK-46875][SQL] When the mode is null, a NullPointException should not be thrown

[panbingkun]

What changes were proposed in this pull request?

The pr aims to provide better prompts when option's mode is null.

Why are the changes needed?

In the original logic, if the mode is null, Spark will throw a NullPointerException, which is obviously unfriendly to the user.

val cars = spark.read
      .format("csv")
      .options(Map("header" -> "true", "mode" -> null))
      .load(testFile(carsFile))
cars.show(false)

Before:

Cannot invoke "String.toUpperCase(java.util.Locale)" because "mode" is null
java.lang.NullPointerException: Cannot invoke "String.toUpperCase(java.util.Locale)" because "mode" is null
	at org.apache.spark.sql.catalyst.util.ParseMode$.fromString(ParseMode.scala:50)
	at org.apache.spark.sql.catalyst.csv.CSVOptions.$anonfun$parseMode$1(CSVOptions.scala:105)
	at scala.Option.map(Option.scala:242)
	at org.apache.spark.sql.catalyst.csv.CSVOptions.<init>(CSVOptions.scala:105)
	at org.apache.spark.sql.catalyst.csv.CSVOptions.<init>(CSVOptions.scala:49)
	at org.apache.spark.sql.execution.datasources.csv.CSVFileFormat.inferSchema(CSVFileFormat.scala:60)

After:
It will fall back to PermissiveMode mode and then display the data normally, as shown below:

18:54:06.727 WARN org.apache.spark.sql.catalyst.util.ParseMode: mode is null and not a valid parse mode. Using PERMISSIVE.

+----+-----+-----+----------------------------------+-----+
|year|make |model|comment                           |blank|
+----+-----+-----+----------------------------------+-----+
|2012|Tesla|S    |No comment                        |NULL |
|1997|Ford |E350 |Go get one now they are going fast|NULL |
|2015|Chevy|Volt |NULL                              |NULL |
+----+-----+-----+----------------------------------+-----+

Does this PR introduce any user-facing change?

Yes, When mode is null, it fallback to PermissiveMode instead of throwing a NullPointerException.

How was this patch tested?

• Add new UT.
• Pass GA.

Was this patch authored or co-authored using generative AI tooling?

No.

[MaxGekk]

Shouldn't we show an error on the wrong error instead of silently fallback to the PERMISSIVE mode? cc @HyukjinKwon

I mean both case - null and a wrong mode.

[panbingkun]

@MaxGekk
In the current logic, when mode is null, the following error is displayed:

[panbingkun]

I suggest we have two options as follows:
1.If the mode is null, it will fall back to PermissiveMode, just like setting a non-null error value for the mode.
2.If mode is null, we throw a user-friendly error prompt instead of a NullPointerException.

The fix change for this PR adopts the first option.

[MaxGekk]

Probably, the fallback to the PERMISSIVE mode looks much more consistent with current approach of not-recognised value. We could stick on it in your PR, but I think it is better to change behaviour and throw appropriate error in both cases.

[MaxGekk]

+1, LGTM. Merging to master.
Thank you, @panbingkun.