Skip to content

Commit

Permalink
[SPARK-49249][SQL][FOLLOWUP] Rename `spark.sql.artifact.isolation.(al…
Browse files Browse the repository at this point in the history
…ways.apply.classloader -> alwaysApplyClassloader)`

### What changes were proposed in this pull request?

This is a follow-up to rename a new configuration to comply Apache Spark config namespace.
- apache#48120

### Why are the changes needed?

Currently, `spark.sql.artifact.isolation.always.apply.classloader` config name introduces redundant namespace `spark.sql.artifact.isolation.always.*` and `spark.sql.artifact.isolation.always.apply.*`.
```
- spark.sql.artifact.isolation.always.apply.classloader
+ spark.sql.artifact.isolation.alwaysApplyClassloader
```

Since we have `spark.sql.artifact.isolation.enabled` already, we had better keep the above in the same namespace `spark.sql.artifact.isolation.*`.

### Does this PR introduce _any_ user-facing change?

No, this is a newly added configuration at Spark 4.0.0

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#49101 from dongjoon-hyun/SPARK-49249.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
  • Loading branch information
dongjoon-hyun authored and LuciferYang committed Dec 7, 2024
1 parent fff6793 commit bb17665
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion python/pyspark/sql/connect/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -1044,7 +1044,7 @@ def _start_connect_server(master: str, opts: Dict[str, Any]) -> None:
default_conf = {
"spark.plugins": "org.apache.spark.sql.connect.SparkConnectPlugin",
"spark.sql.artifact.isolation.enabled": "true",
"spark.sql.artifact.isolation.always.apply.classloader": "true",
"spark.sql.artifact.isolation.alwaysApplyClassloader": "true",
}

if "SPARK_TESTING" in os.environ:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3999,7 +3999,7 @@ object SQLConf {
.createWithDefault(true)

val ARTIFACTS_SESSION_ISOLATION_ALWAYS_APPLY_CLASSLOADER =
buildConf("spark.sql.artifact.isolation.always.apply.classloader")
buildConf("spark.sql.artifact.isolation.alwaysApplyClassloader")
.internal()
.doc("When enabled, the classloader holding per-session artifacts will always be applied " +
"during SQL executions (useful for Spark Connect). When disabled, the classloader will " +
Expand Down

0 comments on commit bb17665

Please sign in to comment.