From 5d8d7e48af470440ae5d5833d43ecd087f910779 Mon Sep 17 00:00:00 2001 From: Raza Jafri Date: Tue, 22 Mar 2022 11:25:46 -0700 Subject: [PATCH 1/3] removing mention of shims folder in docs Signed-off-by: Raza Jafri --- CONTRIBUTING.md | 8 +++----- docs/dev/shims.md | 7 +++---- 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 22f426293ff..adc32c6db6c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -160,11 +160,9 @@ Spark version, open [Maven tool window](https://www.jetbrains.com/help/idea/2021 select one of the `release3xx` profiles (e.g, `release320`) for Apache Spark 3.2.0, and click "Reload" if not triggered automatically. -There is a known issue with the shims/spark3xx submodules. After being enabled once, a module such as shims/spark312 -may remain active in IDEA even though you explicitly disable the Maven profile `release312` in the Maven tool window. -With an extra IDEA shim module loaded the IDEA internal build "Build->Build Project" is likely to fail -(whereas it has no adverse effect on Maven build). As a workaround, locate the pom.xml under the extraneous IDEA module, -right-click on it and select "Maven->Ignore Projects". +There is a known issue where, even after selecting a different maven profile in maven submenu, the source folders from +previously selected profile may remain active. To get around this you have to manually reload the maven project from +the maven side menu If you see Scala symbols unresolved (highlighted red) in IDEA please try the following steps to resolve it: - Make sure there are no relevant poms in "File->Settings->Build Tools->Maven->Ignored Files" diff --git a/docs/dev/shims.md b/docs/dev/shims.md index 97bf661c1fe..89090c8c795 100644 --- a/docs/dev/shims.md +++ b/docs/dev/shims.md @@ -89,10 +89,9 @@ by having the documented facade classes with a shim specifier in their package n The second issue that every parent class/trait in the inheritance graph is loaded using the classloader outside Plugin's control. Therefore, all this bytecode must reside in the conventional jar location, and it must be bitwise-identical across *all* shims. The only way to keep the source code for shared functionality unduplicated, -(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated either in `shims/spark3xx` submodules or over -`sql-plugin/src/main/3*/scala` source code roots) is to delay inheriting `ShuffleManager` until as late as possible, -as close as possible to the facade class where we have to split the source code anyways. Use traits as much -as possible for flexibility. +(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated in `sql-plugin/src/main/3*/scala` source code roots) +is to delay inheriting `ShuffleManager` until as late as possible, as close as possible to the facade class where we +have to split the source code anyway. Use traits as much as possible for flexibility. ### Late Initialization of Public Classes' Ancestors From 4505e9e9ca0e86affdc4b378e21cf2f64d230128 Mon Sep 17 00:00:00 2001 From: Raza Jafri Date: Tue, 22 Mar 2022 11:39:19 -0700 Subject: [PATCH 2/3] empty commit to stop requiring CI Signed-off-by: Raza Jafri From 85ce638c5eccd5cae4fbf3ead1946cbdf8455bc9 Mon Sep 17 00:00:00 2001 From: Raza Jafri Date: Tue, 22 Mar 2022 14:34:29 -0700 Subject: [PATCH 3/3] Update CONTRIBUTING.md Co-authored-by: Jason Lowe --- CONTRIBUTING.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index adc32c6db6c..723858095bb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -160,9 +160,9 @@ Spark version, open [Maven tool window](https://www.jetbrains.com/help/idea/2021 select one of the `release3xx` profiles (e.g, `release320`) for Apache Spark 3.2.0, and click "Reload" if not triggered automatically. -There is a known issue where, even after selecting a different maven profile in maven submenu, the source folders from -previously selected profile may remain active. To get around this you have to manually reload the maven project from -the maven side menu +There is a known issue where, even after selecting a different Maven profile in the Maven submenu, the source folders from +a previously selected profile may remain active. To get around this you have to manually reload the Maven project from +the Maven side menu. If you see Scala symbols unresolved (highlighted red) in IDEA please try the following steps to resolve it: - Make sure there are no relevant poms in "File->Settings->Build Tools->Maven->Ignored Files"