You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Originally reported by @revans2 Describe the bug #4029 fix to rely exclusively on the package phase to avoid concurrent installs to the local maven repo causes the test compile in the second single-threaded phase of buildall resuming from dist https://github.com/NVIDIA/spark-rapids/blob/branch-21.12/build/buildall#L184 to pull non-existent or stale aggrgeator artifact during test compile.
Tests should not rely on aggregator #3932. But this we need a workaround in the meantime.
Steps/Code to reproduce bug
Remove you local m2 cache or point maven.repo.local to an empty dir
run
./build/buildall
Failed to execute goal on project rapids-4-spark_2.12: Could not resolve dependencies for project com.nvidia:rapids-4-spark_2.12:jar:21.12.0-SNAPSHOT: Could not find artifact com.nvidia:rapids-4-spark-aggregator_2.12:jar:spark301:21.12.0-SNAPSHOT in snapshots-repo (https://oss.sonatype.org/content/repositories/snapshots)
Expected behavior
should build correctly. We can leave one shim build thread using install without any risk of concurrent install to the local maven repo.
Environment details (please complete the following information)
local dev
Make sure that the minimum spark version classifier of the aggregator (typically 301) is `install`ed
Closes#4053
Signed-off-by: Gera Shegalov <gera@apache.org>
Originally reported by @revans2
Describe the bug
#4029 fix to rely exclusively on the
package
phase to avoid concurrent installs to the local maven repo causes the test compile in the second single-threaded phase of buildall resuming from dist https://github.com/NVIDIA/spark-rapids/blob/branch-21.12/build/buildall#L184 to pull non-existent or stale aggrgeator artifact during test compile.This is because the maven invocation resuming from dist can no longer see dependencies from the first phase unless they are installed. The dist build itself was not affected by this because we bypass Maven when picking up aggregators for the final parallel worlds assembly https://github.com/NVIDIA/spark-rapids/blob/branch-21.12/dist/maven-antrun/build-parallel-worlds.xml#L73.
Tests should not rely on aggregator #3932. But this we need a workaround in the meantime.
Steps/Code to reproduce bug
Remove you local m2 cache or point maven.repo.local to an empty dir
run
Expected behavior
should build correctly. We can leave one shim build thread using
install
without any risk of concurrent install to the local maven repo.Environment details (please complete the following information)
local dev
Additional context
#3932
The text was updated successfully, but these errors were encountered: