DumpPlatformClassPath.dumpJDK9AndNewerBootClassPath does not work with openjdk16 on OSX

[eustas]

Description of the problem / feature request:

bazel build fails with host openjdk16

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

.bazelrc:

build --define=ABSOLUTE_JAVABASE=/redacted/jdk-16.jdk/Contents/Home
build --javabase=@bazel_tools//tools/jdk:absolute_javabase
build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase
build --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla
build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla

command: bazel build --sandbox_debug ...:all

result:

INFO: Analyzed 17 targets (0 packages loaded, 514 targets configured).
INFO: Found 17 targets...
ERROR: /private/var/tmp/_bazel_eustas/hhh/external/bazel_tools/tools/jdk/BUILD:346:14: Action external/bazel_tools/tools/jdk/platformclasspath.jar failed: (Exit 1): sandbox-exec failed: error executing command 
  (cd /private/var/tmp/_bazel_eustas/hhh/sandbox/darwin-sandbox/50/execroot/xxx_java && \
  exec env - \
    TMPDIR=/var/folders/f8/ttt/T/ \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_eustas/zzz/sandbox/darwin-sandbox/50/sandbox.sb /var/tmp/_bazel_eustas/install/hhh/process-wrapper '--timeout=0' '--kill_delay=15' /redacted/jdk-16.jdk/Contents/Home/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath_classes:/redacted/jdk-16.jdk/Contents/Home/lib/tools.jar DumpPlatformClassPath bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath.jar /redacted/jdk-16.jdk/Contents/Home) sandbox-exec failed: error executing command 
  (cd /private/var/tmp/_bazel_eustas/hhh/sandbox/darwin-sandbox/50/execroot/xxx_java && \
  exec env - \
    TMPDIR=/var/folders/f8/ttt/T/ \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_eustas/hhh/sandbox/darwin-sandbox/50/sandbox.sb /var/tmp/_bazel_eustas/install/hhh/process-wrapper '--timeout=0' '--kill_delay=15' /redacted/jdk-16.jdk/Contents/Home/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath_classes:/redacted/jdk-16.jdk/Contents/Home/lib/tools.jar DumpPlatformClassPath bazel-out/darwin-fastbuild/bin/external/bazel_tools/tools/jdk/platformclasspath.jar /redacted/jdk-16.jdk/Contents/Home)
Exception in thread "main" java.lang.IllegalAccessError: class DumpPlatformClassPath (in unnamed module @0x1b28cdfa) cannot access class com.sun.tools.javac.util.Context (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.util to unnamed module @0x1b28cdfa
	at DumpPlatformClassPath.dumpJDK9AndNewerBootClassPath(DumpPlatformClassPath.java:104)
	at DumpPlatformClassPath.main(DumpPlatformClassPath.java:67)
INFO: Elapsed time: 0,383s, Critical Path: 0,16s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

What operating system are you running Bazel on?

macOS Big Sur 11.2.2

What's the output of bazel info release?

release 4.0.0-homebrew

Have you found anything relevant by searching the web?

Likely, it works on other OS: https://stackoverflow.com/a/65161064

Any other information, logs, or outputs that you want to share?

OpenJDK16 taken from here: https://jdk.java.net/16/
(https://download.java.net/java/GA/jdk16/7863447f0ab643c585b9bdebf67c69db/36/GPL/openjdk-16_osx-x64_bin.tar.gz)

[comius]

We haven't looked yet into JDK16 support, but apparently the internals Bazel is using have changed.

Thanks for the report!

I'd be glad to see some more investigation if the modules just need to be exported and we will accept PRs that will provide early support for JDK16.

[cushon]

cannot access class com.sun.tools.javac.util.Context

That's the result of JEP 396: Strongly Encapsulate JDK Internals by Default

That specific failure can be worked around with the JVM flag --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED.

We'll likely need to start passing similar flags to other tools that use internal javac APIs (including JavaBuilder).

FWIW DumpPlatformClassPath has always been kind of a hack, and it would be better to migrate to one of --release or --system, but --system requires pulling the target JDK into the compilation action, and --release has some limitations like JDK-8206937.

[eustas]

Oh, I see. I was confused by --add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED already set by Bazel, and thought it was enough. Thanks.

[davido]

build --define=ABSOLUTE_JAVABASE=/redacted/jdk-16.jdk/Contents/Home

ABSOLUTE_JAVABASE was removed in Bazel@HEAD (in 2a07fed). So that there is no currently an easy way to support unsupported JDK versions in future Bazel releases, starting from Bazel 5.x.

[comius]

build --define=ABSOLUTE_JAVABASE=/redacted/jdk-16.jdk/Contents/Home

ABSOLUTE_JAVABASE was removed in Bazel@HEAD (in 2a07fed). So that there is no currently an easy way to support unsupported JDK versions in future Bazel releases, starting from Bazel 5.x.

The way to use an unsupported JDK has changed. Adding local_jdk_repository to WORKSPACE, default_java_toolchain targets and registering it in the WORKSPACE, should not be considered too complex in my opinion. Documentation is here
https://docs.bazel.build/versions/master/bazel-and-java.html#configuring-additional-execution-toolchains
https://docs.bazel.build/versions/master/bazel-and-java.html#configuring-additional-compilation-toolchains.

Also adding an extra --add-exports can naturally happen there. I don't think ABSOLUTE_JAVABASE would make this task more simple.

[davido]

@comius

Thanks for clarifying. I will try to put together a change to add support for JDK 16 using remote_java_repository and default_java_toolchain.

@cushon

There is another issue with exporting of jdk.compiler/com.sun.tools.javac.api, so that this diff in Bazel@HEAD fixed it:

diff --git a/tools/jdk/default_java_toolchain.bzl b/tools/jdk/default_java_toolchain.bzl
index 03c0760ff3..dca07e0139 100644
--- a/tools/jdk/default_java_toolchain.bzl
+++ b/tools/jdk/default_java_toolchain.bzl
@@ -233,6 +233,8 @@ def _bootclasspath_impl(ctx):
     args = ctx.actions.args()
     args.add("-XX:+IgnoreUnrecognizedVMOptions")
     args.add("--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED")
+    args.add("--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED")
+    args.add("--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED")
     args.add_joined(
         "-cp",
         [class_outputs[0].dirname, "%s/lib/tools.jar" % host_javabase.java_home],