jlinkModules fails for modules that require automatic modules

[nigredo-tori]

Expected behaviour

When classpath includes a module that requires an automatic module, JlinkPlugin should properly build an image. An example of such module would be

"com.fasterxml.jackson.module" % "jackson-module-paranamer" % "2.10.1"

which depends on the paranamer module, even though the paranamer project has never switched to JPMS.

Actual behaviour

The jdeps call fails:

[info] Running: jdeps --multi-release 11 -R /tmp/sbt_8df286d1/test-jlink-misc/foo/target/scala-2.12/classes /home/nigredo/.ivy2/cache/com.fasterxml.jackson.module/jackson-module-paranamer/bundles/jackson-module-paranamer-2.10.1.jar /home/nigredo/.ivy2/cache/com.fasterxml.jackson.core/jackson-databind/bundles/jackson-databind-2.10.1.jar /home/nigredo/.ivy2/cache/com.fasterxml.jackson.core/jackson-annotations/bundles/jackson-annotations-2.10.1.jar /home/nigredo/.ivy2/cache/com.fasterxml.jackson.core/jackson-core/bundles/jackson-core-2.10.1.jar /home/nigredo/.ivy2/cache/com.thoughtworks.paranamer/paranamer/bundles/paranamer-2.8.jar
[error] Exception in thread "main" java.lang.module.FindException: Module paranamer not found, required by com.fasterxml.jackson.module.paranamer
[error] 	at java.base/java.lang.module.Resolver.findFail(Resolver.java:877)
[error] 	at java.base/java.lang.module.Resolver.resolve(Resolver.java:191)
[error] 	at java.base/java.lang.module.Resolver.resolve(Resolver.java:140)
[error] 	at java.base/java.lang.module.Configuration.resolve(Configuration.java:411)
[error] 	at java.base/java.lang.module.Configuration.resolve(Configuration.java:245)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration.<init>(JdepsConfiguration.java:117)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:563)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:589)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:543)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:519)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.Main.run(Main.java:64)
[error] 	at jdk.jdeps/com.sun.tools.jdeps.Main$JDepsToolProvider.run(Main.java:73)
[error] 	at java.base/java.util.spi.ToolProvider.run(ToolProvider.java:137)
[error] 	at ru.eldis.toollauncher.ToolLauncher.runTool(ToolLauncher.java:68)
[error] 	at ru.eldis.toollauncher.ToolLauncher.lambda$main$1(ToolLauncher.java:33)
[error] 	at ru.eldis.toollauncher.ToolLauncher.main(ToolLauncher.java:48)

Information

This happens on Ubuntu, JDK 11, sbt-native-packager on master (e900351).

Analysis

At the moment, the only sane way to solve this is to override jlinkModules (disabling the jdeps step entirely). This helps, but as more Java libraries switch to JPMS, we're likely to see more issues like this. So it would be nice to be able to handle at least a part of such cases out of the box.

As I've mentioned in #1247 (comment), we can't just put all of our classpath in --module-path (since some JAR names break automatic modules). Now it turns out that we can't put all of it into jdeps arguments, either, since in that case automatic modules will not be recognized. This is inherently messy, since we can't really detect whether a JAR should be a module without looking at other JARs.

I propose we partition class path items depending on whether they can be automatic modules (as in, if they are JARs with "nice" names), put those that match into --module-path, and put everything else into args. There are probably some edge cases that won't be solved with this - so we should at least leave an escape hatch to override/augment this heuristic.

[nigredo-tori]

put those that match into --module-path, and put everything else into args

This also has its issues. I've implemented this (nigredo-tori@c6b8aa9), but if two "automatic" JARs have classes in the same package (split package), we get a conflict. Ultimately, this solves a rare issue by introducing a more frequent one.

I think we've reached the limit of what hacks like this can accomplish. Solving this will have to wait until we can build with Java 9, so that we are able to do something with ModuleDescriptor. Until then, we should at least add a workaround that will allow us to manually resolve issues like the paranamer one.

1. Add support for --module-path to the jdeps call.
2. Figure out the best way to find which JARs in the classpath should be made into automatic modules (by adding them to --module-path). For now it looks like that would be
   • getting the missing module's name from the error message;
   • finding an artifact with this name in the classpath (via metadata);
   • adding it to modulePath and trying again.
3. Document this troubleshooting process, provide convenience helpers if needed.