Try to fix JDK-8240567

Co-authored-by: Christoph Schwentker <siedlerkiller@gmail.com>

doc/bug-8240567.md

@@ -0,0 +1,161 @@
+# Bug fix for bug-8240567.md
+
+This describes the fix for [JDK-8240567](https://bugs.openjdk.org/browse/JDK-8240567).
+
+When jlink is run, the method `SystemModulesPlugin.SystemModulesClassGenerator.genModuleDescriptorsMethod` is executed.
+This method generates a method calling the construction of a `ModuleDescriptor` for each module used. In case
+many modules are used, this method gets larger than 64kb, which is out of the limits of JDK.
+
+The idea of the fix is to move the calls down one call level to helper methods and let
+`genModuleDescriptorsMethod` call these helper methods. The challenge is that no Java code is generated,
+but directly Java byte code.
+
+## Preconditions
+
+To work on this fix, following pre-conditions need to be fulfilled:
+
+- Get [jtreg](https://openjdk.org/projects/code-tools/jtreg/intro.html) build locally on your machine.
+- Configure the JDK build with jtreg (`./configure --with-boot-jdk=... --with-jtreg=/.../jtreg/build/images/jtreg`)
+
+## Determining correct Java byte code
+
+To determine the correct Java byte code, following minimal Java code is used:
+
+```java
+import java.lang.module.ModuleDescriptor;
+
+class MethodCalls {
+
+    public void method0(ModuleDescriptor[] moduleDescriptors) {
+        moduleDescriptors[0] = null;
+        return;
+    }
+
+    public void method1(ModuleDescriptor[] moduleDescriptors) {
+        moduleDescriptors[1] = null;
+        return;
+    }
+
+    public ModuleDescriptor[] moduleDescriptors() {
+        ModuleDescriptor[] result = new ModuleDescriptor[10];
+        method0(result);
+        method1(result);
+        return result;
+    }
+
+    public static void main (String args[]) {
+        MethodCalls methodCalls = new MethodCalls();
+        ModuleDescriptor[] result = methodCalls.moduleDescriptors();
+        System.out.println(result[0]);
+    }
+}
+```
+
+This content is stored in `MethodCalls.java`. After compiling with `javac`, one can generate the byte code
+output using `javap`.
+
+```terminal
+javap -c MethodCalls
+```
+
+The output is as follows:
+
+```text
+Compiled from "MethodCalls.java"
+class MethodCalls {
+  MethodCalls();
+    Code:
+       0: aload_0
+       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
+       4: return
+
+  public void method0(java.lang.module.ModuleDescriptor[]);
+    Code:
+       0: aload_1
+       1: iconst_0
+       2: aconst_null
+       3: aastore
+       4: return
+
+  public void method1(java.lang.module.ModuleDescriptor[]);
+    Code:
+       0: aload_1
+       1: iconst_1
+       2: aconst_null
+       3: aastore
+       4: return
+
+  public java.lang.module.ModuleDescriptor[] moduleDescriptors();
+    Code:
+       0: bipush        10
+       2: anewarray     #7                  // class java/lang/module/ModuleDescriptor
+       5: astore_1
+       6: aload_0
+       7: aload_1
+       8: invokevirtual #9                  // Method method0:([Ljava/lang/module/ModuleDescriptor;)V
+      11: aload_0
+      12: aload_1
+      13: invokevirtual #15                 // Method method1:([Ljava/lang/module/ModuleDescriptor;)V
+      16: aload_1
+      17: areturn
+
+  public static void main(java.lang.String[]);
+    Code:
+       0: new           #10                 // class MethodCalls
+       3: dup
+       4: invokespecial #18                 // Method "<init>":()V
+       7: astore_1
+       8: aload_1
+       9: invokevirtual #19                 // Method moduleDescriptors:()[Ljava/lang/module/ModuleDescriptor;
+      12: astore_2
+      13: getstatic     #23                 // Field java/lang/System.out:Ljava/io/PrintStream;
+      16: aload_2
+      17: iconst_0
+      18: aaload
+      19: invokevirtual #29                 // Method java/io/PrintStream.println:(Ljava/lang/Object;)V
+      22: return
+}
+```
+
+Thus, the sequence to call a method is
+
+```
+aload_0
+aload_1
+invokevirtual #9
+```
+
+`#9` needs to be replaced by the reference of the method to be called.
+
+## Test case
+
+The class `JLink100Modules` contains the test case for the issue. With following command one can compile and run that
+test class:
+
+```terminal
+make test TEST="test/jdk/tools/jlink/JLink100Modules.java"
+```
+
+The test currently fails.
+
+The generated java modules can be investigated in following directory:
+
+```
+build\windows-x86_64-server-release\test-support\jtreg_test_jdk_tools_jlink_JLink100Modules_java\tools\jlink\JLink100Modules\bug8240567
+```
+
+## Manually executing jlink
+
+One can manually execute jlink
+
+```
+build\windows-x86_64-server-release\images\jdk\bin\jlink.exe --output C:\TEMP\out-jlink --module-path build\windows-x86_64-server-release\test-support\jtreg_test_jdk_tools_jlink_JLink100Modules_java\tools\jlink\JLink100Modules\bug8240567\out --add-modules bug8240567x
+```
+
+In case the output is as follows, the patch did not work out properly:
+
+```
+Fehler: jdk.internal.org.objectweb.asm.MethodTooLargeException: Method too large: jdk/internal/module/SystemModules$all.moduleDescriptorsSub1 ([Ljava/lang/module/ModuleDescriptor;)V
+```
+
+With that command, one can do a cycle through code adaption and checking with jlink.

src/jdk.jlink/share/classes/jdk/tools/jlink/internal/plugins/SystemModulesPlugin.java

@@ -666,6 +666,54 @@ private void genIncubatorModules(ClassBuilder clb) {
          * Generate bytecode for moduleDescriptors method
          */
         private void genModuleDescriptorsMethod(ClassBuilder clb) {
+            // split up module infos in "consumable" packages
+            List<List<ModuleInfo>> splitModuleInfos = new ArrayList<>();
+            List<ModuleInfo> currentModuleInfos = new ArrayList<>();
+            splitModuleInfos.add(currentModuleInfos);
+            for (int index = 0; index < moduleInfos.size(); index++) {
+                // The method is "manually split" based on the heuristics that 90 ModuleDescriptors are smaller than 64kb
+                // The number 90 is chosen "randomly" to be below the 64kb limit of a method
+                if ((index % 90 == 0) && (splitModuleInfos.size() > 1)) {
+                    // Prepare new list
+                    currentModuleInfos = new ArrayList<>();
+                    splitModuleInfos.add(currentModuleInfos);
+                }
+                currentModuleInfos.add(moduleInfos.get(index));
+            }
+
+            // generate all helper methods
+            final String helperMethodNamePrefix = "moduleDescriptorsSub";
+            final int[] globalCount = {0};
+            for (final int[] index = {0}; index[0] < splitModuleInfos.size(); index[0]++) {
+                clb.withMethodBody(
+                        helperMethodNamePrefix + index[0],
+                        MethodTypeDesc.of(CD_MODULE_DESCRIPTOR.arrayType()),
+                        ACC_PUBLIC,
+                        cob -> {
+                            List<ModuleInfo> moduleInfosPackage = splitModuleInfos.get(index[0]);
+                            if (index[0] > 0) {
+                                // call last helper method
+                                cob.aload(MD_VAR)
+                                   .invokevirtual(
+                                           this.classDesc,
+                                           helperMethodNamePrefix + (index[0] - 1),
+                                           MethodTypeDesc.of(CD_void, CD_MODULE_DESCRIPTOR.arrayType())
+                                   );
+                            }
+                            for (int j = 0; j < moduleInfosPackage.size(); j++) {
+                                ModuleInfo minfo = moduleInfosPackage.get(j);
+                                new ModuleDescriptorBuilder(cob,
+                                        minfo.descriptor(),
+                                        minfo.packages(),
+                                        globalCount[0]).build();
+                                globalCount[0]++;
+                            }
+                            cob.aload(MD_VAR)
+                               .areturn();
+                        });
+            }
+
+            // generate call to last helper method
             clb.withMethodBody(
                     "moduleDescriptors",
                     MethodTypeDesc.of(CD_MODULE_DESCRIPTOR.arrayType()),
@@ -675,13 +723,13 @@ private void genModuleDescriptorsMethod(ClassBuilder clb) {
                            .anewarray(CD_MODULE_DESCRIPTOR)
                            .astore(MD_VAR);
 
-                        for (int index = 0; index < moduleInfos.size(); index++) {
-                            ModuleInfo minfo = moduleInfos.get(index);
-                            new ModuleDescriptorBuilder(cob,
-                                                        minfo.descriptor(),
-                                                        minfo.packages(),
-                                                        index).build();
-                        }
+                        cob.aload(MD_VAR)
+                           .invokevirtual(
+                                   this.classDesc,
+                                   helperMethodNamePrefix + (splitModuleInfos.size() - 1),
+                                   MethodTypeDesc.of(CD_void, CD_MODULE_DESCRIPTOR.arrayType())
+                           );
+
                         cob.aload(MD_VAR)
                            .areturn();
                     });

test/jdk/tools/jlink/JLink100Modules.java

@@ -0,0 +1,121 @@
+/*
+ * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Arrays;
+import java.util.StringJoiner;
+import java.util.spi.ToolProvider;
+import tests.JImageGenerator;
+import tests.JImageGenerator.JLinkTask;
+/*
+ * @test
+ * @summary Make sure that 100 modules can be linked using jlink.
+ * @bug 8240567
+ * @library ../lib
+ * @modules java.base/jdk.internal.jimage
+ *          jdk.jdeps/com.sun.tools.classfile
+ *          jdk.jlink/jdk.tools.jlink.internal
+ *          jdk.jlink/jdk.tools.jlink.plugin
+ *          jdk.jlink/jdk.tools.jmod
+ *          jdk.jlink/jdk.tools.jimage
+ *          jdk.compiler
+ * @build tests.*
+ * @run main/othervm -verbose:gc -Xmx1g -Xlog:init=debug -XX:+UnlockDiagnosticVMOptions -XX:+BytecodeVerificationLocal JLink100Modules
+ */
+public class JLink100Modules {
+    private static final ToolProvider JAVAC_TOOL = ToolProvider.findFirst("javac")
+                                                               .orElseThrow(() -> new RuntimeException("javac tool not found"));
+    private static final ToolProvider JLINK_TOOL = ToolProvider.findFirst("jlink")
+                                                               .orElseThrow(() -> new RuntimeException("jlink tool not found"));
+
+    static void report(String command, String[] args) {
+        System.out.println(command + " " + String.join(" ", Arrays.asList(args)));
+    }
+
+    static void javac(String[] args) {
+        report("javac", args);
+        JAVAC_TOOL.run(System.out, System.err, args);
+    }
+
+    static void jlink(String[] args) {
+        report("jlink", args);
+        JLINK_TOOL.run(System.out, System.err, args);
+    }
+
+    public static void main(String[] args) throws Exception {
+        Path src = Paths.get("bug8240567");
+
+        StringJoiner mainModuleInfoContent = new StringJoiner(";\n  requires ", "module bug8240567x {\n  requires ", "\n;}");
+
+        // create 100 modules. With this naming schema up to 130 seem to work
+        for (int i = 0; i < 150; i++) {
+            String name = "module" + i + "x";
+            Path moduleDir = Files.createDirectories(src.resolve(name));
+
+            StringBuilder builder = new StringBuilder("module ");
+            builder.append(name).append(" {");
+
+            for (int j = 0; j < i; j++) {
+                builder.append("requires module" + j + "x;");
+            }
+            builder.append("}\n");
+            Files.writeString(moduleDir.resolve("module-info.java"), builder.toString());
+            mainModuleInfoContent.add(name);
+        }
+
+        // create module reading the generated modules
+        Path mainModulePath = src.resolve("bug8240567x");
+        Files.createDirectories(mainModulePath);
+        Path mainModuleInfo = mainModulePath.resolve("module-info.java");
+        Files.writeString(mainModuleInfo, mainModuleInfoContent.toString());
+
+        Path mainClassDir = mainModulePath.resolve("testpackage");
+        Files.createDirectories(mainClassDir);
+
+        Files.writeString(mainClassDir.resolve("JLink100ModulesTest.java"), """
+                package testpackage;
+
+                public class JLink100ModulesTest {
+                    public static void main(String[] args) throws Exception {
+                        System.out.println("JLink100ModulesTest started.");
+                    }
+                }
+                """);
+
+        String out = src.resolve("out").toString();
+
+        javac(new String[] {
+                "-d", out,
+                "--module-source-path", src.toString(),
+                "--module", "bug8240567x"
+        });
+
+        JImageGenerator.getJLinkTask()
+                .modulePath(out)
+                .output(src.resolve("out-jlink"))
+                .addMods("bug8240567x")
+                .call().assertSuccess();
+    }
+}