[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. #3446

olpaw · 2021-06-02T14:24:08Z

Removes the undocumented native-image.properties property ImageIncludeBuiltinModules. It was previously needed for building truffle-based images. Cleaning up class-init for Java 11+ make the existence of ImageIncludeBuiltinModules obsolete. (I.e. if there is truffle, it's classes are subject to class-init)
Fix implementation of ResourceBundle access. Packages that contain ResourceBundles are opened up in their respective modules prior to accessing them from the image-builder. This also fixes JDK bundles are not discovered and added to native image with JDK16 #3404
Add support for building shared library images where the --module-path is used.
Build native-image-agent and native-image-diagnostics-agent via --module-path and test the shared objects in mx hellomodule

olpaw · 2021-06-21T15:59:35Z

@gilles-duboscq please have a look at the changes in sdk/mx.sdk/mx_sdk_vm_impl.py and sdk/mx.sdk/mx_sdk_vm.py if that makes sense to you.

sdk/mx.sdk/mx_sdk_vm.py

sdk/mx.sdk/mx_sdk_vm_impl.py

...evm/src/com.oracle.svm.hosted.jdk11/src/com/oracle/svm/hosted/ClassLoaderSupportFeature.java

substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ClassLoaderSupportFeature.java

olpaw · 2021-06-23T15:37:44Z

Now that a461aa0 is on master we need 4580142 cc @gradinac

olpaw · 2021-06-25T13:25:53Z

7302e45 ... how silly of me, forgetting that not everything is in a java module ... yet 🤡

sdk/mx.sdk/mx_sdk_vm_impl.py

...cle.svm.hosted.jdk11/src/com/oracle/svm/hosted/jdk11/ClassLoaderSupportImplJDK11OrLater.java

…module class-init

…iveimage.driver:com.oracle.svm.driver.metainf

…ying class ClassLoaderSupportFeature

…LL_UNNAMED)

substratevm/src/com.oracle.objectfile.jdk11/src/com/oracle/objectfile/ModuleAccess.java

oracle/graal@1e639c9 from oracle/graal#3446 introduces some more dependencies on Truffle that Mandrel doesn't really depend on

…ation oracle/graal@1e639c9 introduces some more dependencies on Truffle that Mandrel doesn't really depend on oracle/graal@d81403a change the location of the jvmti-agent-base.jar Both changes where part of oracle/graal#3446

…ation oracle/graal@1e639c9 introduces some more dependencies on Truffle that Mandrel doesn't really depend on oracle/graal@d81403a change the location of the jvmti-agent-base.jar Both changes were part of oracle/graal#3446

zakkak · 2021-07-01T14:17:37Z

...cle.svm.hosted.jdk11/src/com/oracle/svm/hosted/jdk11/ClassLoaderSupportImplJDK11OrLater.java

+        }
+        String packageName = packageName(bundleName);
+        if (packageName == null) {
+            throw new MissingResourceException("ResourceBundle does not seem to be a fully qualified class name.", bundleName, locale.toLanguageTag());


Hi @olpaw ,

With this check in place jar files including resource bundles in .properties file format at their root are not supported. Such an example is org.eclipse.yasson (seen failing in quarkusio/quarkus#18305).

Going through ResourceBundle's Javadoc I was not able to find any mention/guidance on avoiding adding ResourceBundles to the root of jar files, so I am wondering if this is something we should relax on GraalVM.

Thanks

@zakkak Yes we should probably add fallback code for that to support that. Please create an issue for me.

Hmm I forgot to mention the issue here.
FWIW the created issue was #3535 and it got fixed by #3540

graalvmbot added the oracle-emp label Jun 2, 2021

olpaw force-pushed the paw/GR-31840 branch from 198457a to ef19782 Compare June 9, 2021 18:21

olpaw linked an issue Jun 9, 2021 that may be closed by this pull request

JDK bundles are not discovered and added to native image with JDK16 #3404

Closed

olpaw force-pushed the paw/GR-31840 branch from d7a8f96 to 280af2f Compare June 10, 2021 07:49

olpaw mentioned this pull request Jun 17, 2021

[GR-30957] Implement Module/ModuleLayer substitutions #3445

Merged

olpaw force-pushed the paw/GR-31840 branch 3 times, most recently from 1c29a4c to ea04d3c Compare June 21, 2021 14:52

olpaw changed the title ~~[GR-31840] Allow native-image shared libs to be built on module-path.~~ [GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path. Jun 21, 2021

olpaw changed the title ~~[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path.~~ [GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. Jun 21, 2021

olpaw requested review from d-kozak, christianwimmer, cstancu and gilles-duboscq June 21, 2021 15:54

gilles-duboscq reviewed Jun 21, 2021

View reviewed changes

sdk/mx.sdk/mx_sdk_vm.py Outdated Show resolved Hide resolved

sdk/mx.sdk/mx_sdk_vm_impl.py Outdated Show resolved Hide resolved

sdk/mx.sdk/mx_sdk_vm_impl.py Show resolved Hide resolved

christianwimmer suggested changes Jun 21, 2021

View reviewed changes

...evm/src/com.oracle.svm.hosted.jdk11/src/com/oracle/svm/hosted/ClassLoaderSupportFeature.java Outdated Show resolved Hide resolved

substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/ClassLoaderSupportFeature.java Outdated Show resolved Hide resolved

olpaw force-pushed the paw/GR-31840 branch from ea04d3c to 09bb627 Compare June 23, 2021 14:59

olpaw force-pushed the paw/GR-31840 branch 2 times, most recently from 909cc97 to d130946 Compare June 24, 2021 09:29

d-kozak reviewed Jun 25, 2021

View reviewed changes

sdk/mx.sdk/mx_sdk_vm_impl.py Outdated Show resolved Hide resolved

d-kozak reviewed Jun 25, 2021

View reviewed changes

sdk/mx.sdk/mx_sdk_vm_impl.py Show resolved Hide resolved

d-kozak reviewed Jun 25, 2021

View reviewed changes

...cle.svm.hosted.jdk11/src/com/oracle/svm/hosted/jdk11/ClassLoaderSupportImplJDK11OrLater.java Show resolved Hide resolved

olpaw added 3 commits June 28, 2021 10:55

Replace ImageIncludeBuiltinModules special handling with generic per-…

bcb3177

…module class-init

Implement ResourceBundle module awareness

3e5e1b8

Remove debugging code

c468644

olpaw added 14 commits June 28, 2021 10:55

Add support for module-path based native-image library builds

0c72b9d

Add missing suite.py moduleInfo for libnative-image-agent on module-path

d81403a

Generate --add-exports from requiresConcealed for macro-options

2daaf63

Add support for building native-image-diagnostics-agent on module-path

ce13fde

Ensure module-path built agents are tested in mx hellomodule

a825ddc

Module org.graalvm.nativeimage.agent.tracing now uses org.graalvm.nat…

f9750cf

…iveimage.driver:com.oracle.svm.driver.metainf

Field main_module is only needed in LauncherConfig

5e5a341

Also build substratevm:SVM_CONFIGURE via module-path

783623f

Dependencies to image_config.jar_distributions only needed for Java > 8

fdae21e

Prevent pylint complaining about line-too-long

3eb54fb

Add missing dependency for lib:native-image-agent

3a2e72a

Use version specific ClassLoaderSupportImpl classes instead of overla…

ad0554a

…ying class ClassLoaderSupportFeature

If bundle is not located in any module get it via classloader (from A…

dfe3980

…LL_UNNAMED)

Fix typo

f4899bc

olpaw force-pushed the paw/GR-31840 branch from 7302e45 to f4899bc Compare June 28, 2021 08:55

olpaw added 2 commits June 28, 2021 13:25

Fix check if module that contains bundle needs to be opened

bf17ca8

Remove invalid dependency from native-image installable

11f03c4

d-kozak reviewed Jun 28, 2021

View reviewed changes

substratevm/src/com.oracle.objectfile.jdk11/src/com/oracle/objectfile/ModuleAccess.java Show resolved Hide resolved

d-kozak approved these changes Jun 28, 2021

View reviewed changes

graalvmbot merged commit d8d9146 into oracle:master Jun 29, 2021

zakkak added a commit to zakkak/mandrel-packaging that referenced this pull request Jun 30, 2021

Remove newly added truffle dependencies

e4773f4

oracle/graal@1e639c9 from oracle/graal#3446 introduces some more dependencies on Truffle that Mandrel doesn't really depend on

zakkak mentioned this pull request Jun 30, 2021

Remove newly added truffle dependencies & update jvmti-agent-base location graalvm/mandrel-packaging#154

Merged

zakkak mentioned this pull request Jul 1, 2021

Main and JSON-B ITs failing with latest GraalVM 21.3-dev quarkusio/quarkus#18305

Closed

zakkak reviewed Jul 1, 2021

View reviewed changes

zakkak mentioned this pull request Jul 1, 2021

native-image no longer supports resource bundles in .properties file format that reside at JAR's root #3535

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. #3446

[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. #3446

olpaw commented Jun 2, 2021 •

edited

Loading

olpaw commented Jun 21, 2021

olpaw commented Jun 23, 2021

olpaw commented Jun 25, 2021 •

edited

Loading

zakkak Jul 1, 2021 •

edited

Loading

olpaw Jul 1, 2021 •

edited

Loading

zakkak Aug 17, 2021

[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. #3446

[GR-31840] [GR-31681] Allow native-image shared libs to be built on module-path and fix resource bundle lookup. #3446

Conversation

olpaw commented Jun 2, 2021 • edited Loading

olpaw commented Jun 21, 2021

olpaw commented Jun 23, 2021

olpaw commented Jun 25, 2021 • edited Loading

zakkak Jul 1, 2021 • edited Loading

Choose a reason for hiding this comment

olpaw Jul 1, 2021 • edited Loading

Choose a reason for hiding this comment

zakkak Aug 17, 2021

Choose a reason for hiding this comment

olpaw commented Jun 2, 2021 •

edited

Loading

olpaw commented Jun 25, 2021 •

edited

Loading

zakkak Jul 1, 2021 •

edited

Loading

olpaw Jul 1, 2021 •

edited

Loading