Document how and where to add custom GraalVM configuration files

[klopfdreh]

Version information

Spring Boot version: 3.3.4

Bug description

When you place a

• reflection-config.json
• proxy-config.json
• resource-config.json
• serialization-config.json

into

src/main/resources/META-INF/native-image/<groupid>/<artifactid>/

and you perform a process-aot with spring-boot-maven-plugin the configuration files that you have defined in your project are not merged with those the plugin is generating.

Instead the plugin overwrites the configuration which clashes with the documentation of https://www.graalvm.org/latest/reference-manual/native-image/metadata/.

The current workaround is that you have to write a RuntimeHintsRegistrar and define the definitions there.

@Configuration
@ImportRuntimeHints(MyRuntimeHints.class)
@Slf4j
public class MyRuntimeHints implements RuntimeHintsRegistrar {
    @Override
    public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
      //....
    }
}

With a RuntimeHintsRegistrar the plugin is adding the entries to the desired files.

[snicoll]

Instead the plugin overwrites the configuration which clashes with the documentation of https://www.graalvm.org/latest/reference-manual/native-image/metadata/.

What part of the doc clashes with this?

I think we're lacking some guidance in this area. It is true that our AOT infrastructure takes control and will generate metadata in src/main/resources/META-INF/native-image/<groupid>/<artifactid>/. These will end up in BOOT-INF/classes iin the repackaged archive.

If you decide to write manual metadata using the same namespace in the same module, they will be overwritten at the moment. You can use a different namespace for this, something like src/main/resources/META-INF/native-image//-manual/. Or you can write the metadata in a different module with its own artifactId and include that in your application. Have you tried any of that?

[klopfdreh]

What part of the doc clashes with this?

It is mentioned that in this folder GraalVM collect metadata for your application automatically.

If you decide to write manual metadata using the same namespace in the same module, they will be overwritten at the moment. You can use a different namespace for this, something like src/main/resources/META-INF/native-image//-manual/. Or you can write the metadata in a different module with its own artifactId and include that in your application. Have you tried any of that?

Sure this is working if you pack it into a separate dependency which is not using the spring-boot-maven-plugin and packed as "normal" jar, but this would require to split the project apart without any reason.

If you use a custom namespace this would require you to use -H:+UnlockExperimentalVMOptions for newer GraalVM versions and to add command line args like -H:ResourceConfigurationResources=META-INF/native-image/custom-definitions/resource-config.json for example to the native-image build and the resources are not picked up automatically.

[snicoll]

It is mentioned that in this folder GraalVM collect metadata for your application automatically.

Sorry I don't see how that section refers to the Spring Boot plugin or where it is said that manual metadata is merged automatically. Can you please quote exactly what you think implies that the Spring Boot plugin "clashes with the documentation"?

this would require to split the project apart without any reason.

For a start, I wasn't aware that GraalVM would let you pick any namespace you want in META-INF/native-image but would not support two. This looks quite strange to me.

Assuming that's what it does, the reason would be exactly that, GraalVM doesn't support picking up such resources from two locations automatically so you have to separate them or configure the plugin.

That's what I'd recommend for the time being anyway. As for merging manual input, we can consider it, I'll flag for team attention to get more feedback from the team.

[klopfdreh]

Sorry I don't see how that section refers to the Spring Boot plugin or where it is said that manual metadata is merged automatically. Can you please quote exactly what you think implies that the Spring Boot plugin "clashes with the documentation"?

So what I meant by this is that the GraalVM documentation explains that if you place the metadata configuration files in this specific folder they are automatically picked up. The spring-boot-maven-plugin however overwrites them and the configuration are not used anymore.

That's what I'd recommend for the time being anyway. As for merging manual input, we can consider it, I'll flag for team attention to get more feedback from the team.

Thanks a lot! 👍

Anyway - we found a workaround so it isn't urgent.

[mhalbritter]

If you use a custom namespace this would require you to use -H:+UnlockExperimentalVMOptions for newer GraalVM versions and to add command line args like -H:ResourceConfigurationResources=META-INF/native-image/custom-definitions/resource-config.json for example to the native-image build and the resources are not picked up automatically.

This is not what I'm seeing. I'm using Java version: 23+37, vendor version: Oracle GraalVM 23+37.1 and I can create META-INF/native-image/foo/bar/reachability-metadata.json and this is used in addition to the metadata the process-aot task has created. (where foo and bar is something different from my group id and artifact).

This also works with Java version: 17.0.12+8-LTS, vendor version: Oracle GraalVM 17.0.12+8.1 using reflect-config.json.

So what Stephane said:

You can use a different namespace for this, something like src/main/resources/META-INF/native-image//-manual/.

is true, isn't it?

[klopfdreh]

Yes you can use a different namespace and as I said it is no problem to do so. Also we just used RuntimeHints and those are adding the definitions to the generated hint files with the groupid and artifactid of our project.

Just wanted to mention the overwriting as this might be confusing.

[wilkinsona]

You said that it worked but that “this would require you to use -H:+UnlockExperimentalVMOptions for newer GraalVM versions and to add command line args like -H:ResourceConfigurationResources=META-INF/native-image/custom-definitions/resource-config.json”. As far as we can tell, those options are not necessary. Can you please clarify what you meant?

[klopfdreh]

Here is a complete log of a build: awslabs/aws-crt-java#834 (comment)

there you can see the hint „ 4 experimental option(s) unlocked:“

It is working without -H:+UnlockExperimentalVMOptions for now but might not working that way in future. It was mentioned in the native build when I remove the unlock option.

[mhalbritter]

I don't think that those warnings are caused by your META-INF/native-image/... files.

I have a project with the following structure:

.../src/main/resources 
> tree
.
├── application.properties
├── META-INF
│   └── native-image
│       ├── my-group
│       │   └── my-artifact
│       │       ├── reachability-metadata.json
│       │       └── reflect-config.json
│       └── my-group2
│           └── my-artifact
│               ├── reachability-metadata.json
│               └── reflect-config.json

(reachability-metadata.json is for GraalVM 23, reflect-config.json for GraalVM 17).

GraalVM 17 doesn't give me any warning, GraalVM 23 shows this:

Warning: Using a deprecated option --report-unsupported-elements-at-runtime from 'META-INF/native-image/com.example/sb-42515/native-image.properties' in 'file:///home/mhalbritter/Downloads/issue-projects/sb-42515/build/resources/aot/'. The option is deprecated and will be removed in the future. The use of unsupported elements is always reported at run time.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
Warning: Option 'DynamicProxyConfigurationResources' is deprecated and might be removed in a future release: This can be caused by a proxy-config.json file in your META-INF directory. Consider including proxy configuration in the reflection section of reachability-metadata.md instead.. Please refer to the GraalVM release notes.

Neither of those warnings are caused by my META-INF/native-image/... setup.

I've attached the build logs and the project, so I think the workaround mentioned by Stephane is valid and will be valid in the future. But I'll double check with the GraalVM native-image team.

graal-17.txt

graal-23.txt

project.zip

[klopfdreh]

Thanks a lot for all the time you already spent!

Just to summarize the workarounds:

1. Use a custom namespace which must not be the groupid / artifactid of the project
2. Provide a RuntimeHintsRegistrar via code and add all hints there

(1) It is not required to add the files provided via custom namespace with for example -H:ResourceConfigurationResources and -H:+UnlockExperimentalVMOptions as they are caught up automatically if META-INF/native-image/<custom_groupid>/<custom_artifactId>/... is used.

[klopfdreh]

I just thought about (1) of the previous comment. Maybe the fix for the spring-boot-maven-plugin could be that if the custom namespaces are caught up automatically to just store the hints generated with process-aot into a custom namespace. So for example if the projects groupid is de-tests and the artifactid is playground the spring-boot-maven-plugin is generating the hints into META-INF/native-image/de-tests/playground-spring-boot/ the suffix could be customizable within the configuration.

Edit: If this is ok for you I could provide the PR 🙂.

[mhalbritter]

The GraalVM team confirmed that it's okay to put multiple folders under META-INF/native-image:

your strategy is good, you can use any folder name as long as nobody else uses it.

And this is also documented on GraalVM side here:

The generated configuration files can be supplied to the native-image tool by placing them in a META-INF/native-image/ directory on the class path. This directory (or any of its subdirectories) is searched for the file named reachability-metadata.json that is then automatically included in the build process. Not all of those files must be present. When multiple files with the same name are found, all of them are considered.

The first thing we should do is to document that problem and the workaround in our documentation.