Java memory calculator MaxMetaspaceSize is causing issue.

[skylartd]

Hi, we recently switch our way of building spring boot docker image using paketo (which is come with spring boot maven plugin).
But the -XX:MaxMetaspaceSize calculated by the memory calculator is a little bit small for our application and causing a lot of OutOfMemoryError: Metaspace. And with the -XX:+ExitOnOutOfMemoryError flag contribute by the java pack, our component shut down and get restart (since it is deployed inside a k8s cluster) a lot.

We monitored our component and the metaspace size is stable after some code execution path so there should be no memory leak.
Can we somehow provide a way customize/choose our own flag instead of sticking with what provided by the memory calculator?

Also, we cannot observe and tune all of our components and set a custom value for -XX:MaxMetaspaceSize since it's kind of an exhaustive task.

[dmikusa]

There are configuration settings to adjust the metaspace calculations.

1. You can set BPL_JVM_LOADED_CLASS_COUNT which overrides the number of classes used in the memory calculator's calculation. Typically, it'll count all of the classes it can find and multiply by 35%. You can set this to a fixed value, but I don't usually recommend this because the class count could unexpectedly change, and this value might be inaccurate. Also, it's unlikely that it's missing classes, rather it's more likely that you have something dynamically generating classes like proxies.

https://github.com/paketo-buildpacks/bellsoft-liberica#configuration
https://paketo.io/docs/howto/java/#configure-the-jvm-at-runtime

2. It's not documented, but we support setting BPL_JVM_CLASS_ADJUSTMENT which allows you to dynamically adjust the metaspace class calculation. You may set this to a static value like 200, which will result in increasing the class count by this number, or you can set it to a percentage like 150% and it'll multiply the class count by this percent (so greater than 100% will increase the class count or lower than 100% will decrease the class count). Also, enabling debug log level will result in printing the memory calculation values so you can see the exact formula being used.

https://github.com/paketo-buildpacks/libjvm/blob/main/helper/memory_calculator.go#L126

3. You can override the memory calculator and set -XX:MaxMetaspaceSize=100M in JAVA_TOOL_OPTIONS and it'll override all of the memory calculator calculations and just use the value you set. So the example above would just use 100M.

I would recommend option 2.) and just increase by 20 or 30% and see if that stabilizes your app. That is likely better long run because if your class count does change, it'll adjust dynamically. That said, you can also just figure out what you need to stabilize the app and set that specific value, i.e. option 3.). I would avoid option 1.) and am just documenting that to be complete.

Hope that helps!

[skylartd]

Thank you @dmikusa for your answer. I tried the option 2 and it works 👍