Allow users to opt-out of some or all java memory calculator settings

[jericop]

One of the challenges of porting java apps built with Dockerfiles to buildpacks is that some developers expect or rely on the java defaults for some memory settings. At present the memory calculator will always be used if the java runtime is provided by any of the paketo buildpacks (bellsoft-liberica, azul-zulu). It is simple enough to override any of the settings through JAVA_TOOL_OPTIONS; however, it is not trivial to remove options altogether. While the memory calculator settings are indeed helpful in providing some guard rails when running apps, I have encountered a few situations where setting any value for a specific option, -XX:MaxDirectMemorySize for example, would cause the app to crash.

As an end user the only way for me to opt out of any of the java memory calculator defaults is to specify a custom container entrypoint that does not use the launcher to start the java app. This means I also have to set PATH and JAVA_HOME, along with any other environment variables. Another way to potentially get around the issue is to have another custom exec.d script/executable that runs after the memory calculator and updates JAVA_TOOL_OPTIONS. Both of these solutions require in-depth knowledge of how buildpacks work and an understanding of the spec.

None of the workarounds are ideal and I think it would be helpful to have some way for users to opt out of the memory calculator either in part or entirely when needed. I think having a disclaimer in a README or documentation will make it clear that a user assumes all responsibility and they should know what they are doing if they choose to go down this road.

[dmikusa]

For the record, the position of the @paketo-buildpacks/java-maintainers is that the memory calculator provides a vital role in producing stable and safe containers.

The JVM is getting better about running in containers, but even in 2024 it does not make any attempts to restrict its memory usage to keep within the memory limits put on your container. What this means is that the JVM will happily keep trying to allocate memory right until the OOM killer steps in and kills it. The memory calculator preallocates memory in a way that will produce a safe configuration, i.e. it will not go above the container limit, and that ensures your app will continue to run so long as you've provisioned your container memory limits correctly.

There is a risk for the Java buildpack to allow users to disable this feature because it makes it very easy for users to create a configuration that will break. There was a time when the memory calculator didn't exist, way long ago and a period where it was not as thorough as it is now. I was involved in many long and arduous troubleshooting sessions during that time trying to help folks figure out why their apps were crashing. It was not fun. The memory calculator in its present state would have prevented much of that effort. If we make it easy for users to disable the memory calculator, we believe that we'll see an increase in similar problems because rather than work out a sustainable memory configuration, users will just disable the memory calculator (it's the easy short-term fix).

Given this, I would be -1 on any sort of environment variable or quick configuration to disable the memory calculator.

If we find that there is a default configuration that has not aged well or a way that the memory calculator's algorithm works which is counter-productive, then we can certainly do some investigations and with supporting evidence make changes to the memory calculator defaults.

We can also look at adding different memory profiles/algorithms to the memory calculator. There is an RFC that's yet to be implemented to add a low-memory profile/algorithm to the calculator. This RFC is to enable the calculator to produce more useful memory configurations when the container memory limit is under 1G of RAM. We could enable other profiles/algorithms that key off of different things in your application, for example, if certain libraries are present then we'd increase the default direct memory.

I'd also be OK with allowing folks to repackage their JVM vendor buildpack of choice without the memory calculator or perhaps in a mode that enables it to be disabled. We would not do this for the standard Paketo buildpack, but I'm not opposed to making it easier for folks to create their own variant of the buildpack that does enable this.

Repackaging is extra work, but it is feasible for an advanced user like yourself to do this and disable the memory calculator. I'm not sure exactly how this would work and we don't have anything quite like that at present, but I'm confident that we could sort something out if this is a path that folks would like us to go down.

[jericop]

Thanks for the thoughtful response and for all your help throughout my buildpacks adoption journey.

At present our primary use-case for opting out of some or all java memory calculator settings is the removal of the -XX:MaxDirectMemorySize setting. Would you considering creating a profile/algorithm for the yet-to-be-implemented RFC that removes the -XX:MaxDirectMemorySize entirely? This would preserve all of the other memory settings and allow users to select that profile when needed.

I have considered repackaging the jvm vendor buildpack, but would prefer to avoid that because it creates the need to fork and maintain a code base and comes with its own set of responsibilities and challenges. It will eventually lead to drift as the paketo jvm buildpacks evolve and there are old forks out there. Have you considered de-coupling the memory calculator entirely into a separate buildpack and having it be part of the java build group order in the builder?

At the end of the day, we want to use paketo buildpacks without duplicating the burden of maintaining code bases, cutting releases, patching vulnerabilities, etc. If users go down the path of repackaging paketo buildpacks, they will have less incentive to contribute back to upstream because they would be using a custom version that only they are benefiting from. Repackaging is not really a solution in my opinion (2 cents).

Again, I appreciate the input and hope to find some common ground that can benefit me and potentially others in the future. The memory calculator is there to help the user, but I do think there should be a way for the user to customize things if needed.

[jericop]

Thanks for the suggestion. Are you thinking of allowing the memory calculator to be omitted entirely through a sed operation during packaging? This approach would potentially work, but I don't think it's ideal because it would require users to explicitly select the jvm vendor buildpack at build time. It's true users can easily override buildpacks at build time, but it's one more thing for a developer to think about.

My goal is to make container builds with buildpacks as easy as possible. Setting an environment variable at build time is a well-established pattern for customizing build behavior, which is my first preference. I would prefer to avoid the need for project.toml files in every repo in order to override the jvm vendor buildpack. This just creates more files to manage and keep up to date if someone pins to a specific version of a jvm vendor buildpack.

From my perspective, if any effort goes into making this possible, why not allow it at run time to make it as user-friendly as possible? I also think it creates tight coupling between build time and run time configuration. A run-time configurable (environment variable) approach means existing default behavior and documentation still applies and is not ambiguous depending on how the buildpack was packaged.

I'm not trying to be negative, but rather trying to list all the potential gotchas with the proposed approached so we can come up with something that works well and we don't regret implementing in the future.

[dmikusa]

Are you thinking of allowing the memory calculator to be omitted entirely through a sed operation during packaging?

Yes, the idea is that if you want to disable it, then you need to put in the extra work to do so. It's not something folks should do on a whim. If someone like yourself has a use case for this and wants to opt-in and accept the risks, I want to make that possible but I don't want this to be something someone posts as an answer on stack overflow and users start copying without understanding the implications.

This also makes it clear when someone has done this because you need to build your copy of the buildpack as a different, not paketobuildpacks/* labeled image.

My goal is to make container builds with buildpacks as easy as possible. Setting an environment variable at build time is a well-established pattern for customizing build behavior, which is my first preference. I would prefer to avoid the need for project.toml files in every repo in order to override the jvm vendor buildpack. This just creates more files to manage and keep up to date if someone pins to a specific version of a jvm vendor buildpack.

I think you can still do that. You would need to repackage the buildpack and then create a builder with this buildpack combination. Then from your user's perspective, it's not really different. There's just a builder that has the memory calculator (standard Paketo builder), which hopefully they use most of the time, and there's this other builder (your custom builder) that they use for the special cases where you need to opt-out of the memory calculator. It's up to you as the operator to dictate to users when they use each builder.

From my perspective, if any effort goes into making this possible, why not allow it at run time to make it as user-friendly as possible?

The trouble with runtime is that it makes it too easy. I want there to be a little work involved here because I don't want the casual user to disable the memory calculator. If it's easy, then you have users disabling it that don't understand what it means. It creates problems that they don't understand and then blame on buildpacks. Then I have to spend time explaining why that's not the case.

In a nutshell, I want to make this possible for you and other advanced users, to support advanced use cases (operators). I don't want it to become that setting people (developers) turn on without understanding it.

[jericop]

Definitely all fair points. I think having this escape hatch will be helpful. I too want developers to use the paketo buildpack with memory calculator by default, and only plan to have the escape hatch available for those that seek it out when needed. Let me know if you want me to be contributing to help make this possible. I would probably seek some guidance, but can certainly help if needed.

Any thoughts on my other suggestion below? If so would these profiles be configurable at runtime?

At present our primary use-case for opting out of some or all java memory calculator settings is the removal of the -XX:MaxDirectMemorySize setting. Would you considering creating a profile/algorithm for the yet-to-be-implemented RFC that removes the -XX:MaxDirectMemorySize entirely? This would preserve all of the other memory settings and allow users to select that profile when needed.

[dmikusa]

Any thoughts on my other suggestion below? If so would these profiles be configurable at runtime?

The problem with this is that it would create a broken profile. It would not account for direct memory, so the limits set by that memory calculator profile would allow for an app to exceed its memory limits and crash. I suppose if this were some region of memory that was fairly static after start-up, I'd be less concerned, but direct memory can grow quite a bit depending on the app. It just seems likely to cause issues, so I'd like to keep this something that takes a little work to be enabled.

[dmikusa]

Let me know if you want me to be contributing to help make this possible. I would probably seek some guidance, but can certainly help if needed.

It'd be wonderful if you can contribute. I think the main thing we'll need to decide on is how we enable/disable this functionality. This way my initial half baked thought.

Change some yet to be determined setting in the buildpack. We'd make this be something that you could sed or easily swap in an automation.

I'd like to steer clear of putting in a configuration setting to buildpack.toml. That would imply it could be changed at runtime. That said, I think we could put some other sort of metadata in buildpack.toml. Maybe make a new section or option like:

[metadata]
  include-files = ["LICENSE", "NOTICE", "README.md", "linux/amd64/bin/build", "linux/amd64/bin/detect", "linux/amd64/bin/main", "linux/amd64/bin/helper", "linux/arm64/bin/build", "linux/arm64/bin/detect", "linux/arm64/bin/main", "linux/arm64/bin/helper", "buildpack.toml"]
  pre-package = "scripts/build.sh"
  features = ["memory-calculator"]

This would require a change somewhere to read this value out of the buildpack.toml file. Then we could have the memory calculator no-op or even better, not install the helper.

Another possibility that might be more complicated would be to do this at the Go level. Go has this //go:build conditional compilation, that I haven't played around with a ton, but maybe we could have it such that you set a flag when building that would not even compile in the memory calculator stuff.

I was also thinking that we could perhaps add something into libpak-tools, so that it handles swapping this configuration setting. That way you don't have to worry about parsing/modifying files, you'd just build using libpak-tools and pass it a flag like --features and it would override the features.

What do you thinks?