Implement Truffle compiler control based on HotSpot's CompileBroker compilation activity

[simonis]

Truffle compilations run in "hosted" mode, i.e. the Truffle runtimes triggers compilations independently of HotSpot's CompileBroker. But the results of Truffle compilations are still stored as ordinary nmethods in HotSpot's code cache (with the help of the JVMCI method jdk.vm.ci.hotspot.HotSpotCodeCacheProvider::installCode()). The regular JIT compilers are controlled by the CompileBroker which is aware of the code cache occupancy. If the code cache runs full, the CompileBroker temporary pauses any subsequent JIT compilations until the code cache gets swept (if running with -XX:+UseCodeCacheFlushing -XX:+MethodFlushing which is the default) or completely shuts down the JIT compilers if running with -XX:+UseCodeCacheFlushing.

Truffle compiled methods can contribute significantly to the overall code cache occupancy and they can trigger JIT compilation stalls if they fill the code cache up. But the Truffle framework itself is neither aware of the current code cache occupancy, nor of the compilation activity of the CompileBroker. If Truffle tries to install a compiled method through JVMCI and the code cache is full, it will silently fail. Currently Truffle interprets such failures as transient errors and basically ignores it. Whenever the corresponding method gets hot again (usually immediately at the next invocation), Truffle will recompile it again just to fail again in the nmethod installation step, if the code cache is still full.

When the code cache is tight, this can lead to situations, where Truffle is unnecessarily and repeatedly compiling methods which can't be installed in the code cache but produce a significant CPU load. Instead, Truffle should poll HotSpot's CompileBroker compilation activity and pause compilations for the time the CompileBroker is pausing JIT compilations (or completely shutdown Truffle compilations if the CompileBroker shut down the JIT compilers).

The corresponding JVMCI change is tracked under JDK-8344727: [JVMCI] Export the CompileBroker compilation activity mode for Truffle compiler control.

This topic has initially been discussed on the Truffle channel in the Graal Slack: https://graalvm.slack.com/archives/CNQSB2DHD/p1724699423251739