[Loom] Excessive JIT memory usage in Skynet test

[fengxue-IS]

When triaging #15939, I noticed that there seem to be excessive allocation from JIT

I was using -Xmx512M option, test resulted in using:
VM:
1.4GB total memory
~512MB heap memory
~500MB thread memory
~300MB class memory
JIT
~15GB total memory
~250MB code cache

Maybe similar to #16148

To reproduce:

Testcase: https://github.com/ibmruntimes/openj9-openjdk-jdk19/blob/openj9/test/jdk/java/lang/Thread/virtual/stress/Skynet.java
compile with javac --enable-preview -source 19 Skynet.java
run with java --enable-preview --Xnocompressedrefs -Xmx512M Skynet

FYI @0xdaryl @tajila

[fengxue-IS]

This may also be one of the reason for the significant perf regression against RI (5-6x) with JIT enabled. When testing with -Xint OpenJ9 has 10-20% better perf than RI

[0xdaryl]

At the moment, Skynet consistently crashes with the latest JDK 19 on x86-64 Linux with either the "Invalid JIT Return Address" problem or a crash in the VM or GC. It is probably worth waiting for those issues to be resolved before spending time investigating this.

[0xdaryl]

I am able to run this on a 96-thread Cascade Lake now. Studying the verbose log memory statistics of a few runs doesn't show anything close to 15GB memory consumption (some profiles at higher opt are in the 250-300MB range).

What technique did you use to measure the memory consumption? In the meantime I'll look at more process-based statistics as well as what the javacore tells me late in a run.

[fengxue-IS]

I was monitor the memory usage via top command on my fyre machine, I also generated a java core to confirm the memory usage.

[0xdaryl]

There does appear to be an excessive allocation problem. 99.2% of all JVM allocations running Skynet are coming from jitGetExceptionTableFromPC, specifically from here:

openj9/runtime/codert_vm/jswalk.c

Line 1500 in 65bd145

artifactSearchCache = j9mem_allocate_memory(JIT_ARTIFACT_SEARCH_CACHE_SIZE * sizeof (TR_jit_artifact_search_cache), OMRMEM_CATEGORY_JIT);

Given the rate of growth I doubt this memory is ever freed. Still figuring out why and what can be done about it.

[0xdaryl]

vmThread->jitArtifactSearchCache is always NULL, leading us to allocate a 4k chunk each time this code is called that is never freed.