jck.mk and 'tests' value of compiler.lang/playlist.xml do not match

[yaqsun]

running jck-compiler-lang-LMBD-group6_multijvm, jck-compiler-lang-LMBD-group7_multijvm and jck-compiler-lang-ANNOT-group5_multijvm results in the following error, so jck.mk and 'tests' value of compiler.lang/playlist.xml were modified.

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
    at JavatestUtil.main(JavatestUtil.java:155)
Starting JCK agent..
Agent started with PID 19175
Starting JCK harness..
cannot find file "/jck/aqa-tests/TKG/../TKG/output_16842250754665/jck-compiler-lang-ANNOT-group5_multijvm_0/generated.jtb" 
Test complete. Stopping JCK Agent..
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1 
at JavatestUtil.main(JavatestUtil.java:155)

[smlambert]

Thanks for your interest @yaqsun

Running *_multijvm test targets requires a specific configuration setup. You will notice in the playlist that they are marked as dev level. At the moment, you should not run dev level.

You are likely running it because you are running make _jck (which picks up and runs all test targets from all levels unless those targets are disabled. 2 ways to proceed here:

1. Run the 3 required top-level targets via 3 commands make _sanity.jck, make _extended.jck, and make _special.jck (we do it this way, as we want to break up and run the tests into smaller sets).
2. Continue to run make _jck and disable the *_multijvm targets for hotspot.

So, instead of any of the changes you are proposing in this PR, I am proposing that the *_multijvm should be disabled for hotspot.

To disable the targets, please add:

			<disable>
				<comment>Disabled temporarily in multijvm mode</comment>
				<impl>hotspot</impl>
			</disable>

inside the <disables> tag, just below where it is disabled for the openj9 impl.

Other things to note, the changes in jck.mk are generated by running the https://github.com/adoptium/aqa-tests/blob/master/jck/splitter/splitter.sh script, so we do not hand edit those entries. If there is an issue with them, we'd look to update splitter.sh.

[karianna]

@yaqsun You'll also need to sign the Eclipse Contributor Agreement

[yaqsun]

Running *_multijvm test targets requires a specific configuration setup. You will notice in the playlist that they are marked as dev level. At the moment, you should not run dev level.

You are likely running it because you are running make _jck (which picks up and runs all test targets from all levels unless those targets are disabled. 2 ways to proceed here:

1. Run the 3 required top-level targets via 3 commands make _sanity.jck, make _extended.jck, and make _special.jck (we do it this way, as we want to break up and run the tests into smaller sets).
2. Continue to run make _jck and disable the *_multijvm targets for hotspot.

So, instead of any of the changes you are proposing in this PR, I am proposing that the *_multijvm should be disabled for hotspot.

To disable the targets, please add:

			<disable>
				<comment>Disabled temporarily in multijvm mode</comment>
				<impl>hotspot</impl>
			</disable>

inside the <disables> tag, just below where it is disabled for the openj9 impl.

Other things to note, the changes in jck.mk are generated by running the https://github.com/adoptium/aqa-tests/blob/master/jck/splitter/splitter.sh script, so we do not hand edit those entries. If there is an issue with them, we'd look to update splitter.sh.

Thank you for your reply and also for your introduction of the two handling methods. @smlambert

[yaqsun]

@yaqsun You'll also need to sign the Eclipse Contributor Agreement

thanks. Done.

[smlambert]

As per #4590 (comment), please update this PR by disabling the targets, instead of the approach currently taken. Or close this and create a new PR with that approach. Thanks!

[smlambert]

thanks!