Functional testing uses wrong test material in release testing

[pshipton]

#2085 recently added functional testing for aarch64 Java 11.

When testing the OpenJ9 0.23 release milestone 2, there is a test failure running functional testing, which is related to using the wrong test material. The test material is from the master branch instead of from https://github.com/eclipse/openj9/tree/v0.23.0-release.

There was a discussion on Slack, copying some of the points here.

Lan Xia
The default Openj9 branch is master unless it is overridden by the upstream job. In this case, it should be set to v0.23.0-release
https://ci.adoptopenjdk.net/job/Test_openjdk11_j9_extended.functional_aarch64_linux/8/rebuild/parameterized

Andrew Leonard
the functional test suites look like they need some additional pipeline support adding for openj9 scm ref?

shelley.lambert
indeed

shelley.lambert
as Lan stated above, just as we have a release_tag for openjdk test material, we will need to add a tag for openj9 test material (AQA pulls test material from a variety of open repos... when we do not know a tag or SHA, we have to assume tip)

Lan Xia
... the upstream build job needs to set the value when triggering test jobs for specific release.

--

The actual (invalid) failures were in the extended.functional suite, VmArgumentTests_0 / testEnvironmentVariableOrdering, testJavaOptions.

[llxia]

I think there are a couple of approaches to resolving this:

1. user set OPENJ9_BRANCH when triggering milestone/release builds.
   The problem is that the user (build launcher) who launches the milestone/release builds needs to be clear what value to put in.

2. the build script contains a map to map between release_tag and OPENJ9_BRANCH as release_tag has to be set for milestone/release builds.
   The user does not need to know the specific value, but a map needs to be maintained in the script. It looks like we can have a generic mapping rule to figure out OPENJ9_BRANCH value based on release_tag. For example, map release_tag=openj9-0.23.0 to OPENJ9_BRANCH=v0.23.0-release

3. use release_tag value for OPENJ9_BRANCH
   This needs openj9 repo and extension repo to be branched with the same name. And update the script to set OPENJ9_BRANCH to release_tag value if it is openj9 build
   https://github.com/AdoptOpenJDK/openjdk-build/blob/a37d55290568c70fe07914609ddd639c3407bc7d/pipelines/build/common/openjdk_build_pipeline.groovy#L189

IMHO, approach 3 is the easiest. The user does not need to know the specific value and the script does not need to maintain the map. However, if for some reason that we cannot have the same branch name, then my second favorite is to have a generic mapping.

[andrew-m-leonard]

@pshipton (3) We could change the extensions release branching to have the same branch name as openj9/omr, I think that makes sense, so: v0.23.0-release

[pshipton]

Sure, I'll use the same name for the 0.24 release.