[Failing Test]: XVR tests running on VR test suites causing test failure

[Abacn]

What happened?

#29924 merged core-construction-java into java-core. After that many validation runner tests failed:

org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$GroupByKeyTest
FAILED	0.076s	0.076s	0.076s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$SingleInputOutputTest
FAILED	0.071s	0.071s	0.071s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$FlattenTest
FAILED	0.066s	0.066s	0.066s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$PythonDependenciesTest
FAILED	0.066s	0.066s	0.066s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$MultiInputOutputWithSideInputTest
FAILED	0.065s	0.065s	0.065s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$CombineGloballyTest
FAILED	0.061s	0.061s	0.061s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$CoGroupByKeyTest
FAILED	0.056s	0.056s	0.056s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$CombinePerKeyTest
FAILED	0.056s	0.056s	0.056s	
org.apache.beam.sdk.util.construction.ValidateRunnerXlangTest$PartitionTest
FAILED	0.051s	0.051s	0.051s	

These tests are not supposed to run in VR test suites. On the other hand, also checked that XVR test suites no longer run these tests.

We should exclude these tests from running in VR while add them back to XVR

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

CC: @kennknowles

[chamikaramj]

XVR test suites no longer run these tests.

Any idea why ? XVR tests are supported to include these tests.

[kennknowles]

Do we need to roll back? Is it a configuration problem or a functionality problem?

[Abacn]

It's a configuration problem. XVR test didn't run tests in core (that moved from runners-core-construction) because the change here: https://github.com/apache/beam/pull/29924/files#r1496122168 . VR tests attempted to run those tests because it always has core test jar in its classpath, and the change added those test originally in runner-core-construction to java-core