[SUREFIRE-1894] Add missing "version" attribute to report XSD

[martinmo]

This adds the missing version attribute for the <testsuite/> element to the schema. As a test, I've just added a copy of the surefire-597/TEST-surefire.MyTest.xml with the additional version="3.0" attribute in <testsuite/>. It will be picked up by the existing SurefireSchemaValidationTest. To catch similar errors in the future, a further validation using a freshly created XML report, instead of static ones, would probably be better. However, I'm not familiar enough with the build setup to implement such a check.

[Tibor17]

Making an XML is not enough. You have to use it in a test. For more information see Surefire597Test.

[martinmo]

Hi, thanks for looking at this! Can you please elaborate why this is not enough? The new XML file is automatically used in SurefireSchemaValidationTest, which picks up all TEST-*.xml files in the test resources. For example, if I revert the fix to the schema file, the test fails and thus reproduces the issue:

[ERROR] Tests run: 13, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.19 s <<< FAILURE! - in org.apache.maven.plugins.surefire.report.JUnit4SuiteTest
[ERROR] org.apache.maven.plugins.surefire.report.SurefireSchemaValidationTest.testValidate_XMLs_against_schema  Time elapsed: 0.221 s  <<< ERROR!
java.lang.AssertionError:
surefire-1894/TEST-surefire.MyTest.xml has violations:
 - ERROR at row:2 col:104 cvc-complex-type.3.2.2: Attribute 'version' is not allowed to appear in element 'testsuite'.
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.apache.maven.plugins.surefire.report.SurefireSchemaValidationTest.testValidate_XMLs_against_schema(SurefireSchemaValidationTest.java:83)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at junit.framework.TestCase.runTest(TestCase.java:177)
	at junit.framework.TestCase.runBare(TestCase.java:142)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:130)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:50)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.shadefire.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:99)
	at org.apache.maven.shadefire.surefire.junit.JUnit3Provider.executeTestSet(JUnit3Provider.java:141)
	at org.apache.maven.shadefire.surefire.junit.JUnit3Provider.invoke(JUnit3Provider.java:114)
	at org.apache.maven.shadefire.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
	at org.apache.maven.shadefire.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
	at org.apache.maven.shadefire.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
	at org.apache.maven.shadefire.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)

Or do you want to test something else? I've looked at Surefire597Test, but it tests the generation of an HTML report from the XML report and this seems unrelated.

[Tibor17]

@martinmo
The test verifies XML against the schema which is very suitable in this case. If the test fails without your fix and passed with it, then it is acceptable. Thx.