From cd36b49b19ee1335a73a05956c1adfb3caea1628 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1=C5=A1=20Rylek?= Date: Wed, 27 Dec 2023 18:30:19 +0100 Subject: [PATCH 1/3] Clean up testgrouping.proj and move JIT/Intrinsics under JIT.1 In my attempts to switch over the remainder of JIT tests to the merged wrapper model I hit a weird bug in two tests, JIT/Intrinsics/TypeIntrinsics_r.csproj and TypeIntrinsics_ro.csproj when running under CG2. I found out that the bug is real even without test merging, it was just hidden due to a weird corner case in the script helixpublishwitharcade.proj: The testgrouping.proj script is used for two purposes - to split some large test groups using the TestProjects item group and to merge multiple legacy test XUnit wrappers for small test groups into a single Helix work item using the XUnitWrapperGrouping item group. As I found out, the somewhat hacky logic in helixpublishwitharcade.proj doesn't directly use the above item groups to construct the Helix work items, it uses a roundabout way of doing that which among others states that the "TestGroup" trait used to filter tests within a single legacy XUnit wrapper at the XUnit console level was only set when there was a single XUnit wrapper in a Helix work item. As it turns out, the PayloadGroup "JIT.2" included two subtrees - the "JIT/Intrinsics" folder and the "JIT/SIMD" subtree. Once the JIT/SIMD subtree was converted to the merged test wrapper model, the JIT/Intrinsics/JIT.Intrinsics.XUnitWrapper.dll remained as the only (legacy) wrapper in the JIT.2 group, thus triggering the logic in helixpublishwitharcade that added the "-trait TestGroup=JIT.2" to the xunit.console execution command line, effectively blocking out all 31 tests in that wrapper. In the first iteration I have only moved the JIT/Intrinsics wrapper under the PayloadGroup "JIT.1" and I have deleted all items referring to tests that have already been merged. My plan is to use this first iteration to verify that runtime-coreclr2 outerloop now fails as it should (because the two problematic TypeIntrinsics tests should start running in CG2 mode with the change). I'll then publish an update to the PR adding an issues.targets blocking clause for the two TypeIntrinsics tests and I'll clean up helixpublishwitharcade.proj by removing the no longer needed logic around TestGroup as all test groups using the TestGrouping item group have already been converted to the merged model. Thanks Tomas --- src/tests/Common/testgrouping.proj | 108 +---------------------------- 1 file changed, 3 insertions(+), 105 deletions(-) diff --git a/src/tests/Common/testgrouping.proj b/src/tests/Common/testgrouping.proj index fe677242679b6..17277e0027457 100644 --- a/src/tests/Common/testgrouping.proj +++ b/src/tests/Common/testgrouping.proj @@ -1,108 +1,11 @@ - - baseservices.threading.generics - $(TestBinDir)baseservices\threading\baseservices.threading.XUnitWrapper.dll - - - - JIT.jit64.hfa - $(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll - - - - JIT.jit64.mcc - $(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll - - - - JIT.jit64.opt - $(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll - - - - JIT.jit64.valuetypes - $(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll - - - - JIT.Regression.CLR-x86-JIT.V1-M09-M11 - $(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll - - - - JIT.Regression.CLR-x86-JIT.V1-M12-M13 - $(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll - - - - JIT.Regression.JitBlue - $(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll - - - - Loader.classloader.generics - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - - Loader.classloader.TypeGeneratorTests.TypeGeneratorTest0-299 - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - - Loader.classloader.TypeGeneratorTests.TypeGeneratorTest300-599 - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - - Loader.classloader.TypeGeneratorTests.TypeGeneratorTest600-899 - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - - Loader.classloader.TypeGeneratorTests.TypeGeneratorTest900-1199 - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - - Loader.classloader.TypeGeneratorTests.TypeGeneratorTest1200-1500 - $(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll - - - + $(TestBinDir)managed\**\*.XUnitWrapper.dll" /> PayloadGroup0 @@ -112,15 +15,10 @@ JIT.1 - - JIT.2 - From 9622800e2ef5af3fce545c9e9c9772e44fa029fb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1=C5=A1=20Rylek?= Date: Wed, 27 Dec 2023 19:09:35 +0100 Subject: [PATCH 2/3] Typo --- src/tests/Common/testgrouping.proj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/Common/testgrouping.proj b/src/tests/Common/testgrouping.proj index 17277e0027457..8df299575532e 100644 --- a/src/tests/Common/testgrouping.proj +++ b/src/tests/Common/testgrouping.proj @@ -5,7 +5,7 @@ $(TestBinDir)hosting\**\*.XUnitWrapper.dll; $(TestBinDir)ilasm\**\*.XUnitWrapper.dll; $(TestBinDir)Loader\**\*.XUnitWrapper.dll; - $(TestBinDir)managed\**\*.XUnitWrapper.dll" /> + $(TestBinDir)managed\**\*.XUnitWrapper.dll"> PayloadGroup0 From 8ad2796ca4da031c85fd68fc0de75df84aaeeba6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tom=C3=A1=C5=A1=20Rylek?= Date: Wed, 27 Dec 2023 21:34:27 +0100 Subject: [PATCH 3/3] Block out TypeIntrinsics_r[o]; clean up helixpublishwitharcade.proj Add issues.targets entries blocking out the tests JIT/Intrinsics/TypeIntrinsics_r[o].csproj that currently fail in CG2 mode. I have also cleaned up the helixpublishwitharcade.proj script by removing parts pertaining to the no longer used TestGrouping logic. Thanks Tomas --- src/tests/Common/helixpublishwitharcade.proj | 28 +------------------- src/tests/issues.targets | 7 +++-- 2 files changed, 6 insertions(+), 29 deletions(-) diff --git a/src/tests/Common/helixpublishwitharcade.proj b/src/tests/Common/helixpublishwitharcade.proj index 8f8ae6c295108..b6774b669c18d 100644 --- a/src/tests/Common/helixpublishwitharcade.proj +++ b/src/tests/Common/helixpublishwitharcade.proj @@ -206,9 +206,6 @@ - <_XUnitWrapperDll Include="%(TestGrouping.XUnitWrapperDll)" Condition="Exists('%(XUnitWrapperDll)')"> - %(TestGroup) - <_XUnitWrapperDll Include="@(XUnitWrapperGrouping)" /> - <_LegacyPayloadFiles Include="$(_FileDirectory)**" Exclude="@(_TestGroupingRelevant)" Condition="'$(_TestGroupingExists)' != 'true'" /> - - <_LegacyPayloadFiles Include="@(_TestGroupingRelevant->WithMetadataValue('TestGroup','$(_PayloadGroup)')->DistinctWithCase())" Condition="'$(_TestGroupingExists)' == 'true'" /> - <_LegacyPayloadFiles Include="$(_FileDirectory)*" Condition="'$(_TestGroupingExists)' == 'true'" /> - <_LegacyPayloadFiles Include="$(_FileDirectory)/*.app" Condition="'$(_TestGroupingExists)' == 'true'" /> - <_LegacyPayloadFiles Include="$(_FileDirectory)/*.app/**" Condition="'$(_TestGroupingExists)' == 'true'" /> - + <_LegacyPayloadFiles Include="$(_FileDirectory)**" /> <_LegacyPayloadFiles Update="@(_LegacyPayloadFiles)"> @@ -792,15 +774,10 @@ $([System.String]::Join(' ', $([System.IO.Directory]::GetFiles(%(FullPath), '*.XUnitWrapper.dll', SearchOption.AllDirectories))).Replace($([MSBuild]::EnsureTrailingSlash(%(FullPath))),'')) - - %(PayloadGroup) - - - $(LegacyPayloadsRootDirectory)\%(PayloadGroup).zip @@ -903,7 +880,6 @@ %(PayloadDirectory) $(_WorkaroundForNuGetMigrationsForPrepending) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) - $(_WorkaroundForNuGetMigrationsForPrepending) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup) $([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes))) coreclr_tests.run.$(TargetOS).$(TargetArchitecture).$(Configuration).mch;coreclr_tests.run.$(TargetOS).$(TargetArchitecture).$(Configuration).log @@ -911,7 +887,6 @@ $([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes))) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) - dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup) @@ -925,7 +900,6 @@ $(AppleTestTarget) $([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes))) $(SigningCommand) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) - $(SigningCommand) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup) diff --git a/src/tests/issues.targets b/src/tests/issues.targets index 12c37bcde107b..521783871076c 100644 --- a/src/tests/issues.targets +++ b/src/tests/issues.targets @@ -657,8 +657,11 @@ https://github.com/dotnet/runtime/issues/38096 - - https://github.com/dotnet/runtime/issues/615 + + https://github.com/dotnet/runtime/issues/96336 + + + https://github.com/dotnet/runtime/issues/96336 Not compatible with crossgen2