Clean up testgrouping.proj and move JIT/Intrinsics under JIT.1 (#96333)

In my attempts to switch over the remainder of JIT tests to the
merged wrapper model I hit a weird bug in two tests,
JIT/Intrinsics/TypeIntrinsics_r.csproj and TypeIntrinsics_ro.csproj
when running under CG2. I found out that the bug is real even without
test merging, it was just hidden due to a weird corner case in the
script helixpublishwitharcade.proj:

The testgrouping.proj script is used for two purposes - to split
some large test groups using the TestProjects item group and to
merge multiple legacy test XUnit wrappers for small test groups
into a single Helix work item using the XUnitWrapperGrouping item
group.

As I found out, the somewhat hacky logic in helixpublishwitharcade.proj
doesn't directly use the above item groups to construct the Helix
work items, it uses a roundabout way of doing that which among others
states that the "TestGroup" trait used to filter tests within a
single legacy XUnit wrapper at the XUnit console level was only
set when there was a single XUnit wrapper in a Helix work item.

As it turns out, the PayloadGroup "JIT.2" included two subtrees -
the "JIT/Intrinsics" folder and the "JIT/SIMD" subtree. Once the
JIT/SIMD subtree was converted to the merged test wrapper model,
the JIT/Intrinsics/JIT.Intrinsics.XUnitWrapper.dll remained as the
only (legacy) wrapper in the JIT.2 group, thus triggering the
logic in helixpublishwitharcade that added the "-trait TestGroup=JIT.2"
to the xunit.console execution command line, effectively blocking
out all 31 tests in that wrapper.

I have moved the JIT/Intrinsics wrapper under the PayloadGroup
"JIT.1" and I have deleted all items referring to tests that have already
been merged. After verifying that the TypeIntrinsics tests started
failing as they should, I have adding an issues.targets blocking clause
for the two TypeIntrinsics tests and I have cleaned up
helixpublishwitharcade.proj by removing the no longer needed logic
around TestGroup as all test groups using the TestGrouping item group
have already been converted to the merged model.

Thanks

Tomas

src/tests/Common/helixpublishwitharcade.proj

@@ -206,9 +206,6 @@
 
   <Target Name="DiscoverLegacyXUnitWrappers">
     <ItemGroup>
-      <_XUnitWrapperDll Include="%(TestGrouping.XUnitWrapperDll)" Condition="Exists('%(XUnitWrapperDll)')">
-        <PayloadGroup>%(TestGroup)</PayloadGroup>
-      </_XUnitWrapperDll>
       <_XUnitWrapperDll Include="@(XUnitWrapperGrouping)" />
 
       <!-- This adds the remaining *.XUnitWrapper.dll files in TestBinDir unless
@@ -235,23 +232,8 @@
       <_XUnitWrapperDll>%(_XUnitWrapperDll.FullPath)</_XUnitWrapperDll>
     </PropertyGroup>
 
-    <ItemGroup>
-      <_TestGroupingRelevant Include="@(TestGrouping->WithMetadataValue('XUnitWrapperDll',$(_XUnitWrapperDll))->Metadata('FullPath'))" />
-    </ItemGroup>
-
-    <PropertyGroup>
-      <_TestGroupingExists>@(_TestGroupingRelevant->AnyHaveMetadataValue('TestGroup','$(_PayloadGroup)'))</_TestGroupingExists>
-    </PropertyGroup>
-
     <ItemGroup Condition="'$(_FileDirectory)' != ''">
-      <!-- If no TestGrouping is defined, all the files under $(_FileDirectory) and its subdirectories goes to the PayloadDirectory. -->
-      <_LegacyPayloadFiles Include="$(_FileDirectory)**" Exclude="@(_TestGroupingRelevant)" Condition="'$(_TestGroupingExists)' != 'true'" />
-      <!-- If there is a TestGrouping, then take only the files that belong to the TestGroup == $(_PayloadGroup). -->
-      <_LegacyPayloadFiles Include="@(_TestGroupingRelevant->WithMetadataValue('TestGroup','$(_PayloadGroup)')->DistinctWithCase())" Condition="'$(_TestGroupingExists)' == 'true'" />
-      <_LegacyPayloadFiles Include="$(_FileDirectory)*" Condition="'$(_TestGroupingExists)' == 'true'" />
-      <_LegacyPayloadFiles Include="$(_FileDirectory)/*.app" Condition="'$(_TestGroupingExists)' == 'true'" />
-      <_LegacyPayloadFiles Include="$(_FileDirectory)/*.app/**" Condition="'$(_TestGroupingExists)' == 'true'" />
-
+      <_LegacyPayloadFiles Include="$(_FileDirectory)**" />
       <_LegacyPayloadFiles Update="@(_LegacyPayloadFiles)">
         <!-- Never use [MSBuild]::MakeRelative here! We have some files containing Unicode characters in their %(FullPath) and
              MakeRelative function calls Escape function internally that replaces all the Unicode characters with %<xx>. -->
@@ -792,15 +774,10 @@
       <XUnitWrapperDlls>$([System.String]::Join(' ', $([System.IO.Directory]::GetFiles(%(FullPath), '*.XUnitWrapper.dll', SearchOption.AllDirectories))).Replace($([MSBuild]::EnsureTrailingSlash(%(FullPath))),''))</XUnitWrapperDlls>
     </LegacyPayloads>
     <LegacyPayloads Update="@(LegacyPayloads)">
-      <!-- Specify the test group from the payload name. For explicitly named groups, PayloadGroup specifies the subset of tests we want to run in this Helix Work Item -->
-      <TestGroup>%(PayloadGroup)</TestGroup>
-      <!-- When Payload contains more than one *.XUnitWrapper.dll TestGroup should not be specified. When we specify multiple assemblies, we aren't trying to run only a subset of tests within one assembly. -->
-      <TestGroup Condition=" $([MSBuild]::ValueOrDefault(%(XUnitWrapperDlls), '').IndexOf('.XUnitWrapper.dll')) != $([MSBuild]::ValueOrDefault(%(XUnitWrapperDlls), '').LastIndexOf('.XUnitWrapper.dll')) "></TestGroup>
       <!--
         PayloadGroup0 is just a synthesized name for a bundle of multiple test assemblies that we send as one payload.
         Normally the condition above catches this case; however, when we're only sending one assembly, passing PayloadGroup0 to Helix means that no tests will be run.
       -->
-      <TestGroup Condition="'%(PayloadGroup)' == 'PayloadGroup0'"></TestGroup>
       <PayloadZipFile>$(LegacyPayloadsRootDirectory)\%(PayloadGroup).zip</PayloadZipFile>
     </LegacyPayloads>
   </ItemGroup>
@@ -903,15 +880,13 @@
     <HelixWorkItem Include="@(LegacyPayloads->Metadata('PayloadGroup'))" Condition="'$(TargetHasHelixXHarnessSdkSupport)' != 'true'">
       <PayloadDirectory>%(PayloadDirectory)</PayloadDirectory>
       <Command>$(_WorkaroundForNuGetMigrationsForPrepending) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs)</Command>
-      <Command Condition=" '%(TestGroup)' != '' ">$(_WorkaroundForNuGetMigrationsForPrepending) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup)</Command>
       <Timeout Condition=" '$(TimeoutPerTestCollectionInMinutes)' != '' ">$([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes)))</Timeout>
       <DownloadFilesFromResults Condition=" '$(SuperPmiCollect)' == 'true' ">coreclr_tests.run.$(TargetOS).$(TargetArchitecture).$(Configuration).mch;coreclr_tests.run.$(TargetOS).$(TargetArchitecture).$(Configuration).log</DownloadFilesFromResults>
     </HelixWorkItem>
 
     <XHarnessApkToTest Include="@(LegacyPayloads->Metadata('PayloadZipFile'))" Condition="'$(TargetsAndroid)' == 'true'">
       <TestTimeout Condition=" '$(TimeoutPerTestCollectionInMinutes)' != '' ">$([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes)))</TestTimeout>
       <CustomCommands>dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs)</CustomCommands>
-      <CustomCommands Condition=" '%(TestGroup)' != '' ">dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup)</CustomCommands>
     </XHarnessApkToTest>
 
     <XHarnessApkToTest Include="@(MergedPayloads->Metadata('PayloadGroup'))" Condition="'$(TargetsAndroid)' == 'true'">
@@ -925,7 +900,6 @@
       <TestTarget>$(AppleTestTarget)</TestTarget>
       <TestTimeout Condition=" '$(TimeoutPerTestCollectionInMinutes)' != '' ">$([System.TimeSpan]::FromMinutes($(TimeoutPerTestCollectionInMinutes)))</TestTimeout>
       <CustomCommands> $(SigningCommand) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs)</CustomCommands>
-      <CustomCommands Condition=" '%(TestGroup)' != '' ">$(SigningCommand) dotnet $(XUnitRunnerDll) %(XUnitWrapperDlls) $(XUnitRunnerArgs) -trait TestGroup=%(TestGroup)</CustomCommands>
     </XHarnessAppBundleToTest>
 
     <XHarnessAppBundleToTest Include="@(MergedPayloads->Metadata('PayloadGroup'))" Condition="'$(TargetsAppleMobile)' == 'true'">

src/tests/Common/testgrouping.proj

@@ -1,108 +1,11 @@
 <Project>
   <ItemGroup>
-    <TestGrouping Include="$(TestBinDir)baseservices\threading\generics\**">
-      <TestGroup>baseservices.threading.generics</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)baseservices\threading\baseservices.threading.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\jit64\hfa\**">
-      <TestGroup>JIT.jit64.hfa</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Condition="'$(TargetOS)' == 'windows'" Include="$(TestBinDir)JIT\jit64\mcc\**">
-      <TestGroup>JIT.jit64.mcc</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\jit64\opt\**">
-      <TestGroup>JIT.jit64.opt</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\jit64\valuetypes\**">
-      <TestGroup>JIT.jit64.valuetypes</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\jit64\JIT.jit64.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\Regression\CLR-x86-JIT\V1-M09*\**;
-                           $(TestBinDir)JIT\Regression\CLR-x86-JIT\V1-M10*\**;
-                           $(TestBinDir)JIT\Regression\CLR-x86-JIT\V1-M11*\**">
-      <TestGroup>JIT.Regression.CLR-x86-JIT.V1-M09-M11</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\Regression\CLR-x86-JIT\V1-M12*\**;
-                           $(TestBinDir)JIT\Regression\CLR-x86-JIT\V1-M13*\**">
-      <TestGroup>JIT.Regression.CLR-x86-JIT.V1-M12-M13</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)JIT\Regression\JitBlue\GitHub_*\**">
-      <TestGroup>JIT.Regression.JitBlue</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)JIT\Regression\JIT.Regression.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\generics\**">
-      <TestGroup>Loader.classloader.generics</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest?\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest1??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest2??\**">
-      <TestGroup>Loader.classloader.TypeGeneratorTests.TypeGeneratorTest0-299</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest3??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest4??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest5??\**">
-      <TestGroup>Loader.classloader.TypeGeneratorTests.TypeGeneratorTest300-599</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest6??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest7??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest8??\**">
-      <TestGroup>Loader.classloader.TypeGeneratorTests.TypeGeneratorTest600-899</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest9??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest10??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest11??\**">
-      <TestGroup>Loader.classloader.TypeGeneratorTests.TypeGeneratorTest900-1199</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <TestGrouping Include="$(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest12??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest13??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest14??\**;
-                           $(TestBinDir)Loader\classloader\TypeGeneratorTests\TypeGeneratorTest1500\**">
-      <TestGroup>Loader.classloader.TypeGeneratorTests.TypeGeneratorTest1200-1500</TestGroup>
-      <XUnitWrapperDll>$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll</XUnitWrapperDll>
-    </TestGrouping>
-
-    <XUnitWrapperGrouping Include="$(TestBinDir)baseservices\compilerservices\*.XUnitWrapper.dll;
-                                   $(TestBinDir)baseservices\exceptions\*.XUnitWrapper.dll;
-                                   $(TestBinDir)baseservices\multidimmarray\*.XUnitWrapper.dll;
-                                   $(TestBinDir)baseservices\TieredCompilation\*.XUnitWrapper.dll;
-                                   $(TestBinDir)baseservices\typeequivalence\*.XUnitWrapper.dll;
-                                   $(TestBinDir)baseservices\varargs\*.XUnitWrapper.dll;
-                                   $(TestBinDir)CoreMangLib\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)Exceptions\**\*.XUnitWrapper.dll;
+    <XUnitWrapperGrouping Include="$(TestBinDir)Exceptions\**\*.XUnitWrapper.dll;
                                    $(TestBinDir)GC\**\*.XUnitWrapper.dll;
                                    $(TestBinDir)hosting\**\*.XUnitWrapper.dll;
                                    $(TestBinDir)ilasm\**\*.XUnitWrapper.dll;
                                    $(TestBinDir)Loader\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)managed\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)readytorun\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)reflection\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)Regressions\**\*.XUnitWrapper.dll;
-                                   $(TestBinDir)tracing\**\*.XUnitWrapper.dll"
-                          Exclude="$(TestBinDir)Loader\classloader\Loader.classloader.XUnitWrapper.dll">
+                                   $(TestBinDir)managed\**\*.XUnitWrapper.dll">
       <PayloadGroup>PayloadGroup0</PayloadGroup>
     </XUnitWrapperGrouping>
 
@@ -112,15 +15,10 @@
 
     <XUnitWrapperGrouping Include="$(TestBinDir)JIT\BBT\*.XUnitWrapper.dll;
                                    $(TestBinDir)JIT\CheckProjects\*.XUnitWrapper.dll;
-                                   $(TestBinDir)JIT\opt\*.XUnitWrapper.dll;
-                                   $(TestBinDir)JIT\Performance\*.XUnitWrapper.dll;
+                                   $(TestBinDir)JIT\Intrinsics\*.XUnitWrapper.dll;
                                    $(TestBinDir)JIT\RyuJIT\*.XUnitWrapper.dll;
                                    $(TestBinDir)JIT\superpmi\*.XUnitWrapper.dll">
       <PayloadGroup>JIT.1</PayloadGroup>
     </XUnitWrapperGrouping>
-    <XUnitWrapperGrouping Include="$(TestBinDir)JIT\Intrinsics\*.XUnitWrapper.dll;
-                                   $(TestBinDir)JIT\SIMD\*.XUnitWrapper.dll">
-      <PayloadGroup>JIT.2</PayloadGroup>
-    </XUnitWrapperGrouping>
   </ItemGroup>
 </Project>

src/tests/issues.targets

@@ -657,8 +657,11 @@
         <ExcludeList Include="$(XunitTestBinBase)/Interop/IJW/FixupCallsHostWhenLoaded/FixupCallsHostWhenLoaded/*">
             <Issue>https://github.com/dotnet/runtime/issues/38096</Issue>
         </ExcludeList>
-        <ExcludeList Include="$(XunitTestBinBase)/JIT/Regression/CLR-x86-JIT/v2.1/DDB/B168384/LdfldaHack/*">
-            <Issue>https://github.com/dotnet/runtime/issues/615</Issue>
+        <ExcludeList Include="$(XunitTestBinBase)/JIT/Intrinsics/TypeIntrinsics_r/*">
+            <Issue>https://github.com/dotnet/runtime/issues/96336</Issue>
+        </ExcludeList>
+        <ExcludeList Include="$(XunitTestBinBase)/JIT/Intrinsics/TypeIntrinsics_ro/*">
+            <Issue>https://github.com/dotnet/runtime/issues/96336</Issue>
         </ExcludeList>
         <ExcludeList Include="$(XunitTestBinBase)/JIT/superpmi/superpmicollect/*">
             <Issue>Not compatible with crossgen2</Issue>