-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable the fundamentals behind libraries tests compiled via crossgen2 #75230
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -101,4 +101,21 @@ dotnet build /t:Test /p:Outerloop=true | |
|
||
### Running tests on a different target framework | ||
|
||
Each test project can potentially have multiple target frameworks. There are some tests that might be OS-specific, or might be testing an API that is available only on some target frameworks, so the `TargetFrameworks` property specifies the valid target frameworks. | ||
Each test project can potentially have multiple target frameworks. There are some tests that might be OS-specific, or might be testing an API that is available only on some target frameworks, so the `TargetFrameworks` property specifies the valid target frameworks. | ||
|
||
### Running tests in custom compilation modes | ||
|
||
There are several custom compilation modes for tests. These are enabled by setting a switch during the config. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What you mean by "during the config"? I assume you are talking about the test invocation? Might make sense to clarify that part. |
||
|
||
| Mode | Description | Prerequisites | ||
| --- | --- | --- | | ||
| TestSingleFile | Test compilation using the single file compilation mode | libs+clr | | ||
| TestNativeAot | Test by compiling the test using NativeAOT | libs+clr.aot | | ||
| TestCrossgen2 | Test compilation of the test/libraries into R2R binaries | libs+clr | | ||
|
||
To run a test in specific mode, simply build the tests after building the prerequisite subsets, and specify the test mode on a command line such as: | ||
```cmd | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. NIT: Try to keep lists and code examples surrounded by blank lines, as per the recommended Markdown guidelines, so that we keep our |
||
dotnet build /t:Test /p:Configuration=Release /p:TestCrossgen2=true | ||
``` | ||
|
||
These tests do not use the standard XUnit test runner, instead they use [SingleFileTestRunner.cs](../../../../src/libraries/Common/tests/SingleFileTestRunner/SingleFileTestRunner.cs). The set of available command line options is limited to `-xml`, `-notrait`, `-class`, `-class-`, `-noclass`, `-method`, `-method-`, `-nomethod`, `-namespace`, `-namespace-`, `-nonamespace`, and `-parallel`. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,6 +8,7 @@ | |
<VSTestNoLogo>true</VSTestNoLogo> | ||
<ILLinkDescriptorsPath>$(MSBuildThisFileDirectory)ILLinkDescriptors\</ILLinkDescriptorsPath> | ||
<TestSingleFile Condition="'$(TestNativeAot)' == 'true'">true</TestSingleFile> | ||
<TestSingleFile Condition="'$(TestCrossgen2)' == 'true'">true</TestSingleFile> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would it make sense to just name this TestReadyToRun? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. That's a better name |
||
</PropertyGroup> | ||
|
||
<PropertyGroup Condition="'$(TargetsMobile)' == 'true'"> | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -27,6 +27,9 @@ public static int Main(string[] args) | |
var asm = typeof(SingleFileTestRunner).Assembly; | ||
Console.WriteLine("Running assembly:" + asm.FullName); | ||
|
||
// The current RemoteExecutor implementation is not compatible with the SingleFileTestRunner | ||
Environment.SetEnvironmentVariable("DOTNET_REMOTEEXECUTOR_SUPPORTED", "0"); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we have a tracking issue for that, to link to? |
||
|
||
var diagnosticSink = new ConsoleDiagnosticMessageSink(); | ||
var testsFinished = new TaskCompletionSource(); | ||
var testSink = new TestMessageSink(); | ||
|
@@ -74,10 +77,58 @@ public static int Main(string[] args) | |
noTraits.Add(traitKeyValue[0], values = new List<string>()); | ||
} | ||
values.Add(traitKeyValue[1]); | ||
i++; | ||
} | ||
if (args[i].Equals("-xml", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
xmlResultFileName=args[i + 1].Trim(); | ||
i++; | ||
} | ||
if (args[i].Equals("-class", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.IncludedClasses.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-noclass", StringComparison.OrdinalIgnoreCase) || args[i].Equals("-class-", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.ExcludedClasses.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-method", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.IncludedMethods.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-nomethod", StringComparison.OrdinalIgnoreCase) || args[i].Equals("-method-", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.ExcludedMethods.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-namespace", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.IncludedNamespaces.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-nonamespace", StringComparison.OrdinalIgnoreCase) || args[i].Equals("-namespace-", StringComparison.OrdinalIgnoreCase)) | ||
{ | ||
filters.ExcludedNamespaces.Add(args[i + 1].Trim()); | ||
i++; | ||
} | ||
if (args[i].Equals("-parallel", StringComparison.OrdinalIgnoreCase)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you for extending this! Especially the I would like us to get rid of this runner eventually (#70432). The xunit runner from Arcade used to support single file testing (including remote executor) but all of that was deleted when UWP support was deleted from Arcade. I'd like to restore it at some point. I wonder though whether crossgen2 testing needs the single file runner - could it get by with the real runner? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suspect it could use the normal file runner, but this looks like it will work, and I don't understand the libraries testing enough to understand how exactly to make this work with the normal runner without doing more research. With the additional arguments, I don't see a huge benefit to making this different from SingleFile or NativeAOT testing. As a side benefit, enabling R2R singlefile testing here becomes extremely easy should we want to do so. |
||
{ | ||
string parallelismArg = args[i + 1].ToLower(); | ||
var (parallelizeAssemblies, parallelizeTestCollections) = parallelismArg switch | ||
{ | ||
"all" => (true, true), | ||
"assemblies" => (true, false), | ||
"collections" => (false, true), | ||
"none" => (false, false), | ||
_ => throw new ArgumentException("unknown parallelism option") | ||
}; | ||
|
||
assemblyConfig.ParallelizeAssembly = parallelizeAssemblies; | ||
assemblyConfig.ParallelizeTestCollections = parallelizeTestCollections; | ||
i++; | ||
} | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ViktorHofer I'm interested in any particular details I missed here. We really need some documentation on this sort of thing, as the behaviors of our tests are not particularly clear to folks who don't look into these systems often.