Use of Skip causes failure with diagnostic verbosity logger
#391
Comments
|
The test doesn't fail:
I cannot repro your non-zero exit code problem either. My prompt shows non-zero exit codes and you can see
Skip reasons are reported as messages so they can be filtered in Test Explorer, per #110 |
|
Ok, try this: It's targeting net8.0 but you can still repro with earlier versions. To repro that, run: |
|
This isn't Disable this by adding |
|
This is your hint that it's Exec doing this:
|
|
Thanks for educating me. Using |
|
Here's another example of this unnecessary and counterproductive parsing: using Xunit.Abstractions;
namespace TestProject1
{
public class UnitTest1
{
readonly ITestOutputHelper helper;
public UnitTest1(ITestOutputHelper helper)
{
this.helper = helper;
}
[Fact]
public void Test1()
{
helper.WriteLine("error: isn't really what's happening here");
}
}
}
That's not even a skipped test, that's a passing test with output that "looks suspicious". |
|
That's a very "helpful" feature. 😉 |
Not necessarily relevant. I just ran dotnet vstest **/bin/**/*Tests.dll --settings:test.runsettings --logger:"console;verbosity=detailed" pointing directly at my test assemblies (no .csproj | *sln as argument). Wathcing it in Process Explorer it is clear that MSBuild is not part of the stack (needed to be sure ;-))
However dotnet vstest **/bin/**/*Tests.dll --settings:test.runsettings --logger:"console;verbosity=normal" works like a charm M |
|
@brumlemann You should open an issue against VSTest. |
When running
dotnet testwith verbosity set to diagnostic, it will cause a non-zero exit code if a test has setFact.Skipto a non-empty value. This is a regression that was introduced by version 2.5.0-pre.27 of xunit.runner.visualstudio.Repro:
[Fact]to[Fact(Skip = "foo")]dotnet test --logger:'console;verbosity=detailed'Output:
Changing
verbositytonormalwill cause it to succeed.The text was updated successfully, but these errors were encountered: