Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Unit Test Provider documentation #2037

Merged

Conversation

SabotageAndi
Copy link
Contributor

Types of changes

  • Bug fix (non-breaking change which fixes an issue).
  • New feature (non-breaking change which adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).
  • Performance improvement
  • Refactoring (so no functional change)
  • Other (docs, build config, etc)

Checklist:

  • I've added tests for my code. (most of the time mandatory)
  • I have added an entry to the changelog. (mandatory)
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@SabotageAndi
Copy link
Contributor Author

@tzongithub You are now my goto reviewer of docu changes ;-) I hope this is Ok for you.

SpecFlow supports NUnit 3.11 or later.

Please add the NuGet package [SpecFlow.NUnit](https://www.nuget.org/packages/SpecFlow.NUnit/) to your project to use xUnit in your project.
You probably also need the NUnit3 TestAdapter NuGet Package [NUnit3TestAdapter](https://www.nuget.org/packages/NUnit3TestAdapter/) in your project that the scenarios will be visible in the Visual Studio Test Explorer.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is generally a better practice to first create a "working" NUnit test project (using the appropriate Visual Studio template) and then add the SpecFlow.NUnit package to it.
I did this recently for the GWT sample and the NUnit template adds 3 packages:

	<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0"/>
	<PackageReference Include="nunit" Version="3.12.0" />
	<PackageReference Include="NUnit3TestAdapter" Version="3.15.1" />

I think all these are necessary to have a properly working test assembly with NUnit and Test Explorer integration.

(If you start from scratch with a simple class library template and add SpecFlow.NUnit it won't work with the Test Explorer, and it is hard to troubleshoot why. It's also hard to figure all those 3 packages for NUnit).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the same setup advice applies also for MsTest2 and xUnit (start with the VS template for the given unit test provider)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NUnit dependency should come from the SpecFlow.NUnit package. The ´NUnit3TestAdapteris not always needed. VS has it's own discoverer and executor integrated. Only for some special features you really need it AFAIK. That theMicrosoft.NET.Test.Sdk` is needed as a direct dependency is new to me. Most of the times it comes multiple times from dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remove from the GWT sample project the Microsoft.NET.Test.SDK package I get this error from discovery:

[06.07.2020 5:05:33.939 PM] Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Unable to find C:\Work\GWTWS_Challenge1\UserRepository.Specs\bin\Debug\netcoreapp3.1\testhost.dll. Please publish your test project and retry.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostPath(String runtimeConfigDevPath, String depsFilePath, String sourceDirectory)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.DiscoverTests(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler)
[06.07.2020 5:05:33.940 PM] ========== Test discovery aborted: 0 Tests found in 102,4 ms ==========

If I remove the NUnit3TestAdapter I get this

[06.07.2020 6:14:32.701 PM] Test project UserRepository.Specs does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.
[06.07.2020 6:14:33.752 PM] No test is available in C:\Work\GWTWS_Challenge1\UserRepository.Specs\bin\Debug\netcoreapp3.1\UserRepository.Specs.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[06.07.2020 6:14:33.776 PM] ========== Test discovery finished: 0 Tests found in 1,1 sec ==========

Since SpecFlow.NUnit brings nunit as dependency nunit can be removed without a problem.
But the two other packages seem to be mandatory in my project setup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, then this changed already again. At least in my head. ;-)
I will update it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the log.
We can double-check the needed packages either by running dotnet new nunit, or creating an NUnit test project with Visual Studio, or looking up the MS documentation for nUnit at https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-nunit or in the NUnit documentation directly at https://github.com/nunit/docs/wiki/.NET-Core-and-.NET-Standard

The MS docs has side-by-side chapters for all 3 unit test providers:
xUnit: https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-dotnet-test
nUnit: https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-nunit
MSTest: https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-mstest

docs/Integrations/NUnit.md Outdated Show resolved Hide resolved
@SabotageAndi SabotageAndi merged commit 8fe50bb into SpecFlowOSS:master Jul 8, 2020
@SabotageAndi SabotageAndi deleted the docs/ImproveUnitTestProviderDocs branch July 8, 2020 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants