Use TestCase with Name instead of Description Attribute for Nunit3 provider. Fix #1225

[goblinmaks]

Types of changes

• Bug fix (non-breaking change which fixes an issue).
• New feature (non-breaking change which adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist:

• I've added tests for my code.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added an entry to the changelog

[goblinmaks]

#1225

[SabotageAndi]

@goblinmaks For some reason the results of the release pipeline are not shown here.
This PR failed a lot of tests. There is somewhere a null-check that throws an exception.

Release pipeline is here: https://dev.azure.com/specflow/SpecFlow/_apps/hub/ms.vss-releaseManagement-web.hub-explorer?releaseId=1003&releaseEnvironmentId=4495&_a=release-pipeline-progress

[goblinmaks]

@SabotageAndi It is not completed commit, I will fix tests, but have additional questions about this PR in #1225 . Could you look at my comment please ?

[goblinmaks]

@SabotageAndi Looks like release pipeline is OK.
https://dev.azure.com/specflow/SpecFlow/_releaseProgress?_a=release-pipeline-progress&releaseId=1015

[SabotageAndi]

Looks good, except the naming in the unit tests.

Could you post a screenshot of how it looks now in the test explorer?

[goblinmaks]

Looks good, except the naming in the unit tests.

Could you post a screenshot of how it looks now in the test explorer?

We have a chance to review this PR soon ?

[SabotageAndi]

Sorry for not getting earlier back to you. I have a lot of work currently to do.

I was not clear enough what I meant with the screenshot. I wanted to see how this change affects the display of the scenarios in the test explorer. Not how the unit tests you added look like.

[goblinmaks]

@SabotageAndi nothing changes after this PR will be applied.

[SabotageAndi]

@goblinmaks Sure things changed. The feature and scenario names are now nice. Before there weren't spaces. Now they are here.
I thought something like this will probably happen. And it changed to be better.

Thanks for your contribution to SpecFlow. If you would like us to send you some SpecFlow stickers as a thank you, please fill out the form here.

[LiohAu]

@goblinmaks Is it normal that the TEST_ATTR constant ("NUnit.Framework.TestAttribute") is not used anymore after your PR ?

Just saw that because I am looking for a way to add the attributes below on the generated class.

[TestFixture(Platform.iOS)]
[TestFixture(Platform.Android)]

Until now, I was using a partial class + an abstract base class to complete the one generated by SpecFlow with those attributes, but it seems nunit does not detect inherited test fixtures anymore :(

[goblinmaks]

@LiohAu

Is it normal that the TEST_ATTR constant ("NUnit.Framework.TestAttribute") is not used anymore after your PR ?

I think it is normal, instead of NUnit.Framework.TestAttribute we start use NUnit.Framework.TestCaseAttribute that is allowed to provide TestName property to engine.

Could you share your code, to provide info how you use additional parameter like "platform" in [TextFixture] attribute. Currently we still generate [TextFixture] code attribute but with parameter TestName [TestFixture(TestName="BlaBla")]

How you use code with my commit ? It is not released now due to SpecFlow CI build error.

[nvborisenko]

I wouldn't recommend to use TestCaseAttribute for regular scenarios. This attribute is designed for data driven tests, when the same test can be executed many times on different sets of data. Using it may cause other issues in nunit reporting, like nunit engine creates new hierarchical level of tests. So, regular scenario in specflow will be considered as a suite w/ single test in terms of nunit.

[goblinmaks]

@nvborisenko
Currently we have no other ability to set Test Name for for NUnit test.
For my point of view TestCaseAttribute should be used for Scenario Outline as minimum.
We can try add TestName to TestAttribute on nunit side, in case if this will be applied we can use it for regular Scenario.

It just my opinion.

[nvborisenko]

Right, this way is better. We will not see collisions in tests tree.

Now you see "good" tests tree in VS Test Explorer because VS applies some magic in building it. VS hides tree nodes. I guess if you execute tests with any runner, you will see unexpected "namespace" for regular scenarios (which are not outlined).

[goblinmaks]

nunit/nunit#3285