Allure-xunit: support for the second reporter and xunit 2.5.0. Updated packaging and AspectInjector (fixes 368)

[delatrie]

This PR lifts the limitation described in #368. Some additional changes also have been made to accommodate xunit's 2.5.0 breaking change on how reporter assemblies are discovered [1] and to fix some issues with NuGet packaging.

Context

Reporters incompatibility

xUnit.net allows only one runner reporter to be executed at a time. A reporter can be specified explicitly by the user or selected implicitly by xUnit based on environmental availability. By default, xUnit selects the first reporter that is environmentally enabled.

If more than one reporter is environmentally enabled, xUnit selects the first one and ignores the rest. Since Allure.XUnit is implemented as an always enabled reporter, there might be a chance another reporter is preferred by xUnit over it.

Typical scenario is running xUnit tests in a CI environment, e.g., in TeamCity. In such a case the teamcity reporter is enabled together with allure and xUnit may select any of them.

It is a quite common scenario though to generate allure report in a CI environment while reporting test results to the CI system as well. This PR adds support for it.

Implementation

Reporters in xUnit.net are easily composable. The trickiest part is to discover the reporter class. We try to find the one that would've been used by xUnit if Allure.XUnit wasn't present. If such reporter exists, it is composed with Allure.XUnit. Otherwise, Allure.XUnit is used alone.

In most cases this is an acceptable approach. It allows automatically detects CI environments and report test results both to allure and the CI server.

However, some flexibility is allowed through the following property in allureConfig.json:

{
    "allure": {
        "xunitRunnerReporter": "auto|none|<switch>|<AQ-classname>"
    }
}

• auto is the default.
• To report to allure only, the none value should be used.
• The reporter can be specified explicitly by its switch.
• The reporter can be specified explicitly by its assembly qualified class name to skip time consuming discovery algorithm and use the given reporter class instead.

In case of auto or <switch> values the following discovery algorithm is used:

1. Iterate through the assemblies in the current application domain (assembly load context in .NET Core, see [2]) omitting obviously unfit ones.
2. For each assembly iterate through its types searching for Xunit.IRunnerReporter implementations.
3. For each found implementation create an instance and check if it should be used:
   1. In the auto mode the reporter is selected if it is environmentally enabled.
   2. If the <switch> mode the reporter is selected if its runner switch matches the specified one (using case-insensitive comparison).

Examples

Examples 1: disable an additional reporter leaving only Allure.XUnit's one

{
    "allure": {
        "xunitRunnerReporter": "none"
    }
}

Example 2: use the teamcity reporter together with Allure.XUnit:

{
    "allure": {
        "xunitRunnerReporter": "teamcity"
    }
}

Example 3: use the assembly qualified class name to quickly create and use the teamcity reporter together with Allure.XUnit:

{
    "allure": {
        "xunitRunnerReporter": "Xunit.Runner.Reporters.TeamCityReporter, xunit.runner.reporters.netcoreapp10, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c"
    }
}

Note

Make sure you use the correct version of the assembly.

Other changes

Adapting to xUnit.net 2.5.0 breaking change

Starting from 2.5.0, xUnit.net only considers assemblies matching the *reporters*.dll pattern. This PR moves the reporter class into the new project Allure.XUnit.Reporters that produces the Allure.XUnit.reporters.dll assembly. This assembly is packed together with Allure.XUnit.dll in the Allure.XUnit package, so the user experience shouldn't be affected.

Introduction of Directory.Build.props

The Directory.Build.props file was introduced to reduce the boilerplate code in .csproj files [3]. It contains the common metadata (e.g., the version), the common properties, reference to the source link and some defaults.

Increase AspectInjector's version to 2.8.2

There is a bug in the underlying Mono.Cecil dependency that excludes the PdbChecksum debug header entry from a generated assembly. Pdb's for such assemblies can't be validated by NuGet and the corresponding .snupkg files are always rejected. See also jbevain/cecil#610. This was fixed in jbevain/cecil#810.

The new version of AspectInjector depends on Mono.Cecil 0.11.5 that includes the fix.

Fully deterministic build

This PR enables fully deterministic build via the -p:ContinuousIntegrationBuild=true in the publishing pipeline [4].

Checklist

• Sign Allure CLA
• Provide unit tests

Fixes #368