Fix identification properties and parameter formatting

[delatrie]

Context

There are issues with the current algorithms behind identification properties (uuid, fullName, historyId) that may lead to unexpected behavior. This includes:

• Different tests might be considered as retries
• Test results of the same parametrized test case might be considered as different test cases.

More details in this issue: #387.

Additionally, all integrations don't set testCaseId (see #373).

Parameter serialization

Parameter values are one of the inputs to the decision on whether two test results are retries of a single test or two different tests of a parametrized test case.

Current formatting implementation though often formats different values into undistinguishable strings. For example, many collections are serialized just as type names. The same goes for user-defined types that don't override ToString.

Additionally, custom type formatters although are present in the code, actually aren't used in the preview branch (see #377).

Implementation details

A new set of API was added for integrations to calculate identification properties:

public static string Allure.Net.Commons.Functions.IdFunctions.CreateUUID();
public static string Allure.Net.Commons.Functions.IdFunctions.CreateFullName(System.Type targetClass);
public static string Allure.Net.Commons.Functions.IdFunctions.CreateFullName(System.Reflection.MethodBase method);
public static string Allure.Net.Commons.Functions.IdFunctions.CreateTestCaseId(string fullName);
public static string Allure.Net.Commons.Functions.IdFunctions.CreateHistoryId(string fullName, System.Collections.Generic.IEnumerable<Parameter> parameters);

UUID

A UUID is created with System.Guid.NewGuid().

Class-based fullName

This fullName can be used as a name of class-level test containers. It has the following format:

<assembly>:<namespace>.<class-name>`<generic-param-count>[<generic-parameters>]

• assembly - the name of the assembly of the class
• namespace - the namespace of the class (optional; excluded with the following dot if the namespace is empty)
• class-name - the name of the class
• generic-param-count - the number of generic parameters of the class
• generic-parameters - the list of the generic parameters or arguments of the class

The generic part starting from the backtick is excluded if the class doesn't have generic parameters.

Method-based fullName

This fullName can be used as a name of a test result that is based on a test method. It has the following format:

<class-fullName>.<method-name>[<generic-parameters>](<parameters>)

• class-fullName - the fullName of the declaring class calculated using the rules above
• generic-parameters - the list of the generic parameter of the method
• parameters - the comma-separated list of the parameters of the method. Each parameter is represented as fullName of its type (assembly name is not included for system types and generic parameter types).

testCaseId

The value of testCaseId is calculated based on fullName. It's required that the fullName doesn't depend on test parameters (fullNames calculated using the functions above satisfy this rule).

MD5 hashing algorithm is used to convert a fullName to the corresponding testCaseId.

historyId

This value depends on the fullName and the list of the parameters of the test. Parameters are sorted by their names. The names are then discarded (only values are used in the calculation).

The actual value is calculated as the MD5 hex digest of the following JSON object:

{
    "fullName": "<fullName>",
    "parameters": "<ordered-parameter-values>"
}

useLegacyIds configuration property

A new configuration property was added to allureConfig.json to use old algorithms for identification properties of test results. This switch can be used as a temporary solution to preserve connection to existing test cases until allure ids are assigned to tests.

Parameters serialization

A new set of API was added for integrations to format step and test parameters:

publis static string Allure.Net.Commons.Functions.FormatFunctions.Format(object? value);
publis static string Allure.Net.Commons.Functions.FormatFunctions.Format(object? value, System.Collections.Generic.IReadOnlyDictionary<System.Type, Allure.Net.Commons.ITypeFormatter> formatters);
public void Allure.Net.Commons.TestResult.AddParameter(string name, object value, System.Collections.Generic.IReadOnlyDictionary<System.Type, Allure.Net.Commons.ITypeFormatter> formatters);

They format the value using the following algorithm:

1. If the type of the value matches a formatter in the formatters dictionary, the matched formatter is used to convert the value to a string.
2. Otherwise, the value is JSON-serialized.

These functions are used in the following contexts in allure-nunit, allure-xunit and allure-specflow:

1. To format a test parameter value when adding the parameter to the test result.
2. To format a step parameter when interpolating it into the step's title.
3. To format a step parameter value when adding the parameter to the step result.

In all contexts all type formatters added to the lifecycle via AllureLifecycle.AddTypeFormatter are used.

Other changes

JSON schemas for configuration and testplan

This PR adds schemas for allureConfig.json and testplan.json files. A schema can be added to the document as follows:

{
    "$schema": "https://raw.githubusercontent.com/allure-framework/allure-csharp/<version>/<package>/Schemas/<schema-name>"
}

For example, to apply the schema v2.10 to Allure.NUnit's configuration file, the following URL can be used:

{
    "$schema": "https://raw.githubusercontent.com/allure-framework/allure-csharp/2.10.0/Allure.NUnit/Schemas/allureConfig.schema.json"
}

The release.yml workflow definition file has been updated to put version tags to schema paths.

Language and framework labels

The following methods were added to the public API for integration authors:

public static Allure.Net.Commons.Label Allure.Net.Commons.Label.Language();
public static Allure.Net.Commons.Label Allure.Net.Commons.Label.Framework(string value);

Allure-nunit, allure-xunit and allure-specflow now define these labels for all test results they create.

Checklist

• Sign Allure CLA
• Provide unit tests

Fixes #345
Implements #373
Fixes #377
Implements #387