Make sure we run all the TFMs on master builds

[andrewlock]

Summary of changes

Update RequiresThoroughTesting() to return true when we're running on master or release/2.x

Reason for change

We should be doing thorough testing on the main branches. I thought we were. I don't quite understand how we're not

Implementation details

Check to see if we're on the main branch for the verison, and if so, do the thorough testing

Test coverage

Can't really test this properly until we merge it...

Other details

I sure hope this doesn't reveal more issues :awkward:

[kevingosse]

We're totally going to remember to update this when releasing 4.0

[andrewlock]

There is no 4.0, remember? :stare:

[andrewlock]

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

• This PR (5990)
• master

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

• Welch test with statistical test for significance of 5%
• Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (71ms)  : 67, 74
     .   : milestone, 71,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (1,088ms)  : 1058, 1119
     .   : milestone, 1088,
    master - mean (1,077ms)  : 1053, 1101
     .   : milestone, 1077,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (109ms)  : 105, 114
     .   : milestone, 109,
    master - mean (109ms)  : 105, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (761ms)  : 741, 781
     .   : milestone, 761,
    master - mean (758ms)  : 739, 778
     .   : milestone, 758,

Loading

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (93ms)  : 89, 97
     .   : milestone, 93,
    master - mean (92ms)  : 90, 95
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (715ms)  : 691, 739
     .   : milestone, 715,
    master - mean (710ms)  : 697, 724
     .   : milestone, 710,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (191ms)  : 187, 195
     .   : milestone, 191,
    master - mean (190ms)  : 187, 192
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (1,161ms)  : 1137, 1185
     .   : milestone, 1161,
    master - mean (1,160ms)  : 1137, 1184
     .   : milestone, 1160,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (274ms)  : 270, 279
     .   : milestone, 274,
    master - mean (274ms)  : 270, 279
     .   : milestone, 274,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (917ms)  : 899, 935
     .   : milestone, 917,
    master - mean (922ms)  : 902, 942
     .   : milestone, 922,

Loading

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5990) - mean (263ms)  : 259, 267
     .   : milestone, 263,
    master - mean (264ms)  : 259, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (5990) - mean (905ms)  : 884, 926
     .   : milestone, 905,
    master - mean (902ms)  : 883, 922
     .   : milestone, 902,

Loading

[datadog-ddstaging]

Datadog Report

Branch report: andrew/ensure-we-run-all-tfms-in-integration-tests-master
Commit report: e3a3368
Test service: dd-trace-dotnet

✅ 0 Failed, 366271 Passed, 2059 Skipped, 14h 53m 53.3s Total Time