Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter out .NET Core 3.1 NuGet tests in prerelease versions #5773

Merged
merged 1 commit into from
Jul 4, 2024

Conversation

andrewlock
Copy link
Member

Summary of changes

Skips the .NET Core 3.1 NuGet dotnet-tool installer tests on prerelease versions

Reason for change

The --prerelease flag isn't available < .NET 6, so we can't easily install the prerelease package in these runtime images

Implementation details

Just skip these smoke tests on <.NET 6. It's only a specific case of the NuGet install with these earlier runtimes, so it's very low risk

Test coverage

It should run in this branch, but when we rebase v3-main on top, it shouldn't run

Other details

@andrewlock andrewlock added the area:tests unit tests, integration tests label Jul 4, 2024
@andrewlock andrewlock requested a review from a team as a code owner July 4, 2024 09:47
@github-actions github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jul 4, 2024
Copy link
Collaborator

@gleocadie gleocadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jul 4, 2024

Datadog Report

Branch report: andrew/ci/skip-netcoreapp31-tests-in-prerelease
Commit report: 93fce3b
Test service: dd-trace-dotnet

✅ 0 Failed, 345583 Passed, 1763 Skipped, 13h 45m 16.94s Total Time

@andrewlock
Copy link
Member Author

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (72ms)  : 65, 80
     .   : milestone, 72,
    master - mean (72ms)  : 63, 82
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (1,018ms)  : 995, 1041
     .   : milestone, 1018,
    master - mean (1,013ms)  : 991, 1035
     .   : milestone, 1013,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (112ms)  : 106, 118
     .   : milestone, 112,
    master - mean (109ms)  : 106, 113
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (706ms)  : 683, 729
     .   : milestone, 706,
    master - mean (708ms)  : 685, 731
     .   : milestone, 708,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (94ms)  : 90, 98
     .   : milestone, 94,
    master - mean (93ms)  : 90, 97
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (662ms)  : 633, 692
     .   : milestone, 662,
    master - mean (658ms)  : 634, 682
     .   : milestone, 658,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (190ms)  : 187, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (1,106ms)  : 1088, 1124
     .   : milestone, 1106,
    master - mean (1,108ms)  : 1088, 1128
     .   : milestone, 1108,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (275ms)  : 270, 279
     .   : milestone, 275,
    master - mean (276ms)  : 272, 279
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (876ms)  : 847, 904
     .   : milestone, 876,
    master - mean (881ms)  : 860, 903
     .   : milestone, 881,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5773) - mean (265ms)  : 260, 270
     .   : milestone, 265,
    master - mean (264ms)  : 260, 267
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (5773) - mean (865ms)  : 833, 898
     .   : milestone, 865,
    master - mean (859ms)  : 837, 882
     .   : milestone, 859,

Loading

@andrewlock andrewlock merged commit 4b8ef2c into master Jul 4, 2024
54 of 57 checks passed
@andrewlock andrewlock deleted the andrew/ci/skip-netcoreapp31-tests-in-prerelease branch July 4, 2024 13:11
@github-actions github-actions bot added this to the vNext-v2 milestone Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages area:tests unit tests, integration tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants