Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run ARM64 integration tests against .NET 8 #4875

Merged
merged 1 commit into from
Nov 17, 2023
Merged

Run ARM64 integration tests against .NET 8 #4875

merged 1 commit into from
Nov 17, 2023

Conversation

andrewlock
Copy link
Member

Summary of changes

Run the ARM64 Linux integration tests against .NET 8

Reason for change

This was missed before

Implementation details

Replaced the .NET 7 job with a .NET 8 job. We could have added an extra one, but seeing as the arm64 VMs are typically the slowest to be available, we probably don't want to add extra pressure here? Chose to remove the .NET 7 one because .NET 5 is the lowest supported version (and exercises a different runtime target). I was torn between .NET 6 and 7, but 7 will be out of support sooner so 🤷‍♂️

Test coverage

This PR is the test...

@andrewlock andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages area:tests unit tests, integration tests labels Nov 16, 2023
@andrewlock andrewlock requested a review from a team as a code owner November 16, 2023 12:51
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 16, 2023 •
edited
Loading

Datadog Report

Branch report: andrew/arm64-dotnet8-test
Commit report: 1ae33a6

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 298690 Passed, 975 Skipped, 1 Regression, 36m 40.19s Wall Time

⌛ Performance Regressions vs Default Branch (1)

  • Baseline_liveheap - scenarios 2.85s (+116.11ms, +4%) - Details

@andrewlock
Copy link
Member Author

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (71ms)  : 63, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (991ms)  : 977, 1004
     .   : milestone, 991,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (106ms)  : 102, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (683ms)  : 668, 697
     .   : milestone, 683,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (89ms)  : 87, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (656ms)  : 632, 679
     .   : milestone, 656,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (188ms)  : 186, 191
     .   : milestone, 188,
    master - mean (188ms)  : 186, 190
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (1,133ms)  : 1107, 1160
     .   : milestone, 1133,
    master - mean (1,133ms)  : 1115, 1150
     .   : milestone, 1133,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (272ms)  : 267, 276
     .   : milestone, 272,
    master - mean (271ms)  : 267, 276
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (1,094ms)  : 1064, 1124
     .   : milestone, 1094,
    master - mean (1,091ms)  : 1064, 1117
     .   : milestone, 1091,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4875) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (262ms)  : 258, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (4875) - mean (1,060ms)  : 1031, 1088
     .   : milestone, 1060,
    master - mean (1,057ms)  : 1030, 1084
     .   : milestone, 1057,
Loading

@andrewlock
Copy link
Member Author

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4875) (11.092M)   : 0, 11092439
    master (11.233M)   : 0, 11233174
    benchmarks/2.9.0 (11.667M)   : 0, 11667357

    section Automatic
    This PR (4875) (7.758M)   : 0, 7758315
    master (7.866M)   : 0, 7866434
    benchmarks/2.9.0 (8.416M)   : 0, 8415997

    section Trace stats
    This PR (4875) (7.934M)   : 0, 7934454
    master (8.098M)   : 0, 8098327

    section Manual
    This PR (4875) (9.796M)   : 0, 9795668
    master (10.188M)   : 0, 10188257

    section Manual + Automatic
    This PR (4875) (7.233M)   : 0, 7233070
    master (7.420M)   : 0, 7420073

    section Version Conflict
    This PR (4875) (6.703M)   : 0, 6702736
    master (6.737M)   : 0, 6737026
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4875) (9.726M)   : 0, 9725530
    master (9.451M)   : 0, 9451069
    benchmarks/2.9.0 (9.769M)   : 0, 9769093

    section Automatic
    This PR (4875) (6.655M)   : 0, 6654980
    master (6.699M)   : 0, 6699099

    section Trace stats
    This PR (4875) (6.938M)   : 0, 6937635
    master (6.903M)   : 0, 6902966

    section Manual
    This PR (4875) (8.416M)   : 0, 8415882
    master (8.349M)   : 0, 8349271

    section Manual + Automatic
    This PR (4875) (6.330M)   : 0, 6330036
    master (6.309M)   : 0, 6308931

    section Version Conflict
    This PR (4875) (5.867M)   : 0, 5866560
    master (5.760M)   : 0, 5760080
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4875) (10.226M)   : 0, 10225747
    benchmarks/2.9.0 (10.674M)   : 0, 10673655

    section Automatic
    This PR (4875) (7.346M)   : 0, 7345739
    benchmarks/2.9.0 (7.746M)   : 0, 7746260

    section Trace stats
    This PR (4875) (7.636M)   : 0, 7635560

    section Manual
    This PR (4875) (9.207M)   : 0, 9207280

    section Manual + Automatic
    This PR (4875) (7.076M)   : 0, 7075591

    section Version Conflict
    This PR (4875) (6.348M)   : 0, 6348305
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.406M)   : 0, 7406280
    benchmarks/2.9.0 (7.862M)   : 0, 7861918

    section No attack
    master (2.083M)   : 0, 2082803
    benchmarks/2.9.0 (3.168M)   : 0, 3167950

    section Attack
    master (1.660M)   : 0, 1659908
    benchmarks/2.9.0 (2.547M)   : 0, 2547274

    section Blocking
    master (3.459M)   : 0, 3458664

    section IAST default
    master (6.539M)   : 0, 6538785

    section IAST full
    master (5.922M)   : 0, 5921560

    section Base vuln
    master (0.958M)   : 0, 958088

    section IAST vuln
    master (0.860M)   : 0, 860156
Loading

@andrewlock
Copy link
Member Author

Benchmarks Report 🐌

Benchmarks for #4875 compared to master:

  • 4 benchmarks are faster, with geometric mean 1.199
  • 1 benchmarks are slower, with geometric mean 1.196
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.65μs 45.8ns 255ns 0.0214 0.00855 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.8μs 60.9ns 400ns 0.0273 0.0109 0 7.58 KB
master StartStopWithChild net472 17.2μs 49.3ns 191ns 1.34 0.343 0.111 7.94 KB
#4875 StartStopWithChild net6.0 8.5μs 42ns 188ns 0.0207 0.00829 0 7.48 KB
#4875 StartStopWithChild netcoreapp3.1 10.6μs 58.3ns 335ns 0.0213 0.0107 0 7.58 KB
#4875 StartStopWithChild net472 17μs 58.1ns 225ns 1.33 0.349 0.102 7.96 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 452μs 261ns 1.01μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 641μs 320ns 1.24μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 805μs 338ns 1.27μs 0.401 0 0 3.3 KB
#4875 WriteAndFlushEnrichedTraces net6.0 476μs 459ns 1.72μs 0 0 0 2.7 KB
#4875 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 204ns 791ns 0 0 0 2.7 KB
#4875 WriteAndFlushEnrichedTraces net472 809μs 379ns 1.42μs 0.406 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4875

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.146 217.42 189.67

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.6μs 15.7ns 60.6ns 0.0218 0 0 2.02 KB
master AllCycleSimpleBody netcoreapp3.1 44.7μs 37.9ns 137ns 0.0224 0 0 2 KB
master AllCycleSimpleBody net472 47μs 10ns 38.7ns 0.323 0 0 2.07 KB
master AllCycleMoreComplexBody net6.0 204μs 58.2ns 210ns 0.103 0 0 8.37 KB
master AllCycleMoreComplexBody netcoreapp3.1 207μs 83.4ns 323ns 0.104 0 0 8.26 KB
master AllCycleMoreComplexBody net472 212μs 116ns 448ns 1.27 0 0 8.43 KB
master ObjectExtractorSimpleBody net6.0 129ns 0.0531ns 0.199ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 217ns 0.0978ns 0.366ns 0.00372 0 0 272 B
master ObjectExtractorSimpleBody net472 155ns 0.0994ns 0.385ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.72μs 1.19ns 4.46ns 0.0531 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.61μs 1.19ns 4.28ns 0.0505 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.54μs 2.83ns 11ns 0.603 0.00529 0 3.8 KB
#4875 AllCycleSimpleBody net6.0 43.6μs 16.4ns 63.6ns 0.0217 0 0 2.02 KB
#4875 AllCycleSimpleBody netcoreapp3.1 44.7μs 166ns 644ns 0.0227 0 0 2 KB
#4875 AllCycleSimpleBody net472 46.6μs 17.3ns 67.1ns 0.319 0 0 2.07 KB
#4875 AllCycleMoreComplexBody net6.0 204μs 57.6ns 223ns 0.102 0 0 8.37 KB
#4875 AllCycleMoreComplexBody netcoreapp3.1 208μs 90.1ns 337ns 0.104 0 0 8.26 KB
#4875 AllCycleMoreComplexBody net472 212μs 59.9ns 216ns 1.27 0 0 8.43 KB
#4875 ObjectExtractorSimpleBody net6.0 130ns 0.0621ns 0.233ns 0.00395 0 0 280 B
#4875 ObjectExtractorSimpleBody netcoreapp3.1 190ns 0.0794ns 0.397ns 0.00374 0 0 272 B
#4875 ObjectExtractorSimpleBody net472 152ns 0.164ns 0.634ns 0.0446 0 0 281 B
#4875 ObjectExtractorMoreComplexBody net6.0 2.72μs 0.846ns 3.05ns 0.0532 0 0 3.78 KB
#4875 ObjectExtractorMoreComplexBody netcoreapp3.1 3.6μs 2.21ns 8.54ns 0.0504 0 0 3.69 KB
#4875 ObjectExtractorMoreComplexBody net472 3.49μs 1.85ns 6.94ns 0.602 0.00523 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 12.9μs 3.92ns 14.7ns 0.131 0 0 9.42 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 20.3μs 8.41ns 31.5ns 0.122 0 0 9.41 KB
master RunWaf(args=NestedMap (10)) net472 30.6μs 14ns 54.1ns 1.49 0 0 9.47 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 75.7μs 81.2ns 314ns 0.189 0 0 15.76 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86.2μs 180ns 672ns 0.214 0 0 15.71 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 101μs 134ns 519ns 2.52 0 0 16.03 KB
master RunWaf(args=NestedMap (100)) net6.0 24.7μs 9.46ns 36.6ns 0.27 0 0 19.66 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 39.6μs 16.6ns 64.1ns 0.277 0 0 20.41 KB
master RunWaf(args=NestedMap (100)) net472 58.5μs 20.9ns 78.3ns 3.26 0.0583 0 20.62 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 90.5μs 181ns 703ns 0.36 0 0 26 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 109μs 228ns 791ns 0.328 0 0 26.71 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 134μs 407ns 1.58μs 4.31 0.0663 0 27.18 KB
master RunWaf(args=NestedMap (20)) net6.0 24.4μs 7.85ns 29.4ns 0.268 0 0 19.39 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 39.2μs 20ns 77.3ns 0.254 0 0 19.83 KB
master RunWaf(args=NestedMap (20)) net472 57.6μs 21.9ns 84.9ns 3.17 0.0576 0 20.03 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 90.6μs 42.6ns 165ns 0.361 0 0 25.74 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 110μs 87.9ns 329ns 0.326 0 0 26.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 133μs 143ns 533ns 4.18 0.0664 0 26.59 KB
#4875 RunWaf(args=NestedMap (10)) net6.0 12.9μs 4.45ns 16.6ns 0.129 0 0 9.42 KB
#4875 RunWaf(args=NestedMap (10)) netcoreapp3.1 20.4μs 5.14ns 19.9ns 0.122 0 0 9.41 KB
#4875 RunWaf(args=NestedMap (10)) net472 30.8μs 9.85ns 36.9ns 1.5 0.0153 0 9.47 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 76.3μs 82.9ns 321ns 0.19 0 0 15.76 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 86μs 88.2ns 342ns 0.175 0 0 15.71 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) net472 100μs 196ns 759ns 2.53 0 0 16.03 KB
#4875 RunWaf(args=NestedMap (100)) net6.0 24.6μs 7.74ns 29ns 0.27 0 0 19.66 KB
#4875 RunWaf(args=NestedMap (100)) netcoreapp3.1 40μs 18.7ns 70.1ns 0.279 0 0 20.41 KB
#4875 RunWaf(args=NestedMap (100)) net472 59.2μs 29.1ns 109ns 3.25 0.0591 0 20.62 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 92.5μs 45.5ns 176ns 0.325 0 0 26 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 110μs 28.8ns 99.8ns 0.331 0 0 26.71 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [23]) net472 132μs 54.1ns 202ns 4.28 0.0659 0 27.18 KB
#4875 RunWaf(args=NestedMap (20)) net6.0 24.1μs 8.1ns 30.3ns 0.276 0 0 19.39 KB
#4875 RunWaf(args=NestedMap (20)) netcoreapp3.1 39μs 9.25ns 35.8ns 0.254 0 0 19.83 KB
#4875 RunWaf(args=NestedMap (20)) net472 58.8μs 22.2ns 86ns 3.16 0.0586 0 20.03 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 91.9μs 300ns 1.16μs 0.361 0 0 25.74 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 110μs 53.9ns 209ns 0.327 0 0 26.14 KB
#4875 RunWafWithAttack(args=Neste(...)tack) [22]) net472 132μs 84.6ns 328ns 4.16 0.0661 0 26.59 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 171μs 117ns 424ns 0.257 0 0 18.04 KB
master SendRequest netcoreapp3.1 190μs 216ns 780ns 0.189 0 0 20.2 KB
master SendRequest net472 0.000676ns 0.000322ns 0.00121ns 0 0 0 0 b
#4875 SendRequest net6.0 171μs 171ns 661ns 0.17 0 0 18.04 KB
#4875 SendRequest netcoreapp3.1 190μs 184ns 688ns 0.189 0 0 20.2 KB
#4875 SendRequest net472 3.32E‑05ns 3.32E‑05ns 0.000124ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #4875

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.61 KB 41.89 KB 278 B 0.67%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 559μs 1.28μs 4.61μs 0.553 0 0 41.55 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 684μs 714ns 2.76μs 0.336 0 0 41.61 KB
master WriteAndFlushEnrichedTraces net472 851μs 3.86μs 14.9μs 8.28 2.48 0.414 53.23 KB
#4875 WriteAndFlushEnrichedTraces net6.0 543μs 571ns 2.21μs 0.576 0 0 41.56 KB
#4875 WriteAndFlushEnrichedTraces netcoreapp3.1 664μs 1.56μs 6.06μs 0.327 0 0 41.89 KB
#4875 WriteAndFlushEnrichedTraces net472 837μs 3.76μs 14.6μs 8.28 2.48 0.414 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.02μs 0.428ns 1.66ns 0.0108 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.35μs 0.457ns 1.71ns 0.0103 0 0 768 B
master ExecuteNonQuery net472 1.55μs 0.328ns 1.23ns 0.115 0 0 730 B
#4875 ExecuteNonQuery net6.0 995ns 0.617ns 2.39ns 0.0106 0 0 768 B
#4875 ExecuteNonQuery netcoreapp3.1 1.26μs 1.64ns 6.35ns 0.0101 0 0 768 B
#4875 ExecuteNonQuery net472 1.61μs 0.883ns 3.18ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4875

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.181 1,306.66 1,106.15

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.08μs 0.659ns 2.37ns 0.0131 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.36μs 0.396ns 1.53ns 0.0123 0 0 936 B
master CallElasticsearch net472 2.36μs 0.708ns 2.74ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.31μs 0.501ns 1.81ns 0.0124 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.48μs 0.835ns 3.01ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.51μs 0.79ns 3.06ns 0.16 0 0 1.01 KB
#4875 CallElasticsearch net6.0 1.1μs 0.417ns 1.62ns 0.0132 0 0 936 B
#4875 CallElasticsearch netcoreapp3.1 1.43μs 1.92ns 7.43ns 0.0128 0 0 936 B
#4875 CallElasticsearch net472 2.37μs 1.12ns 4.04ns 0.151 0 0 955 B
#4875 CallElasticsearchAsync net6.0 1.11μs 0.697ns 2.61ns 0.0124 0 0 912 B
#4875 CallElasticsearchAsync netcoreapp3.1 1.43μs 0.47ns 1.76ns 0.0137 0 0 984 B
#4875 CallElasticsearchAsync net472 2.57μs 1.12ns 4.32ns 0.159 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.29μs 0.786ns 2.94ns 0.013 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.42μs 0.596ns 2.23ns 0.0122 0 0 912 B
master ExecuteAsync net472 1.71μs 0.523ns 1.96ns 0.139 0 0 875 B
#4875 ExecuteAsync net6.0 1.25μs 0.787ns 2.84ns 0.0126 0 0 912 B
#4875 ExecuteAsync netcoreapp3.1 1.44μs 0.869ns 3.36ns 0.0123 0 0 912 B
#4875 ExecuteAsync net472 1.7μs 0.531ns 2.06ns 0.139 0.000853 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.72μs 1.28ns 4.95ns 0.026 0 0 1.9 KB
master SendAsync netcoreapp3.1 4.48μs 3.14ns 12.2ns 0.0315 0 0 2.43 KB
master SendAsync net472 7.07μs 2.28ns 8.83ns 0.473 0 0 2.99 KB
#4875 SendAsync net6.0 3.83μs 2.44ns 9.43ns 0.0248 0 0 1.9 KB
#4875 SendAsync netcoreapp3.1 4.41μs 2.71ns 10.1ns 0.0331 0 0 2.43 KB
#4875 SendAsync net472 7.14μs 3.83ns 14.3ns 0.473 0 0 2.99 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.42μs 0.48ns 1.8ns 0.0222 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 1.89μs 0.721ns 2.79ns 0.0214 0 0 1.57 KB
master EnrichedLog net472 2.41μs 1.35ns 5.04ns 0.236 0 0 1.49 KB
#4875 EnrichedLog net6.0 1.38μs 0.836ns 3.13ns 0.0217 0 0 1.57 KB
#4875 EnrichedLog netcoreapp3.1 2.01μs 0.873ns 3.26ns 0.021 0 0 1.57 KB
#4875 EnrichedLog net472 2.3μs 1.86ns 7.22ns 0.237 0 0 1.49 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 74.1ns 267ns 0 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 116μs 72.2ns 270ns 0 0 0 4.21 KB
master EnrichedLog net472 146μs 62.1ns 241ns 0.655 0.218 0 4.38 KB
#4875 EnrichedLog net6.0 112μs 87.6ns 316ns 0.0558 0 0 4.21 KB
#4875 EnrichedLog netcoreapp3.1 115μs 86.9ns 325ns 0 0 0 4.21 KB
#4875 EnrichedLog net472 146μs 57.3ns 222ns 0.655 0.218 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.88μs 1.11ns 4.31ns 0.0287 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 3.81μs 1.46ns 5.64ns 0.0286 0 0 2.13 KB
master EnrichedLog net472 4.46μs 1.3ns 4.5ns 0.307 0 0 1.93 KB
#4875 EnrichedLog net6.0 2.92μs 1.39ns 5.38ns 0.029 0 0 2.13 KB
#4875 EnrichedLog netcoreapp3.1 3.83μs 0.772ns 2.99ns 0.0286 0 0 2.13 KB
#4875 EnrichedLog net472 4.54μs 1.51ns 5.83ns 0.306 0 0 1.93 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.34μs 0.911ns 3.41ns 0.0154 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.57μs 0.4ns 1.39ns 0.0149 0 0 1.1 KB
master SendReceive net472 1.94μs 6.08ns 23.6ns 0.177 0 0 1.12 KB
#4875 SendReceive net6.0 1.27μs 1.76ns 6.82ns 0.0152 0 0 1.1 KB
#4875 SendReceive netcoreapp3.1 1.61μs 0.432ns 1.62ns 0.0152 0 0 1.1 KB
#4875 SendReceive net472 1.97μs 2.01ns 7.79ns 0.176 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.61μs 0.887ns 3.2ns 0.0213 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.68μs 1.73ns 6.7ns 0.0221 0 0 1.58 KB
master EnrichedLog net472 4.01μs 11.3ns 43.8ns 0.309 0 0 1.96 KB
#4875 EnrichedLog net6.0 2.64μs 0.643ns 2.49ns 0.0211 0 0 1.53 KB
#4875 EnrichedLog netcoreapp3.1 3.49μs 1.84ns 7.15ns 0.0209 0 0 1.58 KB
#4875 EnrichedLog net472 3.98μs 1.17ns 4.37ns 0.311 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4875

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.196 385.80 461.27

Faster 🎉 in #4875

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.371 656.38 478.82
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.115 608.79 546.01

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 386ns 0.0535ns 0.2ns 0.00753 0 0 536 B
master StartFinishSpan netcoreapp3.1 606ns 2.45ns 8.82ns 0.00728 0 0 536 B
master StartFinishSpan net472 618ns 0.246ns 0.887ns 0.0851 0 0 538 B
master StartFinishScope net6.0 654ns 0.831ns 3.22ns 0.00914 0 0 656 B
master StartFinishScope netcoreapp3.1 707ns 0.305ns 1.18ns 0.00881 0 0 656 B
master StartFinishScope net472 820ns 0.687ns 2.66ns 0.0981 0 0 618 B
#4875 StartFinishSpan net6.0 461ns 0.154ns 0.597ns 0.00754 0 0 536 B
#4875 StartFinishSpan netcoreapp3.1 546ns 0.23ns 0.86ns 0.00737 0 0 536 B
#4875 StartFinishSpan net472 604ns 0.374ns 1.45ns 0.0852 0 0 538 B
#4875 StartFinishScope net6.0 479ns 0.132ns 0.51ns 0.00912 0 0 656 B
#4875 StartFinishScope netcoreapp3.1 696ns 0.481ns 1.86ns 0.00898 0 0 656 B
#4875 StartFinishScope net472 783ns 0.548ns 2.12ns 0.0979 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 600ns 0.808ns 3.13ns 0.00927 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 848ns 0.495ns 1.92ns 0.00884 0 0 656 B
master RunOnMethodBegin net472 978ns 0.422ns 1.63ns 0.0979 0 0 618 B
#4875 RunOnMethodBegin net6.0 594ns 0.177ns 0.684ns 0.00913 0 0 656 B
#4875 RunOnMethodBegin netcoreapp3.1 803ns 2.52ns 9.77ns 0.00905 0 0 656 B
#4875 RunOnMethodBegin net472 989ns 0.39ns 1.51ns 0.0977 0 0 618 B

@andrewlock andrewlock merged commit dd1d9ad into master Nov 17, 2023
54 checks passed
@andrewlock andrewlock deleted the andrew/arm64-dotnet8-test branch November 17, 2023 09:50
@github-actions github-actions bot added this to the vNext milestone Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zacharycmontoya zacharycmontoya zacharycmontoya approved these changes

@bouwkast bouwkast bouwkast approved these changes

Assignees
No one assigned
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages area:tests unit tests, integration tests
Projects
None yet
Milestone
2.42.0
Development

Successfully merging this pull request may close these issues.
3 participants
@andrewlock @bouwkast @zacharycmontoya