Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Test Package Versions Bump] Updating package versions #4972

Merged
merged 1 commit into from
Feb 16, 2024
Merged

[Test Package Versions Bump] Updating package versions #4972

merged 1 commit into from
Feb 16, 2024

Conversation

github-actions[bot]
Copy link
Contributor

Updates the package versions for integration tests.

@github-actions github-actions bot requested review from a team as code owners December 14, 2023 11:09
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Dec 14, 2023 •
edited
Loading

Datadog Report

Branch report: bot/test-package-versions-bump
Commit report: 3774fc2

❌ dd-trace-dotnet: 28 Failed (0 Known Flaky), 0 New Flaky, 302850 Passed, 963 Skipped, 54m 26.04s Wall Time

❌ Failed Tests (28)

This report shows up to 5 failed tests.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests - Details

    Expand for error 
     Expected otelSpans.Count() to be 33, but found 32.
 Expected activitySourceSpans.Count() to be 1, but found 2.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests - Details

    Expand for error 
     Expected otelSpans.Count() to be 33, but found 32.
 Expected activitySourceSpans.Count() to be 1, but found 2.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests - Details

    Expand for error 
     Expected otelSpans.Count() to be 33, but found 32.
 Expected activitySourceSpans.Count() to be 1, but found 2.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests - Details

    Expand for error 
     Expected otelSpans.Count() to be 33, but found 32.
 Expected activitySourceSpans.Count() to be 1, but found 2.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests - Details

    Expand for error 
     Expected activitySourceSpans.Count() to be 33, but found 32.
 Expected value to be 1, but found 2.

@andrewlock
Copy link
Member

andrewlock commented Dec 14, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (76ms)  : 67, 84
     .   : milestone, 76,
    master - mean (76ms)  : 67, 85
     .   : milestone, 76,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (983ms)  : 957, 1010
     .   : milestone, 983,
    master - mean (988ms)  : 965, 1011
     .   : milestone, 988,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (111ms)  : 108, 114
     .   : milestone, 111,
    master - mean (110ms)  : 107, 113
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (702ms)  : 676, 727
     .   : milestone, 702,
    master - mean (714ms)  : 694, 734
     .   : milestone, 714,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (93ms)  : 91, 96
     .   : milestone, 93,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (662ms)  : 640, 684
     .   : milestone, 662,
    master - mean (663ms)  : 641, 686
     .   : milestone, 663,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (187ms)  : 183, 190
     .   : milestone, 187,
    master - mean (188ms)  : 184, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (1,024ms)  : 999, 1049
     .   : milestone, 1024,
    master - mean (1,053ms)  : 1027, 1080
     .   : milestone, 1053,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (270ms)  : 264, 276
     .   : milestone, 270,
    master - mean (271ms)  : 265, 278
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (1,004ms)  : 981, 1028
     .   : milestone, 1004,
    master - mean (1,035ms)  : 1001, 1068
     .   : milestone, 1035,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4972) - mean (258ms)  : 252, 264
     .   : milestone, 258,
    master - mean (262ms)  : 258, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (4972) - mean (978ms)  : 953, 1003
     .   : milestone, 978,
    master - mean (1,001ms)  : 978, 1024
     .   : milestone, 1001,
Loading

@andrewlock
Copy link
Member

andrewlock commented Dec 14, 2023 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4972) (10.894M)   : 0, 10894210
    master (10.904M)   : 0, 10903785
    benchmarks/2.9.0 (11.116M)   : 0, 11116034

    section Automatic
    This PR (4972) (7.600M)   : 0, 7600443
    master (7.548M)   : 0, 7547783
    benchmarks/2.9.0 (7.891M)   : 0, 7891342

    section Trace stats
    This PR (4972) (7.814M)   : 0, 7814272
    master (7.902M)   : 0, 7902379

    section Manual
    This PR (4972) (9.572M)   : 0, 9572164
    master (9.632M)   : 0, 9631577

    section Manual + Automatic
    This PR (4972) (7.189M)   : 0, 7189288
    master (7.263M)   : 0, 7262759

    section Version Conflict
    This PR (4972) (6.468M)   : 0, 6468223
    master (6.604M)   : 0, 6603647
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4972) (9.452M)   : 0, 9452129
    master (9.355M)   : 0, 9354781
    benchmarks/2.9.0 (9.562M)   : 0, 9562342

    section Automatic
    This PR (4972) (6.740M)   : 0, 6739705
    master (6.646M)   : 0, 6646272

    section Trace stats
    This PR (4972) (6.878M)   : 0, 6878371
    master (6.782M)   : 0, 6782074

    section Manual
    This PR (4972) (8.162M)   : 0, 8162479
    master (8.426M)   : 0, 8425694

    section Manual + Automatic
    This PR (4972) (6.203M)   : 0, 6202901
    master (6.313M)   : 0, 6313053

    section Version Conflict
    This PR (4972) (5.770M)   : 0, 5769862
    master (5.623M)   : 0, 5622957
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4972) (10.188M)   : 0, 10188425
    master (10.039M)   : 0, 10039221
    benchmarks/2.9.0 (10.060M)   : 0, 10060320

    section Automatic
    This PR (4972) (7.261M)   : 0, 7260595
    master (7.158M)   : 0, 7157695
    benchmarks/2.9.0 (7.535M)   : 0, 7534759

    section Trace stats
    This PR (4972) (7.525M)   : 0, 7525248
    master (7.442M)   : 0, 7442467

    section Manual
    This PR (4972) (9.056M)   : 0, 9056198
    master (8.814M)   : 0, 8814383

    section Manual + Automatic
    This PR (4972) (7.024M)   : 0, 7024362
    master (6.903M)   : 0, 6903478

    section Version Conflict
    This PR (4972) (6.444M)   : 0, 6444262
    master (6.144M)   : 0, 6143514
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.398M)   : 0, 7398268
    benchmarks/2.9.0 (7.618M)   : 0, 7617642

    section No attack
    master (1.836M)   : 0, 1835561
    benchmarks/2.9.0 (3.121M)   : 0, 3120509

    section Attack
    master (1.457M)   : 0, 1457427
    benchmarks/2.9.0 (2.488M)   : 0, 2488303

    section Blocking
    master (3.175M)   : 0, 3174901

    section IAST default
    master (6.462M)   : 0, 6462305

    section IAST full
    master (5.843M)   : 0, 5843231

    section Base vuln
    master (0.991M)   : 0, 991340

    section IAST vuln
    master (0.892M)   : 0, 892185
Loading

@andrewlock
Copy link
Member

andrewlock commented Dec 14, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4972 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.172
  • 2 benchmarks are slower, with geometric mean 1.162
  • 2 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.77μs 48.5ns 314ns 0.0225 0.00898 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 61ns 381ns 0.0326 0.0163 0 7.58 KB
master StartStopWithChild net472 17.2μs 70.3ns 272ns 1.34 0.342 0.111 7.95 KB
#4972 StartStopWithChild net6.0 8.8μs 48.3ns 286ns 0.0221 0.00883 0 7.48 KB
#4972 StartStopWithChild netcoreapp3.1 11.1μs 58.1ns 279ns 0.0275 0.011 0 7.58 KB
#4972 StartStopWithChild net472 17.4μs 52.3ns 203ns 1.36 0.366 0.113 7.96 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 455μs 242ns 938ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 629μs 149ns 556ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 815μs 312ns 1.21μs 0.403 0 0 3.3 KB
#4972 WriteAndFlushEnrichedTraces net6.0 470μs 370ns 1.38μs 0 0 0 2.7 KB
#4972 WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 87.6ns 316ns 0 0 0 2.7 KB
#4972 WriteAndFlushEnrichedTraces net472 792μs 253ns 945ns 0.396 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4972

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.191 214.89 255.99

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.8μs 30.9ns 120ns 0.0195 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 41.3μs 38.2ns 148ns 0.0207 0 0 1.74 KB
master AllCycleSimpleBody net472 44.8μs 18.7ns 72.3ns 0.288 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 200μs 85.1ns 330ns 0.0995 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 211μs 241ns 932ns 0.106 0 0 9.14 KB
master AllCycleMoreComplexBody net472 225μs 152ns 588ns 1.46 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 146ns 0.19ns 0.711ns 0.00396 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 215ns 0.133ns 0.513ns 0.00379 0 0 272 B
master ObjectExtractorSimpleBody net472 165ns 0.316ns 1.22ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 1.79ns 6.44ns 0.0533 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.22μs 1.33ns 4.81ns 0.0506 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.81μs 5.91ns 22.1ns 0.602 0.00568 0 3.8 KB
#4972 AllCycleSimpleBody net6.0 38μs 11.4ns 41ns 0.0194 0 0 1.77 KB
#4972 AllCycleSimpleBody netcoreapp3.1 41.6μs 24.3ns 94.3ns 0.0208 0 0 1.74 KB
#4972 AllCycleSimpleBody net472 44.7μs 30.3ns 117ns 0.267 0 0 1.81 KB
#4972 AllCycleMoreComplexBody net6.0 202μs 164ns 636ns 0.1 0 0 9.25 KB
#4972 AllCycleMoreComplexBody netcoreapp3.1 212μs 149ns 576ns 0.106 0 0 9.14 KB
#4972 AllCycleMoreComplexBody net472 225μs 105ns 408ns 1.46 0 0 9.32 KB
#4972 ObjectExtractorSimpleBody net6.0 142ns 0.191ns 0.739ns 0.00396 0 0 280 B
#4972 ObjectExtractorSimpleBody netcoreapp3.1 256ns 0.173ns 0.668ns 0.00363 0 0 272 B
#4972 ObjectExtractorSimpleBody net472 172ns 0.171ns 0.662ns 0.0446 0 0 281 B
#4972 ObjectExtractorMoreComplexBody net6.0 3.05μs 1.39ns 5.21ns 0.0534 0 0 3.78 KB
#4972 ObjectExtractorMoreComplexBody netcoreapp3.1 4.13μs 1.79ns 6.46ns 0.0497 0 0 3.69 KB
#4972 ObjectExtractorMoreComplexBody net472 3.83μs 2.76ns 10.7ns 0.602 0.00577 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4972

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWaf(args: NestedMap (100))‑net6.0 1.134 98,131.99 111,254.36

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 51.9μs 20.3ns 73.1ns 0.208 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 70.2μs 389ns 2.4μs 0.196 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 95.8μs 29.1ns 113ns 2.55 0.0962 0 16.14 KB
master RunWafTwice(args=NestedMap (10)) net6.0 57.5μs 11.2ns 42.1ns 0.218 0 0 16.6 KB
master RunWafTwice(args=NestedMap (10)) netcoreapp3.1 70.5μs 342ns 1.41μs 0.21 0 0 16.58 KB
master RunWafTwice(args=NestedMap (10)) net472 109μs 309ns 1.16μs 2.65 0.104 0 16.69 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 114μs 232ns 897ns 0.286 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 130μs 652ns 2.99μs 0.254 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 165μs 562ns 2.1μs 3.56 0.162 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 98.1μs 56.8ns 220ns 0.441 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 734ns 5.09μs 0.439 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 192μs 847ns 3.28μs 5.34 0.375 0 33.67 KB
master RunWafTwice(args=NestedMap (100)) net6.0 105μs 56.3ns 195ns 0.47 0 0 33.3 KB
master RunWafTwice(args=NestedMap (100)) netcoreapp3.1 137μs 750ns 4.18μs 0.456 0 0 33.86 KB
master RunWafTwice(args=NestedMap (100)) net472 196μs 56.9ns 213ns 5.37 0.391 0 34.23 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 160μs 41.4ns 155ns 0.498 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 197μs 99.9ns 346ns 0.507 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 257μs 1.12μs 4.33μs 6.31 0.505 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 104μs 596ns 4.42μs 0.443 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 133μs 724ns 4.22μs 0.413 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 188μs 106ns 412ns 5.19 0.37 0 32.63 KB
master RunWafTwice(args=NestedMap (20)) net6.0 112μs 344ns 1.33μs 0.449 0 0 32.72 KB
master RunWafTwice(args=NestedMap (20)) netcoreapp3.1 133μs 82.5ns 309ns 0.398 0 0 32.82 KB
master RunWafTwice(args=NestedMap (20)) net472 194μs 136ns 527ns 5.25 0.389 0 33.19 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 168μs 63.2ns 228ns 0.559 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 203μs 597ns 2.31μs 0.484 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 255μs 537ns 2.08μs 6.12 0.5 0 39.2 KB
#4972 RunWaf(args=NestedMap (10)) net6.0 53.9μs 20.3ns 76.1ns 0.222 0 0 16.06 KB
#4972 RunWaf(args=NestedMap (10)) netcoreapp3.1 68.9μs 367ns 1.87μs 0.208 0 0 16.06 KB
#4972 RunWaf(args=NestedMap (10)) net472 99.9μs 38.7ns 145ns 2.56 0.0949 0 16.14 KB
#4972 RunWafTwice(args=NestedMap (10)) net6.0 54.4μs 9.84ns 34.1ns 0.217 0 0 16.6 KB
#4972 RunWafTwice(args=NestedMap (10)) netcoreapp3.1 74.5μs 362ns 1.49μs 0.228 0 0 16.58 KB
#4972 RunWafTwice(args=NestedMap (10)) net472 108μs 55.6ns 201ns 2.63 0.107 0 16.69 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 108μs 34.2ns 132ns 0.27 0 0 22.41 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 133μs 629ns 2.51μs 0.263 0 0 22.36 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) net472 157μs 40.6ns 152ns 3.61 0.157 0 22.7 KB
#4972 RunWaf(args=NestedMap (100)) net6.0 111μs 78.6ns 305ns 0.439 0 0 32.76 KB
#4972 RunWaf(args=NestedMap (100)) netcoreapp3.1 135μs 757ns 4.79μs 0.448 0 0 33.33 KB
#4972 RunWaf(args=NestedMap (100)) net472 190μs 717ns 2.78μs 5.29 0.371 0 33.67 KB
#4972 RunWafTwice(args=NestedMap (100)) net6.0 104μs 202ns 698ns 0.461 0 0 33.3 KB
#4972 RunWafTwice(args=NestedMap (100)) netcoreapp3.1 138μs 740ns 4.12μs 0.411 0 0 33.86 KB
#4972 RunWafTwice(args=NestedMap (100)) net472 200μs 105ns 406ns 5.39 0.385 0 34.22 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 158μs 346ns 1.34μs 0.552 0 0 39.1 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 201μs 1.08μs 4.69μs 0.49 0 0 39.63 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [23]) net472 259μs 1.08μs 4.19μs 6.34 0.507 0 40.23 KB
#4972 RunWaf(args=NestedMap (20)) net6.0 97.6μs 32.2ns 125ns 0.435 0 0 32.18 KB
#4972 RunWaf(args=NestedMap (20)) netcoreapp3.1 134μs 688ns 3.08μs 0.406 0 0 32.3 KB
#4972 RunWaf(args=NestedMap (20)) net472 191μs 853ns 3.3μs 5.11 0.371 0 32.63 KB
#4972 RunWafTwice(args=NestedMap (20)) net6.0 114μs 59.9ns 224ns 0.451 0 0 32.72 KB
#4972 RunWafTwice(args=NestedMap (20)) netcoreapp3.1 139μs 735ns 4.03μs 0.422 0 0 32.82 KB
#4972 RunWafTwice(args=NestedMap (20)) net472 194μs 49.6ns 192ns 5.27 0.39 0 33.19 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 161μs 338ns 1.31μs 0.482 0 0 38.53 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 200μs 763ns 2.95μs 0.494 0 0 38.6 KB
#4972 RunWafWithAttack(args=Neste(...)tack) [22]) net472 252μs 195ns 756ns 6.19 0.505 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 173μs 187ns 723ns 0.26 0 0 18.25 KB
master SendRequest netcoreapp3.1 197μs 1.09μs 6.52μs 0.193 0 0 20.41 KB
master SendRequest net472 0.000465ns 0.000189ns 0.000733ns 0 0 0 0 b
#4972 SendRequest net6.0 174μs 231ns 895ns 0.175 0 0 18.25 KB
#4972 SendRequest netcoreapp3.1 194μs 282ns 1.09μs 0.193 0 0 20.41 KB
#4972 SendRequest net472 0.0369ns 0.0167ns 0.0669ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 547μs 1μs 3.88μs 0.534 0 0 41.85 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 657μs 951ns 3.68μs 0.334 0 0 41.71 KB
master WriteAndFlushEnrichedTraces net472 839μs 2.91μs 11.3μs 8.28 2.48 0.414 53.25 KB
#4972 WriteAndFlushEnrichedTraces net6.0 555μs 1.18μs 4.55μs 0.573 0 0 41.68 KB
#4972 WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 765ns 2.96μs 0.342 0 0 41.7 KB
#4972 WriteAndFlushEnrichedTraces net472 860μs 4.08μs 16.8μs 8.08 2.55 0.425 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.05μs 0.274ns 0.988ns 0.0106 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.47μs 0.732ns 2.83ns 0.0103 0 0 768 B
master ExecuteNonQuery net472 1.8μs 0.71ns 2.56ns 0.116 0 0 730 B
#4972 ExecuteNonQuery net6.0 1.16μs 0.542ns 2.1ns 0.0111 0 0 768 B
#4972 ExecuteNonQuery netcoreapp3.1 1.47μs 0.397ns 1.43ns 0.0104 0 0 768 B
#4972 ExecuteNonQuery net472 1.79μs 4.11ns 15.9ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 1.18ns 4.56ns 0.013 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.56μs 1.72ns 6.45ns 0.0126 0 0 936 B
master CallElasticsearch net472 2.5μs 0.735ns 2.85ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.3μs 0.558ns 2.16ns 0.0124 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.54μs 0.486ns 1.88ns 0.0131 0 0 984 B
master CallElasticsearchAsync net472 2.77μs 1.15ns 4.44ns 0.16 0 0 1.01 KB
#4972 CallElasticsearch net6.0 1.25μs 0.591ns 2.13ns 0.0132 0 0 936 B
#4972 CallElasticsearch netcoreapp3.1 1.53μs 0.768ns 2.98ns 0.0122 0 0 936 B
#4972 CallElasticsearch net472 2.49μs 1.68ns 6.51ns 0.151 0 0 955 B
#4972 CallElasticsearchAsync net6.0 1.29μs 1.24ns 4.64ns 0.0128 0 0 912 B
#4972 CallElasticsearchAsync netcoreapp3.1 1.6μs 1.8ns 6.96ns 0.0128 0 0 984 B
#4972 CallElasticsearchAsync net472 2.61μs 1.5ns 5.83ns 0.161 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4972

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 1.223 1,479.68 1,210.23

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.48μs 0.646ns 2.24ns 0.0126 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.66μs 1.43ns 5.35ns 0.0124 0 0 912 B
master ExecuteAsync net472 1.83μs 0.483ns 1.81ns 0.139 0 0 875 B
#4972 ExecuteAsync net6.0 1.21μs 0.419ns 1.57ns 0.0126 0 0 912 B
#4972 ExecuteAsync netcoreapp3.1 1.63μs 3.77ns 14.6ns 0.0124 0 0 912 B
#4972 ExecuteAsync net472 1.83μs 1.89ns 7.31ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.29μs 7.03ns 25.3ns 0.0299 0 0 2.1 KB
master SendAsync netcoreapp3.1 5.06μs 2.05ns 7.68ns 0.0354 0 0 2.63 KB
master SendAsync net472 7.8μs 3ns 11.6ns 0.523 0 0 3.31 KB
#4972 SendAsync net6.0 4.19μs 2.07ns 8.02ns 0.0293 0 0 2.1 KB
#4972 SendAsync netcoreapp3.1 4.91μs 1.5ns 5.63ns 0.0343 0 0 2.63 KB
#4972 SendAsync net472 7.71μs 4.73ns 18.3ns 0.525 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #4972

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 204.27 KB 203.09 KB -1.18 KB -0.58%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 61.93 KB 59.26 KB -2.67 KB -4.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 59μs 1.16μs 11.3μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 54μs 244ns 1.17μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.6μs 77.7ns 269ns 0 0 0 61.93 KB
master StringConcatAspectBenchmark net6.0 276μs 6.83μs 66.5μs 0 0 0 212.85 KB
master StringConcatAspectBenchmark netcoreapp3.1 327μs 6.37μs 59.4μs 0 0 0 204.27 KB
master StringConcatAspectBenchmark net472 223μs 962ns 3.47μs 0 0 0 221.18 KB
#4972 StringConcatBenchmark net6.0 61.2μs 948ns 9.33μs 0 0 0 43.44 KB
#4972 StringConcatBenchmark netcoreapp3.1 53.6μs 167ns 645ns 0 0 0 42.64 KB
#4972 StringConcatBenchmark net472 38.3μs 57.1ns 206ns 0 0 0 59.26 KB
#4972 StringConcatAspectBenchmark net6.0 281μs 1.33μs 5.65μs 0 0 0 213.25 KB
#4972 StringConcatAspectBenchmark netcoreapp3.1 293μs 7.75μs 72.7μs 0 0 0 203.09 KB
#4972 StringConcatAspectBenchmark net472 235μs 2.78μs 25.8μs 0 0 0 221.18 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.59μs 0.62ns 2.32ns 0.0223 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.2μs 1.64ns 6.14ns 0.0208 0 0 1.57 KB
master EnrichedLog net472 2.58μs 1.36ns 5.26ns 0.238 0 0 1.5 KB
#4972 EnrichedLog net6.0 1.45μs 0.429ns 1.61ns 0.0218 0 0 1.57 KB
#4972 EnrichedLog netcoreapp3.1 2.13μs 0.817ns 3.06ns 0.0217 0 0 1.57 KB
#4972 EnrichedLog net472 2.57μs 2.43ns 9.08ns 0.238 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 130ns 487ns 0.0567 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 108ns 418ns 0.059 0 0 4.21 KB
master EnrichedLog net472 149μs 119ns 447ns 0.668 0.223 0 4.39 KB
#4972 EnrichedLog net6.0 114μs 108ns 403ns 0 0 0 4.21 KB
#4972 EnrichedLog netcoreapp3.1 121μs 39.8ns 138ns 0 0 0 4.21 KB
#4972 EnrichedLog net472 149μs 72.5ns 271ns 0.669 0.223 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.99μs 0.867ns 3.36ns 0.0299 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.28μs 3.46ns 13.4ns 0.0277 0 0 2.13 KB
master EnrichedLog net472 4.86μs 2.06ns 7.98ns 0.308 0 0 1.95 KB
#4972 EnrichedLog net6.0 3.07μs 1.13ns 4.06ns 0.0292 0 0 2.13 KB
#4972 EnrichedLog netcoreapp3.1 4.25μs 1.54ns 5.96ns 0.0275 0 0 2.13 KB
#4972 EnrichedLog net472 4.93μs 4.27ns 16.5ns 0.309 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.33μs 0.546ns 2.04ns 0.0153 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.69μs 0.803ns 3.01ns 0.0147 0 0 1.1 KB
master SendReceive net472 2.2μs 3.74ns 14.5ns 0.177 0 0 1.12 KB
#4972 SendReceive net6.0 1.37μs 0.705ns 2.73ns 0.0158 0 0 1.1 KB
#4972 SendReceive netcoreapp3.1 1.76μs 1.43ns 5.54ns 0.015 0 0 1.1 KB
#4972 SendReceive net472 2.17μs 1.65ns 6.38ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.72μs 0.774ns 2.9ns 0.0217 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.84μs 1.8ns 6.47ns 0.0213 0 0 1.58 KB
master EnrichedLog net472 4.31μs 1.05ns 4.05ns 0.311 0 0 1.97 KB
#4972 EnrichedLog net6.0 2.85μs 0.913ns 3.54ns 0.0215 0 0 1.53 KB
#4972 EnrichedLog netcoreapp3.1 3.77μs 2.63ns 10.2ns 0.0208 0 0 1.58 KB
#4972 EnrichedLog net472 4.26μs 1.73ns 6.7ns 0.312 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4972

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.124 925.98 823.98

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 462ns 0.13ns 0.504ns 0.00742 0 0 536 B
master StartFinishSpan netcoreapp3.1 753ns 0.697ns 2.61ns 0.00719 0 0 536 B
master StartFinishSpan net472 751ns 0.758ns 2.93ns 0.0854 0 0 538 B
master StartFinishScope net6.0 538ns 0.121ns 0.467ns 0.00905 0 0 656 B
master StartFinishScope netcoreapp3.1 926ns 0.686ns 2.66ns 0.00853 0 0 656 B
master StartFinishScope net472 949ns 0.37ns 1.28ns 0.098 0 0 618 B
#4972 StartFinishSpan net6.0 460ns 0.136ns 0.51ns 0.0076 0 0 536 B
#4972 StartFinishSpan netcoreapp3.1 784ns 0.126ns 0.453ns 0.00706 0 0 536 B
#4972 StartFinishSpan net472 782ns 0.29ns 1.12ns 0.0853 0 0 538 B
#4972 StartFinishScope net6.0 585ns 0.167ns 0.604ns 0.0091 0 0 656 B
#4972 StartFinishScope netcoreapp3.1 823ns 0.455ns 1.7ns 0.00863 0 0 656 B
#4972 StartFinishScope net472 864ns 0.701ns 2.72ns 0.098 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 726ns 0.26ns 1.01ns 0.00903 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.786ns 2.84ns 0.00901 0 0 656 B
master RunOnMethodBegin net472 1.11μs 0.388ns 1.5ns 0.098 0 0 618 B
#4972 RunOnMethodBegin net6.0 735ns 0.215ns 0.832ns 0.0092 0 0 656 B
#4972 RunOnMethodBegin netcoreapp3.1 1.02μs 0.501ns 1.94ns 0.00867 0 0 656 B
#4972 RunOnMethodBegin net472 1.1μs 0.448ns 1.73ns 0.0981 0 0 618 B

@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from cb9fed7 to 3774fc2 Compare December 27, 2023 14:18
@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from 3774fc2 to 001c3f2 Compare January 19, 2024 11:17
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 19, 2024 •
edited
Loading

Datadog Report

Branch report: bot/test-package-versions-bump
Commit report: 001c3f2
Test service: dd-trace-dotnet

❌ 27 Failed (0 Known Flaky), 270602 Passed, 1353 Skipped, 1h 1m 51.91s Wall Time

❌ Failed Tests (27)

This report shows up to 5 failed tests.

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.OpenTelemetrySdkTests

@github-actions github-actions bot force-pushed the bot/test-package-versions-bump branch from 001c3f2 to 441dcc6 Compare February 13, 2024 10:31
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Feb 13, 2024 •
edited
Loading

Datadog Report

Branch report: bot/test-package-versions-bump
Commit report: 4e1606e
Test service: dd-trace-dotnet

✅ 0 Failed, 308998 Passed, 1576 Skipped, 41m 24.17s Wall Time

andrewlock
andrewlock reviewed Feb 16, 2024
View reviewed changes
tracer/dependabot/Datadog.Dependabot.Integrations.csproj
Comment on lines +239 to +242
<!-- Integration: Microsoft.VisualStudio.TestPlatform.Common -->
<!-- Assembly: Microsoft.VisualStudio.TestPlatform.Common -->
<!-- Latest package https://www.nuget.org/packages/Microsoft.VisualStudio.TestPlatform/14.0.0.1 -->
<PackageReference Include="Microsoft.VisualStudio.TestPlatform" Version="14.0.0.1" />
Copy link
Member

@andrewlock andrewlock Feb 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is from #5088

andrewlock
andrewlock approved these changes Feb 16, 2024
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and the important stuff passes

tracer/dependabot/Datadog.Dependabot.Integrations.csproj
Comment on lines +319 to +322
<!-- Integration: Microsoft.AspNetCore.Diagnostics -->
<!-- Assembly: Microsoft.AspNetCore.Diagnostics -->
<!-- Latest package https://www.nuget.org/packages/Microsoft.AspNetCore.Diagnostics/2.2.0 -->
<PackageReference Include="Microsoft.AspNetCore.Diagnostics" Version="2.2.0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is due to #5067

@andrewlock andrewlock merged commit 0ebc75b into master Feb 16, 2024
52 of 55 checks passed
@andrewlock andrewlock deleted the bot/test-package-versions-bump branch February 16, 2024 13:56
@github-actions github-actions bot added this to the vNext milestone Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2.48.0
Development

Successfully merging this pull request may close these issues.
1 participant
@andrewlock