Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IAST] Fix system test weak_cipher system test #6034

Merged
merged 3 commits into from
Sep 16, 2024
Merged

[IAST] Fix system test weak_cipher system test #6034

merged 3 commits into from
Sep 16, 2024

Conversation

daniel-romano-DD
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD commented Sep 16, 2024 •
edited
Loading

Summary of changes

Return first frame of the stack trace as location if no suitable one was found, but no exclusion either

Reason for change

System tests for weak cipher were broken due to a rare situation where no location was detected for the vulnerability

Implementation details

If no excluded frame was found, but the rest were skipped, we are returning the first stack frame

Test coverage

system tests should pass now

Other details

@daniel-romano-DD daniel-romano-DD requested a review from a team as a code owner September 16, 2024 10:15
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Sep 16, 2024 •
edited
Loading

Datadog Report

Branch report: dani/iast/fix_weakcipher_systemtests
Commit report: 8023143
Test service: dd-trace-dotnet

✅ 0 Failed, 364739 Passed, 2068 Skipped, 15h 53m 4.14s Total Time

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (70ms)  : 66, 74
     .   : milestone, 70,
    master - mean (69ms)  : 65, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (1,115ms)  : 1089, 1140
     .   : milestone, 1115,
    master - mean (1,118ms)  : 1098, 1139
     .   : milestone, 1118,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (108ms)  : 104, 112
     .   : milestone, 108,
    master - mean (108ms)  : 106, 111
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (807ms)  : 787, 826
     .   : milestone, 807,
    master - mean (812ms)  : 795, 829
     .   : milestone, 812,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (93ms)  : 88, 97
     .   : milestone, 93,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (756ms)  : 738, 774
     .   : milestone, 756,
    master - mean (762ms)  : 736, 788
     .   : milestone, 762,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (192ms)  : 187, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (1,190ms)  : 1163, 1216
     .   : milestone, 1190,
    master - mean (1,193ms)  : 1172, 1214
     .   : milestone, 1193,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (277ms)  : 272, 283
     .   : milestone, 277,
    master - mean (276ms)  : 273, 279
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (969ms)  : 948, 990
     .   : milestone, 969,
    master - mean (966ms)  : 949, 983
     .   : milestone, 966,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6034) - mean (265ms)  : 261, 270
     .   : milestone, 265,
    master - mean (265ms)  : 260, 270
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6034) - mean (945ms)  : 919, 971
     .   : milestone, 945,
    master - mean (948ms)  : 927, 970
     .   : milestone, 948,
Loading

@andrewlock
Copy link
Member

Benchmarks Report for appsec 🐌

Benchmarks for #6034 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.160
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6034

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.202 195.58 235.05
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472 1.120 4,321.48 4,840.73

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 73.2μs 93.6ns 362ns 0.0723 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 62.7μs 103ns 399ns 0.0936 0 0 6.95 KB
master AllCycleSimpleBody net472 48μs 50.2ns 194ns 1.3 0 0 8.34 KB
master AllCycleMoreComplexBody net6.0 79.5μs 114ns 441ns 0.119 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 69.7μs 126ns 489ns 0.139 0 0 10.36 KB
master AllCycleMoreComplexBody net472 57μs 115ns 444ns 1.87 0.0283 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 147ns 0.183ns 0.708ns 0.00394 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 195ns 0.253ns 0.979ns 0.00373 0 0 272 B
master ObjectExtractorSimpleBody net472 178ns 0.346ns 1.34ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.06μs 1.93ns 7.48ns 0.0535 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.18μs 5.17ns 20ns 0.0507 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.32μs 3.55ns 13.8ns 0.602 0.00655 0 3.8 KB
#6034 AllCycleSimpleBody net6.0 73.2μs 103ns 387ns 0.0735 0 0 6 KB
#6034 AllCycleSimpleBody netcoreapp3.1 63.5μs 111ns 430ns 0.0947 0 0 6.95 KB
#6034 AllCycleSimpleBody net472 48.8μs 102ns 396ns 1.31 0 0 8.33 KB
#6034 AllCycleMoreComplexBody net6.0 79.7μs 181ns 702ns 0.118 0 0 9.51 KB
#6034 AllCycleMoreComplexBody netcoreapp3.1 69.1μs 89.9ns 336ns 0.139 0 0 10.37 KB
#6034 AllCycleMoreComplexBody net472 57.1μs 55ns 213ns 1.86 0.029 0 11.85 KB
#6034 ObjectExtractorSimpleBody net6.0 139ns 0.221ns 0.855ns 0.00396 0 0 280 B
#6034 ObjectExtractorSimpleBody netcoreapp3.1 235ns 0.309ns 1.2ns 0.00369 0 0 272 B
#6034 ObjectExtractorSimpleBody net472 180ns 0.516ns 2ns 0.0446 0 0 281 B
#6034 ObjectExtractorMoreComplexBody net6.0 3.18μs 3.25ns 12.1ns 0.0526 0 0 3.78 KB
#6034 ObjectExtractorMoreComplexBody netcoreapp3.1 4.04μs 9.79ns 37.9ns 0.0502 0 0 3.69 KB
#6034 ObjectExtractorMoreComplexBody net472 4.84μs 3.52ns 13.6ns 0.602 0.00482 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 36.4μs 28.1ns 109ns 0.455 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.8μs 47.1ns 183ns 0.436 0 0 32.4 KB
master EncodeArgs net472 68.4μs 69.9ns 271ns 5.16 0.0683 0 32.5 KB
master EncodeLegacyArgs net6.0 76.5μs 11.5ns 41.3ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 106μs 241ns 901ns 0 0 0 2.15 KB
master EncodeLegacyArgs net472 151μs 57.3ns 222ns 0.3 0 0 2.15 KB
#6034 EncodeArgs net6.0 38.1μs 37.5ns 135ns 0.457 0 0 32.4 KB
#6034 EncodeArgs netcoreapp3.1 54μs 33ns 128ns 0.43 0 0 32.4 KB
#6034 EncodeArgs net472 65.6μs 37.1ns 144ns 5.14 0.0659 0 32.5 KB
#6034 EncodeLegacyArgs net6.0 81.8μs 168ns 649ns 0 0 0 2.14 KB
#6034 EncodeLegacyArgs netcoreapp3.1 105μs 172ns 620ns 0 0 0 2.14 KB
#6034 EncodeLegacyArgs net472 151μs 108ns 417ns 0.305 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 184μs 111ns 430ns 0 0 0 2.42 KB
master RunWafRealisticBenchmark netcoreapp3.1 198μs 184ns 687ns 0 0 0 2.37 KB
master RunWafRealisticBenchmark net472 209μs 35.2ns 136ns 0.312 0 0 2.43 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 61.5ns 238ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 132μs 53.1ns 199ns 0 0 0 1.45 KB
master RunWafRealisticBenchmarkWithAttack net472 139μs 65.6ns 254ns 0.208 0 0 1.48 KB
#6034 RunWafRealisticBenchmark net6.0 186μs 80.4ns 301ns 0 0 0 2.42 KB
#6034 RunWafRealisticBenchmark netcoreapp3.1 200μs 354ns 1.37μs 0 0 0 2.37 KB
#6034 RunWafRealisticBenchmark net472 210μs 36.9ns 138ns 0.314 0 0 2.43 KB
#6034 RunWafRealisticBenchmarkWithAttack net6.0 124μs 81.3ns 315ns 0 0 0 1.46 KB
#6034 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 133μs 83.4ns 323ns 0 0 0 1.45 KB
#6034 RunWafRealisticBenchmarkWithAttack net472 140μs 21.4ns 82.9ns 0.21 0 0 1.48 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6034

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 252.47 KB 253.89 KB 1.42 KB 0.56%

Fewer allocations 🎉 in #6034

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 256.49 KB 253.08 KB -3.41 KB -1.33%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 62.4μs 769ns 7.69μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 54.4μs 245ns 847ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.4μs 113ns 407ns 0 0 0 59.04 KB
master StringConcatAspectBenchmark net6.0 300μs 1.65μs 9.88μs 0 0 0 252.47 KB
master StringConcatAspectBenchmark netcoreapp3.1 355μs 1.92μs 11.4μs 0 0 0 256.49 KB
master StringConcatAspectBenchmark net472 277μs 5.63μs 54.9μs 0 0 0 278.53 KB
#6034 StringConcatBenchmark net6.0 60.3μs 885ns 8.67μs 0 0 0 43.44 KB
#6034 StringConcatBenchmark netcoreapp3.1 60.6μs 862ns 8.41μs 0 0 0 42.64 KB
#6034 StringConcatBenchmark net472 36.7μs 55.5ns 192ns 0 0 0 59.01 KB
#6034 StringConcatAspectBenchmark net6.0 315μs 1.79μs 12.4μs 0 0 0 253.89 KB
#6034 StringConcatAspectBenchmark netcoreapp3.1 336μs 1.21μs 4.18μs 0 0 0 253.08 KB
#6034 StringConcatAspectBenchmark net472 271μs 5.43μs 52.7μs 0 0 0 278.53 KB

@andrewlock
Copy link
Member

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6034) (10.937M)   : 0, 10936723
    master (10.964M)   : 0, 10963679
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6034) (7.344M)   : 0, 7344024
    master (7.342M)   : 0, 7341975
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.624M)   : 0, 7624069

    section Manual
    master (11.004M)   : 0, 11004392

    section Manual + Automatic
    This PR (6034) (6.657M)   : 0, 6656758
    master (6.893M)   : 0, 6893295

    section DD_TRACE_ENABLED=0
    master (10.114M)   : 0, 10113707
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6034) (9.305M)   : 0, 9305228
    master (9.408M)   : 0, 9407832
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6034) (6.506M)   : 0, 6505918
    master (6.464M)   : 0, 6463516

    section Trace stats
    master (6.767M)   : 0, 6767065

    section Manual
    master (9.424M)   : 0, 9424497

    section Manual + Automatic
    This PR (6034) (6.172M)   : 0, 6171656
    master (6.104M)   : 0, 6104219

    section DD_TRACE_ENABLED=0
    master (8.750M)   : 0, 8750160
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6034) (10.126M)   : 0, 10125662
    master (10.312M)   : 0, 10312327
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6034) (6.917M)   : 0, 6917389
    master (6.814M)   : 0, 6814056
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.412M)   : 0, 7412240

    section Manual
    master (10.113M)   : 0, 10113182

    section Manual + Automatic
    This PR (6034) (6.483M)   : 0, 6483390
    master (6.441M)   : 0, 6440538

    section DD_TRACE_ENABLED=0
    master (9.598M)   : 0, 9597874
Loading

@andrewlock
Copy link
Member

Benchmarks Report for tracer 🐌

Benchmarks for #6034 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.115
  • 1 benchmarks are slower, with geometric mean 1.154
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.72μs 44.7ns 369ns 0.0144 0.0072 0 5.43 KB
master StartStopWithChild netcoreapp3.1 10.1μs 52ns 265ns 0.0194 0.00972 0 5.62 KB
master StartStopWithChild net472 16μs 48.2ns 174ns 1.01 0.295 0.0958 6.07 KB
#6034 StartStopWithChild net6.0 7.71μs 43ns 278ns 0.0147 0.00737 0 5.42 KB
#6034 StartStopWithChild netcoreapp3.1 9.8μs 54.6ns 332ns 0.0291 0.0145 0 5.61 KB
#6034 StartStopWithChild net472 15.9μs 43.5ns 168ns 1.02 0.302 0.0953 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 478μs 390ns 1.46μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 637μs 202ns 783ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 820μs 213ns 797ns 0.411 0 0 3.3 KB
#6034 WriteAndFlushEnrichedTraces net6.0 465μs 269ns 1.04μs 0 0 0 2.7 KB
#6034 WriteAndFlushEnrichedTraces netcoreapp3.1 642μs 458ns 1.78μs 0 0 0 2.7 KB
#6034 WriteAndFlushEnrichedTraces net472 844μs 376ns 1.36μs 0.422 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 213μs 1.52μs 15.1μs 0.196 0 0 18.45 KB
master SendRequest netcoreapp3.1 225μs 1.24μs 7.95μs 0.237 0 0 20.61 KB
master SendRequest net472 0.000338ns 0.000161ns 0.000625ns 0 0 0 0 b
#6034 SendRequest net6.0 203μs 1.18μs 11.1μs 0.215 0 0 18.45 KB
#6034 SendRequest netcoreapp3.1 220μs 1.23μs 10.2μs 0.2 0 0 20.61 KB
#6034 SendRequest net472 0.00283ns 0.00111ns 0.0043ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 571μs 3.05μs 15.5μs 0.561 0 0 41.64 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 682μs 2.93μs 10.1μs 0.343 0 0 41.63 KB
master WriteAndFlushEnrichedTraces net472 865μs 2.41μs 8.68μs 8.15 2.26 0.453 53.27 KB
#6034 WriteAndFlushEnrichedTraces net6.0 575μs 2.92μs 12.7μs 0.553 0 0 41.58 KB
#6034 WriteAndFlushEnrichedTraces netcoreapp3.1 665μs 2.76μs 10.7μs 0.345 0 0 41.73 KB
#6034 WriteAndFlushEnrichedTraces net472 859μs 3.85μs 14.9μs 8.39 2.52 0.419 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.33μs 1ns 3.74ns 0.0139 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.78μs 1.47ns 5.71ns 0.0135 0 0 1.02 KB
master ExecuteNonQuery net472 2.1μs 2.03ns 7.87ns 0.156 0 0 987 B
#6034 ExecuteNonQuery net6.0 1.34μs 0.978ns 3.79ns 0.0142 0 0 1.02 KB
#6034 ExecuteNonQuery netcoreapp3.1 1.82μs 2.62ns 10.1ns 0.0136 0 0 1.02 KB
#6034 ExecuteNonQuery net472 2.05μs 2.15ns 8.32ns 0.156 0.00102 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.33μs 0.761ns 2.75ns 0.0132 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.58μs 3.13ns 11.7ns 0.0134 0 0 976 B
master CallElasticsearch net472 2.5μs 1.27ns 4.93ns 0.158 0 0 995 B
master CallElasticsearchAsync net6.0 1.34μs 0.684ns 2.56ns 0.0128 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.77μs 0.604ns 2.26ns 0.0133 0 0 1.02 KB
master CallElasticsearchAsync net472 2.58μs 1.93ns 7.48ns 0.167 0 0 1.05 KB
#6034 CallElasticsearch net6.0 1.22μs 0.724ns 2.71ns 0.0135 0 0 976 B
#6034 CallElasticsearch netcoreapp3.1 1.47μs 1.4ns 5.41ns 0.0132 0 0 976 B
#6034 CallElasticsearch net472 2.45μs 1.26ns 4.86ns 0.157 0.00123 0 995 B
#6034 CallElasticsearchAsync net6.0 1.32μs 1.23ns 4.76ns 0.0132 0 0 952 B
#6034 CallElasticsearchAsync netcoreapp3.1 1.62μs 4.19ns 16.2ns 0.0137 0 0 1.02 KB
#6034 CallElasticsearchAsync net472 2.69μs 5.57ns 21.6ns 0.166 0.00134 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6034

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 1.115 1,690.28 1,515.55

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.25μs 1.51ns 5.85ns 0.0132 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.69μs 0.655ns 2.36ns 0.0127 0 0 952 B
master ExecuteAsync net472 1.77μs 1.84ns 6.88ns 0.145 0.000879 0 915 B
#6034 ExecuteAsync net6.0 1.3μs 0.566ns 2.12ns 0.013 0 0 952 B
#6034 ExecuteAsync netcoreapp3.1 1.52μs 1.85ns 6.91ns 0.013 0 0 952 B
#6034 ExecuteAsync net472 1.81μs 0.618ns 2.39ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.24μs 1.05ns 3.78ns 0.0298 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.15μs 2.98ns 11.6ns 0.0359 0 0 2.76 KB
master SendAsync net472 7.78μs 1.8ns 6.96ns 0.498 0 0 3.15 KB
#6034 SendAsync net6.0 4.16μs 2.39ns 8.94ns 0.0292 0 0 2.22 KB
#6034 SendAsync netcoreapp3.1 5.08μs 2.23ns 8.63ns 0.0355 0 0 2.76 KB
#6034 SendAsync net472 7.83μs 2.52ns 9.42ns 0.496 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.52μs 0.603ns 2.34ns 0.023 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.17μs 0.834ns 3.12ns 0.0227 0 0 1.64 KB
master EnrichedLog net472 2.59μs 0.742ns 2.88ns 0.249 0 0 1.57 KB
#6034 EnrichedLog net6.0 1.5μs 0.791ns 2.96ns 0.0226 0 0 1.64 KB
#6034 EnrichedLog netcoreapp3.1 2.21μs 1.4ns 5.22ns 0.0221 0 0 1.64 KB
#6034 EnrichedLog net472 2.52μs 1.37ns 5.29ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 69.2ns 240ns 0.0571 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 118μs 128ns 494ns 0.0585 0 0 4.28 KB
master EnrichedLog net472 146μs 44.9ns 174ns 0.654 0.218 0 4.46 KB
#6034 EnrichedLog net6.0 114μs 171ns 639ns 0 0 0 4.28 KB
#6034 EnrichedLog netcoreapp3.1 117μs 94.1ns 365ns 0.0587 0 0 4.28 KB
#6034 EnrichedLog net472 146μs 39.2ns 141ns 0.658 0.219 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.97μs 0.76ns 2.94ns 0.0311 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.26μs 1.19ns 4.59ns 0.0298 0 0 2.2 KB
master EnrichedLog net472 4.87μs 1.36ns 5.28ns 0.321 0 0 2.02 KB
#6034 EnrichedLog net6.0 3.15μs 1.91ns 7.14ns 0.0312 0 0 2.2 KB
#6034 EnrichedLog netcoreapp3.1 4.07μs 1.95ns 7.55ns 0.0285 0 0 2.2 KB
#6034 EnrichedLog net472 4.91μs 1.25ns 4.84ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.35μs 0.828ns 3.21ns 0.0158 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.79μs 1.5ns 5.82ns 0.0151 0 0 1.14 KB
master SendReceive net472 2.06μs 0.448ns 1.74ns 0.183 0 0 1.16 KB
#6034 SendReceive net6.0 1.34μs 1.09ns 4.22ns 0.0161 0 0 1.14 KB
#6034 SendReceive netcoreapp3.1 1.76μs 1.08ns 4.05ns 0.0159 0 0 1.14 KB
#6034 SendReceive net472 2.04μs 0.518ns 1.94ns 0.183 0.00102 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.89μs 0.811ns 3.04ns 0.0231 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.02μs 1.75ns 6.8ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.4μs 2.4ns 9.29ns 0.323 0 0 2.04 KB
#6034 EnrichedLog net6.0 2.83μs 0.821ns 3.07ns 0.0225 0 0 1.6 KB
#6034 EnrichedLog netcoreapp3.1 3.88μs 0.883ns 3.06ns 0.0213 0 0 1.65 KB
#6034 EnrichedLog net472 4.37μs 1.11ns 4.01ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 411ns 0.249ns 0.964ns 0.00806 0 0 576 B
master StartFinishSpan netcoreapp3.1 633ns 0.455ns 1.76ns 0.00761 0 0 576 B
master StartFinishSpan net472 718ns 0.408ns 1.58ns 0.0917 0 0 578 B
master StartFinishScope net6.0 481ns 0.368ns 1.42ns 0.00965 0 0 696 B
master StartFinishScope netcoreapp3.1 669ns 0.384ns 1.38ns 0.00932 0 0 696 B
master StartFinishScope net472 916ns 1.28ns 4.94ns 0.104 0 0 658 B
#6034 StartFinishSpan net6.0 421ns 0.27ns 0.974ns 0.00807 0 0 576 B
#6034 StartFinishSpan netcoreapp3.1 633ns 0.618ns 2.31ns 0.00793 0 0 576 B
#6034 StartFinishSpan net472 680ns 0.748ns 2.9ns 0.0915 0 0 578 B
#6034 StartFinishScope net6.0 480ns 0.471ns 1.82ns 0.00994 0 0 696 B
#6034 StartFinishScope netcoreapp3.1 677ns 0.756ns 2.93ns 0.00929 0 0 696 B
#6034 StartFinishScope net472 918ns 1.68ns 6.51ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6034

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.154 655.62 756.37

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 655ns 0.3ns 1.12ns 0.00982 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 876ns 0.687ns 2.57ns 0.00951 0 0 696 B
master RunOnMethodBegin net472 1.07μs 0.891ns 3.45ns 0.104 0 0 658 B
#6034 RunOnMethodBegin net6.0 757ns 0.585ns 2.26ns 0.00951 0 0 696 B
#6034 RunOnMethodBegin netcoreapp3.1 848ns 0.847ns 3.28ns 0.00929 0 0 696 B
#6034 RunOnMethodBegin net472 1.07μs 1.13ns 4.37ns 0.104 0 0 658 B

@andrewlock andrewlock merged commit 965a822 into master Sep 16, 2024
73 of 74 checks passed
@andrewlock andrewlock deleted the dani/iast/fix_weakcipher_systemtests branch September 16, 2024 12:27
@github-actions github-actions bot added this to the vNext-v3 milestone Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.3.0
Development

Successfully merging this pull request may close these issues.
3 participants
@daniel-romano-DD @andrewlock @NachoEchevarria