Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ASM] Add null check for content body before running security checks #4950

Merged
merged 3 commits into from
Dec 4, 2023
Merged

[ASM] Add null check for content body before running security checks #4950

merged 3 commits into from
Dec 4, 2023

Conversation

zacharycmontoya
Copy link
Collaborator

Summary of changes

Adds a null check on the content body before running security checks.

Reason for change

When a controller returns a Microsoft.AspNetCore.Mvc.ObjectResult with a null object, the Value field will end up being null. This was not being checked before and could lead to a NullReferenceException when analyzing the content body. This PR adds the check and adds a regression test for this scenario.

Implementation details

Adds nullability to the updated files and ensures that the Microsoft.AspNetCore.Mvc.ObjectResult has a non-null body before running security checks on the body.

Test coverage

Adds a new test case (with corresponding snapshots) for test case Datadog.Trace.Security.IntegrationTests.ApiSecurity.AspNetCoreApiSecurity.TestApiSecurityScan that tests the null body behavior. Before the fix, we throw a NullReferenceException and return a 500 status code. After the fix, we return a 204 status code (as expected).

Other details

N/A

@zacharycmontoya zacharycmontoya self-assigned this Dec 2, 2023
@zacharycmontoya zacharycmontoya requested review from a team as code owners December 2, 2023 00:39
lucaspimentel
lucaspimentel approved these changes Dec 2, 2023
View reviewed changes
Copy link
Member

@lucaspimentel lucaspimentel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Dec 2, 2023 •
edited
Loading

Datadog Report

Branch report: zach.montoya/null-ref-apisec
Commit report: 7d5847a

❄️ dd-trace-dotnet: 0 Failed, 2 New Flaky, 302723 Passed, 1026 Skipped, 39m 14.59s Wall Time

New Flaky Tests (2)

  • DetectAgentUrl - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.AgentConnectivityCheckTests - Last Failure

    Expand for error 
     Timeout when waiting for the target process to start

  • DetectContinousProfilerState - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.ProcessBasicChecksTests - Last Failure

    Expand for error 
     Timeout when waiting for the target process to start

@andrewlock
Copy link
Member

andrewlock commented Dec 2, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (71ms)  : 62, 80
     .   : milestone, 71,
    master - mean (71ms)  : 61, 82
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (995ms)  : 978, 1013
     .   : milestone, 995,
    master - mean (995ms)  : 974, 1015
     .   : milestone, 995,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (106ms)  : 103, 109
     .   : milestone, 106,
    master - mean (106ms)  : 101, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (684ms)  : 669, 700
     .   : milestone, 684,
    master - mean (685ms)  : 670, 700
     .   : milestone, 685,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (89ms)  : 85, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (665ms)  : 639, 691
     .   : milestone, 665,
    master - mean (661ms)  : 636, 685
     .   : milestone, 661,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (188ms)  : 185, 192
     .   : milestone, 188,
    master - mean (188ms)  : 185, 192
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (1,131ms)  : 1110, 1152
     .   : milestone, 1131,
    master - mean (1,128ms)  : 1102, 1153
     .   : milestone, 1128,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (271ms)  : 268, 275
     .   : milestone, 271,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (1,089ms)  : 1062, 1117
     .   : milestone, 1089,
    master - mean (1,086ms)  : 1062, 1110
     .   : milestone, 1086,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4950) - mean (262ms)  : 256, 267
     .   : milestone, 262,
    master - mean (261ms)  : 256, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (4950) - mean (1,057ms)  : 1036, 1079
     .   : milestone, 1057,
    master - mean (1,052ms)  : 1022, 1083
     .   : milestone, 1052,
Loading

@andrewlock
Copy link
Member

andrewlock commented Dec 2, 2023 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4950) (11.309M)   : 0, 11308779
    master (11.626M)   : 0, 11626386
    benchmarks/2.9.0 (11.328M)   : 0, 11327845

    section Automatic
    This PR (4950) (7.759M)   : 0, 7758593
    master (7.992M)   : 0, 7991701
    benchmarks/2.9.0 (8.136M)   : 0, 8136367

    section Trace stats
    This PR (4950) (8.165M)   : 0, 8164514
    master (8.096M)   : 0, 8095574

    section Manual
    This PR (4950) (9.827M)   : 0, 9826716
    master (9.977M)   : 0, 9976603

    section Manual + Automatic
    This PR (4950) (7.362M)   : 0, 7362332
    master (7.452M)   : 0, 7451788

    section Version Conflict
    This PR (4950) (6.662M)   : 0, 6662452
    master (6.671M)   : 0, 6670594
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4950) (9.653M)   : 0, 9652766
    master (9.409M)   : 0, 9408968
    benchmarks/2.9.0 (9.420M)   : 0, 9419924

    section Automatic
    This PR (4950) (6.386M)   : 0, 6386052
    master (6.357M)   : 0, 6356859

    section Trace stats
    This PR (4950) (6.941M)   : 0, 6941256
    master (6.816M)   : 0, 6816099

    section Manual
    This PR (4950) (8.271M)   : 0, 8271214
    master (8.210M)   : 0, 8210002

    section Manual + Automatic
    This PR (4950) (6.287M)   : 0, 6287171
    master (6.279M)   : 0, 6278638

    section Version Conflict
    This PR (4950) (5.623M)   : 0, 5622533
    master (5.563M)   : 0, 5562691
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4950) (10.659M)   : 0, 10658615
    master (9.164M)   : 0, 9163580
    benchmarks/2.9.0 (9.398M)   : 0, 9397967

    section Automatic
    This PR (4950) (7.519M)   : 0, 7518953
    master (6.719M)   : 0, 6718549
    benchmarks/2.9.0 (6.738M)   : 0, 6737745

    section Trace stats
    This PR (4950) (7.894M)   : 0, 7894249
    master (7.093M)   : 0, 7092954

    section Manual
    This PR (4950) (9.331M)   : 0, 9330981
    master (8.069M)   : 0, 8069148

    section Manual + Automatic
    This PR (4950) (7.216M)   : 0, 7215729
    master (6.254M)   : 0, 6253624

    section Version Conflict
    This PR (4950) (6.528M)   : 0, 6528321
    master (5.744M)   : 0, 5743710
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4950) (7.410M)   : 0, 7410392
    master (7.235M)   : 0, 7235099
    benchmarks/2.9.0 (7.548M)   : 0, 7547683

    section No attack
    This PR (4950) (1.977M)   : 0, 1976742
    master (1.933M)   : 0, 1932936
    benchmarks/2.9.0 (3.158M)   : 0, 3158374

    section Attack
    This PR (4950) (1.568M)   : 0, 1568011
    master (1.547M)   : 0, 1547342
    benchmarks/2.9.0 (2.521M)   : 0, 2520552

    section Blocking
    This PR (4950) (3.252M)   : 0, 3252136
    master (3.194M)   : 0, 3194354

    section IAST default
    This PR (4950) (6.618M)   : 0, 6618079
    master (6.257M)   : 0, 6256506

    section IAST full
    This PR (4950) (5.750M)   : 0, 5749576
    master (5.759M)   : 0, 5759115

    section Base vuln
    This PR (4950) (0.956M)   : 0, 956147
    master (0.955M)   : 0, 955244

    section IAST vuln
    This PR (4950) (0.876M)   : 0, 875751
    master (0.873M)   : 0, 873378
Loading

@andrewlock
Copy link
Member

andrewlock commented Dec 2, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4950 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.207
  • 3 benchmarks are slower, with geometric mean 1.251
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.68μs 45.9ns 294ns 0.0258 0.0129 0 7.48 KB
master StartStopWithChild netcoreapp3.1 11.1μs 56.9ns 261ns 0.0316 0.0158 0 7.57 KB
master StartStopWithChild net472 16.7μs 55.7ns 216ns 1.31 0.326 0.1 7.94 KB
#4950 StartStopWithChild net6.0 8.68μs 46.1ns 244ns 0.0336 0.0168 0.00419 7.47 KB
#4950 StartStopWithChild netcoreapp3.1 10.6μs 33.6ns 116ns 0.0261 0.0104 0 7.58 KB
#4950 StartStopWithChild net472 17μs 38.6ns 150ns 1.35 0.368 0.103 7.96 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 457μs 272ns 1.02μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 633μs 291ns 1.13μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 778μs 310ns 1.16μs 0.386 0 0 3.3 KB
#4950 WriteAndFlushEnrichedTraces net6.0 449μs 199ns 771ns 0 0 0 2.7 KB
#4950 WriteAndFlushEnrichedTraces netcoreapp3.1 639μs 210ns 812ns 0 0 0 2.7 KB
#4950 WriteAndFlushEnrichedTraces net472 775μs 201ns 779ns 0.386 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.6μs 12.3ns 46.1ns 0.022 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 47.2μs 151ns 544ns 0.0231 0 0 1.74 KB
master AllCycleSimpleBody net472 49.3μs 26.5ns 99.2ns 0.277 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 214μs 112ns 419ns 0.107 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 226μs 102ns 396ns 0.115 0 0 9.14 KB
master AllCycleMoreComplexBody net472 237μs 92.9ns 360ns 1.42 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 129ns 0.0652ns 0.252ns 0.00393 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 184ns 0.092ns 0.344ns 0.00371 0 0 272 B
master ObjectExtractorSimpleBody net472 151ns 0.108ns 0.419ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.86μs 1.02ns 3.94ns 0.0528 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.75μs 1.42ns 5.31ns 0.0506 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.38μs 1.69ns 6.31ns 0.602 0.00507 0 3.8 KB
#4950 AllCycleSimpleBody net6.0 43.3μs 119ns 461ns 0.0218 0 0 1.77 KB
#4950 AllCycleSimpleBody netcoreapp3.1 47.4μs 47.3ns 183ns 0.0237 0 0 1.74 KB
#4950 AllCycleSimpleBody net472 49.7μs 13.9ns 53.9ns 0.267 0 0 1.81 KB
#4950 AllCycleMoreComplexBody net6.0 212μs 54.1ns 202ns 0.107 0 0 9.25 KB
#4950 AllCycleMoreComplexBody netcoreapp3.1 226μs 178ns 690ns 0.112 0 0 9.14 KB
#4950 AllCycleMoreComplexBody net472 239μs 83.9ns 303ns 1.44 0 0 9.32 KB
#4950 ObjectExtractorSimpleBody net6.0 130ns 0.0596ns 0.215ns 0.00394 0 0 280 B
#4950 ObjectExtractorSimpleBody netcoreapp3.1 182ns 0.144ns 0.538ns 0.00368 0 0 272 B
#4950 ObjectExtractorSimpleBody net472 152ns 0.0678ns 0.263ns 0.0446 0 0 281 B
#4950 ObjectExtractorMoreComplexBody net6.0 2.75μs 1.12ns 4.19ns 0.0522 0 0 3.78 KB
#4950 ObjectExtractorMoreComplexBody netcoreapp3.1 3.82μs 1.26ns 4.71ns 0.0497 0 0 3.69 KB
#4950 ObjectExtractorMoreComplexBody net472 3.38μs 1.44ns 5.57ns 0.602 0.00507 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 48.8μs 24.8ns 96.1ns 0.218 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 71.4μs 96.7ns 362ns 0.215 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 94.4μs 261ns 905ns 2.52 0.0935 0 16.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 114μs 58.8ns 204ns 0.301 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 140μs 478ns 1.72μs 0.265 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 167μs 815ns 3.26μs 3.53 0.164 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 95.4μs 76.3ns 275ns 0.467 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 137μs 154ns 595ns 0.411 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 183μs 86.6ns 324ns 5.32 0.367 0 33.67 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 175μs 214ns 802ns 0.524 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 202μs 987ns 4.19μs 0.491 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 260μs 337ns 1.3μs 6.31 0.515 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 107μs 102ns 394ns 0.427 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 126μs 652ns 3.06μs 0.438 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 189μs 894ns 3.46μs 5.11 0.365 0 32.63 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 167μs 47.5ns 184ns 0.504 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 207μs 1.08μs 5.72μs 0.49 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 259μs 107ns 414ns 6.22 0.519 0 39.2 KB
#4950 RunWaf(args=NestedMap (10)) net6.0 53.1μs 13ns 45.1ns 0.211 0 0 16.06 KB
#4950 RunWaf(args=NestedMap (10)) netcoreapp3.1 68.2μs 390ns 2.97μs 0.207 0 0 16.06 KB
#4950 RunWaf(args=NestedMap (10)) net472 94.4μs 24.7ns 92.5ns 2.55 0.0944 0 16.14 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 115μs 27.3ns 106ns 0.288 0 0 22.41 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 137μs 608ns 3.04μs 0.276 0 0 22.36 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) net472 173μs 53.7ns 193ns 3.55 0.165 0 22.7 KB
#4950 RunWaf(args=NestedMap (100)) net6.0 95μs 15.4ns 55.4ns 0.428 0 0 32.76 KB
#4950 RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 712ns 3.9μs 0.443 0 0 33.33 KB
#4950 RunWaf(args=NestedMap (100)) net472 192μs 1.05μs 7.94μs 5.31 0.373 0 33.67 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 168μs 270ns 1.04μs 0.5 0 0 39.1 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 206μs 739ns 2.86μs 0.507 0 0 39.63 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [23]) net472 259μs 63.9ns 230ns 6.3 0.514 0 40.23 KB
#4950 RunWaf(args=NestedMap (20)) net6.0 102μs 11.4ns 41ns 0.449 0 0 32.18 KB
#4950 RunWaf(args=NestedMap (20)) netcoreapp3.1 132μs 677ns 3.25μs 0.394 0 0 32.3 KB
#4950 RunWaf(args=NestedMap (20)) net472 186μs 867ns 3.36μs 5.11 0.365 0 32.63 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 167μs 75.7ns 283ns 0.518 0 0 38.53 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 207μs 481ns 1.86μs 0.494 0 0 38.6 KB
#4950 RunWafWithAttack(args=Neste(...)tack) [22]) net472 268μs 100ns 376ns 6.11 0.52 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 172μs 99.5ns 385ns 0.257 0 0 18.25 KB
master SendRequest netcoreapp3.1 193μs 186ns 721ns 0.192 0 0 20.41 KB
master SendRequest net472 0.000131ns 0.000101ns 0.000365ns 0 0 0 0 b
#4950 SendRequest net6.0 174μs 128ns 463ns 0.174 0 0 18.25 KB
#4950 SendRequest netcoreapp3.1 192μs 352ns 1.36μs 0.191 0 0 20.41 KB
#4950 SendRequest net472 0.000163ns 0.000113ns 0.000423ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #4950

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.72 KB 41.4 KB -323 B -0.77%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 574μs 395ns 1.53μs 0.546 0 0 41.72 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 659μs 372ns 1.39μs 0.327 0 0 41.71 KB
master WriteAndFlushEnrichedTraces net472 805μs 3.46μs 13.4μs 8.2 2.34 0.391 53.23 KB
#4950 WriteAndFlushEnrichedTraces net6.0 542μs 1.76μs 6.57μs 0.553 0 0 41.4 KB
#4950 WriteAndFlushEnrichedTraces netcoreapp3.1 664μs 412ns 1.54μs 0.334 0 0 41.78 KB
#4950 WriteAndFlushEnrichedTraces net472 844μs 3.18μs 12.3μs 8.17 2.45 0.408 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.03μs 0.604ns 2.34ns 0.0109 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.48μs 0.465ns 1.68ns 0.0104 0 0 768 B
master ExecuteNonQuery net472 1.73μs 3.42ns 13.3ns 0.115 0 0 730 B
#4950 ExecuteNonQuery net6.0 1.08μs 0.478ns 1.85ns 0.0104 0 0 768 B
#4950 ExecuteNonQuery netcoreapp3.1 1.41μs 2.13ns 8.25ns 0.0106 0 0 768 B
#4950 ExecuteNonQuery net472 1.79μs 4.13ns 16ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4950

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.175 1,351.28 1,150.03

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.35μs 0.598ns 2.24ns 0.0128 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.5μs 1.02ns 3.82ns 0.012 0 0 936 B
master CallElasticsearch net472 2.49μs 3.93ns 14.7ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.27μs 0.502ns 1.88ns 0.0126 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.67μs 0.697ns 2.7ns 0.0134 0 0 984 B
master CallElasticsearchAsync net472 2.64μs 3.39ns 13.1ns 0.16 0.00131 0 1.01 KB
#4950 CallElasticsearch net6.0 1.15μs 1.04ns 3.87ns 0.0133 0 0 936 B
#4950 CallElasticsearch netcoreapp3.1 1.53μs 1.2ns 4.66ns 0.0129 0 0 936 B
#4950 CallElasticsearch net472 2.47μs 2.17ns 8.41ns 0.151 0 0 955 B
#4950 CallElasticsearchAsync net6.0 1.26μs 0.375ns 1.4ns 0.0125 0 0 912 B
#4950 CallElasticsearchAsync netcoreapp3.1 1.56μs 0.658ns 2.46ns 0.0133 0 0 984 B
#4950 CallElasticsearchAsync net472 2.71μs 0.866ns 3.35ns 0.16 0.00135 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.26μs 1.16ns 4.51ns 0.0125 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.58μs 0.947ns 3.28ns 0.012 0 0 912 B
master ExecuteAsync net472 1.77μs 1.76ns 6.33ns 0.139 0 0 875 B
#4950 ExecuteAsync net6.0 1.31μs 0.421ns 1.52ns 0.0125 0 0 912 B
#4950 ExecuteAsync netcoreapp3.1 1.56μs 3.89ns 15.1ns 0.0124 0 0 912 B
#4950 ExecuteAsync net472 1.87μs 1.05ns 4.07ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.1μs 1.18ns 4.24ns 0.0292 0 0 2.1 KB
master SendAsync netcoreapp3.1 5μs 1.32ns 4.95ns 0.0353 0 0 2.63 KB
master SendAsync net472 7.63μs 7.41ns 28.7ns 0.524 0 0 3.31 KB
#4950 SendAsync net6.0 4.06μs 2.36ns 9.12ns 0.0282 0 0 2.1 KB
#4950 SendAsync netcoreapp3.1 5μs 2.46ns 9.54ns 0.035 0 0 2.63 KB
#4950 SendAsync net472 7.78μs 3.91ns 15.1ns 0.525 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #4950

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑netcoreapp3.1 1.450 70,400.00 102,100.00 multimodal

Faster 🎉 in #4950

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 1.239 62,200.00 50,200.00

More allocations ⚠️ in #4950

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 59.7 KB 60.02 KB 320 B 0.54%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 62.4μs 783ns 7.6μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 63μs 960ns 9.6μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.7μs 128ns 479ns 0 0 0 59.7 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 65μs 281ns 1.01μs 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 70.6μs 303ns 1.09μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 62.2μs 295ns 1.18μs 0 0 0 57.34 KB
#4950 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 58.1μs 715ns 6.9μs 0 0 0 43.44 KB
#4950 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 61.8μs 723ns 7.12μs 0 0 0 42.64 KB
#4950 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 37.9μs 46.8ns 162ns 0 0 0 60.02 KB
#4950 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 91.7μs 2.66μs 26.6μs 0 0 0 43.29 KB
#4950 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 103μs 2.68μs 26.8μs 0 0 0 42.64 KB
#4950 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 50.2μs 55.7ns 193ns 0 0 0 57.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.6μs 0.78ns 2.81ns 0.0216 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.28μs 0.858ns 3.21ns 0.0205 0 0 1.57 KB
master EnrichedLog net472 2.67μs 10.8ns 42ns 0.238 0 0 1.5 KB
#4950 EnrichedLog net6.0 1.44μs 0.636ns 2.46ns 0.0216 0 0 1.57 KB
#4950 EnrichedLog netcoreapp3.1 2.15μs 0.669ns 2.59ns 0.0215 0 0 1.57 KB
#4950 EnrichedLog net472 2.59μs 2.02ns 7.55ns 0.238 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 114μs 101ns 389ns 0.057 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 118μs 60.4ns 226ns 0 0 0 4.21 KB
master EnrichedLog net472 151μs 94.5ns 354ns 0.679 0.226 0 4.39 KB
#4950 EnrichedLog net6.0 113μs 185ns 691ns 0.0562 0 0 4.21 KB
#4950 EnrichedLog netcoreapp3.1 118μs 157ns 610ns 0 0 0 4.21 KB
#4950 EnrichedLog net472 151μs 90.5ns 339ns 0.69 0.23 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.02μs 0.898ns 3.48ns 0.0302 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.15μs 2.65ns 9.9ns 0.0269 0 0 2.13 KB
master EnrichedLog net472 4.83μs 3.57ns 13.8ns 0.309 0 0 1.95 KB
#4950 EnrichedLog net6.0 2.92μs 1.12ns 4.2ns 0.0291 0 0 2.13 KB
#4950 EnrichedLog netcoreapp3.1 4.16μs 1.8ns 6.97ns 0.029 0 0 2.13 KB
#4950 EnrichedLog net472 4.87μs 2.35ns 8.49ns 0.308 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.31μs 0.694ns 2.6ns 0.0157 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.77μs 0.5ns 1.94ns 0.0142 0 0 1.1 KB
master SendReceive net472 2.33μs 1.76ns 6.59ns 0.177 0 0 1.12 KB
#4950 SendReceive net6.0 1.41μs 0.645ns 2.5ns 0.0155 0 0 1.1 KB
#4950 SendReceive netcoreapp3.1 1.68μs 1.12ns 4.18ns 0.0151 0 0 1.1 KB
#4950 SendReceive net472 2.13μs 2.95ns 11.1ns 0.176 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.69μs 0.497ns 1.79ns 0.0215 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.88μs 3.9ns 15.1ns 0.0209 0 0 1.58 KB
master EnrichedLog net472 4.3μs 1.02ns 3.8ns 0.312 0 0 1.97 KB
#4950 EnrichedLog net6.0 2.63μs 1.54ns 5.96ns 0.021 0 0 1.53 KB
#4950 EnrichedLog netcoreapp3.1 3.95μs 0.941ns 3.52ns 0.0196 0 0 1.58 KB
#4950 EnrichedLog net472 4.33μs 1.71ns 6.64ns 0.311 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 468ns 0.185ns 0.715ns 0.00737 0 0 536 B
master StartFinishSpan netcoreapp3.1 689ns 0.243ns 0.908ns 0.00709 0 0 536 B
master StartFinishSpan net472 788ns 0.135ns 0.523ns 0.0851 0 0 538 B
master StartFinishScope net6.0 643ns 0.262ns 0.979ns 0.00922 0 0 656 B
master StartFinishScope netcoreapp3.1 771ns 0.24ns 0.867ns 0.00903 0 0 656 B
master StartFinishScope net472 940ns 0.195ns 0.755ns 0.0979 0 0 618 B
#4950 StartFinishSpan net6.0 501ns 0.136ns 0.509ns 0.00757 0 0 536 B
#4950 StartFinishSpan netcoreapp3.1 652ns 0.234ns 0.875ns 0.00741 0 0 536 B
#4950 StartFinishSpan net472 745ns 1.25ns 4.69ns 0.0853 0 0 538 B
#4950 StartFinishScope net6.0 614ns 0.145ns 0.542ns 0.0093 0 0 656 B
#4950 StartFinishScope netcoreapp3.1 841ns 0.162ns 0.583ns 0.00852 0 0 656 B
#4950 StartFinishScope net472 916ns 0.617ns 2.39ns 0.0978 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4950

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.213 592.17 718.04
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.113 1,048.71 1,166.77

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 592ns 0.263ns 1.02ns 0.00916 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 939ns 1.82ns 7.04ns 0.00878 0 0 656 B
master RunOnMethodBegin net472 1.05μs 1.08ns 4.19ns 0.0981 0 0 618 B
#4950 RunOnMethodBegin net6.0 718ns 0.168ns 0.652ns 0.00931 0 0 656 B
#4950 RunOnMethodBegin netcoreapp3.1 949ns 2.87ns 11.1ns 0.00866 0 0 656 B
#4950 RunOnMethodBegin net472 1.17μs 0.426ns 1.65ns 0.098 0 0 618 B

NachoEchevarria
NachoEchevarria approved these changes Dec 4, 2023
View reviewed changes
Copy link
Contributor

@NachoEchevarria NachoEchevarria left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!!

@andrewlock andrewlock merged commit d791201 into master Dec 4, 2023
56 checks passed
@andrewlock andrewlock deleted the zach.montoya/null-ref-apisec branch December 4, 2023 14:41
@github-actions github-actions bot added this to the vNext milestone Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@anna-git anna-git anna-git approved these changes

@lucaspimentel lucaspimentel lucaspimentel approved these changes

Assignees

@zacharycmontoya zacharycmontoya

Labels
area:asm type:bug
Projects
None yet
Milestone
2.43.0
Development

Successfully merging this pull request may close these issues.
5 participants
@zacharycmontoya @andrewlock @lucaspimentel @anna-git @NachoEchevarria