Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor MockTracerAgent to allow sending custom responses for any endpoint #4997

Merged
merged 10 commits into from
Dec 28, 2023
Merged

Refactor MockTracerAgent to allow sending custom responses for any endpoint #4997

merged 10 commits into from
Dec 28, 2023

Conversation

andrewlock
Copy link
Member

Summary of changes

Refactors MockTracerAgent slightly to allow returning a custom response/status code for any endpoint

Reason for change

I wanted to do this for the tracer flare and realised it would involve duplicating a bunch of stuff

Implementation details

  • Allow registering a generic custom response for any endpoint
  • Current limitation is that it uses this response for all requests to that endpoint, but that's all we need atm
  • Removed (now obsolete) RcmResponse and AgentBehaviour fields/properties

Test coverage

Covered by existing tests

Other details

Prerequiste for

Stacked on

@andrewlock andrewlock added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:tests unit tests, integration tests type:refactor labels Dec 21, 2023
@andrewlock andrewlock requested a review from a team as a code owner December 21, 2023 14:34
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Dec 21, 2023 •
edited
Loading

Datadog Report

Branch report: andrew/tracer_flare/mock-agent-responses
Commit report: 7921b39

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 307410 Passed, 994 Skipped, 31m 0.01s Wall Time

@andrewlock
Copy link
Member Author

andrewlock commented Dec 21, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (72ms)  : 61, 83
     .   : milestone, 72,
    master - mean (71ms)  : 63, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (1,032ms)  : 1015, 1048
     .   : milestone, 1032,
    master - mean (1,018ms)  : 1001, 1034
     .   : milestone, 1018,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (107ms)  : 103, 111
     .   : milestone, 107,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (733ms)  : 711, 755
     .   : milestone, 733,
    master - mean (721ms)  : 699, 744
     .   : milestone, 721,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (91ms)  : 88, 93
     .   : milestone, 91,
    master - mean (90ms)  : 83, 96
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (697ms)  : 676, 719
     .   : milestone, 697,
    master - mean (692ms)  : 671, 714
     .   : milestone, 692,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (189ms)  : 186, 192
     .   : milestone, 189,
    master - mean (188ms)  : 186, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (1,129ms)  : 1109, 1149
     .   : milestone, 1129,
    master - mean (1,126ms)  : 1098, 1154
     .   : milestone, 1126,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (273ms)  : 269, 276
     .   : milestone, 273,
    master - mean (273ms)  : 267, 278
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (1,085ms)  : 1056, 1115
     .   : milestone, 1085,
    master - mean (1,083ms)  : 1058, 1108
     .   : milestone, 1083,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4997) - mean (263ms)  : 259, 267
     .   : milestone, 263,
    master - mean (262ms)  : 259, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (4997) - mean (1,055ms)  : 1031, 1079
     .   : milestone, 1055,
    master - mean (1,047ms)  : 1023, 1071
     .   : milestone, 1047,
Loading

@andrewlock andrewlock force-pushed the andrew/tracer_flare/chunked_encoding branch from 2a76f33 to dbdfbb6 Compare December 21, 2023 16:44
@andrewlock andrewlock force-pushed the andrew/tracer_flare/mock-agent-responses branch from 9ebb12e to e9a19d0 Compare December 21, 2023 16:45
This was referenced Dec 21, 2023
Merged
Merged
Merged
Merged
Merged
@andrewlock
Copy link
Member Author

andrewlock commented Dec 21, 2023 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4997) (10.832M)   : 0, 10832165
    master (11.023M)   : 0, 11023309
    benchmarks/2.9.0 (10.850M)   : 0, 10849500

    section Automatic
    This PR (4997) (7.331M)   : 0, 7330940
    master (7.466M)   : 0, 7466308
    benchmarks/2.9.0 (7.823M)   : 0, 7822730

    section Trace stats
    This PR (4997) (7.678M)   : 0, 7678064
    master (7.718M)   : 0, 7717767

    section Manual
    This PR (4997) (9.563M)   : 0, 9562983
    master (9.430M)   : 0, 9430114

    section Manual + Automatic
    This PR (4997) (6.953M)   : 0, 6953069
    master (7.071M)   : 0, 7071290

    section Version Conflict
    This PR (4997) (6.260M)   : 0, 6259972
    master (6.346M)   : 0, 6345816
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4997) (9.517M)   : 0, 9516989
    master (9.715M)   : 0, 9714686
    benchmarks/2.9.0 (9.680M)   : 0, 9679996

    section Automatic
    This PR (4997) (6.558M)   : 0, 6557721
    master (6.423M)   : 0, 6423186

    section Trace stats
    This PR (4997) (6.865M)   : 0, 6864834
    master (6.963M)   : 0, 6963293

    section Manual
    This PR (4997) (8.144M)   : 0, 8144326
    master (8.450M)   : 0, 8449696

    section Manual + Automatic
    This PR (4997) (6.328M)   : 0, 6327826
    master (5.985M)   : 0, 5984843

    section Version Conflict
    This PR (4997) (5.606M)   : 0, 5606010
    master (5.754M)   : 0, 5753645
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4997) (9.655M)   : 0, 9654880
    master (10.285M)   : 0, 10285399
    benchmarks/2.9.0 (10.863M)   : 0, 10863181

    section Automatic
    This PR (4997) (6.790M)   : crit ,0, 6790239
    master (7.225M)   : 0, 7224692
    benchmarks/2.9.0 (8.130M)   : 0, 8129748

    section Trace stats
    This PR (4997) (6.969M)   : crit ,0, 6968683
    master (7.543M)   : 0, 7542799

    section Manual
    This PR (4997) (8.614M)   : 0, 8614061
    master (8.984M)   : 0, 8984036

    section Manual + Automatic
    This PR (4997) (7.128M)   : 0, 7128454
    master (6.977M)   : 0, 6976681

    section Version Conflict
    This PR (4997) (6.339M)   : 0, 6338971
    master (6.364M)   : 0, 6363807
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.339M)   : 0, 7339251
    benchmarks/2.9.0 (7.835M)   : 0, 7834738

    section No attack
    master (1.982M)   : 0, 1982279
    benchmarks/2.9.0 (3.201M)   : 0, 3200714

    section Attack
    master (1.591M)   : 0, 1591401
    benchmarks/2.9.0 (2.469M)   : 0, 2468672

    section Blocking
    master (3.230M)   : 0, 3229524

    section IAST default
    master (6.528M)   : 0, 6527810

    section IAST full
    master (5.692M)   : 0, 5691891

    section Base vuln
    master (0.961M)   : 0, 960692

    section IAST vuln
    master (0.859M)   : 0, 859242
Loading

@andrewlock
Copy link
Member Author

andrewlock commented Dec 21, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4997 compared to master:

  • 4 benchmarks are faster, with geometric mean 1.174
  • 2 benchmarks are slower, with geometric mean 1.138
  • 1 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.71μs 48.1ns 289ns 0.0212 0.00849 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.7μs 52.7ns 303ns 0.0176 0.00587 0 7.58 KB
master StartStopWithChild net472 16.9μs 50.7ns 196ns 1.34 0.362 0.109 7.94 KB
#4997 StartStopWithChild net6.0 8.55μs 44.4ns 259ns 0.0172 0.00861 0 7.49 KB
#4997 StartStopWithChild netcoreapp3.1 10.9μs 57ns 267ns 0.0212 0.00531 0 7.58 KB
#4997 StartStopWithChild net472 17.1μs 53.3ns 206ns 1.34 0.343 0.103 7.96 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 452μs 257ns 994ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 627μs 249ns 963ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 791μs 429ns 1.55μs 0.391 0 0 3.3 KB
#4997 WriteAndFlushEnrichedTraces net6.0 466μs 498ns 1.93μs 0 0 0 2.7 KB
#4997 WriteAndFlushEnrichedTraces netcoreapp3.1 617μs 224ns 839ns 0 0 0 2.7 KB
#4997 WriteAndFlushEnrichedTraces net472 785μs 377ns 1.41μs 0.391 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4997

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.255 234.25 186.66

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 43.6μs 143ns 554ns 0.0221 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 46.9μs 45.3ns 163ns 0.0235 0 0 1.74 KB
master AllCycleSimpleBody net472 49.3μs 122ns 471ns 0.269 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 216μs 89.2ns 334ns 0.108 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 228μs 102ns 381ns 0.112 0 0 9.14 KB
master AllCycleMoreComplexBody net472 239μs 72ns 279ns 1.43 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 133ns 0.0263ns 0.0982ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 234ns 0.155ns 0.599ns 0.00366 0 0 272 B
master ObjectExtractorSimpleBody net472 153ns 0.113ns 0.437ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.75μs 0.724ns 2.61ns 0.0539 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.64μs 0.857ns 3.21ns 0.0508 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.54μs 5.65ns 21.9ns 0.602 0.0051 0 3.8 KB
#4997 AllCycleSimpleBody net6.0 42.8μs 95.5ns 370ns 0.0212 0 0 1.77 KB
#4997 AllCycleSimpleBody netcoreapp3.1 46.8μs 33ns 123ns 0 0 0 1.74 KB
#4997 AllCycleSimpleBody net472 49.5μs 15.2ns 59ns 0.267 0 0 1.81 KB
#4997 AllCycleMoreComplexBody net6.0 214μs 272ns 1.05μs 0.106 0 0 9.25 KB
#4997 AllCycleMoreComplexBody netcoreapp3.1 226μs 165ns 619ns 0.112 0 0 9.14 KB
#4997 AllCycleMoreComplexBody net472 238μs 67.8ns 263ns 1.45 0 0 9.32 KB
#4997 ObjectExtractorSimpleBody net6.0 131ns 0.055ns 0.206ns 0.00396 0 0 280 B
#4997 ObjectExtractorSimpleBody netcoreapp3.1 187ns 0.1ns 0.388ns 0.00377 0 0 272 B
#4997 ObjectExtractorSimpleBody net472 154ns 0.0928ns 0.359ns 0.0446 0 0 281 B
#4997 ObjectExtractorMoreComplexBody net6.0 2.93μs 1.08ns 4.04ns 0.0528 0 0 3.78 KB
#4997 ObjectExtractorMoreComplexBody netcoreapp3.1 3.71μs 1.25ns 4.82ns 0.0499 0 0 3.69 KB
#4997 ObjectExtractorMoreComplexBody net472 3.46μs 1.84ns 6.89ns 0.603 0.00518 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 51.7μs 158ns 612ns 0.224 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 67.3μs 386ns 3.18μs 0.193 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 94μs 76.5ns 296ns 2.54 0.0941 0 16.14 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 119μs 574ns 2.29μs 0.285 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 137μs 675ns 2.87μs 0.278 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 173μs 100ns 389ns 3.57 0.166 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 95.9μs 27.3ns 102ns 0.431 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 736ns 5.05μs 0.434 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 183μs 123ns 476ns 5.29 0.365 0 33.67 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 163μs 55ns 198ns 0.528 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 213μs 314ns 1.22μs 0.523 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 270μs 172ns 666ns 6.3 0.525 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 105μs 222ns 860ns 0.425 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 130μs 630ns 2.52μs 0.399 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 182μs 140ns 523ns 5.17 0.363 0 32.63 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 163μs 181ns 700ns 0.492 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 212μs 1.1μs 5.03μs 0.511 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 260μs 120ns 450ns 6.11 0.52 0 39.2 KB
#4997 RunWaf(args=NestedMap (10)) net6.0 49.7μs 115ns 503ns 0.225 0 0 16.06 KB
#4997 RunWaf(args=NestedMap (10)) netcoreapp3.1 71.6μs 271ns 1.05μs 0.21 0 0 16.06 KB
#4997 RunWaf(args=NestedMap (10)) net472 96.7μs 506ns 2.43μs 2.56 0.0949 0 16.14 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 121μs 487ns 1.75μs 0.248 0 0 22.41 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 141μs 756ns 4.21μs 0.274 0 0 22.36 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) net472 174μs 65.7ns 237ns 3.55 0.165 0 22.7 KB
#4997 RunWaf(args=NestedMap (100)) net6.0 94.9μs 87.6ns 328ns 0.418 0 0 32.76 KB
#4997 RunWaf(args=NestedMap (100)) netcoreapp3.1 133μs 407ns 1.63μs 0.448 0 0 33.33 KB
#4997 RunWaf(args=NestedMap (100)) net472 188μs 841ns 3.26μs 5.32 0.373 0 33.67 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 179μs 432ns 1.67μs 0.527 0 0 39.1 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 211μs 857ns 3.32μs 0.533 0 0 39.63 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [23]) net472 258μs 239ns 925ns 6.34 0.518 0 40.23 KB
#4997 RunWaf(args=NestedMap (20)) net6.0 107μs 593ns 3.93μs 0.414 0 0 32.18 KB
#4997 RunWaf(args=NestedMap (20)) netcoreapp3.1 126μs 92.7ns 347ns 0.374 0 0 32.3 KB
#4997 RunWaf(args=NestedMap (20)) net472 187μs 902ns 3.49μs 5.12 0.366 0 32.63 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 177μs 121ns 469ns 0.52 0 0 38.53 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 215μs 1.1μs 4.91μs 0.512 0 0 38.6 KB
#4997 RunWafWithAttack(args=Neste(...)tack) [22]) net472 274μs 1.16μs 4.51μs 6.21 0.529 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 169μs 165ns 641ns 0.255 0 0 18.25 KB
master SendRequest netcoreapp3.1 190μs 363ns 1.41μs 0.189 0 0 20.41 KB
master SendRequest net472 0.000619ns 0.00024ns 0.00093ns 0 0 0 0 b
#4997 SendRequest net6.0 167μs 115ns 416ns 0.167 0 0 18.25 KB
#4997 SendRequest netcoreapp3.1 191μs 237ns 919ns 0.191 0 0 20.41 KB
#4997 SendRequest net472 0.00107ns 0.000253ns 0.000981ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 540μs 282ns 1.09μs 0.521 0 0 41.5 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 1.16μs 4.49μs 0.331 0 0 41.68 KB
master WriteAndFlushEnrichedTraces net472 851μs 3.65μs 14.1μs 8.3 2.62 0.437 53.23 KB
#4997 WriteAndFlushEnrichedTraces net6.0 545μs 218ns 846ns 0.546 0 0 41.67 KB
#4997 WriteAndFlushEnrichedTraces netcoreapp3.1 647μs 426ns 1.47μs 0.326 0 0 41.62 KB
#4997 WriteAndFlushEnrichedTraces net472 873μs 4.27μs 18.6μs 8.45 2.53 0.422 53.23 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.08μs 0.488ns 1.83ns 0.011 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.44μs 0.9ns 3.49ns 0.0101 0 0 768 B
master ExecuteNonQuery net472 1.78μs 0.784ns 2.83ns 0.116 0 0 730 B
#4997 ExecuteNonQuery net6.0 1.06μs 0.411ns 1.54ns 0.0106 0 0 768 B
#4997 ExecuteNonQuery netcoreapp3.1 1.44μs 0.928ns 3.47ns 0.01 0 0 768 B
#4997 ExecuteNonQuery net472 1.83μs 1.14ns 4.12ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4997

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.133 1,124.11 1,273.52

Faster 🎉 in #4997

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.167 1,336.39 1,144.88

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.12μs 0.421ns 1.57ns 0.0132 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.51μs 0.633ns 2.19ns 0.0122 0 0 936 B
master CallElasticsearch net472 2.59μs 0.55ns 2.06ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.34μs 0.71ns 2.65ns 0.0127 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.66μs 0.766ns 2.97ns 0.0133 0 0 984 B
master CallElasticsearchAsync net472 2.61μs 0.474ns 1.83ns 0.16 0 0 1.01 KB
#4997 CallElasticsearch net6.0 1.27μs 0.393ns 1.36ns 0.0128 0 0 936 B
#4997 CallElasticsearch netcoreapp3.1 1.51μs 0.959ns 3.71ns 0.0128 0 0 936 B
#4997 CallElasticsearch net472 2.45μs 1.06ns 3.98ns 0.151 0 0 955 B
#4997 CallElasticsearchAsync net6.0 1.14μs 0.259ns 0.969ns 0.0126 0 0 912 B
#4997 CallElasticsearchAsync netcoreapp3.1 1.55μs 0.97ns 3.76ns 0.0132 0 0 984 B
#4997 CallElasticsearchAsync net472 2.73μs 0.941ns 3.52ns 0.16 0.00137 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.27μs 0.912ns 3.53ns 0.0127 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.58μs 1.05ns 4.08ns 0.0125 0 0 912 B
master ExecuteAsync net472 1.75μs 0.488ns 1.82ns 0.138 0 0 875 B
#4997 ExecuteAsync net6.0 1.38μs 0.74ns 2.87ns 0.0124 0 0 912 B
#4997 ExecuteAsync netcoreapp3.1 1.66μs 1.24ns 4.81ns 0.0119 0 0 912 B
#4997 ExecuteAsync net472 1.88μs 0.822ns 3.18ns 0.139 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.16μs 0.771ns 2.89ns 0.0292 0 0 2.1 KB
master SendAsync netcoreapp3.1 4.84μs 1.72ns 6.66ns 0.0362 0 0 2.63 KB
master SendAsync net472 7.74μs 2.82ns 10.9ns 0.524 0 0 3.31 KB
#4997 SendAsync net6.0 4.1μs 5.13ns 19.2ns 0.0302 0 0 2.1 KB
#4997 SendAsync netcoreapp3.1 5.01μs 3.45ns 13.3ns 0.0349 0 0 2.63 KB
#4997 SendAsync net472 7.73μs 2.54ns 9.83ns 0.523 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #4997

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 1.138 73,650.00 64,700.00 bimodal

More allocations ⚠️ in #4997

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 65.54 KB 73.73 KB 8.19 KB 12.50%

Fewer allocations 🎉 in #4997

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472 59.7 KB 57.97 KB -1.74 KB -2.91%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 52.6μs 302ns 2.14μs 0 0 0 43.44 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 62.2μs 794ns 7.82μs 0 0 0 42.64 KB
master StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38.3μs 114ns 428ns 0 0 0 59.7 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 99.8μs 2.81μs 28.1μs 0 0 0 43.29 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 109μs 4.13μs 40.9μs 0 0 0 42.64 KB
master StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 76.1μs 845ns 8.1μs 0 0 0 65.54 KB
#4997 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net6.0 60.1μs 765ns 7.58μs 0 0 0 43.44 KB
#4997 StringConcatBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 54.5μs 254ns 1.24μs 0 0 0 42.64 KB
#4997 StringConcatBenchmark(parameters=Syste(...)ring] [48]) net472 38.8μs 212ns 1.23μs 0 0 0 57.97 KB
#4997 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net6.0 96.4μs 2.49μs 24.9μs 0 0 0 43.29 KB
#4997 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) netcoreapp3.1 114μs 4.11μs 40.9μs 0 0 0 42.64 KB
#4997 StringConcatAspectBenchmark(parameters=Syste(...)ring] [48]) net472 63.5μs 966ns 9.47μs 0 0 0 73.73 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.54μs 0.691ns 2.59ns 0.0216 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.28μs 0.834ns 3.12ns 0.0213 0 0 1.57 KB
master EnrichedLog net472 2.55μs 2.7ns 10.5ns 0.238 0 0 1.5 KB
#4997 EnrichedLog net6.0 1.53μs 0.348ns 1.35ns 0.0222 0 0 1.57 KB
#4997 EnrichedLog netcoreapp3.1 2.16μs 1.38ns 4.77ns 0.0209 0 0 1.57 KB
#4997 EnrichedLog net472 2.7μs 2.35ns 8.79ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 115μs 111ns 431ns 0 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 120μs 144ns 557ns 0.0601 0 0 4.21 KB
master EnrichedLog net472 150μs 170ns 659ns 0.669 0.223 0 4.39 KB
#4997 EnrichedLog net6.0 113μs 160ns 620ns 0.0568 0 0 4.21 KB
#4997 EnrichedLog netcoreapp3.1 118μs 197ns 765ns 0.0592 0 0 4.21 KB
#4997 EnrichedLog net472 146μs 117ns 452ns 0.656 0.219 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.05μs 1.5ns 5.6ns 0.029 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.31μs 10.6ns 41.1ns 0.0281 0 0 2.13 KB
master EnrichedLog net472 4.85μs 2.31ns 8.94ns 0.308 0 0 1.95 KB
#4997 EnrichedLog net6.0 3.12μs 1.49ns 5.76ns 0.0304 0 0 2.13 KB
#4997 EnrichedLog netcoreapp3.1 4.2μs 0.909ns 3.4ns 0.0272 0 0 2.13 KB
#4997 EnrichedLog net472 4.83μs 0.734ns 2.75ns 0.308 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.25μs 2.31ns 8.95ns 0.015 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.81μs 0.44ns 1.71ns 0.0145 0 0 1.1 KB
master SendReceive net472 2.14μs 3.06ns 11.8ns 0.177 0 0 1.12 KB
#4997 SendReceive net6.0 1.38μs 0.994ns 3.72ns 0.0152 0 0 1.1 KB
#4997 SendReceive netcoreapp3.1 1.77μs 1ns 3.89ns 0.0151 0 0 1.1 KB
#4997 SendReceive net472 2.12μs 2.11ns 8.18ns 0.177 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.69μs 0.906ns 3.39ns 0.0205 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 4.02μs 1.84ns 7.12ns 0.02 0 0 1.58 KB
master EnrichedLog net472 4.35μs 2.03ns 7.85ns 0.311 0 0 1.97 KB
#4997 EnrichedLog net6.0 2.79μs 8.83ns 38.5ns 0.0216 0 0 1.53 KB
#4997 EnrichedLog netcoreapp3.1 4.1μs 1.11ns 4.29ns 0.0205 0 0 1.58 KB
#4997 EnrichedLog net472 4.49μs 1.63ns 5.89ns 0.31 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4997

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.138 875.14 769.33

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 452ns 0.162ns 0.628ns 0.00749 0 0 536 B
master StartFinishSpan netcoreapp3.1 704ns 0.154ns 0.576ns 0.0073 0 0 536 B
master StartFinishSpan net472 768ns 0.262ns 1.01ns 0.0853 0 0 538 B
master StartFinishScope net6.0 545ns 0.153ns 0.592ns 0.00904 0 0 656 B
master StartFinishScope netcoreapp3.1 874ns 0.666ns 2.49ns 0.0087 0 0 656 B
master StartFinishScope net472 900ns 0.329ns 1.27ns 0.0981 0 0 618 B
#4997 StartFinishSpan net6.0 471ns 0.243ns 0.909ns 0.00758 0 0 536 B
#4997 StartFinishSpan netcoreapp3.1 671ns 0.305ns 1.1ns 0.00732 0 0 536 B
#4997 StartFinishSpan net472 729ns 0.641ns 2.48ns 0.0853 0 0 538 B
#4997 StartFinishScope net6.0 576ns 0.288ns 1.08ns 0.00912 0 0 656 B
#4997 StartFinishScope netcoreapp3.1 769ns 0.172ns 0.595ns 0.00874 0 0 656 B
#4997 StartFinishScope net472 976ns 0.867ns 3.36ns 0.0981 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4997

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 1.142 882.56 1,008.02

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 625ns 0.177ns 0.661ns 0.00906 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 883ns 0.265ns 1.03ns 0.00884 0 0 656 B
master RunOnMethodBegin net472 1.14μs 0.382ns 1.48ns 0.0978 0 0 618 B
#4997 RunOnMethodBegin net6.0 690ns 0.389ns 1.51ns 0.00929 0 0 656 B
#4997 RunOnMethodBegin netcoreapp3.1 1.01μs 0.254ns 0.951ns 0.00902 0 0 656 B
#4997 RunOnMethodBegin net472 1.14μs 0.492ns 1.91ns 0.0978 0 0 618 B

kevingosse
kevingosse reviewed Dec 22, 2023
View reviewed changes
tracer/test/Datadog.Trace.TestHelpers/MockTracerAgent.cs
SendResponse = sendResponse,
StatusCode = statusCode
};
return CustomResponses.TryGetValue(responseType, out var custom)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the previous version, the status code was set explicitly to 200 for trace commands. Now it's not set anymore. I wonder if that's the reason why the test is failing, even though I feel like the status code should be 200 by default anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now it's not set anymore

Not true, the MockTracerResponse defaults to sending a 200 🙂 As it turns out, the problem was actually that we don't set 200 when an exception is thrown any more 😅 Anyway, should be fixed in #4988

@andrewlock andrewlock force-pushed the andrew/tracer_flare/chunked_encoding branch from dbdfbb6 to 46e27a2 Compare December 27, 2023 10:54
@andrewlock andrewlock force-pushed the andrew/tracer_flare/mock-agent-responses branch from e9a19d0 to fd5485a Compare December 27, 2023 10:54
@andrewlock
Vendor in PushStreamContent to avoid in-memory buffering
6bf7969 
Based on (and simplified from)
- https://github.com/aspnet/AspNetWebStack/blob/1231b77d79956152831b75ad7f094f844251b97f/src/System.Net.Http.Formatting/Internal/DelegatingStream.cs
- https://github.com/aspnet/AspNetWebStack/blob/1231b77d79956152831b75ad7f094f844251b97f/src/System.Net.Http.Formatting/PushStreamContent.cs
@andrewlock
Add support for streaming responses
5ed3009
@andrewlock
Don't require the caller to call close on the stream (as it seems weird)
6cb2e19
@andrewlock
Update trimming file
2eea2b2
@andrewlock
Add a chunked encoding stream writer
a018de5
@andrewlock
Update the built-in HttpClient to use the chunked encoding writer for…
0df770d 
… push stream content
@andrewlock
Add support for reading chunked encoded streams to MockHttpParser
3f76f58
@andrewlock
Add tests for chunked encoding
679fb6b
@andrewlock
Update trimming file
7d3b360
@andrewlock
Refactor MockTracerAgent to allow sending custom responses for any en…
7921b39 
…dpoint
@andrewlock andrewlock force-pushed the andrew/tracer_flare/chunked_encoding branch from 46e27a2 to 7d3b360 Compare December 27, 2023 13:31
@andrewlock andrewlock force-pushed the andrew/tracer_flare/mock-agent-responses branch from fd5485a to 7921b39 Compare December 27, 2023 13:31
bouwkast
bouwkast approved these changes Dec 27, 2023
View reviewed changes
Copy link
Contributor

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Base automatically changed from andrew/tracer_flare/chunked_encoding to master December 28, 2023 11:33
@andrewlock andrewlock merged commit 3189baf into master Dec 28, 2023
37 of 51 checks passed
@andrewlock andrewlock deleted the andrew/tracer_flare/mock-agent-responses branch December 28, 2023 11:35
@github-actions github-actions bot added this to the vNext milestone Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bouwkast bouwkast bouwkast approved these changes

@kevingosse kevingosse kevingosse approved these changes

Assignees
No one assigned
Labels
area:tests unit tests, integration tests area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:refactor
Projects
None yet
Milestone
2.45.0
Development

Successfully merging this pull request may close these issues.
3 participants
@andrewlock @bouwkast @kevingosse