Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Exception Debugging] Introducing the Exception Debugging product #5163

Merged
merged 27 commits into from
Mar 20, 2024
Merged

[Exception Debugging] Introducing the Exception Debugging product #5163

merged 27 commits into from
Mar 20, 2024

Conversation

GreenMatan
Copy link
Contributor

@GreenMatan GreenMatan commented Feb 8, 2024 •
edited
Loading

Summary of changes

Implemented the core of Exception Debugging according to my internal Exception Debugging V1 RFC.

Exception Debugging enables customers to pinpoint the root cause and sequence of events leading to exceptions visible in Error Tracking. This is done by capturing and attaching the variables of each method participating in those exceptions, showing the causality chain as if the customer had a debugger attached to the application at the point in time when the exception occurred.

When an exception occurs and reaches the service entry span, we analyze its call stack, instrumenting each frame. On subsequent occurrences of the same exception, snapshots of each frame (apart 3rd party code) are captured and uploaded. This approach allows customers to trace the variables and method flow leading to the exception.

For V1 we support 5xx errors, thus the lifecycle of the root entry span is followed. The duration and async context in which the root span is active is where Exception Debugging is allowed to act. When the root span is marked with an error, we analyze the call stack of the exception for tracking / report snapshots of already tracked exceptions.

Reason for change

Assist customers to pinpoint the root cause of exceptions thrown in their application, that lead to 5xx.

Implementation details

The instrumentation of Exception Debugging relies on Dynamic Instrumentation's Method Probes.
A hook was added to the marking logic of spans as errored to analyze and instrument all 1st party frames. Upon second encounter of the same exceptions, all the variables of each frame participating in the exception are collected and uploaded as debugger snapshots to the logs intake.

Test coverage

Subsequent PR.

@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch 2 times, most recently from 2a604b0 to 41ec64a Compare February 8, 2024 09:32
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Feb 8, 2024 •
edited
Loading

Datadog Report

Branch report: matang/exception-debugging-kickoff
Commit report: 893aa9b
Test service: dd-trace-dotnet

✅ 0 Failed, 330442 Passed, 2079 Skipped, 39m 58.09s Wall Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • ObfuscateWithDefaultPattern - Datadog.Trace.Tests.Util.Http.EvidenceRedactorTests - Last Failure

    Expand for error 
     Expected string to be "key1=val1&<redacted>&key2=val2" with a length of 30, but "" has a length of 0, differs near "" (index 0).

@andrewlock
Copy link
Member

andrewlock commented Feb 8, 2024 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (73ms)  : 64, 83
     .   : milestone, 73,
    master - mean (74ms)  : 66, 83
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (1,000ms)  : 984, 1017
     .   : milestone, 1000,
    master - mean (1,000ms)  : 981, 1019
     .   : milestone, 1000,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (111ms)  : 108, 114
     .   : milestone, 111,
    master - mean (110ms)  : 107, 114
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (719ms)  : 694, 744
     .   : milestone, 719,
    master - mean (716ms)  : 694, 738
     .   : milestone, 716,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (94ms)  : 91, 98
     .   : milestone, 94,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (673ms)  : 650, 695
     .   : milestone, 673,
    master - mean (672ms)  : 653, 691
     .   : milestone, 672,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (188ms)  : 185, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (1,071ms)  : 1046, 1097
     .   : milestone, 1071,
    master - mean (1,071ms)  : 1054, 1089
     .   : milestone, 1071,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (271ms)  : 267, 276
     .   : milestone, 271,
    master - mean (270ms)  : 265, 276
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (875ms)  : 850, 900
     .   : milestone, 875,
    master - mean (875ms)  : 855, 895
     .   : milestone, 875,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5163) - mean (262ms)  : 258, 267
     .   : milestone, 262,
    master - mean (260ms)  : 256, 264
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (5163) - mean (861ms)  : 838, 883
     .   : milestone, 861,
    master - mean (855ms)  : 825, 884
     .   : milestone, 855,
Loading

@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch from 41ec64a to 7c3e5b8 Compare February 8, 2024 10:27
@andrewlock
Copy link
Member

andrewlock commented Feb 8, 2024 •
edited
Loading

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5163) (11.112M)   : 0, 11112380
    master (11.275M)   : 0, 11275303
    benchmarks/2.9.0 (10.971M)   : 0, 10971099

    section Automatic
    This PR (5163) (7.665M)   : 0, 7664836
    master (7.666M)   : 0, 7665532
    benchmarks/2.9.0 (7.985M)   : 0, 7985222

    section Trace stats
    This PR (5163) (8.018M)   : 0, 8017982
    master (7.921M)   : 0, 7921134

    section Manual
    This PR (5163) (9.786M)   : 0, 9785581
    master (9.714M)   : 0, 9713676

    section Manual + Automatic
    This PR (5163) (7.264M)   : 0, 7263907
    master (7.349M)   : 0, 7349366

    section Version Conflict
    This PR (5163) (6.535M)   : 0, 6534780
    master (6.632M)   : 0, 6632189
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5163) (9.642M)   : 0, 9641506
    master (9.545M)   : 0, 9544752
    benchmarks/2.9.0 (9.554M)   : 0, 9554391

    section Automatic
    This PR (5163) (6.559M)   : 0, 6559386
    master (6.521M)   : 0, 6520796

    section Trace stats
    This PR (5163) (6.961M)   : 0, 6961376
    master (6.754M)   : 0, 6754071

    section Manual
    This PR (5163) (8.272M)   : 0, 8272342
    master (8.175M)   : 0, 8174781

    section Manual + Automatic
    This PR (5163) (5.971M)   : crit ,0, 5971445
    master (6.335M)   : 0, 6334868

    section Version Conflict
    This PR (5163) (5.714M)   : 0, 5714227
    master (5.782M)   : 0, 5781917
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5163) (9.879M)   : 0, 9879474
    master (10.170M)   : 0, 10170429
    benchmarks/2.9.0 (10.017M)   : 0, 10017367

    section Automatic
    This PR (5163) (7.026M)   : 0, 7026036
    master (7.080M)   : 0, 7079776
    benchmarks/2.9.0 (7.417M)   : 0, 7416538

    section Trace stats
    This PR (5163) (7.475M)   : 0, 7474719
    master (7.506M)   : 0, 7506143

    section Manual
    This PR (5163) (8.661M)   : 0, 8660927
    master (8.778M)   : 0, 8778038

    section Manual + Automatic
    This PR (5163) (6.856M)   : 0, 6856190
    master (6.847M)   : 0, 6847478

    section Version Conflict
    This PR (5163) (6.137M)   : 0, 6136886
    master (6.163M)   : 0, 6162877
Loading

@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch from 7c3e5b8 to 57fc175 Compare February 8, 2024 15:10
@andrewlock
Copy link
Member

andrewlock commented Feb 9, 2024 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #5163 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.226
  • 4 benchmarks are slower, with geometric mean 1.194
  • 36 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.43μs 44.8ns 269ns 0.0368 0.0123 0 7.49 KB
master StartStopWithChild netcoreapp3.1 10.5μs 58.6ns 389ns 0.0312 0.0156 0 7.59 KB
master StartStopWithChild net472 16.8μs 61.7ns 239ns 1.32 0.325 0.108 7.96 KB
#5163 StartStopWithChild net6.0 9μs 49.3ns 274ns 0.0212 0.00848 0 7.51 KB
#5163 StartStopWithChild netcoreapp3.1 10.6μs 55.5ns 293ns 0.0317 0.0158 0 7.6 KB
#5163 StartStopWithChild net472 17μs 45.8ns 177ns 1.33 0.334 0.109 7.97 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 443μs 468ns 1.81μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 616μs 194ns 725ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 805μs 194ns 752ns 0.401 0 0 3.3 KB
#5163 WriteAndFlushEnrichedTraces net6.0 443μs 315ns 1.18μs 0 0 0 2.7 KB
#5163 WriteAndFlushEnrichedTraces netcoreapp3.1 627μs 274ns 1.06μs 0 0 0 2.7 KB
#5163 WriteAndFlushEnrichedTraces net472 803μs 394ns 1.52μs 0.401 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5163

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1 1.226 251.06 204.82

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 40.5μs 21.1ns 79.1ns 0.0204 0 0 2.36 KB
master AllCycleSimpleBody netcoreapp3.1 44μs 71.6ns 258ns 0.0219 0 0 2.34 KB
master AllCycleSimpleBody net472 47.6μs 20.4ns 79.2ns 0.38 0 0 2.41 KB
master AllCycleMoreComplexBody net6.0 213μs 183ns 710ns 0.107 0 0 9.84 KB
master AllCycleMoreComplexBody netcoreapp3.1 227μs 215ns 832ns 0.113 0 0 9.73 KB
master AllCycleMoreComplexBody net472 243μs 72.6ns 272ns 1.57 0 0 9.91 KB
master ObjectExtractorSimpleBody net6.0 145ns 0.167ns 0.623ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 252ns 0.398ns 1.54ns 0.00364 0 0 272 B
master ObjectExtractorSimpleBody net472 164ns 0.082ns 0.318ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3μs 1.54ns 5.95ns 0.0526 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.09μs 2.5ns 9.68ns 0.0512 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.8μs 1.89ns 7.07ns 0.603 0.00574 0 3.8 KB
#5163 AllCycleSimpleBody net6.0 40.9μs 28.3ns 110ns 0.0205 0 0 2.37 KB
#5163 AllCycleSimpleBody netcoreapp3.1 43.7μs 20.8ns 78ns 0.022 0 0 2.34 KB
#5163 AllCycleSimpleBody net472 47.3μs 109ns 424ns 0.377 0 0 2.42 KB
#5163 AllCycleMoreComplexBody net6.0 213μs 129ns 448ns 0.116 0 0 9.85 KB
#5163 AllCycleMoreComplexBody netcoreapp3.1 225μs 175ns 678ns 0.113 0 0 9.74 KB
#5163 AllCycleMoreComplexBody net472 243μs 399ns 1.54μs 1.56 0 0 9.92 KB
#5163 ObjectExtractorSimpleBody net6.0 140ns 0.0799ns 0.299ns 0.00394 0 0 280 B
#5163 ObjectExtractorSimpleBody netcoreapp3.1 205ns 0.0605ns 0.21ns 0.00371 0 0 272 B
#5163 ObjectExtractorSimpleBody net472 172ns 0.174ns 0.673ns 0.0446 0 0 281 B
#5163 ObjectExtractorMoreComplexBody net6.0 3μs 1.05ns 3.93ns 0.0536 0 0 3.78 KB
#5163 ObjectExtractorMoreComplexBody netcoreapp3.1 4.06μs 2.09ns 8.08ns 0.0506 0 0 3.69 KB
#5163 ObjectExtractorMoreComplexBody net472 3.86μs 2.28ns 8.84ns 0.603 0.00576 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 52.9μs 54.5ns 211ns 0.423 0 0 30.94 KB
master EncodeArgs netcoreapp3.1 69.2μs 43.2ns 156ns 0.41 0 0 31.47 KB
master EncodeArgs net472 85.2μs 65.2ns 244ns 5.12 0.0847 0 32.27 KB
master EncodeLegacyArgs net6.0 124μs 356ns 1.33μs 0.436 0 0 33.89 KB
master EncodeLegacyArgs netcoreapp3.1 156μs 172ns 619ns 0.475 0 0 34.09 KB
master EncodeLegacyArgs net472 213μs 285ns 1.1μs 5.48 0.422 0 35 KB
#5163 EncodeArgs net6.0 52.2μs 95ns 355ns 0.439 0 0 30.94 KB
#5163 EncodeArgs netcoreapp3.1 70.1μs 104ns 403ns 0.421 0 0 31.47 KB
#5163 EncodeArgs net472 85.4μs 88.7ns 344ns 5.1 0.0849 0 32.27 KB
#5163 EncodeLegacyArgs net6.0 125μs 485ns 1.88μs 0.491 0 0 33.89 KB
#5163 EncodeLegacyArgs netcoreapp3.1 155μs 526ns 2.04μs 0.455 0 0 34.09 KB
#5163 EncodeLegacyArgs net472 217μs 72.8ns 282ns 5.53 0.425 0 34.99 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 190μs 152ns 588ns 0.0953 0 0 6.51 KB
master RunWafRealisticBenchmark netcoreapp3.1 204μs 235ns 911ns 0 0 0 6.49 KB
master RunWafRealisticBenchmark net472 226μs 187ns 725ns 1.02 0 0 6.59 KB
master RunWafRealisticBenchmarkWithAttack net6.0 129μs 155ns 598ns 0 0 0 4.15 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 135μs 77.5ns 290ns 0 0 0 4.14 KB
master RunWafRealisticBenchmarkWithAttack net472 148μs 59.1ns 221ns 0.594 0 0 4.19 KB
#5163 RunWafRealisticBenchmark net6.0 191μs 187ns 722ns 0.0944 0 0 6.51 KB
#5163 RunWafRealisticBenchmark netcoreapp3.1 204μs 189ns 732ns 0 0 0 6.49 KB
#5163 RunWafRealisticBenchmark net472 223μs 141ns 545ns 1 0 0 6.59 KB
#5163 RunWafRealisticBenchmarkWithAttack net6.0 127μs 215ns 834ns 0.0634 0 0 4.15 KB
#5163 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 134μs 158ns 613ns 0 0 0 4.14 KB
#5163 RunWafRealisticBenchmarkWithAttack net472 148μs 105ns 405ns 0.664 0 0 4.19 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 175μs 123ns 477ns 0.175 0 0 18.26 KB
master SendRequest netcoreapp3.1 191μs 300ns 1.16μs 0.19 0 0 20.42 KB
master SendRequest net472 0.000896ns 0.000432ns 0.00156ns 0 0 0 0 b
#5163 SendRequest net6.0 172μs 180ns 698ns 0.257 0 0 18.28 KB
#5163 SendRequest netcoreapp3.1 195μs 277ns 1.07μs 0.193 0 0 20.44 KB
#5163 SendRequest net472 0.000814ns 0.000299ns 0.00116ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 545μs 1.27μs 4.91μs 0.543 0 0 41.55 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 654μs 916ns 3.55μs 0.326 0 0 41.68 KB
master WriteAndFlushEnrichedTraces net472 864μs 3.93μs 15.2μs 8.19 2.59 0.431 53.25 KB
#5163 WriteAndFlushEnrichedTraces net6.0 522μs 895ns 3.35μs 0.532 0 0 41.65 KB
#5163 WriteAndFlushEnrichedTraces netcoreapp3.1 635μs 1.08μs 4.18μs 0.324 0 0 41.79 KB
#5163 WriteAndFlushEnrichedTraces net472 844μs 2.78μs 10.8μs 8.33 2.5 0.417 53.24 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 738 B 746 B 8 B 1.08%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 776 B 784 B 8 B 1.03%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 776 B 784 B 8 B 1.03%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.12μs 0.435ns 1.68ns 0.0107 0 0 776 B
master ExecuteNonQuery netcoreapp3.1 1.52μs 0.795ns 3.08ns 0.0106 0 0 776 B
master ExecuteNonQuery net472 1.71μs 4.19ns 16.2ns 0.117 0 0 738 B
#5163 ExecuteNonQuery net6.0 1.17μs 0.584ns 2.26ns 0.0111 0 0 784 B
#5163 ExecuteNonQuery netcoreapp3.1 1.45μs 0.759ns 2.94ns 0.0108 0 0 784 B
#5163 ExecuteNonQuery net472 1.84μs 3.03ns 11.7ns 0.118 0 0 746 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 920 B 928 B 8 B 0.87%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 944 B 952 B 8 B 0.85%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 944 B 952 B 8 B 0.85%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 963 B 971 B 8 B 0.83%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 992 B 1 KB 8 B 0.81%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.02 KB 1.03 KB 8 B 0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.26μs 0.684ns 2.56ns 0.0134 0 0 944 B
master CallElasticsearch netcoreapp3.1 1.56μs 1.79ns 6.95ns 0.0124 0 0 944 B
master CallElasticsearch net472 2.49μs 1.59ns 6.16ns 0.153 0 0 963 B
master CallElasticsearchAsync net6.0 1.26μs 0.948ns 3.55ns 0.0127 0 0 920 B
master CallElasticsearchAsync netcoreapp3.1 1.66μs 0.674ns 2.61ns 0.0132 0 0 992 B
master CallElasticsearchAsync net472 2.59μs 0.585ns 2.03ns 0.162 0 0 1.02 KB
#5163 CallElasticsearch net6.0 1.24μs 0.414ns 1.55ns 0.0131 0 0 952 B
#5163 CallElasticsearch netcoreapp3.1 1.58μs 0.61ns 2.28ns 0.0128 0 0 952 B
#5163 CallElasticsearch net472 2.48μs 3.48ns 13.5ns 0.154 0 0 971 B
#5163 CallElasticsearchAsync net6.0 1.27μs 0.606ns 2.18ns 0.013 0 0 928 B
#5163 CallElasticsearchAsync netcoreapp3.1 1.69μs 0.5ns 1.87ns 0.0135 0 0 1 KB
#5163 CallElasticsearchAsync net472 2.68μs 1.18ns 4.57ns 0.162 0 0 1.03 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net472 883 B 891 B 8 B 0.91%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 920 B 928 B 8 B 0.87%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 920 B 928 B 8 B 0.87%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.32μs 0.56ns 2.17ns 0.0127 0 0 920 B
master ExecuteAsync netcoreapp3.1 1.6μs 1.42ns 5.31ns 0.0127 0 0 920 B
master ExecuteAsync net472 1.8μs 1ns 3.76ns 0.14 0 0 883 B
#5163 ExecuteAsync net6.0 1.39μs 0.468ns 1.75ns 0.0126 0 0 928 B
#5163 ExecuteAsync netcoreapp3.1 1.6μs 1.03ns 3.86ns 0.0122 0 0 928 B
#5163 ExecuteAsync net472 1.83μs 0.523ns 1.96ns 0.141 0 0 891 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.22μs 2.95ns 10.6ns 0.0279 0 0 2.1 KB
master SendAsync netcoreapp3.1 5.09μs 1.94ns 7.52ns 0.0357 0 0 2.64 KB
master SendAsync net472 7.64μs 1.99ns 7.72ns 0.525 0 0 3.31 KB
#5163 SendAsync net6.0 4.13μs 1.91ns 7.16ns 0.0288 0 0 2.11 KB
#5163 SendAsync netcoreapp3.1 4.96μs 2.44ns 9.45ns 0.0348 0 0 2.65 KB
#5163 SendAsync net472 7.73μs 2.11ns 8.17ns 0.526 0 0 3.32 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #5163

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 1.246 50,500.00 62,900.00 bimodal

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 206.24 KB 213.81 KB 7.57 KB 3.67%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 202.57 KB 206.68 KB 4.11 KB 2.03%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 59.07 KB 59.74 KB 672 B 1.14%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 50.2μs 212ns 764ns 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 55.6μs 542ns 5.42μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.3μs 98.4ns 355ns 0 0 0 59.07 KB
master StringConcatAspectBenchmark net6.0 273μs 1.17μs 5.73μs 0 0 0 206.24 KB
master StringConcatAspectBenchmark netcoreapp3.1 285μs 1.4μs 9.21μs 0 0 0 202.57 KB
master StringConcatAspectBenchmark net472 239μs 2.71μs 25.7μs 0 0 0 221.18 KB
#5163 StringConcatBenchmark net6.0 62.4μs 827ns 8.19μs 0 0 0 43.44 KB
#5163 StringConcatBenchmark netcoreapp3.1 53.2μs 265ns 1.09μs 0 0 0 42.64 KB
#5163 StringConcatBenchmark net472 38μs 90.9ns 328ns 0 0 0 59.74 KB
#5163 StringConcatAspectBenchmark net6.0 278μs 1.42μs 9.75μs 0 0 0 213.81 KB
#5163 StringConcatAspectBenchmark netcoreapp3.1 276μs 1.77μs 17μs 0 0 0 206.68 KB
#5163 StringConcatAspectBenchmark net472 242μs 2.48μs 23.3μs 0 0 0 221.18 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.51 KB 1.52 KB 16 B 1.06%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.58 KB 1.59 KB 16 B 1.01%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.58 KB 1.59 KB 16 B 1.01%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.53μs 0.794ns 3.07ns 0.0222 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 2.21μs 1.38ns 5.35ns 0.0209 0 0 1.58 KB
master EnrichedLog net472 2.56μs 1ns 3.76ns 0.239 0 0 1.51 KB
#5163 EnrichedLog net6.0 1.48μs 0.732ns 2.74ns 0.0223 0 0 1.59 KB
#5163 EnrichedLog netcoreapp3.1 2.15μs 1.25ns 4.67ns 0.0216 0 0 1.59 KB
#5163 EnrichedLog net472 2.63μs 2.86ns 11.1ns 0.242 0 0 1.52 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 131ns 506ns 0.0566 0 0 4.22 KB
master EnrichedLog netcoreapp3.1 119μs 118ns 425ns 0 0 0 4.22 KB
master EnrichedLog net472 149μs 175ns 653ns 0.667 0.222 0 4.4 KB
#5163 EnrichedLog net6.0 115μs 81.8ns 306ns 0.0577 0 0 4.23 KB
#5163 EnrichedLog netcoreapp3.1 117μs 152ns 590ns 0.0584 0 0 4.23 KB
#5163 EnrichedLog net472 147μs 103ns 397ns 0.659 0.22 0 4.41 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 1.95 KB 1.97 KB 16 B 0.82%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.14 KB 2.15 KB 16 B 0.75%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.14 KB 2.15 KB 16 B 0.75%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.02μs 1.61ns 6.04ns 0.0287 0 0 2.14 KB
master EnrichedLog netcoreapp3.1 4.22μs 2.89ns 11.2ns 0.0295 0 0 2.14 KB
master EnrichedLog net472 4.73μs 1.64ns 6.14ns 0.309 0 0 1.95 KB
#5163 EnrichedLog net6.0 3.09μs 0.796ns 2.98ns 0.031 0 0 2.15 KB
#5163 EnrichedLog netcoreapp3.1 4.24μs 1.38ns 5.16ns 0.0276 0 0 2.15 KB
#5163 EnrichedLog net472 5.03μs 2.36ns 9.13ns 0.312 0 0 1.97 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.11 KB 1.12 KB 8 B 0.72%
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1 1.11 KB 1.12 KB 8 B 0.72%
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472 1.12 KB 1.13 KB 8 B 0.71%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.4μs 0.92ns 3.56ns 0.0154 0 0 1.11 KB
master SendReceive netcoreapp3.1 1.82μs 1.77ns 6.87ns 0.0148 0 0 1.11 KB
master SendReceive net472 2.01μs 1.68ns 6.49ns 0.178 0 0 1.12 KB
#5163 SendReceive net6.0 1.37μs 0.778ns 3.01ns 0.0158 0 0 1.12 KB
#5163 SendReceive netcoreapp3.1 1.93μs 2.03ns 7.85ns 0.0146 0 0 1.12 KB
#5163 SendReceive net472 2.13μs 2.31ns 8.96ns 0.18 0 0 1.13 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.54 KB 1.55 KB 16 B 1.04%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.58 KB 1.6 KB 16 B 1.01%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 1.97 KB 1.99 KB 16 B 0.81%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.88μs 0.877ns 3.4ns 0.0216 0 0 1.54 KB
master EnrichedLog netcoreapp3.1 3.87μs 1.69ns 6.54ns 0.0214 0 0 1.58 KB
master EnrichedLog net472 4.34μs 1.56ns 5.83ns 0.312 0 0 1.97 KB
#5163 EnrichedLog net6.0 2.8μs 0.449ns 1.56ns 0.0211 0 0 1.55 KB
#5163 EnrichedLog netcoreapp3.1 3.92μs 1.84ns 7.14ns 0.0203 0 0 1.6 KB
#5163 EnrichedLog net472 4.34μs 1.96ns 7.6ns 0.315 0 0 1.99 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #5163

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 1.193 666.16 794.89
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.164 514.36 598.75

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 544 B 552 B 8 B 1.47%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 544 B 552 B 8 B 1.47%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 546 B 554 B 8 B 1.47%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 626 B 634 B 8 B 1.28%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 664 B 672 B 8 B 1.20%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 664 B 672 B 8 B 1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 527ns 1.23ns 4.75ns 0.00769 0 0 544 B
master StartFinishSpan netcoreapp3.1 673ns 0.872ns 3.26ns 0.00745 0 0 544 B
master StartFinishSpan net472 667ns 1.39ns 5.38ns 0.0867 0 0 546 B
master StartFinishScope net6.0 514ns 1.08ns 4.18ns 0.00915 0 0 664 B
master StartFinishScope netcoreapp3.1 868ns 1.68ns 6.51ns 0.00895 0 0 664 B
master StartFinishScope net472 970ns 1.42ns 5.51ns 0.0993 0 0 626 B
#5163 StartFinishSpan net6.0 576ns 0.798ns 3.09ns 0.00761 0 0 552 B
#5163 StartFinishSpan netcoreapp3.1 708ns 3.32ns 12.8ns 0.00749 0 0 552 B
#5163 StartFinishSpan net472 793ns 2.07ns 7.73ns 0.0878 0 0 554 B
#5163 StartFinishScope net6.0 599ns 0.643ns 2.49ns 0.0094 0 0 672 B
#5163 StartFinishScope netcoreapp3.1 807ns 0.553ns 2.14ns 0.00914 0 0 672 B
#5163 StartFinishScope net472 958ns 1.22ns 4.71ns 0.1 0 0 634 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #5163

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.174 598.38 702.31

More allocations ⚠️ in #5163

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 626 B 634 B 8 B 1.28%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 664 B 672 B 8 B 1.20%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 664 B 672 B 8 B 1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 598ns 1.58ns 6.12ns 0.00928 0 0 664 B
master RunOnMethodBegin netcoreapp3.1 963ns 2.89ns 11.2ns 0.00881 0 0 664 B
master RunOnMethodBegin net472 1.14μs 2ns 7.76ns 0.0995 0 0 626 B
#5163 RunOnMethodBegin net6.0 702ns 0.349ns 1.35ns 0.00944 0 0 672 B
#5163 RunOnMethodBegin netcoreapp3.1 899ns 1.21ns 4.7ns 0.00897 0 0 672 B
#5163 RunOnMethodBegin net472 1.1μs 2.63ns 10.2ns 0.1 0 0 634 B

@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch 9 times, most recently from a135834 to 615f65d Compare February 12, 2024 12:20
@GreenMatan GreenMatan changed the title [Dynamic Instrumentation/Exception Debugging] Implemented the core of Exception Debugging [Exception Debugging] Introducing the Exception Debugging product Feb 12, 2024
@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch 11 times, most recently from 2a03f0d to cab399b Compare February 19, 2024 10:18
@GreenMatan
Changed the approach in which the ShadowStack is initialized & cleane…
6bde411 
…d for in flight requests - basing it on the root span lifecycle
@GreenMatan
Added 3rd party libraries detection + corrected async local cleanups
be2c034
@GreenMatan
Separated Dynamic Instrumentation's enablement from the Exception Deb…
3b22433 
…ugger
@GreenMatan
Added Exception Debugging to telemetry
ab3c866
@GreenMatan
Adapted the hashing usages on the existing vendored in Fnv1aHash algo…
bef6832 
…rithm instead of a customized implementation of it
@GreenMatan
Addressed PR comment - ditched Span.ExceptionDebugging and inlined th…
b8bea42 
…e interiors
@GreenMatan
Improved MethodExtensions.GetFullyQualifiedName according to PR feedback
43815e7
@GreenMatan
Simplified .NET Core versioning check
35cda2f
@GreenMatan
Removed the new field _shadowStack from Span.cs with the premise that…
d49d6d2 
… AsyncLocal will track the flow correctly
@GreenMatan
Addressed PR comment
4cafcfd
@GreenMatan
Excluded Exception Debugging from telemetry reporting
5f42bf7
@GreenMatan
Rebasing fixes + Addressing PR comment regarding field size
7415225
@GreenMatan
Fixed Probe Statuses of the Dynamic Instrumentation, normalizing INST…
2291020 
…RUMENTED (new status) to INSTSALLED
@GreenMatan
Fixed a small typo
7a945bb
@GreenMatan
Fixed an issue with timing @duration + approvals + rebase fixes
6779ac7
@GreenMatan
Added #nullable enable on top of each new file + adjusted the code ac…
f329904 
…cordingly
@GreenMatan
Renamed DD_EXCEPTION_DEBUGGING_CAPTURE_FULL_CALLSTACK -> DD_EXCEPTION…
ebc2427 
…_DEBUGGING_CAPTURE_FULL_CALLSTACK_ENABLED
@GreenMatan
added m_taskId into the scrubbed values
893aa9b
@GreenMatan GreenMatan force-pushed the matang/exception-debugging-kickoff branch from 0ba2553 to 893aa9b Compare March 20, 2024 07:59
@GreenMatan GreenMatan merged commit af7e9d9 into master Mar 20, 2024
58 of 61 checks passed
@GreenMatan GreenMatan deleted the matang/exception-debugging-kickoff branch March 20, 2024 09:38
@github-actions github-actions bot added this to the vNext milestone Mar 20, 2024
@GreenMatan GreenMatan mentioned this pull request Mar 20, 2024
Merged
GreenMatan added a commit that referenced this pull request Mar 20, 2024
@GreenMatan
Fixed a small bug slipped in as part of the nullability fixes done in…
f230812 
… PR #5163 + added diagnosting tags to Exception Debugging when the log level is set to Debug
@GreenMatan GreenMatan mentioned this pull request Mar 20, 2024
Merged
GreenMatan added a commit that referenced this pull request Mar 20, 2024
@GreenMatan
Fixed a small bug slipped in as part of the nullability fixes done in…
d474d46 
… PR #5163 + added diagnosting tags to Exception Debugging when the log level is set to Debug
GreenMatan added a commit that referenced this pull request Mar 20, 2024
@GreenMatan
Fixed a small bug slipped in as part of the nullability fixes done in…
5c9f6c9 
… PR #5163 + added diagnosting tags to Exception Debugging when the log level is set to Debug
@GreenMatan GreenMatan mentioned this pull request Apr 4, 2024
Merged
@andrewlock andrewlock modified the milestones: vNext, vNext-v2 Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock andrewlock approved these changes

@dudikeleti dudikeleti dudikeleti approved these changes

Assignees
No one assigned
Labels
area:debugger
Projects
None yet
Milestone
2.50.0
Development

Successfully merging this pull request may close these issues.
3 participants
@GreenMatan @andrewlock @dudikeleti