Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix error checking for CallTargetBubbleUpException #5836

Merged
merged 2 commits into from
Aug 2, 2024

Conversation

kevingosse
Copy link
Collaborator

Summary of changes

Fix error checking when emitting the CallTargetBubbleUpException code.
As a bonus, add synchronization when creating the metadata tokens.

Reason for change

If for some reason we fail to create the metadata tokens, we could create an invalid EH clause, which causes an access violation later on.

Implementation details

Converted the calltarget_tokens mutex to a recursive mutex, because the EnsureBaseCalltargetTokens methods call each other.

Test coverage

Hard to test, but coincidentally I discovered that the the added synchronization fixes a crash when running powershell with SentinelOne activated.

Other details

Fixes #5799

@kevingosse kevingosse requested a review from a team as a code owner July 31, 2024 16:47
@andrewlock
Copy link
Member

andrewlock commented Jul 31, 2024

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (74ms)  : 62, 86
     .   : milestone, 74,
    master - mean (75ms)  : 63, 87
     .   : milestone, 75,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (1,043ms)  : 1024, 1061
     .   : milestone, 1043,
    master - mean (1,042ms)  : 1022, 1061
     .   : milestone, 1042,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (109ms)  : 106, 113
     .   : milestone, 109,
    master - mean (109ms)  : 105, 114
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (730ms)  : 709, 751
     .   : milestone, 730,
    master - mean (731ms)  : 712, 749
     .   : milestone, 731,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (92ms)  : 88, 97
     .   : milestone, 92,
    master - mean (92ms)  : 88, 96
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (679ms)  : 654, 703
     .   : milestone, 679,
    master - mean (677ms)  : 650, 704
     .   : milestone, 677,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (192ms)  : 189, 196
     .   : milestone, 192,
    master - mean (192ms)  : 188, 197
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (1,147ms)  : 1123, 1171
     .   : milestone, 1147,
    master - mean (1,145ms)  : 1114, 1176
     .   : milestone, 1145,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (275ms)  : 270, 280
     .   : milestone, 275,
    master - mean (277ms)  : 272, 282
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (904ms)  : 876, 933
     .   : milestone, 904,
    master - mean (907ms)  : 874, 939
     .   : milestone, 907,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5836) - mean (266ms)  : 261, 270
     .   : milestone, 266,
    master - mean (267ms)  : 262, 271
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (5836) - mean (884ms)  : 861, 906
     .   : milestone, 884,
    master - mean (884ms)  : 861, 907
     .   : milestone, 884,

Loading

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jul 31, 2024

Datadog Report

Branch report: kevin/bubble_metadata
Commit report: 5fb4e1d
Test service: dd-trace-dotnet

✅ 0 Failed, 350536 Passed, 2248 Skipped, 23h 39m 17.67s Total Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.StackExchangeRedisTests - Last Failure

    Expand for error
     Results do not match.
     Differences:
     Received: StackExchangeRedisTests.Latest.SchemaV1.received.txt
     Verified: StackExchangeRedisTests.Latest.SchemaV1.verified.txt
     Received Content:
     [
       {
         TraceId: Id_1,
         SpanId: Id_2,
         Name: redis.command,
     ...
    

@andrewlock
Copy link
Member

Benchmarks Report for tracer 🐌

Benchmarks for #5836 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.170
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.7μs 42ns 249ns 0.0151 0.00753 0 5.42 KB
master StartStopWithChild netcoreapp3.1 9.84μs 53.9ns 324ns 0.0193 0.00963 0 5.62 KB
master StartStopWithChild net472 16.2μs 72.9ns 291ns 1.03 0.316 0.103 6.06 KB
#5836 StartStopWithChild net6.0 7.73μs 44.1ns 312ns 0.0111 0.00371 0 5.42 KB
#5836 StartStopWithChild netcoreapp3.1 10μs 55.9ns 349ns 0.0143 0.00477 0 5.62 KB
#5836 StartStopWithChild net472 16μs 48.7ns 182ns 1.01 0.295 0.0958 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 461μs 293ns 1.06μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 636μs 511ns 1.98μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 845μs 752ns 2.91μs 0.422 0 0 3.3 KB
#5836 WriteAndFlushEnrichedTraces net6.0 458μs 362ns 1.4μs 0 0 0 2.7 KB
#5836 WriteAndFlushEnrichedTraces netcoreapp3.1 637μs 661ns 2.56μs 0 0 0 2.7 KB
#5836 WriteAndFlushEnrichedTraces net472 844μs 826ns 2.98μs 0.422 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 200μs 1.17μs 11.3μs 0.187 0 0 18.45 KB
master SendRequest netcoreapp3.1 216μs 1.24μs 9.65μs 0.246 0 0 20.61 KB
master SendRequest net472 0.000625ns 0.00045ns 0.00168ns 0 0 0 0 b
#5836 SendRequest net6.0 200μs 1.17μs 11.2μs 0.188 0 0 18.45 KB
#5836 SendRequest netcoreapp3.1 216μs 1.17μs 7.96μs 0.216 0 0 20.61 KB
#5836 SendRequest net472 0.000834ns 0.000421ns 0.00152ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 584μs 2.61μs 9.78μs 0.579 0 0 41.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 701μs 3.9μs 23.7μs 0.332 0 0 41.78 KB
master WriteAndFlushEnrichedTraces net472 888μs 3.66μs 14.2μs 8.42 2.66 0.443 53.29 KB
#5836 WriteAndFlushEnrichedTraces net6.0 574μs 2.81μs 15.4μs 0.293 0 0 41.55 KB
#5836 WriteAndFlushEnrichedTraces netcoreapp3.1 700μs 3.37μs 13.9μs 0.336 0 0 41.76 KB
#5836 WriteAndFlushEnrichedTraces net472 889μs 2.54μs 9.5μs 8.3 2.62 0.437 53.32 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.3μs 1.85ns 7.18ns 0.0142 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.71μs 3.07ns 11.1ns 0.0137 0 0 1.02 KB
master ExecuteNonQuery net472 2.05μs 1.8ns 6.98ns 0.157 0 0 987 B
#5836 ExecuteNonQuery net6.0 1.3μs 1.42ns 5.3ns 0.0143 0 0 1.02 KB
#5836 ExecuteNonQuery netcoreapp3.1 1.65μs 2.3ns 8.89ns 0.0139 0 0 1.02 KB
#5836 ExecuteNonQuery net472 1.94μs 2.09ns 8.11ns 0.157 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.17μs 0.344ns 1.24ns 0.0135 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.48μs 0.774ns 2.9ns 0.0133 0 0 976 B
master CallElasticsearch net472 2.47μs 1.39ns 5.2ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.3μs 1ns 3.88ns 0.0131 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.53μs 1.41ns 5.29ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.69μs 1.4ns 5.25ns 0.166 0.00134 0 1.05 KB
#5836 CallElasticsearch net6.0 1.27μs 0.932ns 3.61ns 0.0133 0 0 976 B
#5836 CallElasticsearch netcoreapp3.1 1.5μs 3.85ns 14.4ns 0.0126 0 0 976 B
#5836 CallElasticsearch net472 2.46μs 1.44ns 5.58ns 0.158 0.00123 0 995 B
#5836 CallElasticsearchAsync net6.0 1.38μs 0.633ns 2.45ns 0.0131 0 0 952 B
#5836 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.641ns 2.4ns 0.014 0 0 1.02 KB
#5836 CallElasticsearchAsync net472 2.62μs 1.36ns 4.91ns 0.167 0.00131 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.27μs 0.504ns 1.89ns 0.0134 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.63μs 0.504ns 1.82ns 0.0131 0 0 952 B
master ExecuteAsync net472 1.74μs 0.698ns 2.7ns 0.145 0 0 915 B
#5836 ExecuteAsync net6.0 1.29μs 0.417ns 1.56ns 0.013 0 0 952 B
#5836 ExecuteAsync netcoreapp3.1 1.65μs 0.977ns 3.78ns 0.0132 0 0 952 B
#5836 ExecuteAsync net472 1.69μs 0.574ns 2.07ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.22μs 1.64ns 6.37ns 0.0318 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.02μs 3.87ns 15ns 0.0376 0 0 2.76 KB
master SendAsync net472 7.74μs 3.37ns 13.1ns 0.498 0 0 3.15 KB
#5836 SendAsync net6.0 4.17μs 1.96ns 7.34ns 0.0313 0 0 2.22 KB
#5836 SendAsync netcoreapp3.1 5.14μs 2.93ns 10.2ns 0.036 0 0 2.76 KB
#5836 SendAsync net472 7.8μs 2.93ns 11.3ns 0.496 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.45μs 0.501ns 1.94ns 0.0233 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.14μs 2.21ns 8.57ns 0.0223 0 0 1.64 KB
master EnrichedLog net472 2.57μs 1.24ns 4.79ns 0.25 0 0 1.57 KB
#5836 EnrichedLog net6.0 1.61μs 6.91ns 26.8ns 0.0229 0 0 1.64 KB
#5836 EnrichedLog netcoreapp3.1 2.28μs 0.928ns 3.47ns 0.0226 0 0 1.64 KB
#5836 EnrichedLog net472 2.52μs 1.92ns 7.44ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 250ns 967ns 0.056 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 119μs 277ns 1.07μs 0.0599 0 0 4.28 KB
master EnrichedLog net472 148μs 197ns 764ns 0.664 0.221 0 4.46 KB
#5836 EnrichedLog net6.0 116μs 98.1ns 380ns 0.0582 0 0 4.28 KB
#5836 EnrichedLog netcoreapp3.1 120μs 153ns 573ns 0.0603 0 0 4.28 KB
#5836 EnrichedLog net472 147μs 380ns 1.47μs 0.671 0.224 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.26μs 1.03ns 3.99ns 0.0302 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.22μs 2.79ns 10.4ns 0.0294 0 0 2.2 KB
master EnrichedLog net472 4.98μs 9.62ns 37.2ns 0.32 0 0 2.02 KB
#5836 EnrichedLog net6.0 3.21μs 1.27ns 4.77ns 0.0307 0 0 2.2 KB
#5836 EnrichedLog netcoreapp3.1 4.18μs 1.45ns 5.63ns 0.0293 0 0 2.2 KB
#5836 EnrichedLog net472 4.75μs 2.65ns 10.3ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.43μs 1.26ns 4.89ns 0.0158 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.67μs 1.04ns 3.76ns 0.0158 0 0 1.14 KB
master SendReceive net472 2.05μs 1.09ns 4.08ns 0.183 0.00102 0 1.16 KB
#5836 SendReceive net6.0 1.49μs 0.861ns 3.22ns 0.0156 0 0 1.14 KB
#5836 SendReceive netcoreapp3.1 1.71μs 0.805ns 3.12ns 0.0146 0 0 1.14 KB
#5836 SendReceive net472 2.14μs 1.2ns 4.65ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.8μs 1.14ns 4.42ns 0.0224 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.02μs 2.32ns 8.67ns 0.022 0 0 1.65 KB
master EnrichedLog net472 4.45μs 1.3ns 4.67ns 0.322 0 0 2.04 KB
#5836 EnrichedLog net6.0 2.75μs 0.932ns 3.61ns 0.0219 0 0 1.6 KB
#5836 EnrichedLog netcoreapp3.1 3.76μs 1.47ns 5.69ns 0.0207 0 0 1.65 KB
#5836 EnrichedLog net472 4.38μs 2.38ns 9.21ns 0.322 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5836

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.170 512.79 599.93

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 431ns 0.714ns 2.67ns 0.00804 0 0 576 B
master StartFinishSpan netcoreapp3.1 577ns 0.864ns 3.35ns 0.00785 0 0 576 B
master StartFinishSpan net472 658ns 0.773ns 2.79ns 0.0915 0 0 578 B
master StartFinishScope net6.0 513ns 1.18ns 4.43ns 0.00983 0 0 696 B
master StartFinishScope netcoreapp3.1 710ns 0.792ns 2.96ns 0.00938 0 0 696 B
master StartFinishScope net472 943ns 1.03ns 3.98ns 0.104 0 0 658 B
#5836 StartFinishSpan net6.0 398ns 0.0979ns 0.379ns 0.00801 0 0 576 B
#5836 StartFinishSpan netcoreapp3.1 566ns 0.138ns 0.536ns 0.00773 0 0 576 B
#5836 StartFinishSpan net472 672ns 0.616ns 2.39ns 0.0916 0 0 578 B
#5836 StartFinishScope net6.0 600ns 0.164ns 0.614ns 0.00964 0 0 696 B
#5836 StartFinishScope netcoreapp3.1 707ns 0.273ns 1.06ns 0.00959 0 0 696 B
#5836 StartFinishScope net472 838ns 3.9ns 15.6ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 689ns 0.775ns 3ns 0.00962 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 885ns 1.11ns 4.28ns 0.00956 0 0 696 B
master RunOnMethodBegin net472 1.13μs 0.368ns 1.43ns 0.104 0 0 658 B
#5836 RunOnMethodBegin net6.0 658ns 0.415ns 1.55ns 0.00984 0 0 696 B
#5836 RunOnMethodBegin netcoreapp3.1 955ns 1.12ns 4.35ns 0.0091 0 0 696 B
#5836 RunOnMethodBegin net472 1.14μs 1.15ns 4.29ns 0.105 0 0 658 B

Copy link
Contributor

@GreenMatan GreenMatan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kevingosse kevingosse requested a review from a team as a code owner August 1, 2024 13:02
@kevingosse kevingosse merged commit 035dfb1 into master Aug 2, 2024
70 of 74 checks passed
@kevingosse kevingosse deleted the kevin/bubble_metadata branch August 2, 2024 14:06
@github-actions github-actions bot added this to the vNext-v3 milestone Aug 2, 2024
kevingosse added a commit that referenced this pull request Aug 2, 2024
## Summary of changes

Fix error checking when emitting the CallTargetBubbleUpException code.
As a bonus, add synchronization when creating the metadata tokens.

## Reason for change

If for some reason we fail to create the metadata tokens, we could
create an invalid EH clause, which causes an access violation later on.

## Implementation details

Converted the calltarget_tokens mutex to a recursive mutex, because the
`EnsureBaseCalltargetTokens` methods call each other.

## Test coverage

Hard to test, but coincidentally I discovered that the the added
synchronization fixes a crash when running powershell with SentinelOne
activated.

## Other details

Fixes #5799
kevingosse added a commit that referenced this pull request Aug 8, 2024
## Summary of changes

Fix error checking when emitting the CallTargetBubbleUpException code.
As a bonus, add synchronization when creating the metadata tokens.

## Reason for change

If for some reason we fail to create the metadata tokens, we could
create an invalid EH clause, which causes an access violation later on.

## Implementation details

Converted the calltarget_tokens mutex to a recursive mutex, because the
`EnsureBaseCalltargetTokens` methods call each other.

## Test coverage

Hard to test, but coincidentally I discovered that the the added
synchronization fixes a crash when running powershell with SentinelOne
activated.

## Other details

Fixes #5799
Backport of #5836 to v2.
@andrewlock andrewlock added type:bug area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) labels Aug 14, 2024
@andrewlock
Copy link
Member

dotnet/runtime#105710

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crash in ILRewriter for a .NET 6 app in w3wp
3 participants