Revert recent native library changes #5767

andrewlock · 2024-07-03T08:51:01Z

Summary of changes

Reverts recent changes to the native side

Reason for change

We've seen crashes on the native side in smoke tests etc that we haven't got to the bottom of yet. Out of caution, choosing to revert these changes to investigate further

Implementation details

Reverted the following PRs:

Test coverage

This is the main test, will run a full installer test suite on this PR to confirm we don't see the issues immediately. Will require further monitoring to look for other issues

Other details

This reverts commit 421a272.

This reverts commit 4203f40.

…ame method (#5533)" This reverts commit 61603e1.

github-actions · 2024-07-03T08:51:49Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

2 occurrences of :

-  {
-    TraceId: Id_1,
-    SpanId: Id_2,
-    Name: aspnet_core.request,
-    Resource: GET /iast/customattribute,
-    Service: Samples.Security.AspNetCore5,
-    Type: web,
-    Tags: {
-      aspnet_core.endpoint: Samples.Security.AspNetCore5.Controllers.IastController.CustomAttribute (Samples.Security.AspNetCore5),
-      aspnet_core.route: iast/customattribute,
-      component: aspnet_core,
-      env: integration_tests,
-      http.method: GET,
-      http.request.headers.host: localhost:00000,
-      http.route: iast/customattribute,
-      http.status_code: 200,
-      http.url: http://localhost:00000/Iast/CustomAttribute?userName=Vicent,
-      http.useragent: Mistake Not...,
-      language: dotnet,
-      runtime-id: Guid_1,
-      span.kind: server,
-      _dd.iast.enabled: 1,
-      _dd.iast.json:
[...]
-  "vulnerabilities": [
-    {
-      "type": "PATH_TRAVERSAL",
-      "hash": -2017497883,
-      "location": {
-        "spanId": XXX,
-        "path": "Samples.Security.AspNetCore5.Controllers.IastController",
-        "method": "GetCustomString"
-      },
-      "evidence": {
-        "valueParts": [
-          {
-            "value": "Vicent",
-            "source": 0
-          }
-        ]
-      }
-    }
-  ],
-  "sources": [
-    {
-      "origin": "http.request.parameter",
-      "name": "userName",
-      "value": "Vicent"
-    }
-  ]
[...]
-    },
-    Metrics: {
-      process_id: 0,
-      _dd.top_level: 1.0,
-      _dd.tracer_kr: 1.0,
-      _sampling_priority_v1: 1.0
-    }
-  },
-  {
-    TraceId: Id_1,
-    SpanId: Id_3,
-    Name: aspnet_core_mvc.request,
-    Resource: GET /iast/customattribute,
-    Service: Samples.Security.AspNetCore5,
-    Type: web,
-    ParentId: Id_2,
-    Tags: {
-      aspnet_core.action: customattribute,
-      aspnet_core.controller: iast,
-      aspnet_core.route: iast/customattribute,
-      component: aspnet_core,
-      env: integration_tests,
-      language: dotnet,
-      span.kind: server
-    }
-  },
-  {
-    TraceId: Id_1,
-    SpanId: Id_4,
-    Name: span.custom.attribute,
-    Resource: IastController.GetCustomString,
-    Service: Samples.Security.AspNetCore5,
-    ParentId: Id_3,
-    Tags: {
-      component: trace,
-      env: integration_tests,
-      language: dotnet
-    }
-  }

daniel-romano-DD

:_

datadog-ddstaging · 2024-07-03T09:19:47Z

Datadog Report

Branch report: andrew/revert-native-changes
Commit report: 86c0a8e
Test service: dd-trace-dotnet

✅ 0 Failed, 304018 Passed, 2179 Skipped, 24h 2m 50.41s Total Time
⌛ 1 Performance Regression

⌛ Performance Regressions vs Default Branch (1)

ObjectExtractorMoreComplexBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark 4.58µs (+603ns, +15%) - Details

andrewlock · 2024-07-03T09:20:58Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (73ms)  : 65, 81
     .   : milestone, 73,
    master - mean (75ms)  : 62, 88
     .   : milestone, 75,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (1,017ms)  : crit, 988, 1047
     .   : crit, milestone, 1017,
    master - mean (918ms)  : 897, 938
     .   : milestone, 918,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (110ms)  : 107, 114
     .   : milestone, 110,
    master - mean (110ms)  : 106, 113
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (703ms)  : crit, 682, 723
     .   : crit, milestone, 703,
    master - mean (635ms)  : 618, 653
     .   : milestone, 635,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (93ms)  : 90, 96
     .   : milestone, 93,
    master - mean (93ms)  : 90, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (658ms)  : crit, 639, 678
     .   : crit, milestone, 658,
    master - mean (598ms)  : 583, 612
     .   : milestone, 598,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (191ms)  : 187, 194
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (1,104ms)  : crit, 1077, 1132
     .   : crit, milestone, 1104,
    master - mean (1,011ms)  : 987, 1036
     .   : milestone, 1011,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (275ms)  : 271, 279
     .   : milestone, 275,
    master - mean (275ms)  : 272, 279
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (872ms)  : crit, 851, 893
     .   : crit, milestone, 872,
    master - mean (823ms)  : 793, 853
     .   : milestone, 823,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5767) - mean (264ms)  : 260, 268
     .   : milestone, 264,
    master - mean (267ms)  : 263, 271
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (5767) - mean (859ms)  : crit, 835, 884
     .   : crit, milestone, 859,
    master - mean (803ms)  : 770, 836
     .   : milestone, 803,

andrewlock · 2024-07-03T09:44:54Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5767) (11.914M)   : 0, 11914341
    master (12.102M)   : 0, 12102247
    benchmarks/2.9.0 (11.967M)   : 0, 11966853

    section Automatic
    This PR (5767) (8.068M)   : 0, 8068287
    master (8.089M)   : 0, 8088693
    benchmarks/2.9.0 (8.405M)   : 0, 8404892

    section Trace stats
    master (8.443M)   : 0, 8442998

    section Manual
    This PR (5767) (10.182M)   : 0, 10182034
    master (10.275M)   : 0, 10274868

    section Manual + Automatic
    This PR (5767) (7.521M)   : 0, 7521335
    master (7.533M)   : 0, 7533118

    section Version Conflict
    master (6.842M)   : 0, 6842477

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5767) (9.778M)   : 0, 9777776
    master (9.603M)   : 0, 9602520
    benchmarks/2.9.0 (9.255M)   : 0, 9255366

    section Automatic
    This PR (5767) (6.679M)   : 0, 6679275
    master (6.457M)   : 0, 6457136

    section Trace stats
    master (6.985M)   : 0, 6985383

    section Manual
    This PR (5767) (8.563M)   : 0, 8563172
    master (8.151M)   : 0, 8151308

    section Manual + Automatic
    This PR (5767) (6.099M)   : 0, 6099340
    master (6.255M)   : 0, 6255035

    section Version Conflict
    master (5.671M)   : 0, 5670523

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5767) (10.117M)   : 0, 10116751
    master (10.319M)   : 0, 10319080
    benchmarks/2.9.0 (10.302M)   : 0, 10302396

    section Automatic
    This PR (5767) (7.165M)   : 0, 7164807
    master (7.083M)   : 0, 7083219
    benchmarks/2.9.0 (7.635M)   : 0, 7634994

    section Trace stats
    master (7.566M)   : 0, 7566202

    section Manual
    This PR (5767) (8.827M)   : 0, 8827434
    master (8.698M)   : 0, 8698395

    section Manual + Automatic
    This PR (5767) (6.930M)   : 0, 6930037
    master (6.776M)   : 0, 6776250

    section Version Conflict
    master (6.140M)   : 0, 6139972

andrewlock · 2024-07-03T09:59:34Z

Benchmarks Report for appsec 🐌

Benchmarks for #5767 compared to master:

1 benchmarks are slower, with geometric mean 1.159
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5767

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑netcoreapp3.1	1.159	3,946.26	4,575.46

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	71.5μs	43.3ns	168ns	0.0712	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	61μs	84.2ns	326ns	0.0909	0	6.95 KB
master	`AllCycleSimpleBody`	net472	46.9μs	37.4ns	140ns	1.31	0	8.34 KB
master	`AllCycleMoreComplexBody`	net6.0	77μs	63.7ns	238ns	0.115	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	67.5μs	83.9ns	325ns	0.135	0	10.37 KB
master	`AllCycleMoreComplexBody`	net472	53.7μs	39.9ns	154ns	1.88	0.0268	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	146ns	0.116ns	0.435ns	0.00397	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	213ns	0.23ns	0.862ns	0.00365	0	272 B
master	`ObjectExtractorSimpleBody`	net472	169ns	0.128ns	0.496ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.04μs	1.87ns	7.01ns	0.0534	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.95μs	2.32ns	8.67ns	0.0495	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.75μs	2.34ns	8.75ns	0.602	0.00562	3.8 KB
#5767	`AllCycleSimpleBody`	net6.0	72.4μs	71.5ns	277ns	0.073	0	6 KB
#5767	`AllCycleSimpleBody`	netcoreapp3.1	62.5μs	75.2ns	281ns	0.0922	0	6.95 KB
#5767	`AllCycleSimpleBody`	net472	47.1μs	48.5ns	175ns	1.31	0	8.34 KB
#5767	`AllCycleMoreComplexBody`	net6.0	77.6μs	86.5ns	324ns	0.116	0	9.51 KB
#5767	`AllCycleMoreComplexBody`	netcoreapp3.1	69.1μs	88.1ns	341ns	0.139	0	10.37 KB
#5767	`AllCycleMoreComplexBody`	net472	55.2μs	37ns	143ns	1.88	0.0276	11.85 KB
#5767	`ObjectExtractorSimpleBody`	net6.0	145ns	0.154ns	0.597ns	0.00398	0	280 B
#5767	`ObjectExtractorSimpleBody`	netcoreapp3.1	201ns	0.207ns	0.802ns	0.00376	0	272 B
#5767	`ObjectExtractorSimpleBody`	net472	170ns	0.171ns	0.663ns	0.0446	0	281 B
#5767	`ObjectExtractorMoreComplexBody`	net6.0	3.02μs	0.78ns	2.81ns	0.0529	0	3.78 KB
#5767	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.58μs	2.39ns	8.94ns	0.0505	0	3.69 KB
#5767	`ObjectExtractorMoreComplexBody`	net472	3.78μs	1.28ns	4.94ns	0.602	0.00566	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	38.2μs	16.8ns	62.9ns	0.457	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	53.9μs	22.5ns	87.1ns	0.431	0	32.4 KB
master	`EncodeArgs`	net472	66.9μs	40.9ns	158ns	5.13	0.0667	32.5 KB
master	`EncodeLegacyArgs`	net6.0	78.7μs	83.5ns	312ns	0.0397	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	106μs	100ns	376ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	157μs	111ns	429ns	0.313	0	2.15 KB
#5767	`EncodeArgs`	net6.0	37.9μs	15.9ns	59.4ns	0.456	0	32.4 KB
#5767	`EncodeArgs`	netcoreapp3.1	53.9μs	28.3ns	106ns	0.43	0	32.4 KB
#5767	`EncodeArgs`	net472	67.3μs	42.3ns	164ns	5.14	0.0671	32.5 KB
#5767	`EncodeLegacyArgs`	net6.0	77.1μs	402ns	1.8μs	0	0	2.14 KB
#5767	`EncodeLegacyArgs`	netcoreapp3.1	106μs	271ns	1.05μs	0	0	2.14 KB
#5767	`EncodeLegacyArgs`	net472	157μs	59.8ns	216ns	0.312	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	182μs	24.5ns	94.9ns	0	2.42 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	194μs	104ns	387ns	0	2.37 KB
master	`RunWafRealisticBenchmark`	net472	206μs	66.6ns	249ns	0.309	2.43 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	124ns	466ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	128μs	34.1ns	128ns	0	1.45 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	138μs	128ns	477ns	0.207	1.48 KB
#5767	`RunWafRealisticBenchmark`	net6.0	181μs	66.9ns	259ns	0	2.42 KB
#5767	`RunWafRealisticBenchmark`	netcoreapp3.1	192μs	96ns	359ns	0	2.37 KB
#5767	`RunWafRealisticBenchmark`	net472	206μs	55.1ns	206ns	0.309	2.43 KB
#5767	`RunWafRealisticBenchmarkWithAttack`	net6.0	121μs	104ns	404ns	0	1.46 KB
#5767	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	130μs	137ns	529ns	0	1.45 KB
#5767	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	33.8ns	127ns	0.208	1.48 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5767

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	264.38 KB	270.09 KB	5.7 KB	2.16%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	56.7μs	580ns	5.72μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	59.4μs	670ns	6.64μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.9μs	89.3ns	322ns	59.26 KB
master	`StringConcatAspectBenchmark`	net6.0	297μs	1.22μs	6.8μs	264.38 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	331μs	1.85μs	11.2μs	252.37 KB
master	`StringConcatAspectBenchmark`	net472	309μs	6.27μs	61.8μs	278.53 KB
#5767	`StringConcatBenchmark`	net6.0	59.9μs	638ns	6.28μs	43.44 KB
#5767	`StringConcatBenchmark`	netcoreapp3.1	60.9μs	837ns	8.28μs	42.64 KB
#5767	`StringConcatBenchmark`	net472	38μs	110ns	413ns	59.54 KB
#5767	`StringConcatAspectBenchmark`	net6.0	306μs	5.3μs	50.8μs	270.09 KB
#5767	`StringConcatAspectBenchmark`	netcoreapp3.1	302μs	4.81μs	47.9μs	252.54 KB
#5767	`StringConcatAspectBenchmark`	net472	302μs	6.42μs	62.5μs	278.53 KB

andrewlock · 2024-07-03T10:53:54Z

Benchmarks Report for tracer 🐌

Benchmarks for #5767 compared to master:

1 benchmarks are slower, with geometric mean 1.149
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.95μs	43ns	292ns	0.011	0.00367	0	5.42 KB
master	`StartStopWithChild`	netcoreapp3.1	9.62μs	51.2ns	290ns	0.0237	0.00949	0	5.61 KB
master	`StartStopWithChild`	net472	16.2μs	49.9ns	193ns	1.03	0.304	0.0881	6.07 KB
#5767	`StartStopWithChild`	net6.0	7.91μs	45.4ns	346ns	0.00749	0.00374	0	5.43 KB
#5767	`StartStopWithChild`	netcoreapp3.1	10.3μs	58.2ns	399ns	0.0209	0.00522	0	5.62 KB
#5767	`StartStopWithChild`	net472	16.4μs	71.9ns	278ns	1.03	0.32	0.0903	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	502μs	301ns	1.17μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	640μs	102ns	382ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	814μs	245ns	918ns	0.406	3.3 KB
#5767	`WriteAndFlushEnrichedTraces`	net6.0	471μs	257ns	996ns	0	2.7 KB
#5767	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	622μs	292ns	1.13μs	0	2.7 KB
#5767	`WriteAndFlushEnrichedTraces`	net472	829μs	391ns	1.51μs	0.411	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	173μs	234ns	906ns	0.258	18.46 KB
master	`SendRequest`	netcoreapp3.1	193μs	271ns	1.02μs	0.193	20.61 KB
master	`SendRequest`	net472	8.77E‑05ns	4.23E‑05ns	0.000164ns	0	0 b
#5767	`SendRequest`	net6.0	172μs	223ns	863ns	0.171	18.45 KB
#5767	`SendRequest`	netcoreapp3.1	192μs	226ns	874ns	0.192	20.61 KB
#5767	`SendRequest`	net472	0.000434ns	0.000164ns	0.000634ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5767

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.84 KB	41.6 KB	-241 B	-0.58%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	543μs	1.58μs	6.13μs	0.543	0	0	41.84 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	657μs	1.2μs	4.67μs	0.329	0	0	41.72 KB
master	`WriteAndFlushEnrichedTraces`	net472	827μs	3.61μs	14μs	8.22	2.47	0.411	53.21 KB
#5767	`WriteAndFlushEnrichedTraces`	net6.0	535μs	724ns	2.8μs	0.539	0	0	41.6 KB
#5767	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	639μs	1.36μs	5.25μs	0.322	0	0	41.65 KB
#5767	`WriteAndFlushEnrichedTraces`	net472	867μs	2.47μs	9.57μs	8.36	2.64	0.44	53.27 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.27μs	0.71ns	2.75ns	0.0142	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.74μs	0.559ns	2.02ns	0.0131	1.02 KB
master	`ExecuteNonQuery`	net472	1.98μs	1.42ns	5.13ns	0.156	987 B
#5767	`ExecuteNonQuery`	net6.0	1.29μs	0.905ns	3.51ns	0.0142	1.02 KB
#5767	`ExecuteNonQuery`	netcoreapp3.1	1.66μs	1.18ns	4.56ns	0.0131	1.02 KB
#5767	`ExecuteNonQuery`	net472	1.95μs	0.473ns	1.83ns	0.157	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.14μs	0.702ns	2.72ns	0.0137	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.61μs	0.473ns	1.83ns	0.0136	976 B
master	`CallElasticsearch`	net472	2.7μs	2.88ns	11.2ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.27μs	0.566ns	2.19ns	0.0134	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	1.39ns	5.39ns	0.0142	1.02 KB
master	`CallElasticsearchAsync`	net472	2.62μs	2.23ns	8.65ns	0.167	1.05 KB
#5767	`CallElasticsearch`	net6.0	1.24μs	0.567ns	2.12ns	0.0137	976 B
#5767	`CallElasticsearch`	netcoreapp3.1	1.5μs	0.475ns	1.71ns	0.0135	976 B
#5767	`CallElasticsearch`	net472	2.61μs	2.07ns	8.01ns	0.158	995 B
#5767	`CallElasticsearchAsync`	net6.0	1.19μs	0.406ns	1.52ns	0.0132	952 B
#5767	`CallElasticsearchAsync`	netcoreapp3.1	1.6μs	0.446ns	1.61ns	0.0137	1.02 KB
#5767	`CallElasticsearchAsync`	net472	2.56μs	1.09ns	3.93ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5767

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.149	1,175.99	1,351.23

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.18μs	0.506ns	1.96ns	0.0135	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.62μs	0.934ns	3.62ns	0.013	952 B
master	`ExecuteAsync`	net472	1.85μs	3.43ns	12.8ns	0.145	915 B
#5767	`ExecuteAsync`	net6.0	1.35μs	1.11ns	4.17ns	0.0128	952 B
#5767	`ExecuteAsync`	netcoreapp3.1	1.65μs	9.09ns	81.3ns	0.0128	952 B
#5767	`ExecuteAsync`	net472	1.8μs	0.643ns	2.41ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.2μs	2.14ns	8.3ns	0.0294	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.06μs	3.09ns	11.1ns	0.0378	2.76 KB
master	`SendAsync`	net472	7.65μs	2.08ns	8.07ns	0.5	3.15 KB
#5767	`SendAsync`	net6.0	4.17μs	11.7ns	45.3ns	0.0318	2.22 KB
#5767	`SendAsync`	netcoreapp3.1	5.16μs	1.74ns	6.74ns	0.0364	2.76 KB
#5767	`SendAsync`	net472	7.54μs	3.17ns	12.3ns	0.496	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.53μs	0.458ns	1.72ns	0.023	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.17μs	0.939ns	3.51ns	0.0214	1.64 KB
master	`EnrichedLog`	net472	2.44μs	1.86ns	7.2ns	0.25	1.57 KB
#5767	`EnrichedLog`	net6.0	1.46μs	0.38ns	1.42ns	0.0228	1.64 KB
#5767	`EnrichedLog`	netcoreapp3.1	2.14μs	0.774ns	3ns	0.0225	1.64 KB
#5767	`EnrichedLog`	net472	2.46μs	1ns	3.62ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	114μs	130ns	505ns	0.0565	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	118μs	151ns	584ns	0.0591	0	4.28 KB
master	`EnrichedLog`	net472	147μs	169ns	653ns	0.668	0.223	4.46 KB
#5767	`EnrichedLog`	net6.0	117μs	474ns	1.71μs	0.0609	0	4.28 KB
#5767	`EnrichedLog`	netcoreapp3.1	118μs	158ns	614ns	0.0583	0	4.28 KB
#5767	`EnrichedLog`	net472	147μs	106ns	367ns	0.663	0.221	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.08μs	1.2ns	4.48ns	0.0309	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.17μs	1.36ns	5.1ns	0.0291	2.2 KB
master	`EnrichedLog`	net472	4.86μs	2.51ns	9.74ns	0.32	2.02 KB
#5767	`EnrichedLog`	net6.0	3.02μs	2.1ns	8.12ns	0.0305	2.2 KB
#5767	`EnrichedLog`	netcoreapp3.1	4.38μs	1.06ns	4.12ns	0.0284	2.2 KB
#5767	`EnrichedLog`	net472	4.81μs	2.29ns	8.88ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.32μs	0.445ns	1.66ns	0.016	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.71μs	0.603ns	2.34ns	0.0155	1.14 KB
master	`SendReceive`	net472	2.07μs	0.775ns	3ns	0.183	1.16 KB
#5767	`SendReceive`	net6.0	1.34μs	0.796ns	2.98ns	0.016	1.14 KB
#5767	`SendReceive`	netcoreapp3.1	1.77μs	0.777ns	2.91ns	0.0151	1.14 KB
#5767	`SendReceive`	net472	2.06μs	1.41ns	5.27ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.88μs	0.788ns	2.95ns	0.0217	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.72μs	1.51ns	5.83ns	0.0206	1.65 KB
master	`EnrichedLog`	net472	4.43μs	1.58ns	6.13ns	0.323	2.04 KB
#5767	`EnrichedLog`	net6.0	2.78μs	0.743ns	2.78ns	0.0223	1.6 KB
#5767	`EnrichedLog`	netcoreapp3.1	3.78μs	1.64ns	6.34ns	0.0207	1.65 KB
#5767	`EnrichedLog`	net472	4.4μs	2.23ns	8.64ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	408ns	0.147ns	0.57ns	0.00815	576 B
master	`StartFinishSpan`	netcoreapp3.1	648ns	0.302ns	1.17ns	0.00779	576 B
master	`StartFinishSpan`	net472	721ns	0.309ns	1.2ns	0.0918	578 B
master	`StartFinishScope`	net6.0	479ns	0.174ns	0.675ns	0.00986	696 B
master	`StartFinishScope`	netcoreapp3.1	770ns	0.252ns	0.977ns	0.00966	696 B
master	`StartFinishScope`	net472	836ns	0.424ns	1.64ns	0.104	658 B
#5767	`StartFinishSpan`	net6.0	398ns	0.162ns	0.628ns	0.00799	576 B
#5767	`StartFinishSpan`	netcoreapp3.1	608ns	0.32ns	1.24ns	0.00772	576 B
#5767	`StartFinishSpan`	net472	675ns	0.93ns	3.6ns	0.0915	578 B
#5767	`StartFinishScope`	net6.0	531ns	0.188ns	0.727ns	0.00977	696 B
#5767	`StartFinishScope`	netcoreapp3.1	716ns	0.54ns	2.09ns	0.0093	696 B
#5767	`StartFinishScope`	net472	850ns	0.275ns	1.06ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	663ns	0.421ns	1.63ns	0.00966	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	874ns	0.438ns	1.7ns	0.00978	696 B
master	`RunOnMethodBegin`	net472	1.08μs	0.393ns	1.42ns	0.104	658 B
#5767	`RunOnMethodBegin`	net6.0	698ns	0.196ns	0.759ns	0.00972	696 B
#5767	`RunOnMethodBegin`	netcoreapp3.1	927ns	0.358ns	1.39ns	0.00935	696 B
#5767	`RunOnMethodBegin`	net472	1.05μs	0.43ns	1.66ns	0.104	658 B

…5768) ## Summary of changes Reverts recent change that moved definitions to the native side ## Reason for change We've seen crashes on the native side in smoke tests etc that we haven't got to the bottom of yet. Out of caution, choosing to revert these changes to investigate further ## Implementation details Reverted the following PRs: - #5757 (introduced to fix a bug in 5592) - #5592 ## Test coverage This is the main test, will run a full installer test suite on this PR to confirm we don't see the issues immediately. Will require further monitoring to look for other issues ## Other details A less drastic version of - #5767 as we expect the issue is in the moved definitions.

andrewlock · 2024-07-03T14:37:26Z

Closing in favor of #5768

andrewlock added 3 commits July 3, 2024 09:43

Revert "Lock access to rejitters (#5757)"

f0fbba1

This reverts commit 421a272.

Revert "[Tracer] Move definitions to native side (#5592)"

578b1d3

This reverts commit 4203f40.

Revert "[Tracer] Rejit refactor to support multiple products in the s…

86c0a8e

…ame method (#5533)" This reverts commit 61603e1.

andrewlock added the area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) label Jul 3, 2024

andrewlock requested review from a team as code owners July 3, 2024 08:51

andrewlock changed the title ~~Revert recent native changes~~ Revert recent native library changes Jul 3, 2024

daniel-romano-DD approved these changes Jul 3, 2024

View reviewed changes

NachoEchevarria approved these changes Jul 3, 2024

View reviewed changes

lucaspimentel approved these changes Jul 3, 2024

View reviewed changes

andrewlock mentioned this pull request Jul 3, 2024

Revert native changes that moved the definitions to the native side #5768

Merged

andrewlock closed this Jul 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revert recent native library changes #5767

Revert recent native library changes #5767

andrewlock commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

daniel-romano-DD left a comment

datadog-ddstaging bot commented Jul 3, 2024 •

edited

Loading

andrewlock commented Jul 3, 2024

andrewlock commented Jul 3, 2024

andrewlock commented Jul 3, 2024

Slower ⚠️ in #5767

Raw results

Raw results

Raw results

More allocations ⚠️ in #5767

Raw results

andrewlock commented Jul 3, 2024

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5767

Raw results

Raw results

Raw results

Slower ⚠️ in #5767

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

andrewlock commented Jul 3, 2024

Revert recent native library changes #5767

Revert recent native library changes #5767

Conversation

andrewlock commented Jul 3, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

github-actions bot commented Jul 3, 2024

Snapshots difference summary

daniel-romano-DD left a comment

Choose a reason for hiding this comment

datadog-ddstaging bot commented Jul 3, 2024 • edited Loading

Datadog Report

⌛ Performance Regressions vs Default Branch (1)

andrewlock commented Jul 3, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jul 3, 2024

Throughput/Crank Report:zap:

andrewlock commented Jul 3, 2024

Benchmarks Report for appsec 🐌

Benchmark details

Slower ⚠️ in #5767

Raw results

Raw results

Raw results

More allocations ⚠️ in #5767

Raw results

andrewlock commented Jul 3, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5767

Raw results

Raw results

Raw results

Slower ⚠️ in #5767

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

andrewlock commented Jul 3, 2024

datadog-ddstaging bot commented Jul 3, 2024 •

edited

Loading