Config refactor - Add telemetry to otel config (#5717 -> v2) #5864

andrewlock · 2024-08-08T09:51:53Z

Summary of changes

Records logs and metrics for when otel config is overriden by datadog config, or when it's invalid

Reason for change

This was originally part of #5661, but wasn't possible due to concerns with the fact ConfigurationBuilder is used in critical "startup" paths for the tracer, so can result in recursion if we're not careful.

Implementation details

The crux of the implementation is that we "store" error and metrics for writing at the point we know it's safe. This is similar to what we already do for configuration telemetry.

(Technically I think we Could directly access the TelemetryFactory.Metrics, but this is "safer", and makes it easier to test we're sending the right metrics

Test coverage

Added unit tests to all of the existing settings tests where we override configuration to confirm that it works as expected

Other details

The metrics etc are all ported from #5661, but there are currently some issues with the proposed values:

The metrics aren't added to the intake yet, so need updating there (and copying here)
The tag names aren't "valid" for standard telemetry metrics, they should be lowercase, and shouldn't use .. I would suggest changing these to be lowercase and replacing . with _, but either way this should happen after they've been OK'd in the intake.

Original PR:

Config refactor - Add telemetry to otel config #5717

Records logs and metrics for when otel config is overriden by datadog config, or when it's invalid This was originally part of #5661, but wasn't possible due to concerns with the fact `ConfigurationBuilder` is used in critical "startup" paths for the tracer, so can result in recursion if we're not careful. The crux of the implementation is that we "store" error and metrics for writing at the point we know it's safe. This is _similar_ to what we already do for configuration telemetry. (Technically I think we _Could_ directly access the `TelemetryFactory.Metrics`, but this is "safer", and makes it easier to test we're sending the right metrics Added unit tests to all of the existing settings tests where we override configuration to confirm that it works as expected The metrics etc are all ported from #5661, but there are currently some issues with the proposed values: - [x] The metrics aren't added to the intake yet, so need updating there (and copying here) - [x] The tag names aren't "valid" for standard telemetry metrics, they should be lowercase, and shouldn't use `.`. I would suggest changing these to be lowercase and replacing `.` with `_`, but either way this should happen after they've been OK'd in the intake. Part of a big ole stack: - #5713 - #5714 - #5715 - #5716 - #5717 (this PR)

andrewlock · 2024-08-08T10:18:59Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (73ms)  : 60, 86
     .   : milestone, 73,
    master - mean (73ms)  : 63, 83
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (1,027ms)  : 1007, 1047
     .   : milestone, 1027,
    master - mean (1,069ms)  : 1045, 1092
     .   : milestone, 1069,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (110ms)  : 106, 113
     .   : milestone, 110,
    master - mean (109ms)  : 104, 114
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (718ms)  : 688, 747
     .   : milestone, 718,
    master - mean (750ms)  : 727, 773
     .   : milestone, 750,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (92ms)  : 89, 95
     .   : milestone, 92,
    master - mean (93ms)  : 89, 97
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (672ms)  : 649, 696
     .   : milestone, 672,
    master - mean (703ms)  : 683, 723
     .   : milestone, 703,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (192ms)  : 189, 196
     .   : milestone, 192,
    master - mean (192ms)  : 188, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (1,127ms)  : 1107, 1148
     .   : milestone, 1127,
    master - mean (1,166ms)  : 1139, 1193
     .   : milestone, 1166,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (278ms)  : 273, 283
     .   : milestone, 278,
    master - mean (277ms)  : 271, 283
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (885ms)  : 858, 912
     .   : milestone, 885,
    master - mean (918ms)  : 897, 938
     .   : milestone, 918,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5864) - mean (267ms)  : 262, 272
     .   : milestone, 267,
    master - mean (267ms)  : 263, 270
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (5864) - mean (873ms)  : 852, 894
     .   : milestone, 873,
    master - mean (904ms)  : 879, 929
     .   : milestone, 904,

datadog-ddstaging · 2024-08-08T10:57:08Z

Datadog Report

Branch report: andrew/config-refactor/5-record-otel-telemetry-v2
Commit report: 7fb3c6b
Test service: dd-trace-dotnet

✅ 0 Failed, 348246 Passed, 1774 Skipped, 15h 13m 54.1s Total Time
⌛ 15 Performance Regressions

⌛ Performance Regressions vs Default Branch (15)

This report shows up to 5 performance regressions.

Baseline_garbagecollections - scenarios 1.09s (+87.4ms, +9%) - Details
Profiler_cpu_walltime_old_stackwalk - scenarios 3.25s (+909.27ms, +39%) - Details
Profiler_cpu_walltime - scenarios 3.23s (+890.83ms, +38%) - Details
Profiler_walltime - scenarios 3.24s (+892.07ms, +38%) - Details
Profiler_cpu_walltime_timer_create - scenarios 3.23s (+892.09ms, +38%) - Details

andrewlock · 2024-08-08T11:08:01Z

Benchmarks Report for appsec 🐌

Benchmarks for #5864 compared to master:

1 benchmarks are faster, with geometric mean 1.173
1 benchmarks have fewer allocations
2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	74.4μs	94.4ns	366ns	0.0727	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	63.2μs	71.9ns	278ns	0.095	0	6.95 KB
master	`AllCycleSimpleBody`	net472	47.9μs	81.4ns	315ns	1.31	0	8.34 KB
master	`AllCycleMoreComplexBody`	net6.0	77.6μs	70.5ns	264ns	0.117	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	69.5μs	98.2ns	380ns	0.139	0	10.36 KB
master	`AllCycleMoreComplexBody`	net472	55.3μs	70.8ns	274ns	1.86	0.0277	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	139ns	0.161ns	0.623ns	0.00394	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	246ns	1.06ns	4.1ns	0.00376	0	272 B
master	`ObjectExtractorSimpleBody`	net472	163ns	0.108ns	0.405ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.11μs	1.59ns	6.15ns	0.0529	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.96μs	2.24ns	8.37ns	0.0516	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.88μs	4.57ns	17.7ns	0.601	0.00584	3.8 KB
#5864	`AllCycleSimpleBody`	net6.0	73.4μs	73.7ns	255ns	0.073	0	6.01 KB
#5864	`AllCycleSimpleBody`	netcoreapp3.1	62.2μs	70.9ns	275ns	0.0929	0	6.95 KB
#5864	`AllCycleSimpleBody`	net472	47.8μs	38.9ns	135ns	1.32	0	8.34 KB
#5864	`AllCycleMoreComplexBody`	net6.0	78.5μs	74.4ns	278ns	0.118	0	9.51 KB
#5864	`AllCycleMoreComplexBody`	netcoreapp3.1	70.3μs	102ns	395ns	0.14	0	10.36 KB
#5864	`AllCycleMoreComplexBody`	net472	55μs	41.5ns	161ns	1.88	0.0277	11.85 KB
#5864	`ObjectExtractorSimpleBody`	net6.0	140ns	0.156ns	0.603ns	0.00395	0	280 B
#5864	`ObjectExtractorSimpleBody`	netcoreapp3.1	236ns	0.109ns	0.394ns	0.00369	0	272 B
#5864	`ObjectExtractorSimpleBody`	net472	167ns	0.217ns	0.841ns	0.0446	0	281 B
#5864	`ObjectExtractorMoreComplexBody`	net6.0	3.06μs	0.942ns	3.52ns	0.0523	0	3.78 KB
#5864	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.95μs	2.59ns	9.68ns	0.0512	0	3.69 KB
#5864	`ObjectExtractorMoreComplexBody`	net472	3.85μs	2.21ns	8.54ns	0.602	0.00581	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	36.5μs	23.9ns	92.7ns	0.456	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.3μs	70.4ns	273ns	0.432	0	32.4 KB
master	`EncodeArgs`	net472	65.8μs	37.1ns	139ns	5.16	0.0661	32.5 KB
master	`EncodeLegacyArgs`	net6.0	70.9μs	15.1ns	58.6ns	0.0353	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	104μs	92.2ns	319ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	151μs	118ns	458ns	0.303	0	2.15 KB
#5864	`EncodeArgs`	net6.0	40.1μs	12.7ns	49.2ns	0.461	0	32.4 KB
#5864	`EncodeArgs`	netcoreapp3.1	54.5μs	28ns	108ns	0.433	0	32.4 KB
#5864	`EncodeArgs`	net472	65.3μs	37.3ns	139ns	5.14	0.065	32.5 KB
#5864	`EncodeLegacyArgs`	net6.0	73.9μs	126ns	487ns	0	0	2.14 KB
#5864	`EncodeLegacyArgs`	netcoreapp3.1	109μs	347ns	1.34μs	0	0	2.14 KB
#5864	`EncodeLegacyArgs`	net472	153μs	104ns	404ns	0.306	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	183μs	89.8ns	348ns	0	2.42 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	199μs	251ns	974ns	0	2.37 KB
master	`RunWafRealisticBenchmark`	net472	208μs	103ns	397ns	0.312	2.43 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	117ns	455ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	132μs	274ns	1.06μs	0	1.45 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	19.9ns	71.7ns	0.208	1.48 KB
#5864	`RunWafRealisticBenchmark`	net6.0	181μs	112ns	434ns	0	2.42 KB
#5864	`RunWafRealisticBenchmark`	netcoreapp3.1	194μs	117ns	453ns	0	2.37 KB
#5864	`RunWafRealisticBenchmark`	net472	206μs	136ns	525ns	0.307	2.43 KB
#5864	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	112ns	433ns	0	1.46 KB
#5864	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	131μs	78.8ns	305ns	0	1.45 KB
#5864	`RunWafRealisticBenchmarkWithAttack`	net472	140μs	48.9ns	190ns	0.208	1.48 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #5864

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	1.173	61,300.00	52,250.00	bimodal

More allocations ⚠️ in #5864

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	253.98 KB	262.44 KB	8.46 KB	3.33%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	59.01 KB	60.56 KB	1.55 KB	2.63%

Fewer allocations 🎉 in #5864

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	265.14 KB	255.39 KB	-9.74 KB	-3.68%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	60.6μs	653ns	6.53μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	62.1μs	871ns	8.53μs	42.64 KB
master	`StringConcatBenchmark`	net472	38.7μs	186ns	769ns	59.01 KB
master	`StringConcatAspectBenchmark`	net6.0	313μs	1.65μs	10.7μs	265.14 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	347μs	1.95μs	13.1μs	253.98 KB
master	`StringConcatAspectBenchmark`	net472	291μs	7.42μs	72.7μs	278.53 KB
#5864	`StringConcatBenchmark`	net6.0	52.3μs	206ns	825ns	43.44 KB
#5864	`StringConcatBenchmark`	netcoreapp3.1	60.5μs	831ns	8.1μs	42.64 KB
#5864	`StringConcatBenchmark`	net472	38μs	139ns	519ns	60.56 KB
#5864	`StringConcatAspectBenchmark`	net6.0	286μs	5.67μs	56.1μs	255.39 KB
#5864	`StringConcatAspectBenchmark`	netcoreapp3.1	341μs	1.61μs	6.01μs	262.44 KB
#5864	`StringConcatAspectBenchmark`	net472	295μs	7.86μs	76.6μs	278.53 KB

andrewlock · 2024-08-08T11:26:43Z

Benchmarks Report for tracer 🐌

Benchmarks for #5864 compared to master:

1 benchmarks are faster, with geometric mean 1.186
1 benchmarks are slower, with geometric mean 1.133
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.88μs	45.1ns	325ns	0.0157	0.00785	0	5.43 KB
master	`StartStopWithChild`	netcoreapp3.1	9.79μs	54.8ns	342ns	0.0238	0.00954	0	5.62 KB
master	`StartStopWithChild`	net472	16.2μs	71.5ns	311ns	1.03	0.324	0.0973	6.07 KB
#5864	`StartStopWithChild`	net6.0	7.68μs	43ns	295ns	0.0116	0.00771	0	5.43 KB
#5864	`StartStopWithChild`	netcoreapp3.1	9.93μs	54.9ns	364ns	0.0203	0.0102	0	5.61 KB
#5864	`StartStopWithChild`	net472	16μs	52.6ns	197ns	1.01	0.294	0.0952	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	474μs	229ns	859ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	644μs	466ns	1.8μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	829μs	581ns	2.25μs	0.414	3.3 KB
#5864	`WriteAndFlushEnrichedTraces`	net6.0	480μs	275ns	1.06μs	0	2.7 KB
#5864	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	664μs	1.71μs	6.62μs	0	2.7 KB
#5864	`WriteAndFlushEnrichedTraces`	net472	818μs	688ns	2.57μs	0.411	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	206μs	1.58μs	15.7μs	0.2	18.45 KB
master	`SendRequest`	netcoreapp3.1	221μs	1.29μs	11.7μs	0.217	20.61 KB
master	`SendRequest`	net472	0.00207ns	0.000804ns	0.00301ns	0	0 b
#5864	`SendRequest`	net6.0	197μs	1.11μs	8.46μs	0.192	18.45 KB
#5864	`SendRequest`	netcoreapp3.1	227μs	1.32μs	11.4μs	0.22	20.61 KB
#5864	`SendRequest`	net472	0.000565ns	0.000498ns	0.0018ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5864

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.67 KB	41.42 KB	-247 B	-0.59%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	565μs	2.68μs	11.1μs	0.571	0	0	41.67 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	690μs	3.69μs	20.2μs	0.327	0	0	41.79 KB
master	`WriteAndFlushEnrichedTraces`	net472	878μs	3.88μs	14.5μs	8.13	2.57	0.428	53.32 KB
#5864	`WriteAndFlushEnrichedTraces`	net6.0	560μs	2.76μs	12μs	0.561	0	0	41.42 KB
#5864	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	733μs	4.93μs	48.3μs	0.338	0	0	41.76 KB
#5864	`WriteAndFlushEnrichedTraces`	net472	879μs	2.71μs	10.1μs	8.08	2.55	0.425	53.3 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.2μs	1.17ns	4.54ns	0.0144	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.68μs	1.56ns	6.04ns	0.0136	1.02 KB
master	`ExecuteNonQuery`	net472	2.01μs	3.46ns	13.4ns	0.157	987 B
#5864	`ExecuteNonQuery`	net6.0	1.23μs	0.857ns	3.32ns	0.0142	1.02 KB
#5864	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	2.27ns	8.79ns	0.0139	1.02 KB
#5864	`ExecuteNonQuery`	net472	1.99μs	2.41ns	8.69ns	0.156	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5864

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0	1.133	1,197.78	1,357.35

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.2μs	0.56ns	2.02ns	0.0138	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.6μs	0.879ns	3.29ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.6μs	1.22ns	4.71ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.29μs	0.553ns	2.14ns	0.0129	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.69μs	0.763ns	2.86ns	0.0135	1.02 KB
master	`CallElasticsearchAsync`	net472	2.61μs	1.94ns	7.5ns	0.167	1.05 KB
#5864	`CallElasticsearch`	net6.0	1.36μs	1.21ns	4.7ns	0.0137	976 B
#5864	`CallElasticsearch`	netcoreapp3.1	1.53μs	0.643ns	2.4ns	0.013	976 B
#5864	`CallElasticsearch`	net472	2.53μs	0.803ns	2.9ns	0.158	995 B
#5864	`CallElasticsearchAsync`	net6.0	1.27μs	0.584ns	2.26ns	0.013	952 B
#5864	`CallElasticsearchAsync`	netcoreapp3.1	1.69μs	0.687ns	2.57ns	0.0135	1.02 KB
#5864	`CallElasticsearchAsync`	net472	2.59μs	1.51ns	5.85ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.25μs	1.24ns	4.65ns	0.0133	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.62μs	1.12ns	4.21ns	0.0124	952 B
master	`ExecuteAsync`	net472	1.65μs	1.22ns	4.72ns	0.145	915 B
#5864	`ExecuteAsync`	net6.0	1.23μs	0.983ns	3.68ns	0.0131	952 B
#5864	`ExecuteAsync`	netcoreapp3.1	1.55μs	0.853ns	3.3ns	0.0124	952 B
#5864	`ExecuteAsync`	net472	1.81μs	0.349ns	1.31ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.03μs	2.24ns	8.37ns	0.0321	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.07μs	2.28ns	8.84ns	0.0356	2.76 KB
master	`SendAsync`	net472	7.74μs	2.23ns	8.36ns	0.499	3.15 KB
#5864	`SendAsync`	net6.0	4.08μs	1.77ns	6.61ns	0.0305	2.22 KB
#5864	`SendAsync`	netcoreapp3.1	5.12μs	2.08ns	7.51ns	0.036	2.76 KB
#5864	`SendAsync`	net472	7.82μs	4.27ns	16.5ns	0.497	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.6μs	0.943ns	3.53ns	0.0232	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.21μs	1.28ns	4.78ns	0.0211	1.64 KB
master	`EnrichedLog`	net472	2.67μs	5.61ns	21.7ns	0.249	1.57 KB
#5864	`EnrichedLog`	net6.0	1.5μs	6.4ns	24.8ns	0.0226	1.64 KB
#5864	`EnrichedLog`	netcoreapp3.1	2.31μs	1.01ns	3.92ns	0.022	1.64 KB
#5864	`EnrichedLog`	net472	2.53μs	1.01ns	3.92ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	116μs	202ns	782ns	0.0581	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	216ns	836ns	0.0606	0	4.28 KB
master	`EnrichedLog`	net472	150μs	147ns	569ns	0.662	0.221	4.46 KB
#5864	`EnrichedLog`	net6.0	116μs	227ns	819ns	0.0573	0	4.28 KB
#5864	`EnrichedLog`	netcoreapp3.1	120μs	142ns	551ns	0	0	4.28 KB
#5864	`EnrichedLog`	net472	149μs	394ns	1.53μs	0.671	0.224	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.08μs	0.97ns	3.76ns	0.0308	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.17μs	1.42ns	5.48ns	0.0294	2.2 KB
master	`EnrichedLog`	net472	4.81μs	1.12ns	4.19ns	0.319	2.02 KB
#5864	`EnrichedLog`	net6.0	3.1μs	1.16ns	4.49ns	0.031	2.2 KB
#5864	`EnrichedLog`	netcoreapp3.1	4.23μs	2.14ns	8.29ns	0.0297	2.2 KB
#5864	`EnrichedLog`	net472	4.84μs	1.63ns	6.32ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.37μs	0.688ns	2.57ns	0.0164	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.74μs	1.15ns	4.46ns	0.0157	0	1.14 KB
master	`SendReceive`	net472	2.08μs	0.837ns	3.24ns	0.183	0.00104	1.16 KB
#5864	`SendReceive`	net6.0	1.34μs	0.606ns	2.1ns	0.0161	0	1.14 KB
#5864	`SendReceive`	netcoreapp3.1	1.76μs	1.17ns	4.52ns	0.0159	0	1.14 KB
#5864	`SendReceive`	net472	2.16μs	0.992ns	3.84ns	0.183	0.00107	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.99μs	3.19ns	12ns	0.0223	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.83μs	2.2ns	8.53ns	0.0211	1.65 KB
master	`EnrichedLog`	net472	4.41μs	1.44ns	5.37ns	0.322	2.04 KB
#5864	`EnrichedLog`	net6.0	2.8μs	0.941ns	3.64ns	0.0211	1.6 KB
#5864	`EnrichedLog`	netcoreapp3.1	3.87μs	1.25ns	4.85ns	0.0211	1.65 KB
#5864	`EnrichedLog`	net472	4.39μs	3.32ns	12.9ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5864

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.186	564.93	476.49

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	395ns	0.154ns	0.577ns	0.00816	576 B
master	`StartFinishSpan`	netcoreapp3.1	555ns	0.482ns	1.87ns	0.00776	576 B
master	`StartFinishSpan`	net472	629ns	0.207ns	0.802ns	0.0916	578 B
master	`StartFinishScope`	net6.0	565ns	0.341ns	1.28ns	0.00984	696 B
master	`StartFinishScope`	netcoreapp3.1	751ns	0.348ns	1.35ns	0.00943	696 B
master	`StartFinishScope`	net472	814ns	0.428ns	1.66ns	0.104	658 B
#5864	`StartFinishSpan`	net6.0	397ns	0.189ns	0.708ns	0.00801	576 B
#5864	`StartFinishSpan`	netcoreapp3.1	582ns	0.601ns	2.33ns	0.00762	576 B
#5864	`StartFinishSpan`	net472	613ns	0.148ns	0.554ns	0.0915	578 B
#5864	`StartFinishScope`	net6.0	476ns	0.181ns	0.701ns	0.00977	696 B
#5864	`StartFinishScope`	netcoreapp3.1	788ns	0.449ns	1.74ns	0.00947	696 B
#5864	`StartFinishScope`	net472	848ns	1.34ns	5.2ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	656ns	0.483ns	1.87ns	0.00987	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	887ns	0.42ns	1.57ns	0.00931	696 B
master	`RunOnMethodBegin`	net472	1.15μs	0.328ns	1.27ns	0.104	658 B
#5864	`RunOnMethodBegin`	net6.0	686ns	0.366ns	1.42ns	0.00964	696 B
#5864	`RunOnMethodBegin`	netcoreapp3.1	891ns	0.954ns	3.57ns	0.00946	696 B
#5864	`RunOnMethodBegin`	net472	1.09μs	0.254ns	0.952ns	0.104	658 B

daniel-romano-DD

👍

andrewlock requested review from a team as code owners August 8, 2024 09:51

andrewlock added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:refactor area:opentelemetry OpenTelemetry support labels Aug 8, 2024

zacharycmontoya approved these changes Aug 8, 2024

View reviewed changes

daniel-romano-DD approved these changes Aug 8, 2024

View reviewed changes

andrewlock merged commit a84edbe into release/2.x Aug 9, 2024
61 of 64 checks passed

andrewlock deleted the andrew/config-refactor/5-record-otel-telemetry-v2 branch August 9, 2024 15:08

github-actions bot added this to the vNext-v2 milestone Aug 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Config refactor - Add telemetry to otel config (#5717 -> v2) #5864

Config refactor - Add telemetry to otel config (#5717 -> v2) #5864

andrewlock commented Aug 8, 2024

andrewlock commented Aug 8, 2024

datadog-ddstaging bot commented Aug 8, 2024 •

edited

Loading

andrewlock commented Aug 8, 2024

Raw results

Raw results

Raw results

Faster 🎉 in #5864

More allocations ⚠️ in #5864

Fewer allocations 🎉 in #5864

Raw results

andrewlock commented Aug 8, 2024

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5864

Raw results

Raw results

Slower ⚠️ in #5864

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5864

Raw results

Raw results

daniel-romano-DD left a comment

Config refactor - Add telemetry to otel config (#5717 -> v2) #5864

Config refactor - Add telemetry to otel config (#5717 -> v2) #5864

Conversation

andrewlock commented Aug 8, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock commented Aug 8, 2024

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Aug 8, 2024 • edited Loading

Datadog Report

⌛ Performance Regressions vs Default Branch (15)

andrewlock commented Aug 8, 2024

Benchmarks Report for appsec 🐌

Benchmark details

Raw results

Raw results

Raw results

Faster 🎉 in #5864

More allocations ⚠️ in #5864

Fewer allocations 🎉 in #5864

Raw results

andrewlock commented Aug 8, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5864

Raw results

Raw results

Slower ⚠️ in #5864

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5864

Raw results

Raw results

daniel-romano-DD left a comment

Choose a reason for hiding this comment

datadog-ddstaging bot commented Aug 8, 2024 •

edited

Loading