Single-step guard rails: Update `RuntimeInformation` to include "inferred" runtime version #5636

andrewlock · 2024-05-31T16:22:22Z

Summary of changes

Update the RuntimeInformation struct` to include the "inferred" version of .NET we're using

Reason for change

We use the corProfilerInfoUnk->QueryInterface APIs to infer which version of .NET we are actually using, because pCorProfilerInfo->GetRuntimeInformation doesn't always return reliable information. This PR stores that value as a string that can be used later (by guard rails).

Implementation details

Added a string field to the RuntimeInformation struct
Return the inferred version (if any) when walking through the QueryInterface() calls
Fallback to the inferred version as the "detected" version of .NET for "early" versions of .NET Core/.NET Framework

Test coverage

The field isn't used anywhere in this PR, so as long as everything still works, it's ok. The behaviour is covered implicitly by subsequent tests later in the PR stack.

Other details

Prerequisite for single-step guard rails work stack

datadog-ddstaging · 2024-05-31T16:35:18Z

Datadog Report

Branch report: andrew/ssi-guard-rails/2-return-inferred-runtime
Commit report: 3d9d3b6
Test service: dd-trace-dotnet

✅ 0 Failed, 339776 Passed, 1851 Skipped, 15h 0m 59.22s Total Time

andrewlock · 2024-05-31T16:52:27Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (75ms)  : 67, 82
     .   : milestone, 75,
    master - mean (73ms)  : 60, 86
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (976ms)  : 954, 998
     .   : milestone, 976,
    master - mean (983ms)  : 957, 1009
     .   : milestone, 983,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (109ms)  : 107, 111
     .   : milestone, 109,
    master - mean (110ms)  : 107, 112
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (690ms)  : 670, 711
     .   : milestone, 690,
    master - mean (681ms)  : 663, 700
     .   : milestone, 681,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (92ms)  : 90, 95
     .   : milestone, 92,
    master - mean (93ms)  : 90, 96
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (650ms)  : 623, 676
     .   : milestone, 650,
    master - mean (642ms)  : 621, 664
     .   : milestone, 642,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (193ms)  : 189, 196
     .   : milestone, 193,
    master - mean (192ms)  : 189, 195
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (1,075ms)  : 1056, 1095
     .   : milestone, 1075,
    master - mean (1,081ms)  : 1056, 1105
     .   : milestone, 1081,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (282ms)  : 274, 289
     .   : milestone, 282,
    master - mean (277ms)  : 273, 281
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (862ms)  : 836, 889
     .   : milestone, 862,
    master - mean (867ms)  : 848, 886
     .   : milestone, 867,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5636) - mean (267ms)  : 261, 272
     .   : milestone, 267,
    master - mean (267ms)  : 262, 273
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (5636) - mean (851ms)  : 826, 876
     .   : milestone, 851,
    master - mean (856ms)  : 821, 892
     .   : milestone, 856,

andrewlock · 2024-05-31T17:35:08Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5636) (11.026M)   : 0, 11025515
    master (11.065M)   : 0, 11065495
    benchmarks/2.9.0 (11.482M)   : 0, 11482120

    section Automatic
    This PR (5636) (7.610M)   : 0, 7610417
    master (7.418M)   : 0, 7417628
    benchmarks/2.9.0 (8.261M)   : 0, 8261135

    section Trace stats
    master (8.056M)   : 0, 8056029

    section Manual
    This PR (5636) (9.896M)   : 0, 9895769
    master (10.033M)   : 0, 10033034

    section Manual + Automatic
    This PR (5636) (7.239M)   : 0, 7238513
    master (7.235M)   : 0, 7235194

    section Version Conflict
    master (6.380M)   : 0, 6380131

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5636) (9.679M)   : 0, 9678752
    master (9.553M)   : 0, 9553241
    benchmarks/2.9.0 (9.752M)   : 0, 9752478

    section Automatic
    This PR (5636) (6.637M)   : 0, 6636570
    master (6.596M)   : 0, 6596030

    section Trace stats
    master (6.872M)   : 0, 6872094

    section Manual
    This PR (5636) (8.396M)   : 0, 8395916
    master (8.293M)   : 0, 8292775

    section Manual + Automatic
    This PR (5636) (5.994M)   : 0, 5994044
    master (6.239M)   : 0, 6238573

    section Version Conflict
    master (5.662M)   : 0, 5662295

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5636) (9.773M)   : 0, 9772661
    master (10.238M)   : 0, 10238153
    benchmarks/2.9.0 (9.820M)   : 0, 9820306

    section Automatic
    This PR (5636) (6.881M)   : 0, 6881167
    master (7.036M)   : 0, 7036074
    benchmarks/2.9.0 (7.183M)   : 0, 7183038

    section Trace stats
    master (7.383M)   : 0, 7383437

    section Manual
    This PR (5636) (8.593M)   : 0, 8592535
    master (8.662M)   : 0, 8661613

    section Manual + Automatic
    This PR (5636) (6.656M)   : 0, 6656385
    master (6.646M)   : 0, 6646387

    section Version Conflict
    master (6.192M)   : 0, 6192345

andrewlock · 2024-05-31T17:52:23Z

Benchmarks Report for appsec 🐌

Benchmarks for #5636 compared to master:

1 benchmarks are faster, with geometric mean 1.203
1 benchmarks have fewer allocations
2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	74.3μs	97.3ns	364ns	0.074	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	64μs	77.7ns	291ns	0.0954	0	6.94 KB
master	`AllCycleSimpleBody`	net472	50.5μs	80.1ns	300ns	1.31	0	8.33 KB
master	`AllCycleMoreComplexBody`	net6.0	80μs	62.8ns	243ns	0.12	0	9.5 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	71.3μs	78.1ns	281ns	0.108	0	10.36 KB
master	`AllCycleMoreComplexBody`	net472	57.8μs	79.4ns	307ns	1.87	0.0288	11.84 KB
master	`ObjectExtractorSimpleBody`	net6.0	144ns	0.149ns	0.557ns	0.00394	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	212ns	0.134ns	0.502ns	0.00382	0	272 B
master	`ObjectExtractorSimpleBody`	net472	182ns	0.39ns	1.51ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.04μs	2.09ns	7.81ns	0.0535	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.02μs	2.71ns	10.5ns	0.0503	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.98μs	4.05ns	15.7ns	0.602	0.00594	3.8 KB
#5636	`AllCycleSimpleBody`	net6.0	72.1μs	85.4ns	331ns	0.0716	0	6 KB
#5636	`AllCycleSimpleBody`	netcoreapp3.1	61.3μs	72ns	279ns	0.0913	0	6.94 KB
#5636	`AllCycleSimpleBody`	net472	48μs	72.9ns	282ns	1.32	0	8.33 KB
#5636	`AllCycleMoreComplexBody`	net6.0	78.3μs	131ns	508ns	0.114	0	9.5 KB
#5636	`AllCycleMoreComplexBody`	netcoreapp3.1	69.7μs	82.4ns	319ns	0.14	0	10.36 KB
#5636	`AllCycleMoreComplexBody`	net472	55.7μs	59.3ns	230ns	1.86	0.0278	11.84 KB
#5636	`ObjectExtractorSimpleBody`	net6.0	145ns	0.172ns	0.666ns	0.00393	0	280 B
#5636	`ObjectExtractorSimpleBody`	netcoreapp3.1	197ns	0.141ns	0.546ns	0.00371	0	272 B
#5636	`ObjectExtractorSimpleBody`	net472	169ns	0.36ns	1.39ns	0.0446	0	281 B
#5636	`ObjectExtractorMoreComplexBody`	net6.0	3.01μs	2.07ns	7.75ns	0.0528	0	3.78 KB
#5636	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.92μs	1.96ns	7.61ns	0.051	0	3.69 KB
#5636	`ObjectExtractorMoreComplexBody`	net472	3.75μs	2.33ns	9.03ns	0.602	0.00561	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	41.3μs	21.5ns	83.1ns	0.454	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	56.3μs	62.1ns	241ns	0.447	0	32.4 KB
master	`EncodeArgs`	net472	71.1μs	62.2ns	241ns	5.15	0.0711	32.5 KB
master	`EncodeLegacyArgs`	net6.0	71.8μs	91.2ns	353ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	105μs	119ns	459ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	155μs	118ns	459ns	0.308	0	2.15 KB
#5636	`EncodeArgs`	net6.0	40.1μs	25.4ns	98.2ns	0.442	0	32.4 KB
#5636	`EncodeArgs`	netcoreapp3.1	55.8μs	27.2ns	105ns	0.442	0	32.4 KB
#5636	`EncodeArgs`	net472	68.2μs	37ns	143ns	5.14	0.0685	32.5 KB
#5636	`EncodeLegacyArgs`	net6.0	72.2μs	167ns	626ns	0	0	2.14 KB
#5636	`EncodeLegacyArgs`	netcoreapp3.1	104μs	177ns	687ns	0	0	2.14 KB
#5636	`EncodeLegacyArgs`	net472	153μs	135ns	522ns	0.304	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	186μs	127ns	492ns	0	2.42 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	199μs	195ns	731ns	0	2.37 KB
master	`RunWafRealisticBenchmark`	net472	218μs	126ns	486ns	0.326	2.43 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	90.4ns	350ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	133μs	231ns	893ns	0	1.45 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	144μs	174ns	675ns	0.216	1.48 KB
#5636	`RunWafRealisticBenchmark`	net6.0	184μs	74.5ns	289ns	0	2.42 KB
#5636	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	168ns	629ns	0	2.37 KB
#5636	`RunWafRealisticBenchmark`	net472	216μs	85.1ns	330ns	0.324	2.43 KB
#5636	`RunWafRealisticBenchmarkWithAttack`	net6.0	124μs	56.9ns	213ns	0	1.46 KB
#5636	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	131μs	333ns	1.29μs	0	1.45 KB
#5636	`RunWafRealisticBenchmarkWithAttack`	net472	143μs	122ns	472ns	0.214	1.48 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #5636

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	1.203	45,600.00	37,900.00

More allocations ⚠️ in #5636

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	56.65 KB	59.22 KB	2.58 KB	4.55%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	252.13 KB	254.74 KB	2.62 KB	1.04%

Fewer allocations 🎉 in #5636

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472	278.53 KB	275.76 KB	-2.77 KB	-0.99%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	60.4μs	730ns	7.27μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	65.4μs	908ns	9.03μs	42.64 KB
master	`StringConcatBenchmark`	net472	46.5μs	367ns	3.58μs	56.65 KB
master	`StringConcatAspectBenchmark`	net6.0	336μs	1.86μs	13.5μs	256.05 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	342μs	3.08μs	29.5μs	252.13 KB
master	`StringConcatAspectBenchmark`	net472	332μs	7.22μs	69.6μs	278.53 KB
#5636	`StringConcatBenchmark`	net6.0	51.9μs	271ns	1.36μs	43.44 KB
#5636	`StringConcatBenchmark`	netcoreapp3.1	57.3μs	549ns	5.38μs	42.64 KB
#5636	`StringConcatBenchmark`	net472	37.9μs	85.9ns	321ns	59.22 KB
#5636	`StringConcatAspectBenchmark`	net6.0	305μs	1.69μs	10.8μs	255.34 KB
#5636	`StringConcatAspectBenchmark`	netcoreapp3.1	336μs	1.82μs	13.7μs	254.74 KB
#5636	`StringConcatAspectBenchmark`	net472	295μs	6μs	58.5μs	275.76 KB

andrewlock · 2024-05-31T19:27:09Z

Benchmarks Report for tracer 🐌

Benchmarks for #5636 compared to master:

1 benchmarks are faster, with geometric mean 1.154
2 benchmarks are slower, with geometric mean 1.127
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.85μs	48.1ns	263ns	0.0214	0.00854	0	7.56 KB
master	`StartStopWithChild`	netcoreapp3.1	10.7μs	58.9ns	358ns	0.0359	0.0154	0	7.65 KB
master	`StartStopWithChild`	net472	17.3μs	58.3ns	226ns	1.35	0.354	0.103	8.06 KB
#5636	`StartStopWithChild`	net6.0	8.71μs	48.7ns	300ns	0.0212	0.00847	0	7.55 KB
#5636	`StartStopWithChild`	netcoreapp3.1	10.8μs	56.4ns	287ns	0.0223	0.0111	0	7.64 KB
#5636	`StartStopWithChild`	net472	17μs	65.5ns	253ns	1.36	0.387	0.109	8.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	472μs	168ns	629ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	640μs	303ns	1.17μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	839μs	289ns	1.12μs	0.417	3.3 KB
#5636	`WriteAndFlushEnrichedTraces`	net6.0	481μs	454ns	1.7μs	0	2.7 KB
#5636	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	636μs	238ns	892ns	0	2.7 KB
#5636	`WriteAndFlushEnrichedTraces`	net472	850μs	302ns	1.17μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	172μs	241ns	933ns	0.256	18.44 KB
master	`SendRequest`	netcoreapp3.1	192μs	231ns	864ns	0.192	20.6 KB
master	`SendRequest`	net472	0.000276ns	0.000116ns	0.000434ns	0	0 b
#5636	`SendRequest`	net6.0	173μs	246ns	951ns	0.172	18.44 KB
#5636	`SendRequest`	netcoreapp3.1	195μs	320ns	1.24μs	0.195	20.6 KB
#5636	`SendRequest`	net472	7.22E‑05ns	4.09E‑05ns	0.000153ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5636

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.57 KB	41.82 KB	251 B	0.60%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	554μs	350ns	1.36μs	0.551	0	0	41.57 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	647μs	1.3μs	5.05μs	0.324	0	0	41.82 KB
master	`WriteAndFlushEnrichedTraces`	net472	847μs	4.21μs	17.9μs	8.17	2.45	0.408	53.26 KB
#5636	`WriteAndFlushEnrichedTraces`	net6.0	567μs	1.84μs	7.13μs	0.563	0	0	41.82 KB
#5636	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	662μs	977ns	3.79μs	0.329	0	0	41.67 KB
#5636	`WriteAndFlushEnrichedTraces`	net472	859μs	4.12μs	15.9μs	8.45	2.53	0.422	53.23 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.09μs	0.743ns	2.88ns	0.0114	808 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.47μs	0.626ns	2.42ns	0.0108	808 B
master	`ExecuteNonQuery`	net472	1.71μs	1.68ns	6.5ns	0.122	770 B
#5636	`ExecuteNonQuery`	net6.0	1.07μs	0.814ns	3.15ns	0.0112	808 B
#5636	`ExecuteNonQuery`	netcoreapp3.1	1.44μs	0.753ns	2.92ns	0.0108	808 B
#5636	`ExecuteNonQuery`	net472	1.71μs	2.14ns	8.31ns	0.122	770 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.22μs	0.965ns	3.61ns	0.0135	0	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.51μs	1.76ns	6.57ns	0.0127	0	976 B
master	`CallElasticsearch`	net472	2.56μs	1.7ns	6.13ns	0.158	0	995 B
master	`CallElasticsearchAsync`	net6.0	1.16μs	0.362ns	1.36ns	0.0133	0	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.65μs	1.72ns	6.65ns	0.0141	0	1.02 KB
master	`CallElasticsearchAsync`	net472	2.62μs	5.79ns	22.4ns	0.166	0.00133	1.05 KB
#5636	`CallElasticsearch`	net6.0	1.14μs	0.357ns	1.29ns	0.0137	0	976 B
#5636	`CallElasticsearch`	netcoreapp3.1	1.45μs	1.9ns	7.37ns	0.013	0	976 B
#5636	`CallElasticsearch`	net472	2.38μs	1.66ns	6.42ns	0.158	0.00119	995 B
#5636	`CallElasticsearchAsync`	net6.0	1.26μs	0.774ns	2.9ns	0.0133	0	952 B
#5636	`CallElasticsearchAsync`	netcoreapp3.1	1.64μs	0.755ns	2.82ns	0.014	0	1.02 KB
#5636	`CallElasticsearchAsync`	net472	2.57μs	1.54ns	5.98ns	0.166	0.00129	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5636

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.141	1,170.83	1,335.41

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteAsync`	net6.0	1.17μs	0.676ns	2.62ns	0.0135	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.68μs	1.53ns	5.73ns	0.0128	0	952 B
master	`ExecuteAsync`	net472	1.79μs	0.479ns	1.73ns	0.145	0	915 B
#5636	`ExecuteAsync`	net6.0	1.34μs	0.668ns	2.31ns	0.0133	0	952 B
#5636	`ExecuteAsync`	netcoreapp3.1	1.61μs	1.44ns	5.41ns	0.013	0	952 B
#5636	`ExecuteAsync`	net472	1.83μs	1.15ns	4.47ns	0.145	0.000915	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.1μs	1.33ns	4.98ns	0.0307	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.17μs	4.5ns	17.4ns	0.0363	2.76 KB
master	`SendAsync`	net472	7.47μs	2.09ns	8.11ns	0.496	3.12 KB
#5636	`SendAsync`	net6.0	4.15μs	2.34ns	8.77ns	0.0315	2.22 KB
#5636	`SendAsync`	netcoreapp3.1	5.17μs	1.75ns	6.57ns	0.0361	2.76 KB
#5636	`SendAsync`	net472	7.55μs	2.3ns	8.91ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.45μs	0.932ns	3.49ns	0.0231	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.21μs	0.984ns	3.81ns	0.022	1.64 KB
master	`EnrichedLog`	net472	2.49μs	1.05ns	4.08ns	0.249	1.57 KB
#5636	`EnrichedLog`	net6.0	1.5μs	0.629ns	2.44ns	0.0232	1.64 KB
#5636	`EnrichedLog`	netcoreapp3.1	2.22μs	1.76ns	6.58ns	0.0217	1.64 KB
#5636	`EnrichedLog`	net472	2.44μs	0.872ns	3.38ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	117μs	184ns	713ns	0.0579	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	139ns	520ns	0	0	4.28 KB
master	`EnrichedLog`	net472	150μs	70.1ns	272ns	0.673	0.224	4.46 KB
#5636	`EnrichedLog`	net6.0	116μs	81.5ns	316ns	0.0581	0	4.28 KB
#5636	`EnrichedLog`	netcoreapp3.1	121μs	124ns	481ns	0	0	4.28 KB
#5636	`EnrichedLog`	net472	150μs	95.9ns	371ns	0.675	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.03μs	0.311ns	1.2ns	0.0306	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.24μs	1.78ns	6.4ns	0.0296	2.2 KB
master	`EnrichedLog`	net472	4.77μs	1.56ns	6.06ns	0.321	2.02 KB
#5636	`EnrichedLog`	net6.0	3.04μs	1.05ns	4.05ns	0.0305	2.2 KB
#5636	`EnrichedLog`	netcoreapp3.1	4.17μs	1.51ns	5.85ns	0.0292	2.2 KB
#5636	`EnrichedLog`	net472	4.94μs	1.54ns	5.77ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.35μs	0.799ns	3.09ns	0.0162	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.82μs	0.987ns	3.82ns	0.0154	0	1.14 KB
master	`SendReceive`	net472	2.08μs	1.34ns	5.17ns	0.183	0.00106	1.16 KB
#5636	`SendReceive`	net6.0	1.25μs	0.474ns	1.84ns	0.0163	0	1.14 KB
#5636	`SendReceive`	netcoreapp3.1	1.76μs	0.762ns	2.95ns	0.0149	0	1.14 KB
#5636	`SendReceive`	net472	2.12μs	0.925ns	3.58ns	0.183	0	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.8μs	0.697ns	2.61ns	0.0224	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.83μs	2.59ns	10ns	0.0229	1.65 KB
master	`EnrichedLog`	net472	4.44μs	2.67ns	10.4ns	0.322	2.04 KB
#5636	`EnrichedLog`	net6.0	2.8μs	0.647ns	2.33ns	0.0226	1.6 KB
#5636	`EnrichedLog`	netcoreapp3.1	3.85μs	1.58ns	5.93ns	0.0213	1.65 KB
#5636	`EnrichedLog`	net472	4.32μs	3.43ns	13.3ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5636

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.113	474.90	528.65

Faster 🎉 in #5636

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.154	656.23	568.63

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	389ns	0.0922ns	0.345ns	0.00801	576 B
master	`StartFinishSpan`	netcoreapp3.1	548ns	0.856ns	3.32ns	0.00766	576 B
master	`StartFinishSpan`	net472	657ns	0.475ns	1.84ns	0.0918	578 B
master	`StartFinishScope`	net6.0	475ns	0.274ns	1.06ns	0.00966	696 B
master	`StartFinishScope`	netcoreapp3.1	709ns	0.519ns	2.01ns	0.00954	696 B
master	`StartFinishScope`	net472	870ns	0.92ns	3.56ns	0.104	658 B
#5636	`StartFinishSpan`	net6.0	394ns	0.131ns	0.491ns	0.00794	576 B
#5636	`StartFinishSpan`	netcoreapp3.1	558ns	0.39ns	1.51ns	0.00794	576 B
#5636	`StartFinishSpan`	net472	570ns	0.7ns	2.71ns	0.0918	578 B
#5636	`StartFinishScope`	net6.0	529ns	0.116ns	0.449ns	0.00967	696 B
#5636	`StartFinishScope`	netcoreapp3.1	760ns	1.01ns	3.93ns	0.00954	696 B
#5636	`StartFinishScope`	net472	833ns	0.866ns	3.35ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	590ns	0.326ns	1.26ns	0.0098	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	936ns	1.16ns	4.5ns	0.00971	696 B
master	`RunOnMethodBegin`	net472	1.07μs	0.563ns	2.18ns	0.104	658 B
#5636	`RunOnMethodBegin`	net6.0	565ns	0.206ns	0.799ns	0.00961	696 B
#5636	`RunOnMethodBegin`	netcoreapp3.1	986ns	0.465ns	1.74ns	0.00941	696 B
#5636	`RunOnMethodBegin`	net472	1.07μs	0.33ns	1.28ns	0.104	658 B

gleocadie

LGTM

gleocadie · 2024-06-03T14:44:02Z

shared/src/Datadog.Trace.ClrProfiler.Native/cor_profiler.h


        RuntimeInformation() :
            runtime_type((COR_PRF_RUNTIME_TYPE) 0x0), major_version(0), minor_version(0), build_version(0), qfe_version(0)
        {
        }

        RuntimeInformation(COR_PRF_RUNTIME_TYPE runtime_type, USHORT major_version, USHORT minor_version,
-                           USHORT build_version, USHORT qfe_version) :
+                           USHORT build_version, USHORT qfe_version, const std::string inferred_version) :


Nit: const here is not necessary, since it's a copy.

Ah, should it be a copy? That may be my mistake 😅

…e we've already calculated this)

andrewlock added the area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) label May 31, 2024

andrewlock requested a review from a team as a code owner May 31, 2024 16:22

This was referenced May 31, 2024

Single-step guard rails: Move version.h to shared code #5635

Merged

Single-step guard rails: telemetry #5637

Merged

andrewlock mentioned this pull request May 31, 2024

Record SSI injection values in configuration #5611

Merged

kevingosse approved these changes Jun 3, 2024

View reviewed changes

gleocadie approved these changes Jun 3, 2024

View reviewed changes

Base automatically changed from andrew/ssi-guard-rails/1-move-version to master June 4, 2024 12:26

andrewlock requested review from a team as code owners June 4, 2024 12:26

andrewlock added 2 commits June 4, 2024 13:29

Update RuntimeInformation to include "inferred" runtime version (wher…

f12e27c

…e we've already calculated this)

Tweaks based on PR feedback

3d9d3b6

andrewlock force-pushed the andrew/ssi-guard-rails/2-return-inferred-runtime branch from 8d9af1c to 3d9d3b6 Compare June 4, 2024 12:29

github-actions bot added the area:shared-components label Jun 4, 2024

andrewlock removed request for a team June 4, 2024 13:18

andrewlock merged commit 0cb80c6 into master Jun 5, 2024
65 checks passed

andrewlock deleted the andrew/ssi-guard-rails/2-return-inferred-runtime branch June 5, 2024 10:19

github-actions bot added this to the vNext-v2 milestone Jun 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Single-step guard rails: Update `RuntimeInformation` to include "inferred" runtime version #5636

Single-step guard rails: Update `RuntimeInformation` to include "inferred" runtime version #5636

andrewlock commented May 31, 2024 •

edited

Loading

datadog-ddstaging bot commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024

Raw results

Raw results

Raw results

Faster 🎉 in #5636

More allocations ⚠️ in #5636

Fewer allocations 🎉 in #5636

Raw results

andrewlock commented May 31, 2024 •

edited

Loading

Raw results

Raw results

Raw results

More allocations ⚠️ in #5636

Raw results

Raw results

Raw results

Slower ⚠️ in #5636

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5636

Faster 🎉 in #5636

Raw results

Raw results

gleocadie left a comment

gleocadie Jun 3, 2024

andrewlock Jun 3, 2024

Single-step guard rails: Update RuntimeInformation to include "inferred" runtime version #5636

Single-step guard rails: Update RuntimeInformation to include "inferred" runtime version #5636

Conversation

andrewlock commented May 31, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented May 31, 2024 • edited Loading

Datadog Report

andrewlock commented May 31, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented May 31, 2024 • edited Loading

Throughput/Crank Report:zap:

andrewlock commented May 31, 2024

Benchmarks Report for appsec 🐌

Benchmark details

Raw results

Raw results

Raw results

Faster 🎉 in #5636

More allocations ⚠️ in #5636

Fewer allocations 🎉 in #5636

Raw results

andrewlock commented May 31, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #5636

Raw results

Raw results

Raw results

Slower ⚠️ in #5636

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5636

Faster 🎉 in #5636

Raw results

Raw results

gleocadie left a comment

Choose a reason for hiding this comment

gleocadie Jun 3, 2024

Choose a reason for hiding this comment

andrewlock Jun 3, 2024

Choose a reason for hiding this comment

Single-step guard rails: Update `RuntimeInformation` to include "inferred" runtime version #5636

Single-step guard rails: Update `RuntimeInformation` to include "inferred" runtime version #5636

andrewlock commented May 31, 2024 •

edited

Loading

datadog-ddstaging bot commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024 •

edited

Loading

andrewlock commented May 31, 2024 •

edited

Loading