[ASM] Fix Null reference exception error #5664

NachoEchevarria · 2024-06-07T13:26:01Z

Summary of changes

This PR fixes some NullReferenceException Error seen in production.

These are two different stacks that had the error:

System.NullReferenceException
at REDACTED
at System.Collections.Generic.ObjectEqualityComparer1.GetHashCode(T obj) at System.Collections.Generic.HashSet1.InternalGetHashCode(T item)
at System.Collections.Generic.HashSet1.AddIfNotPresent(T value) at Datadog.Trace.AppSec.ObjectExtractor.ExtractProperties(Object body, Int32 depth, HashSet1 visited)
at Datadog.Trace.AppSec.ObjectExtractor.ExtractListOrArray(Object value, Int32 depth, HashSet1 visited) at Datadog.Trace.AppSec.ObjectExtractor.ExtractDictionary(Object value, Type dictType, Int32 depth, HashSet1 visited)
at Datadog.Trace.AppSec.ControllerContextExtensions.MonitorBodyAndPathParams(IControllerContext controllerContext, IDictionary`2 parameters, String peekScopeKey)

System.NullReferenceException
at REDACTED
at System.Collections.Generic.ObjectEqualityComparer1.GetHashCode(T obj) at System.Collections.Generic.HashSet1.InternalGetHashCode(T item)
at System.Collections.Generic.HashSet1.Contains(T item) at Datadog.Trace.AppSec.ObjectExtractor.ExtractProperties(Object body, Int32 depth, HashSet1 visited)
at Datadog.Trace.AppSec.ObjectExtractor.ExtractType(Type itemType, Object value, Int32 depth, HashSet`1 visited)

If we take a look at the library code, we can see this:

ObjectEqualityComparer`1.GetHashCode has this code:
public override int GetHashCode(T obj)
{
return obj?.GetHashCode() ?? 0;
}

Sending null values as a body in the method ExtractProperties does not launch any exception.

While we cannot see it because the stack is redacted, it looks like the exception is thrown in a custom implementation of GetHashCode(). The error is pretty persistent and have seen several times, so it makes sense to protect the method ExtractProperties against this particular case.

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging · 2024-06-07T13:52:01Z

Datadog Report

Branch report: nacho/FixNullReferenceExObjectEx
Commit report: e243671
Test service: dd-trace-dotnet

✅ 0 Failed, 343575 Passed, 1680 Skipped, 14h 36m 43.59s Total Time

andrewlock · 2024-06-07T13:56:47Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (73ms)  : 63, 82
     .   : milestone, 73,
    master - mean (75ms)  : 61, 90
     .   : milestone, 75,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (980ms)  : 963, 996
     .   : milestone, 980,
    master - mean (978ms)  : 947, 1010
     .   : milestone, 978,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (109ms)  : 106, 113
     .   : milestone, 109,
    master - mean (109ms)  : 107, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (691ms)  : 674, 709
     .   : milestone, 691,
    master - mean (688ms)  : 669, 707
     .   : milestone, 688,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (93ms)  : 90, 96
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (650ms)  : 624, 676
     .   : milestone, 650,
    master - mean (643ms)  : 622, 663
     .   : milestone, 643,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (192ms)  : 188, 196
     .   : milestone, 192,
    master - mean (192ms)  : 188, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (1,079ms)  : 1058, 1100
     .   : milestone, 1079,
    master - mean (1,066ms)  : 1041, 1090
     .   : milestone, 1066,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (276ms)  : 272, 280
     .   : milestone, 276,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (864ms)  : 842, 886
     .   : milestone, 864,
    master - mean (864ms)  : 840, 888
     .   : milestone, 864,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5664) - mean (268ms)  : 261, 274
     .   : milestone, 268,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (5664) - mean (858ms)  : 825, 891
     .   : milestone, 858,
    master - mean (855ms)  : 828, 881
     .   : milestone, 855,

robertpi · 2024-06-07T15:16:30Z

tracer/src/Datadog.Trace/AppSec/ObjectExtractor.cs

+            catch
+            {
+                // Contains and Add call GetHashCode which can throw an exception if has a custom implementation
+                // If visited is empty, we could potentially get the exception only when calling Add


So this is client's implementation of .GetHashCode() that is throwing the exception? I wonder if there are other places where we handle client objects and this could occur?

I wonder if there are other places where we handle client objects and this could occur?

Technically pretty much anywhere we're calling ToString() comes to mind, but I don't know if it's worth explicitly guarding against it at a low level like this unless we have evidence of it? And instead relying on high-level existing safeguards?

andrewlock · 2024-06-07T18:53:51Z

Benchmarks Report for appsec 🐌

Benchmarks for #5664 compared to master:

1 benchmarks are faster, with geometric mean 1.136
1 benchmarks have fewer allocations
2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	72.8μs	51.1ns	184ns	0.0732	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	62.6μs	53.4ns	207ns	0.093	0	6.94 KB
master	`AllCycleSimpleBody`	net472	48.5μs	52.5ns	197ns	1.32	0	8.33 KB
master	`AllCycleMoreComplexBody`	net6.0	79.5μs	73.5ns	265ns	0.119	0	9.5 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	69.8μs	70.9ns	275ns	0.139	0	10.36 KB
master	`AllCycleMoreComplexBody`	net472	55.7μs	44.3ns	172ns	1.86	0.0278	11.84 KB
master	`ObjectExtractorSimpleBody`	net6.0	141ns	0.284ns	1.1ns	0.00396	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	222ns	0.33ns	1.24ns	0.00375	0	272 B
master	`ObjectExtractorSimpleBody`	net472	170ns	0.166ns	0.643ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.09μs	3.93ns	14.7ns	0.0537	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.06μs	2.1ns	8.12ns	0.0507	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.85μs	1.71ns	6.62ns	0.602	0.00577	3.8 KB
#5664	`AllCycleSimpleBody`	net6.0	72.1μs	112ns	432ns	0.0718	0	6 KB
#5664	`AllCycleSimpleBody`	netcoreapp3.1	61.6μs	85ns	318ns	0.0917	0	6.94 KB
#5664	`AllCycleSimpleBody`	net472	48.7μs	28.3ns	102ns	1.3	0	8.33 KB
#5664	`AllCycleMoreComplexBody`	net6.0	77.2μs	72.1ns	270ns	0.117	0	9.5 KB
#5664	`AllCycleMoreComplexBody`	netcoreapp3.1	69.2μs	75.5ns	283ns	0.139	0	10.36 KB
#5664	`AllCycleMoreComplexBody`	net472	56μs	49.4ns	185ns	1.87	0.0283	11.84 KB
#5664	`ObjectExtractorSimpleBody`	net6.0	141ns	0.136ns	0.51ns	0.00393	0	280 B
#5664	`ObjectExtractorSimpleBody`	netcoreapp3.1	221ns	0.234ns	0.907ns	0.00369	0	272 B
#5664	`ObjectExtractorSimpleBody`	net472	174ns	0.229ns	0.888ns	0.0446	0	281 B
#5664	`ObjectExtractorMoreComplexBody`	net6.0	3.05μs	2.18ns	8.17ns	0.0534	0	3.78 KB
#5664	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.94μs	1.73ns	6.71ns	0.0494	0	3.69 KB
#5664	`ObjectExtractorMoreComplexBody`	net472	3.87μs	2.83ns	10.6ns	0.601	0.0058	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5664

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0	1.136	79,949.00	70,356.82

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	39.5μs	27.7ns	107ns	0.456	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.1μs	14.1ns	54.5ns	0.431	0	32.4 KB
master	`EncodeArgs`	net472	65.7μs	34.4ns	129ns	5.14	0.0655	32.5 KB
master	`EncodeLegacyArgs`	net6.0	79.9μs	105ns	407ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	104μs	110ns	425ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	154μs	66.4ns	257ns	0.31	0	2.15 KB
#5664	`EncodeArgs`	net6.0	37.5μs	15.6ns	60.4ns	0.453	0	32.4 KB
#5664	`EncodeArgs`	netcoreapp3.1	54.9μs	28.2ns	106ns	0.436	0	32.4 KB
#5664	`EncodeArgs`	net472	66.3μs	30.8ns	115ns	5.14	0.0664	32.5 KB
#5664	`EncodeLegacyArgs`	net6.0	70.3μs	17.8ns	66.6ns	0	0	2.14 KB
#5664	`EncodeLegacyArgs`	netcoreapp3.1	105μs	349ns	1.35μs	0	0	2.14 KB
#5664	`EncodeLegacyArgs`	net472	155μs	81.1ns	314ns	0.311	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	183μs	126ns	456ns	0	2.42 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	198μs	226ns	877ns	0	2.37 KB
master	`RunWafRealisticBenchmark`	net472	215μs	56.6ns	219ns	0.321	2.43 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	89.1ns	345ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	130μs	185ns	693ns	0	1.45 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	143μs	55.3ns	214ns	0.214	1.48 KB
#5664	`RunWafRealisticBenchmark`	net6.0	185μs	212ns	820ns	0	2.42 KB
#5664	`RunWafRealisticBenchmark`	netcoreapp3.1	195μs	292ns	1.09μs	0	2.37 KB
#5664	`RunWafRealisticBenchmark`	net472	215μs	120ns	465ns	0.323	2.43 KB
#5664	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	200ns	773ns	0	1.46 KB
#5664	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	129μs	56.6ns	212ns	0	1.45 KB
#5664	`RunWafRealisticBenchmarkWithAttack`	net472	141μs	41ns	148ns	0.212	1.48 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5664

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	253.88 KB	262.87 KB	8.99 KB	3.54%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	57.9 KB	59.22 KB	1.32 KB	2.28%

Fewer allocations 🎉 in #5664

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	264.84 KB	254.98 KB	-9.86 KB	-3.72%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	58.3μs	784ns	7.76μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	60.1μs	759ns	7.51μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.9μs	72.3ns	250ns	57.9 KB
master	`StringConcatAspectBenchmark`	net6.0	308μs	1.59μs	10.3μs	264.84 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	343μs	1.66μs	6.85μs	253.88 KB
master	`StringConcatAspectBenchmark`	net472	286μs	6.09μs	58.1μs	278.53 KB
#5664	`StringConcatBenchmark`	net6.0	52.2μs	239ns	893ns	43.44 KB
#5664	`StringConcatBenchmark`	netcoreapp3.1	59.7μs	659ns	6.55μs	42.64 KB
#5664	`StringConcatBenchmark`	net472	38.3μs	140ns	506ns	59.22 KB
#5664	`StringConcatAspectBenchmark`	net6.0	279μs	4.41μs	41.4μs	254.98 KB
#5664	`StringConcatAspectBenchmark`	netcoreapp3.1	337μs	1.72μs	12.4μs	262.87 KB
#5664	`StringConcatAspectBenchmark`	net472	254μs	1.26μs	5.48μs	278.53 KB

andrewlock · 2024-06-07T19:09:54Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5664) (11.919M)   : 0, 11919010
    master (11.871M)   : 0, 11871366
    benchmarks/2.9.0 (11.919M)   : 0, 11919161

    section Automatic
    This PR (5664) (8.035M)   : 0, 8034822
    master (8.096M)   : 0, 8095854
    benchmarks/2.9.0 (8.337M)   : 0, 8336508

    section Trace stats
    master (8.307M)   : 0, 8307336

    section Manual
    This PR (5664) (10.257M)   : 0, 10257305
    master (10.399M)   : 0, 10398602

    section Manual + Automatic
    This PR (5664) (7.487M)   : 0, 7486977
    master (7.649M)   : 0, 7649350

    section Version Conflict
    master (6.876M)   : 0, 6876355

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5664) (9.482M)   : 0, 9482385
    master (9.597M)   : 0, 9596857
    benchmarks/2.9.0 (9.646M)   : 0, 9645709

    section Automatic
    This PR (5664) (6.728M)   : 0, 6728051
    master (6.640M)   : 0, 6640085

    section Trace stats
    master (7.020M)   : 0, 7019523

    section Manual
    This PR (5664) (8.394M)   : 0, 8394473
    master (8.363M)   : 0, 8362991

    section Manual + Automatic
    This PR (5664) (6.280M)   : 0, 6279551
    master (6.245M)   : 0, 6244614

    section Version Conflict
    master (5.601M)   : 0, 5601482

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5664) (10.000M)   : 0, 10000341
    master (10.167M)   : 0, 10167200
    benchmarks/2.9.0 (10.036M)   : 0, 10035695

    section Automatic
    This PR (5664) (7.174M)   : 0, 7173633
    master (7.140M)   : 0, 7139547
    benchmarks/2.9.0 (7.475M)   : 0, 7475419

    section Trace stats
    master (7.447M)   : 0, 7447083

    section Manual
    This PR (5664) (8.888M)   : 0, 8888121
    master (8.810M)   : 0, 8810177

    section Manual + Automatic
    This PR (5664) (6.901M)   : 0, 6901005
    master (6.884M)   : 0, 6883942

    section Version Conflict
    master (6.169M)   : 0, 6168609

andrewlock · 2024-06-07T20:57:28Z

Benchmarks Report for tracer 🐌

Benchmarks for #5664 compared to master:

1 benchmarks are faster, with geometric mean 1.168
1 benchmarks are slower, with geometric mean 1.125
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.67μs	43.3ns	203ns	0.0213	0.00852	0	7.68 KB
master	`StartStopWithChild`	netcoreapp3.1	10.8μs	52.4ns	222ns	0.0221	0.0111	0	7.78 KB
master	`StartStopWithChild`	net472	17.3μs	44.7ns	173ns	1.36	0.319	0.101	8.2 KB
#5664	`StartStopWithChild`	net6.0	8.96μs	46.7ns	264ns	0.0353	0.0132	0.00441	7.68 KB
#5664	`StartStopWithChild`	netcoreapp3.1	11.1μs	57.6ns	270ns	0.0219	0.0109	0	7.77 KB
#5664	`StartStopWithChild`	net472	17.4μs	72.4ns	280ns	1.37	0.349	0.113	8.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	505μs	512ns	1.91μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	629μs	206ns	769ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	841μs	245ns	950ns	0.419	3.3 KB
#5664	`WriteAndFlushEnrichedTraces`	net6.0	476μs	384ns	1.44μs	0	2.7 KB
#5664	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	642μs	227ns	849ns	0	2.7 KB
#5664	`WriteAndFlushEnrichedTraces`	net472	848μs	236ns	883ns	0.425	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	173μs	255ns	989ns	0.259	18.44 KB
master	`SendRequest`	netcoreapp3.1	192μs	253ns	982ns	0.193	20.6 KB
master	`SendRequest`	net472	0.000137ns	9.3E‑05ns	0.00036ns	0	0 b
#5664	`SendRequest`	net6.0	172μs	187ns	723ns	0.173	18.44 KB
#5664	`SendRequest`	netcoreapp3.1	190μs	346ns	1.34μs	0.19	20.6 KB
#5664	`SendRequest`	net472	0.000614ns	0.000403ns	0.0014ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5664

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.84 KB	41.62 KB	-222 B	-0.53%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	567μs	655ns	2.54μs	0.558	0	0	41.82 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	652μs	841ns	2.91μs	0.324	0	0	41.84 KB
master	`WriteAndFlushEnrichedTraces`	net472	866μs	3.05μs	11.4μs	8.45	2.53	0.422	53.25 KB
#5664	`WriteAndFlushEnrichedTraces`	net6.0	565μs	1.1μs	4.28μs	0.539	0	0	41.8 KB
#5664	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	1.04μs	4.02μs	0.338	0	0	41.62 KB
#5664	`WriteAndFlushEnrichedTraces`	net472	849μs	4.05μs	15.7μs	8.13	2.57	0.428	53.27 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.07μs	1.02ns	3.96ns	0.0112	808 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.45μs	0.692ns	2.59ns	0.0109	808 B
master	`ExecuteNonQuery`	net472	1.71μs	1.76ns	6.83ns	0.122	770 B
#5664	`ExecuteNonQuery`	net6.0	1.16μs	0.766ns	2.97ns	0.0111	808 B
#5664	`ExecuteNonQuery`	netcoreapp3.1	1.45μs	0.465ns	1.74ns	0.0109	808 B
#5664	`ExecuteNonQuery`	net472	1.71μs	1.52ns	5.91ns	0.122	770 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.32μs	0.717ns	2.68ns	0.0139	0	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.58μs	0.666ns	2.4ns	0.0128	0	976 B
master	`CallElasticsearch`	net472	2.54μs	2.03ns	7.85ns	0.157	0.00127	995 B
master	`CallElasticsearchAsync`	net6.0	1.34μs	0.762ns	2.95ns	0.0134	0	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.72μs	0.834ns	3.12ns	0.0138	0	1.02 KB
master	`CallElasticsearchAsync`	net472	2.55μs	0.773ns	2.79ns	0.166	0.00128	1.05 KB
#5664	`CallElasticsearch`	net6.0	1.22μs	0.404ns	1.51ns	0.0139	0	976 B
#5664	`CallElasticsearch`	netcoreapp3.1	1.63μs	0.609ns	2.28ns	0.0131	0	976 B
#5664	`CallElasticsearch`	net472	2.52μs	2.13ns	8.24ns	0.158	0	995 B
#5664	`CallElasticsearchAsync`	net6.0	1.27μs	0.723ns	2.7ns	0.0134	0	952 B
#5664	`CallElasticsearchAsync`	netcoreapp3.1	1.77μs	0.987ns	3.69ns	0.0141	0	1.02 KB
#5664	`CallElasticsearchAsync`	net472	2.52μs	2.52ns	9.77ns	0.167	0	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteAsync`	net6.0	1.23μs	1.25ns	4.69ns	0.0133	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.53μs	2.7ns	10.5ns	0.0131	0	952 B
master	`ExecuteAsync`	net472	1.8μs	1.61ns	6.25ns	0.145	0.000905	915 B
#5664	`ExecuteAsync`	net6.0	1.17μs	0.879ns	3.41ns	0.0134	0	952 B
#5664	`ExecuteAsync`	netcoreapp3.1	1.56μs	2.46ns	9.54ns	0.0129	0	952 B
#5664	`ExecuteAsync`	net472	1.75μs	1.19ns	4.47ns	0.145	0.000874	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.05μs	0.864ns	3.11ns	0.0304	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.01μs	1.29ns	5.01ns	0.0376	2.76 KB
master	`SendAsync`	net472	7.6μs	2.47ns	9.57ns	0.493	3.12 KB
#5664	`SendAsync`	net6.0	4.13μs	1.47ns	5.71ns	0.0309	2.22 KB
#5664	`SendAsync`	netcoreapp3.1	5.13μs	2.7ns	10.5ns	0.0386	2.76 KB
#5664	`SendAsync`	net472	7.67μs	2.23ns	8.65ns	0.496	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5664

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1	1.125	2,163.79	2,433.36

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.51μs	0.46ns	1.72ns	0.0234	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.16μs	3.1ns	11.2ns	0.0218	1.64 KB
master	`EnrichedLog`	net472	2.61μs	2.63ns	9.12ns	0.249	1.57 KB
#5664	`EnrichedLog`	net6.0	1.57μs	0.413ns	1.43ns	0.0227	1.64 KB
#5664	`EnrichedLog`	netcoreapp3.1	2.43μs	0.942ns	3.65ns	0.0219	1.64 KB
#5664	`EnrichedLog`	net472	2.42μs	2.53ns	9.8ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	115μs	411ns	1.59μs	0.0563	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	222ns	831ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	116ns	401ns	0.68	0.227	4.46 KB
#5664	`EnrichedLog`	net6.0	118μs	340ns	1.32μs	0.0577	0	4.28 KB
#5664	`EnrichedLog`	netcoreapp3.1	123μs	558ns	2.16μs	0.0606	0	4.28 KB
#5664	`EnrichedLog`	net472	151μs	400ns	1.55μs	0.673	0.224	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.09μs	0.98ns	3.67ns	0.0308	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.06μs	1.64ns	6.36ns	0.0303	2.2 KB
master	`EnrichedLog`	net472	4.89μs	5.16ns	20ns	0.319	2.02 KB
#5664	`EnrichedLog`	net6.0	3.15μs	1.08ns	3.91ns	0.0307	2.2 KB
#5664	`EnrichedLog`	netcoreapp3.1	4.13μs	1.01ns	3.92ns	0.0288	2.2 KB
#5664	`EnrichedLog`	net472	4.98μs	1.41ns	5.45ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.27μs	0.587ns	2.28ns	0.0159	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.64μs	3.61ns	14ns	0.0155	0	1.14 KB
master	`SendReceive`	net472	1.92μs	1.36ns	5.09ns	0.183	0.000968	1.16 KB
#5664	`SendReceive`	net6.0	1.35μs	2.71ns	10.5ns	0.0162	0	1.14 KB
#5664	`SendReceive`	netcoreapp3.1	1.8μs	1.49ns	5.58ns	0.0152	0	1.14 KB
#5664	`SendReceive`	net472	2.07μs	3.86ns	13.9ns	0.183	0	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.79μs	1.22ns	4.74ns	0.0223	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	4μs	1.81ns	7ns	0.022	1.65 KB
master	`EnrichedLog`	net472	4.37μs	8.46ns	31.7ns	0.323	2.04 KB
#5664	`EnrichedLog`	net6.0	2.75μs	2.98ns	11.5ns	0.0221	1.6 KB
#5664	`EnrichedLog`	netcoreapp3.1	3.92μs	2.89ns	11.2ns	0.0216	1.65 KB
#5664	`EnrichedLog`	net472	4.32μs	2.56ns	9.93ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	398ns	0.367ns	1.42ns	0.00812	576 B
master	`StartFinishSpan`	netcoreapp3.1	605ns	0.334ns	1.25ns	0.0079	576 B
master	`StartFinishSpan`	net472	606ns	0.505ns	1.95ns	0.0916	578 B
master	`StartFinishScope`	net6.0	540ns	0.129ns	0.483ns	0.00972	696 B
master	`StartFinishScope`	netcoreapp3.1	694ns	0.136ns	0.51ns	0.00946	696 B
master	`StartFinishScope`	net472	838ns	0.333ns	1.29ns	0.104	658 B
#5664	`StartFinishSpan`	net6.0	394ns	0.096ns	0.359ns	0.0081	576 B
#5664	`StartFinishSpan`	netcoreapp3.1	575ns	0.761ns	2.95ns	0.00785	576 B
#5664	`StartFinishSpan`	net472	631ns	0.412ns	1.59ns	0.0915	578 B
#5664	`StartFinishScope`	net6.0	588ns	0.186ns	0.722ns	0.00988	696 B
#5664	`StartFinishScope`	netcoreapp3.1	724ns	0.402ns	1.56ns	0.00947	696 B
#5664	`StartFinishScope`	net472	834ns	0.543ns	2.1ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5664

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.168	687.16	588.57

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	688ns	0.225ns	0.872ns	0.00966	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	891ns	0.294ns	1.14ns	0.00948	696 B
master	`RunOnMethodBegin`	net472	1.15μs	0.424ns	1.64ns	0.104	658 B
#5664	`RunOnMethodBegin`	net6.0	588ns	0.283ns	1.06ns	0.00977	696 B
#5664	`RunOnMethodBegin`	netcoreapp3.1	985ns	0.209ns	0.811ns	0.00931	696 B
#5664	`RunOnMethodBegin`	net472	1.08μs	0.456ns	1.76ns	0.104	658 B

Add fix and tests

e243671

NachoEchevarria changed the title ~~Add fix and tests~~ [ASM] Fix Null reference exception Jun 7, 2024

NachoEchevarria changed the title ~~[ASM] Fix Null reference exception~~ [ASM] Fix Null reference exception error Jun 7, 2024

NachoEchevarria marked this pull request as ready for review June 7, 2024 14:43

NachoEchevarria requested a review from a team as a code owner June 7, 2024 14:43

robertpi reviewed Jun 7, 2024

View reviewed changes

robertpi approved these changes Jun 7, 2024

View reviewed changes

andrewlock approved these changes Jun 12, 2024

View reviewed changes

NachoEchevarria merged commit 66699dd into master Jun 12, 2024
58 checks passed

NachoEchevarria deleted the nacho/FixNullReferenceExObjectEx branch June 12, 2024 08:00

github-actions bot added this to the vNext-v2 milestone Jun 12, 2024

andrewlock pushed a commit that referenced this pull request Jun 14, 2024

Add fix and tests (#5664)

a489d7b

andrewlock added type:bug area:asm labels Jun 17, 2024

andrewlock modified the milestones: vNext-v2, 2.53.1 Jun 17, 2024

andrewlock added the identified-by:telemetry label Jun 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Fix Null reference exception error #5664

[ASM] Fix Null reference exception error #5664

NachoEchevarria commented Jun 7, 2024 •

edited

Loading

datadog-ddstaging bot commented Jun 7, 2024 •

edited

Loading

andrewlock commented Jun 7, 2024

robertpi Jun 7, 2024

andrewlock Jun 10, 2024

andrewlock commented Jun 7, 2024

Raw results

Faster 🎉 in #5664

Raw results

Raw results

More allocations ⚠️ in #5664

Fewer allocations 🎉 in #5664

Raw results

andrewlock commented Jun 7, 2024

andrewlock commented Jun 7, 2024

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5664

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5664

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5664

Raw results

[ASM] Fix Null reference exception error #5664

[ASM] Fix Null reference exception error #5664

Conversation

NachoEchevarria commented Jun 7, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Jun 7, 2024 • edited Loading

Datadog Report

andrewlock commented Jun 7, 2024

Execution-Time Benchmarks Report ⏱️

robertpi Jun 7, 2024

Choose a reason for hiding this comment

andrewlock Jun 10, 2024

Choose a reason for hiding this comment

andrewlock commented Jun 7, 2024

Benchmarks Report for appsec 🐌

Benchmark details

Raw results

Faster 🎉 in #5664

Raw results

Raw results

More allocations ⚠️ in #5664

Fewer allocations 🎉 in #5664

Raw results

andrewlock commented Jun 7, 2024

Throughput/Crank Report:zap:

andrewlock commented Jun 7, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #5664

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5664

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5664

Raw results

NachoEchevarria commented Jun 7, 2024 •

edited

Loading

datadog-ddstaging bot commented Jun 7, 2024 •

edited

Loading