[ASM] Fix bug RC empty key #6058

e-n-0 · 2024-09-19T16:58:34Z

Summary of changes

Fixed a regression bug.
The key variable for the process update wasn't the right one.

Bug explanation

The Path of the file wasn't get from the right variable.
result.TypedFile is a JToken, getting the Path from this object would get the path of the JSON token. The path is an empty string and totally not the info we wanted to get.

Test coverage

Add a way to set custom headers instead of the default Mistake Not... to trigger a blocking event from the default ruleset
Added an integration test where it tests deletion and switch back to default rules but keep updated blocking action.

datadog-ddstaging · 2024-09-19T17:26:21Z

Datadog Report

Branch report: flavien/system-tests-bug
Commit report: a34e78f
Test service: dd-trace-dotnet

✅ 0 Failed, 371021 Passed, 2079 Skipped, 16h 25m 52.73s Total Time

andrewlock · 2024-09-19T17:26:23Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (70ms)  : 67, 72
     .   : milestone, 70,
    master - mean (70ms)  : 67, 73
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (1,099ms)  : 1080, 1117
     .   : milestone, 1099,
    master - mean (1,105ms)  : 1067, 1143
     .   : milestone, 1105,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (108ms)  : 105, 112
     .   : milestone, 108,
    master - mean (109ms)  : 105, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (762ms)  : 750, 774
     .   : milestone, 762,
    master - mean (766ms)  : 751, 782
     .   : milestone, 766,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (721ms)  : 706, 736
     .   : milestone, 721,
    master - mean (727ms)  : 707, 747
     .   : milestone, 727,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (190ms)  : 187, 193
     .   : milestone, 190,
    master - mean (191ms)  : 187, 195
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (1,192ms)  : 1169, 1214
     .   : milestone, 1192,
    master - mean (1,202ms)  : 1165, 1240
     .   : milestone, 1202,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (276ms)  : 273, 280
     .   : milestone, 276,
    master - mean (276ms)  : 270, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (935ms)  : 916, 955
     .   : milestone, 935,
    master - mean (938ms)  : 918, 958
     .   : milestone, 938,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6058) - mean (265ms)  : 261, 268
     .   : milestone, 265,
    master - mean (265ms)  : 261, 269
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6058) - mean (921ms)  : 904, 939
     .   : milestone, 921,
    master - mean (922ms)  : 906, 939
     .   : milestone, 922,

andrewlock · 2024-09-19T17:55:16Z

Benchmarks Report for appsec 🐌

Benchmarks for #6058 compared to master:

1 benchmarks are faster, with geometric mean 1.185
1 benchmarks are slower, with geometric mean 1.142
1 benchmarks have fewer allocations
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6058

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1	1.142	189.33	216.21

Faster 🎉 in #6058

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.185	209.22	176.49

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	73μs	87.5ns	339ns	0.0722	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	61.3μs	87.8ns	340ns	0.0922	0	6.95 KB
master	`AllCycleSimpleBody`	net472	48.1μs	51.7ns	200ns	1.3	0	8.34 KB
master	`AllCycleMoreComplexBody`	net6.0	78.3μs	114ns	443ns	0.117	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	69μs	111ns	432ns	0.137	0	10.36 KB
master	`AllCycleMoreComplexBody`	net472	55.5μs	71.7ns	278ns	1.87	0.0279	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	139ns	0.0954ns	0.369ns	0.00396	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	189ns	0.359ns	1.39ns	0.00367	0	272 B
master	`ObjectExtractorSimpleBody`	net472	209ns	0.158ns	0.614ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3μs	1.9ns	7.34ns	0.0525	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.03μs	1.93ns	6.95ns	0.0485	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.89μs	2.36ns	8.81ns	0.602	0.00586	3.8 KB
#6058	`AllCycleSimpleBody`	net6.0	72.9μs	117ns	453ns	0.0726	0	6 KB
#6058	`AllCycleSimpleBody`	netcoreapp3.1	63.6μs	89.5ns	347ns	0.0953	0	6.95 KB
#6058	`AllCycleSimpleBody`	net472	48.1μs	22.7ns	81.7ns	1.32	0	8.34 KB
#6058	`AllCycleMoreComplexBody`	net6.0	80.8μs	119ns	459ns	0.117	0	9.51 KB
#6058	`AllCycleMoreComplexBody`	netcoreapp3.1	69.4μs	116ns	448ns	0.138	0	10.36 KB
#6058	`AllCycleMoreComplexBody`	net472	54.7μs	51.1ns	191ns	1.88	0.0273	11.85 KB
#6058	`ObjectExtractorSimpleBody`	net6.0	139ns	0.135ns	0.505ns	0.0039	0	280 B
#6058	`ObjectExtractorSimpleBody`	netcoreapp3.1	216ns	0.395ns	1.53ns	0.00371	0	272 B
#6058	`ObjectExtractorSimpleBody`	net472	176ns	0.228ns	0.883ns	0.0446	0	281 B
#6058	`ObjectExtractorMoreComplexBody`	net6.0	3.06μs	2.3ns	8.59ns	0.0534	0	3.78 KB
#6058	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.93μs	2.1ns	7.86ns	0.0487	0	3.69 KB
#6058	`ObjectExtractorMoreComplexBody`	net472	3.8μs	3.21ns	12.4ns	0.602	0.0057	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	35.9μs	12.4ns	48.1ns	0.449	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.5μs	22.6ns	87.4ns	0.435	0	32.4 KB
master	`EncodeArgs`	net472	66.3μs	34.3ns	133ns	5.15	0.0661	32.5 KB
master	`EncodeLegacyArgs`	net6.0	70.2μs	15.6ns	60.6ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	106μs	374ns	1.45μs	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	152μs	64.2ns	249ns	0.302	0	2.15 KB
#6058	`EncodeArgs`	net6.0	38μs	28.2ns	109ns	0.449	0	32.4 KB
#6058	`EncodeArgs`	netcoreapp3.1	54.1μs	15.3ns	57.2ns	0.431	0	32.4 KB
#6058	`EncodeArgs`	net472	65.8μs	41.9ns	162ns	5.13	0.0658	32.5 KB
#6058	`EncodeLegacyArgs`	net6.0	76.6μs	35.2ns	137ns	0	0	2.14 KB
#6058	`EncodeLegacyArgs`	netcoreapp3.1	105μs	359ns	1.39μs	0	0	2.14 KB
#6058	`EncodeLegacyArgs`	net472	152μs	151ns	586ns	0.304	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	184μs	57ns	197ns	0	2.44 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	235ns	909ns	0	2.39 KB
master	`RunWafRealisticBenchmark`	net472	208μs	127ns	477ns	0.314	2.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	44.5ns	166ns	0	1.47 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	129μs	93.5ns	350ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	53.1ns	206ns	0.209	1.49 KB
#6058	`RunWafRealisticBenchmark`	net6.0	185μs	51.8ns	194ns	0	2.44 KB
#6058	`RunWafRealisticBenchmark`	netcoreapp3.1	199μs	280ns	1.08μs	0	2.39 KB
#6058	`RunWafRealisticBenchmark`	net472	208μs	45.6ns	164ns	0.311	2.46 KB
#6058	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	37.1ns	139ns	0	1.47 KB
#6058	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	130μs	69ns	267ns	0	1.46 KB
#6058	`RunWafRealisticBenchmarkWithAttack`	net472	138μs	25.3ns	91.3ns	0.207	1.49 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6058

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	255.36 KB	263.99 KB	8.63 KB	3.38%

Fewer allocations 🎉 in #6058

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	59.39 KB	59.04 KB	-352 B	-0.59%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	59.4μs	775ns	7.75μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	60.1μs	722ns	7.11μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.3μs	150ns	562ns	59.39 KB
master	`StringConcatAspectBenchmark`	net6.0	291μs	5.71μs	56.6μs	255.36 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	334μs	1.75μs	11.8μs	253.17 KB
master	`StringConcatAspectBenchmark`	net472	269μs	5.14μs	49.8μs	278.53 KB
#6058	`StringConcatBenchmark`	net6.0	60.3μs	628ns	6.12μs	43.44 KB
#6058	`StringConcatBenchmark`	netcoreapp3.1	62.7μs	795ns	7.91μs	42.64 KB
#6058	`StringConcatBenchmark`	net472	37.3μs	79.7ns	287ns	59.04 KB
#6058	`StringConcatAspectBenchmark`	net6.0	306μs	1.73μs	11.2μs	263.99 KB
#6058	`StringConcatAspectBenchmark`	netcoreapp3.1	334μs	1.53μs	6.67μs	253.21 KB
#6058	`StringConcatAspectBenchmark`	net472	236μs	939ns	3.25μs	278.53 KB

andrewlock · 2024-09-19T17:56:39Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6058) (11.020M)   : 0, 11020144
    master (11.101M)   : 0, 11101013
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6058) (7.291M)   : 0, 7291344
    master (7.322M)   : 0, 7321945
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.648M)   : 0, 7647979

    section Manual
    master (11.159M)   : 0, 11158874

    section Manual + Automatic
    This PR (6058) (6.699M)   : 0, 6699016
    master (6.756M)   : 0, 6755889

    section DD_TRACE_ENABLED=0
    master (10.104M)   : 0, 10103630

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6058) (9.504M)   : 0, 9503984
    master (9.538M)   : 0, 9538166
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6058) (6.672M)   : 0, 6671570
    master (6.560M)   : 0, 6559669

    section Trace stats
    master (6.899M)   : 0, 6898524

    section Manual
    master (9.530M)   : 0, 9529944

    section Manual + Automatic
    This PR (6058) (6.152M)   : 0, 6151691
    master (6.111M)   : 0, 6111240

    section DD_TRACE_ENABLED=0
    master (8.837M)   : 0, 8836832

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6058) (10.141M)   : 0, 10140511
    master (10.017M)   : 0, 10017307
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6058) (6.581M)   : 0, 6580786
    master (6.715M)   : 0, 6715040
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.357M)   : 0, 7357178

    section Manual
    master (9.959M)   : 0, 9958707

    section Manual + Automatic
    This PR (6058) (6.095M)   : 0, 6094768
    master (6.139M)   : 0, 6139159

    section DD_TRACE_ENABLED=0
    master (9.384M)   : 0, 9383899

andrewlock · 2024-09-19T18:28:17Z

Benchmarks Report for tracer 🐌

Benchmarks for #6058 compared to master:

All benchmarks have the same speed
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.88μs	43.3ns	256ns	0.0162	0.00812	0	5.43 KB
master	`StartStopWithChild`	netcoreapp3.1	9.88μs	55.7ns	390ns	0.0189	0.00945	0	5.61 KB
master	`StartStopWithChild`	net472	15.8μs	52.8ns	205ns	1.02	0.3	0.103	6.07 KB
#6058	`StartStopWithChild`	net6.0	7.74μs	43.7ns	290ns	0.0157	0.00785	0	5.43 KB
#6058	`StartStopWithChild`	netcoreapp3.1	10μs	53.5ns	268ns	0.0203	0.0102	0	5.61 KB
#6058	`StartStopWithChild`	net472	16μs	52.1ns	202ns	1.01	0.292	0.0948	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	484μs	485ns	1.88μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	650μs	417ns	1.56μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	835μs	498ns	1.93μs	0.417	3.3 KB
#6058	`WriteAndFlushEnrichedTraces`	net6.0	459μs	258ns	966ns	0	2.7 KB
#6058	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	638μs	351ns	1.36μs	0	2.7 KB
#6058	`WriteAndFlushEnrichedTraces`	net472	836μs	640ns	2.39μs	0.417	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	207μs	1.21μs	11.1μs	0.202	18.45 KB
master	`SendRequest`	netcoreapp3.1	237μs	1.35μs	9.76μs	0.215	20.61 KB
master	`SendRequest`	net472	0.00304ns	0.00115ns	0.00444ns	0	0 b
#6058	`SendRequest`	net6.0	205μs	1.1μs	6.23μs	0.219	18.45 KB
#6058	`SendRequest`	netcoreapp3.1	228μs	1.22μs	7.79μs	0.223	20.61 KB
#6058	`SendRequest`	net472	0.000745ns	0.000461ns	0.00166ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	577μs	3.1μs	17.8μs	0.566	0	0	41.57 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	686μs	3.28μs	14.3μs	0.347	0	0	41.64 KB
master	`WriteAndFlushEnrichedTraces`	net472	875μs	3.4μs	12.7μs	8.09	2.25	0.45	53.33 KB
#6058	`WriteAndFlushEnrichedTraces`	net6.0	576μs	2.86μs	12.1μs	0.546	0	0	41.74 KB
#6058	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	689μs	3.61μs	18.7μs	0.332	0	0	41.76 KB
#6058	`WriteAndFlushEnrichedTraces`	net472	863μs	4.11μs	16.9μs	8.45	2.53	0.422	53.29 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.34μs	1.32ns	4.95ns	0.0142	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.71μs	1.13ns	4.39ns	0.0137	1.02 KB
master	`ExecuteNonQuery`	net472	2.13μs	2.16ns	8.36ns	0.156	987 B
#6058	`ExecuteNonQuery`	net6.0	1.33μs	0.864ns	3.35ns	0.0141	1.02 KB
#6058	`ExecuteNonQuery`	netcoreapp3.1	1.73μs	1.39ns	5.39ns	0.0131	1.02 KB
#6058	`ExecuteNonQuery`	net472	2.21μs	1.45ns	5.6ns	0.156	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.22μs	3.91ns	14.6ns	0.0139	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.53μs	0.951ns	3.56ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.46μs	1.66ns	6.41ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.17μs	1.32ns	4.93ns	0.0136	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	0.823ns	3.08ns	0.0134	1.02 KB
master	`CallElasticsearchAsync`	net472	2.55μs	1.79ns	6.95ns	0.166	1.05 KB
#6058	`CallElasticsearch`	net6.0	1.14μs	0.846ns	3.17ns	0.0137	976 B
#6058	`CallElasticsearch`	netcoreapp3.1	1.47μs	3.79ns	14.7ns	0.0132	976 B
#6058	`CallElasticsearch`	net472	2.52μs	1.69ns	6.56ns	0.157	995 B
#6058	`CallElasticsearchAsync`	net6.0	1.29μs	0.689ns	2.67ns	0.0135	952 B
#6058	`CallElasticsearchAsync`	netcoreapp3.1	1.64μs	0.866ns	3.24ns	0.0139	1.02 KB
#6058	`CallElasticsearchAsync`	net472	2.63μs	1.52ns	5.88ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.23μs	0.572ns	2.14ns	0.0134	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.61μs	2.86ns	11.1ns	0.0127	952 B
master	`ExecuteAsync`	net472	1.81μs	0.978ns	3.79ns	0.145	915 B
#6058	`ExecuteAsync`	net6.0	1.2μs	0.62ns	2.32ns	0.0133	952 B
#6058	`ExecuteAsync`	netcoreapp3.1	1.62μs	1.09ns	4.24ns	0.0129	952 B
#6058	`ExecuteAsync`	net472	1.75μs	0.544ns	2.11ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.26μs	1.23ns	4.6ns	0.0298	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.07μs	4.13ns	16ns	0.0354	2.76 KB
master	`SendAsync`	net472	7.85μs	1.34ns	5.2ns	0.497	3.15 KB
#6058	`SendAsync`	net6.0	4.24μs	1.22ns	4.72ns	0.0318	2.22 KB
#6058	`SendAsync`	netcoreapp3.1	5.14μs	3.39ns	13.1ns	0.0357	2.76 KB
#6058	`SendAsync`	net472	7.64μs	1.39ns	5.19ns	0.499	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.56μs	0.743ns	2.88ns	0.0227	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.2μs	1.15ns	4.46ns	0.022	1.64 KB
master	`EnrichedLog`	net472	2.6μs	1.54ns	5.96ns	0.249	1.57 KB
#6058	`EnrichedLog`	net6.0	1.46μs	0.518ns	1.94ns	0.0227	1.64 KB
#6058	`EnrichedLog`	netcoreapp3.1	2.35μs	0.854ns	3.2ns	0.0224	1.64 KB
#6058	`EnrichedLog`	net472	2.56μs	1.42ns	5.5ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	113μs	120ns	464ns	0.0566	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	119μs	106ns	411ns	0.0598	0	4.28 KB
master	`EnrichedLog`	net472	145μs	67.6ns	244ns	0.656	0.219	4.46 KB
#6058	`EnrichedLog`	net6.0	114μs	137ns	529ns	0.0564	0	4.28 KB
#6058	`EnrichedLog`	netcoreapp3.1	118μs	164ns	637ns	0	0	4.28 KB
#6058	`EnrichedLog`	net472	147μs	125ns	469ns	0.656	0.219	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.09μs	0.928ns	3.59ns	0.0308	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.41μs	1.9ns	7.37ns	0.0287	2.2 KB
master	`EnrichedLog`	net472	4.81μs	1.3ns	4.85ns	0.32	2.02 KB
#6058	`EnrichedLog`	net6.0	3.2μs	0.871ns	3.37ns	0.0305	2.2 KB
#6058	`EnrichedLog`	netcoreapp3.1	4.24μs	1.81ns	7.01ns	0.0294	2.2 KB
#6058	`EnrichedLog`	net472	4.94μs	0.983ns	3.81ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.36μs	0.7ns	2.62ns	0.0156	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.73μs	1.22ns	4.58ns	0.0155	0	1.14 KB
master	`SendReceive`	net472	2.17μs	1.16ns	4.32ns	0.183	0.00109	1.16 KB
#6058	`SendReceive`	net6.0	1.41μs	0.573ns	2.22ns	0.0162	0	1.14 KB
#6058	`SendReceive`	netcoreapp3.1	1.72μs	0.644ns	2.49ns	0.0155	0	1.14 KB
#6058	`SendReceive`	net472	2.11μs	1.18ns	4.26ns	0.183	0.00105	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.77μs	10.3ns	40.1ns	0.022	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.77μs	1.36ns	5.08ns	0.0209	1.65 KB
master	`EnrichedLog`	net472	4.33μs	1.67ns	6.46ns	0.323	2.04 KB
#6058	`EnrichedLog`	net6.0	2.72μs	0.886ns	3.31ns	0.0218	1.6 KB
#6058	`EnrichedLog`	netcoreapp3.1	3.8μs	1.82ns	7.06ns	0.0208	1.65 KB
#6058	`EnrichedLog`	net472	4.41μs	3.69ns	13.8ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	407ns	0.361ns	1.4ns	0.00818	576 B
master	`StartFinishSpan`	netcoreapp3.1	594ns	1.01ns	3.93ns	0.00798	576 B
master	`StartFinishSpan`	net472	654ns	1.56ns	5.84ns	0.0917	578 B
master	`StartFinishScope`	net6.0	477ns	0.235ns	0.911ns	0.00966	696 B
master	`StartFinishScope`	netcoreapp3.1	759ns	1.06ns	4.09ns	0.00938	696 B
master	`StartFinishScope`	net472	888ns	1.04ns	4.01ns	0.105	658 B
#6058	`StartFinishSpan`	net6.0	443ns	0.399ns	1.54ns	0.00803	576 B
#6058	`StartFinishSpan`	netcoreapp3.1	597ns	0.935ns	3.62ns	0.0077	576 B
#6058	`StartFinishSpan`	net472	629ns	0.617ns	2.39ns	0.0916	578 B
#6058	`StartFinishScope`	net6.0	496ns	0.261ns	1.01ns	0.00971	696 B
#6058	`StartFinishScope`	netcoreapp3.1	741ns	0.501ns	1.8ns	0.00952	696 B
#6058	`StartFinishScope`	net472	827ns	0.419ns	1.62ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	676ns	0.514ns	1.99ns	0.00988	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	982ns	0.697ns	2.61ns	0.00936	696 B
master	`RunOnMethodBegin`	net472	1.11μs	0.875ns	3.39ns	0.105	658 B
#6058	`RunOnMethodBegin`	net6.0	643ns	0.344ns	1.29ns	0.00958	696 B
#6058	`RunOnMethodBegin`	netcoreapp3.1	895ns	0.712ns	2.76ns	0.00955	696 B
#6058	`RunOnMethodBegin`	net472	1.06μs	0.813ns	3.15ns	0.104	658 B

github-actions · 2024-09-20T14:28:08Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

+  },
+  {
+    TraceId: Id_7,
+    SpanId: Id_8,
+    Name: aspnet_core.request,
+    Resource: GET /home/index,
+    Service: Samples.Security.AspNetCore5,
+    Type: web,
+    Tags: {
+      actor.ip: 86.242.244.246,
+      appsec.event: true,
+      aspnet_core.endpoint: Samples.Security.AspNetCore5.Controllers.HomeController.Index (Samples.Security.AspNetCore5),
+      aspnet_core.route: {controller=home}/{action=index}/{id?},
+      component: aspnet_core,
+      env: integration_tests,
+      http.client_ip: 127.0.0.1,
+      http.endpoint: {controller=home}/{action=index}/{id?},
+      http.method: GET,
+      http.request.headers.host: localhost:00000,
+      http.request.headers.user-agent: dd-test-scanner-log-block,
+      http.request.headers.x-forwarded-for: 86.242.244.246,
+      http.response.headers.content-type: text/html; charset=utf-8,
+      http.route: {controller=home}/{action=index}/{id?},
+      http.status_code: 200,
+      http.url: http://localhost:00000/,
+      http.useragent: dd-test-scanner-log-block,
+      language: dotnet,
+      network.client.ip: 127.0.0.1,
+      runtime-id: Guid_1,
+      span.kind: server,
+      _dd.appsec.json: {"triggers":[{"rule":{"id":"new-test-non-blocking","name":"Datadog test scanner - NON blocking version: user-agent","tags":{"category":"attack_attempt","type":"attack_tool"}},"rule_matches":[{"operator":"match_regex","operator_value":"^dd-test-scanner-log-block(?:$|/|\\s)","parameters":[{"address":"server.request.headers.no_cookies","highlight":["dd-test-scanner-log-block"],"key_path":["user-agent","0"],"value":"dd-test-scanner-log-block"}]}]}]},
+      _dd.origin: appsec,
+      _dd.runtime_family: dotnet
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.appsec.enabled: 1.0,
+      _dd.appsec.waf.duration: 0.0,
+      _dd.appsec.waf.duration_ext: 0.0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 2.0
+    },
+    MetaStruct: {
+      appsec: 
+    }
+  },
+  {
+    TraceId: Id_9,
+    SpanId: Id_10,
+    Name: aspnet_core.request,
+    Resource: GET /,
+    Service: Samples.Security.AspNetCore5,
+    Type: web,
+    Tags: {
+      actor.ip: 86.242.244.246,
+      appsec.blocked: true,
+      appsec.event: true,
+      component: aspnet_core,
+      env: integration_tests,
+      http.client_ip: 127.0.0.1,
+      http.method: GET,
+      http.request.headers.host: localhost:00000,
+      http.request.headers.user-agent: dd-test-scanner-log-block,
+      http.request.headers.x-forwarded-for: 86.242.244.246,
+      http.response.headers.content-type: application/json,
+      http.status_code: 405,
+      http.url: http://localhost:00000/,
+      http.useragent: dd-test-scanner-log-block,
+      language: dotnet,
+      network.client.ip: 127.0.0.1,
+      runtime-id: Guid_1,
+      span.kind: server,
+      _dd.appsec.json: {"triggers":[{"rule":{"id":"ua0-600-56x","name":"Datadog test scanner - blocking version: user-agent","tags":{"category":"attack_attempt","type":"attack_tool"}},"rule_matches":[{"operator":"match_regex","operator_value":"^dd-test-scanner-log-block(?:$|/|\\s)","parameters":[{"address":"server.request.headers.no_cookies","highlight":["dd-test-scanner-log-block"],"key_path":["user-agent","0"],"value":"dd-test-scanner-log-block"}]}]}]},
+      _dd.origin: appsec,
+      _dd.runtime_family: dotnet
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.appsec.enabled: 1.0,
+      _dd.appsec.waf.duration: 0.0,
+      _dd.appsec.waf.duration_ext: 0.0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 2.0
+    },
+    MetaStruct: {
+      appsec: 
+    }
+  },
+  {
+    TraceId: Id_11,
+    SpanId: Id_12,
+    Name: aspnet_core.request,
+    Resource: GET /,
+    Service: Samples.Security.AspNetCore5,
+    Type: web,
+    Tags: {
+      actor.ip: 86.242.244.246,
+      appsec.blocked: true,
+      appsec.event: true,
+      component: aspnet_core,
+      env: integration_tests,
+      http.client_ip: 127.0.0.1,
+      http.method: GET,
+      http.request.headers.host: localhost:00000,
+      http.request.headers.user-agent: dd-test-scanner-log-block,
+      http.request.headers.x-forwarded-for: 86.242.244.246,
+      http.response.headers.content-type: application/json,
+      http.status_code: 403,
+      http.url: http://localhost:00000/,
+      http.useragent: dd-test-scanner-log-block,
+      language: dotnet,
+      network.client.ip: 127.0.0.1,
+      runtime-id: Guid_1,
+      span.kind: server,
+      _dd.appsec.json: {"triggers":[{"rule":{"id":"ua0-600-56x","name":"Datadog test scanner - blocking version: user-agent","tags":{"category":"attack_attempt","type":"attack_tool"}},"rule_matches":[{"operator":"match_regex","operator_value":"^dd-test-scanner-log-block(?:$|/|\\s)","parameters":[{"address":"server.request.headers.no_cookies","highlight":["dd-test-scanner-log-block"],"key_path":["user-agent","0"],"value":"dd-test-scanner-log-block"}]}]}]},
+      _dd.origin: appsec,
+      _dd.runtime_family: dotnet
+    },
+    Metrics: {
+      process_id: 0,
+      _dd.appsec.enabled: 1.0,
+      _dd.appsec.waf.duration: 0.0,
+      _dd.appsec.waf.duration_ext: 0.0,
+      _dd.top_level: 1.0,
+      _dd.tracer_kr: 1.0,
+      _sampling_priority_v1: 2.0
+    },
+    MetaStruct: {
+      appsec: 
+    }

robertpi · 2024-09-23T11:27:41Z

tracer/src/Datadog.Trace/AppSec/Rcm/AsmDdProduct.cs

@@ -33,7 +33,7 @@ public void ProcessUpdates(ConfigurationStatus configurationStatus, List<RemoteC
                ruleSet = RuleSet.From(result.TypedFile);
            }

-            configurationStatus.RulesByFile[result.TypedFile.Path] = ruleSet;
+            configurationStatus.RulesByFile[firstFile.Path.Path] = ruleSet;


So, this is the only change necessary to fix the bug? It would be nice to see a more developed explaination of why it fixes the bug in the PR summary.

The Path of the file wasn't get from the right variable.
result.TypedFile is a JToken, getting the Path from this object would get the path of the JSON token (that is an empty string here and totally not the info we wanted to get)

anna-git

LGTM , few nits

tracer/test/Datadog.Trace.Security.IntegrationTests/AspNetBase.cs

tracer/test/Datadog.Trace.Security.IntegrationTests/Rcm/AspNetCore5AsmRemoteRules.cs

github-actions bot added the area:asm label Sep 19, 2024

e-n-0 self-assigned this Sep 19, 2024

e-n-0 force-pushed the flavien/system-tests-bug branch from c276ae9 to 5addae2 Compare September 20, 2024 14:27

e-n-0 force-pushed the flavien/system-tests-bug branch from 5addae2 to 24555a0 Compare September 23, 2024 09:08

e-n-0 marked this pull request as ready for review September 23, 2024 10:48

e-n-0 requested a review from a team as a code owner September 23, 2024 10:48

robertpi reviewed Sep 23, 2024

View reviewed changes

anna-git approved these changes Sep 23, 2024

View reviewed changes

e-n-0 added 2 commits September 24, 2024 12:02

Fix path + Add tests

5868e9d

Apply changes

a34e78f

e-n-0 force-pushed the flavien/system-tests-bug branch from f5ef3fe to a34e78f Compare September 24, 2024 10:02

e-n-0 merged commit 9dc9b77 into master Sep 24, 2024
72 of 74 checks passed

e-n-0 deleted the flavien/system-tests-bug branch September 24, 2024 15:27

github-actions bot added this to the vNext-v3 milestone Sep 24, 2024

e-n-0 mentioned this pull request Sep 30, 2024

[dotnet] Enable fixed RC tests DataDog/system-tests#3130

Merged

7 tasks

andrewlock added the type:bug label Oct 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Fix bug RC empty key #6058

[ASM] Fix bug RC empty key #6058

e-n-0 commented Sep 19, 2024 •

edited

Loading

datadog-ddstaging bot commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

Slower ⚠️ in #6058

Faster 🎉 in #6058

Raw results

Raw results

Raw results

More allocations ⚠️ in #6058

Fewer allocations 🎉 in #6058

Raw results

andrewlock commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

github-actions bot commented Sep 20, 2024 •

edited

Loading

robertpi Sep 23, 2024

e-n-0 Sep 23, 2024

anna-git left a comment

[ASM] Fix bug RC empty key #6058

[ASM] Fix bug RC empty key #6058

Conversation

e-n-0 commented Sep 19, 2024 • edited Loading

Summary of changes

Bug explanation

Test coverage

datadog-ddstaging bot commented Sep 19, 2024 • edited Loading

Datadog Report

andrewlock commented Sep 19, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Sep 19, 2024 • edited Loading

Benchmarks Report for appsec 🐌

Benchmark details

Slower ⚠️ in #6058

Faster 🎉 in #6058

Raw results

Raw results

Raw results

More allocations ⚠️ in #6058

Fewer allocations 🎉 in #6058

Raw results

andrewlock commented Sep 19, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Sep 19, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

github-actions bot commented Sep 20, 2024 • edited Loading

Snapshots difference summary

robertpi Sep 23, 2024

Choose a reason for hiding this comment

e-n-0 Sep 23, 2024

Choose a reason for hiding this comment

anna-git left a comment

Choose a reason for hiding this comment

e-n-0 commented Sep 19, 2024 •

edited

Loading

datadog-ddstaging bot commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

andrewlock commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading