[ASM] Add IAST header tainting integration tests #4814

NachoEchevarria · 2023-11-03T16:06:29Z

Summary of changes

We already support the tainting of headers in IAST. We had some unit tests but we did not have integrations tests. This PR adds those tests.

Reason for change

To increase the quality of our testing.

Implementation details

Test coverage

Other details

datadog-ddstaging · 2023-11-03T16:29:28Z

Datadog Report

Branch report: nacho/HeaderTaintingIntegrationTests
Commit report: 4bd2b6d

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 296951 Passed, 946 Skipped, 39m 13.42s Wall Time

andrewlock · 2023-11-03T16:35:04Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (70ms)  : 64, 76
     .   : milestone, 70,
    master - mean (71ms)  : 62, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (999ms)  : 978, 1020
     .   : milestone, 999,
    master - mean (1,004ms)  : 981, 1027
     .   : milestone, 1004,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (106ms)  : 103, 109
     .   : milestone, 106,
    master - mean (106ms)  : 103, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (688ms)  : 669, 707
     .   : milestone, 688,
    master - mean (687ms)  : 671, 702
     .   : milestone, 687,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (91ms)  : 87, 94
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (656ms)  : 634, 677
     .   : milestone, 656,
    master - mean (655ms)  : 635, 674
     .   : milestone, 655,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (187ms)  : 185, 190
     .   : milestone, 187,
    master - mean (187ms)  : 185, 189
     .   : milestone, 187,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (1,134ms)  : 1116, 1152
     .   : milestone, 1134,
    master - mean (1,129ms)  : 1110, 1148
     .   : milestone, 1129,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (271ms)  : 267, 274
     .   : milestone, 271,
    master - mean (271ms)  : 266, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (1,095ms)  : 1068, 1122
     .   : milestone, 1095,
    master - mean (1,093ms)  : 1068, 1117
     .   : milestone, 1093,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4814) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (261ms)  : 256, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (4814) - mean (1,062ms)  : 1032, 1092
     .   : milestone, 1062,
    master - mean (1,065ms)  : 1037, 1093
     .   : milestone, 1065,

andrewlock · 2023-11-03T17:46:25Z

Benchmarks Report 🐌

Benchmarks for #4814 compared to master:

1 benchmarks are faster, with geometric mean 1.113
2 benchmarks are slower, with geometric mean 1.123
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.23μs	46.1ns	291ns	0.0242	0.0121	0	7.29 KB
master	`StartStopWithChild`	netcoreapp3.1	9.92μs	50.4ns	242ns	0.0249	0.00995	0	7.39 KB
master	`StartStopWithChild`	net472	15.8μs	37.4ns	145ns	1.29	0.329	0.102	7.66 KB
#4814	`StartStopWithChild`	net6.0	8.17μs	44.9ns	262ns	0.02	0.008	0	7.29 KB
#4814	`StartStopWithChild`	netcoreapp3.1	10μs	44.4ns	172ns	0.0243	0.00974	0	7.38 KB
#4814	`StartStopWithChild`	net472	15.5μs	70.2ns	272ns	1.29	0.312	0.101	7.66 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	465μs	197ns	764ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	626μs	296ns	1.15μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	806μs	310ns	1.12μs	0.401	3.3 KB
#4814	`WriteAndFlushEnrichedTraces`	net6.0	463μs	227ns	877ns	0	2.7 KB
#4814	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	629μs	249ns	966ns	0	2.7 KB
#4814	`WriteAndFlushEnrichedTraces`	net472	795μs	377ns	1.46μs	0.398	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	43.9μs	16.4ns	59ns	0.0222	0	2.03 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	46.2μs	184ns	713ns	0.0225	0	2.01 KB
master	`AllCycleSimpleBody`	net472	47.1μs	19.3ns	74.7ns	0.324	0	2.08 KB
master	`AllCycleMoreComplexBody`	net6.0	229μs	86.3ns	334ns	0.115	0	8.63 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	234μs	192ns	745ns	0	0	8.52 KB
master	`AllCycleMoreComplexBody`	net472	239μs	57.6ns	223ns	1.31	0	8.7 KB
master	`ObjectExtractorSimpleBody`	net6.0	121ns	0.076ns	0.294ns	0.00393	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	178ns	0.0703ns	0.263ns	0.00375	0	272 B
master	`ObjectExtractorSimpleBody`	net472	146ns	0.112ns	0.42ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.07μs	1.27ns	4.76ns	0.0537	0	3.88 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.12μs	2.32ns	8.99ns	0.0517	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	net472	4.16μs	2.66ns	10.3ns	0.617	0.00621	3.89 KB
#4814	`AllCycleSimpleBody`	net6.0	44μs	178ns	666ns	0.0222	0	2.03 KB
#4814	`AllCycleSimpleBody`	netcoreapp3.1	45.2μs	20.4ns	76.3ns	0.0228	0	2.01 KB
#4814	`AllCycleSimpleBody`	net472	47.4μs	99.3ns	385ns	0.31	0	2.08 KB
#4814	`AllCycleMoreComplexBody`	net6.0	229μs	80.8ns	291ns	0.115	0	8.63 KB
#4814	`AllCycleMoreComplexBody`	netcoreapp3.1	234μs	117ns	453ns	0.117	0	8.52 KB
#4814	`AllCycleMoreComplexBody`	net472	237μs	135ns	523ns	1.31	0	8.7 KB
#4814	`ObjectExtractorSimpleBody`	net6.0	120ns	0.138ns	0.515ns	0.00393	0	280 B
#4814	`ObjectExtractorSimpleBody`	netcoreapp3.1	175ns	0.0991ns	0.371ns	0.00371	0	272 B
#4814	`ObjectExtractorSimpleBody`	net472	146ns	0.0797ns	0.287ns	0.0446	0	281 B
#4814	`ObjectExtractorMoreComplexBody`	net6.0	3.04μs	1.25ns	4.69ns	0.0548	0	3.88 KB
#4814	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.28μs	1.37ns	5.14ns	0.0507	0	3.78 KB
#4814	`ObjectExtractorMoreComplexBody`	net472	4.15μs	2.38ns	8.91ns	0.617	0.00619	3.89 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`RunWaf(args=NestedMap (10))`	net6.0	12.7μs	3.2ns	12.4ns	0.133	0	9.42 KB
master	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	19.5μs	4.4ns	17.1ns	0.126	0	9.42 KB
master	`RunWaf(args=NestedMap (10))`	net472	28.7μs	16.7ns	62.3ns	1.5	0	9.48 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	75.7μs	29.8ns	115ns	0.189	0	15.77 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	85.2μs	239ns	828ns	0.211	0	15.72 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	98μs	92.8ns	360ns	2.54	0	16.04 KB
master	`RunWaf(args=NestedMap (100))`	net6.0	24μs	9.2ns	34.4ns	0.277	0	19.66 KB
master	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	38.2μs	13.2ns	49.3ns	0.268	0	20.42 KB
master	`RunWaf(args=NestedMap (100))`	net472	54.9μs	20ns	77.3ns	3.26	0.0274	20.63 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	91.6μs	181ns	702ns	0.368	0	26.01 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	109μs	175ns	654ns	0.325	0	26.72 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	135μs	51.3ns	199ns	4.26	0.0666	27.19 KB
master	`RunWaf(args=NestedMap (20))`	net6.0	23.5μs	7.08ns	27.4ns	0.269	0	19.4 KB
master	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	37.8μs	17.4ns	67.5ns	0.263	0	19.84 KB
master	`RunWaf(args=NestedMap (20))`	net472	53.8μs	18.7ns	69.9ns	3.18	0.0539	20.04 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	89.4μs	40.1ns	150ns	0.358	0	25.74 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	106μs	57ns	206ns	0.326	0	26.14 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	131μs	385ns	1.44μs	4.22	0.064	26.6 KB
#4814	`RunWaf(args=NestedMap (10))`	net6.0	12.9μs	2.65ns	9.57ns	0.134	0	9.42 KB
#4814	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	19.4μs	2.93ns	11ns	0.127	0	9.42 KB
#4814	`RunWaf(args=NestedMap (10))`	net472	28μs	17.1ns	66.4ns	1.5	0	9.48 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	77μs	53.2ns	206ns	0.196	0	15.77 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	86μs	142ns	493ns	0.212	0	15.72 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	98.4μs	31.7ns	123ns	2.5	0	16.04 KB
#4814	`RunWaf(args=NestedMap (100))`	net6.0	24.1μs	16.4ns	61.5ns	0.278	0	19.66 KB
#4814	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	38.4μs	12.1ns	47ns	0.268	0	20.42 KB
#4814	`RunWaf(args=NestedMap (100))`	net472	55.9μs	25.9ns	100ns	3.26	0.0568	20.63 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	93.4μs	151ns	565ns	0.329	0	26.01 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	108μs	51.9ns	201ns	0.326	0	26.72 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	132μs	42.1ns	146ns	4.26	0.0655	27.19 KB
#4814	`RunWaf(args=NestedMap (20))`	net6.0	24μs	27.7ns	107ns	0.274	0	19.4 KB
#4814	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	37.4μs	12.3ns	47.8ns	0.262	0	19.84 KB
#4814	`RunWaf(args=NestedMap (20))`	net472	55.1μs	19.2ns	74.2ns	3.16	0.055	20.04 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	89.7μs	72.9ns	282ns	0.358	0	25.74 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	109μs	86.8ns	313ns	0.328	0	26.14 KB
#4814	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	132μs	96.3ns	373ns	4.16	0.065	26.6 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	170μs	223ns	865ns	0.253	18.04 KB
master	`SendRequest`	netcoreapp3.1	189μs	384ns	1.49μs	0.188	20.2 KB
master	`SendRequest`	net472	0.000128ns	9.73E‑05ns	0.000364ns	0	0 b
#4814	`SendRequest`	net6.0	167μs	154ns	595ns	0.25	18.04 KB
#4814	`SendRequest`	netcoreapp3.1	192μs	336ns	1.3μs	0.191	20.2 KB
#4814	`SendRequest`	net472	0.000752ns	0.000331ns	0.00128ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #4814

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.59 KB	41.81 KB	218 B	0.52%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	540μs	1.18μs	4.56μs	0.532	0	0	41.66 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	667μs	1.05μs	4.08μs	0.332	0	0	41.59 KB
master	`WriteAndFlushEnrichedTraces`	net472	835μs	2.89μs	11.2μs	8.33	2.5	0.417	53.25 KB
#4814	`WriteAndFlushEnrichedTraces`	net6.0	538μs	278ns	964ns	0.536	0	0	41.51 KB
#4814	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	671μs	2.88μs	11.1μs	0.332	0	0	41.81 KB
#4814	`WriteAndFlushEnrichedTraces`	net472	819μs	3.75μs	14.5μs	8.12	2.44	0.406	53.24 KB

Benchmarks.Trace.DbCommandBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4814

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0	1.120	992.17	1,111.34

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	992ns	0.908ns	3.52ns	0.011	768 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.33μs	0.593ns	2.3ns	0.01	768 B
master	`ExecuteNonQuery`	net472	1.65μs	1.16ns	4.36ns	0.116	730 B
#4814	`ExecuteNonQuery`	net6.0	1.11μs	0.869ns	3.36ns	0.0105	768 B
#4814	`ExecuteNonQuery`	netcoreapp3.1	1.36μs	0.955ns	3.57ns	0.0101	768 B
#4814	`ExecuteNonQuery`	net472	1.61μs	0.952ns	3.3ns	0.116	730 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.25μs	0.568ns	2.12ns	0.0132	0	936 B
master	`CallElasticsearch`	netcoreapp3.1	1.37μs	0.584ns	2.18ns	0.0123	0	936 B
master	`CallElasticsearch`	net472	2.21μs	0.673ns	2.61ns	0.151	0.00117	955 B
master	`CallElasticsearchAsync`	net6.0	1.18μs	0.701ns	2.62ns	0.0125	0	912 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.56μs	0.52ns	1.95ns	0.0134	0	984 B
master	`CallElasticsearchAsync`	net472	2.4μs	0.498ns	1.86ns	0.161	0	1.01 KB
#4814	`CallElasticsearch`	net6.0	1.2μs	0.417ns	1.5ns	0.0132	0	936 B
#4814	`CallElasticsearch`	netcoreapp3.1	1.39μs	1.03ns	3.87ns	0.0125	0	936 B
#4814	`CallElasticsearch`	net472	2.41μs	0.821ns	2.84ns	0.151	0.00121	955 B
#4814	`CallElasticsearchAsync`	net6.0	1.26μs	0.53ns	2.05ns	0.0131	0	912 B
#4814	`CallElasticsearchAsync`	netcoreapp3.1	1.45μs	1.62ns	6.06ns	0.0131	0	984 B
#4814	`CallElasticsearchAsync`	net472	2.46μs	1.62ns	6.28ns	0.16	0	1.01 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.27μs	1.46ns	5.65ns	0.0128	912 B
master	`ExecuteAsync`	netcoreapp3.1	1.44μs	0.554ns	2.15ns	0.0121	912 B
master	`ExecuteAsync`	net472	1.73μs	1.01ns	3.92ns	0.138	875 B
#4814	`ExecuteAsync`	net6.0	1.27μs	1.09ns	4.07ns	0.0125	912 B
#4814	`ExecuteAsync`	netcoreapp3.1	1.51μs	2.45ns	9.48ns	0.012	912 B
#4814	`ExecuteAsync`	net472	1.7μs	0.609ns	2.36ns	0.138	875 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	3.71μs	1.55ns	5.99ns	0.0266	1.9 KB
master	`SendAsync`	netcoreapp3.1	4.4μs	0.999ns	3.87ns	0.033	2.43 KB
master	`SendAsync`	net472	7.01μs	2.82ns	10.9ns	0.474	2.99 KB
#4814	`SendAsync`	net6.0	3.72μs	8.31ns	32.2ns	0.0261	1.9 KB
#4814	`SendAsync`	netcoreapp3.1	4.54μs	10.8ns	41.8ns	0.0336	2.43 KB
#4814	`SendAsync`	net472	7.13μs	3.08ns	11.9ns	0.473	2.99 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.3μs	0.251ns	0.868ns	0.0221	1.57 KB
master	`EnrichedLog`	netcoreapp3.1	1.93μs	1.69ns	6.31ns	0.0212	1.57 KB
master	`EnrichedLog`	net472	2.21μs	1.35ns	5.23ns	0.236	1.49 KB
#4814	`EnrichedLog`	net6.0	1.39μs	0.845ns	3.27ns	0.0218	1.57 KB
#4814	`EnrichedLog`	netcoreapp3.1	1.91μs	1.72ns	6.44ns	0.021	1.57 KB
#4814	`EnrichedLog`	net472	2.21μs	1.37ns	5.32ns	0.236	1.49 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	112μs	102ns	353ns	0.0557	0	4.21 KB
master	`EnrichedLog`	netcoreapp3.1	117μs	120ns	465ns	0	0	4.21 KB
master	`EnrichedLog`	net472	146μs	93.7ns	363ns	0.657	0.219	4.38 KB
#4814	`EnrichedLog`	net6.0	112μs	151ns	544ns	0.0559	0	4.21 KB
#4814	`EnrichedLog`	netcoreapp3.1	118μs	436ns	1.69μs	0.0588	0	4.21 KB
#4814	`EnrichedLog`	net472	149μs	76.6ns	297ns	0.671	0.224	4.38 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.08μs	1.23ns	4.75ns	0.0299	2.13 KB
master	`EnrichedLog`	netcoreapp3.1	3.78μs	0.931ns	3.6ns	0.0284	2.13 KB
master	`EnrichedLog`	net472	4.6μs	2.25ns	8.72ns	0.307	1.93 KB
#4814	`EnrichedLog`	net6.0	2.79μs	1.05ns	3.78ns	0.0294	2.13 KB
#4814	`EnrichedLog`	netcoreapp3.1	3.76μs	4.87ns	18.9ns	0.0278	2.13 KB
#4814	`EnrichedLog`	net472	4.53μs	6.78ns	26.3ns	0.307	1.93 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.26μs	0.765ns	2.96ns	0.0157	0	1.1 KB
master	`SendReceive`	netcoreapp3.1	1.6μs	0.753ns	2.82ns	0.0145	0	1.1 KB
master	`SendReceive`	net472	2.03μs	1.31ns	4.91ns	0.177	0	1.12 KB
#4814	`SendReceive`	net6.0	1.33μs	1.44ns	5.58ns	0.0152	0	1.1 KB
#4814	`SendReceive`	netcoreapp3.1	1.51μs	1.29ns	5ns	0.0144	0	1.1 KB
#4814	`SendReceive`	net472	1.97μs	2.82ns	10.9ns	0.177	0.000982	1.12 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.54μs	0.948ns	3.55ns	0.0217	1.53 KB
master	`EnrichedLog`	netcoreapp3.1	3.61μs	0.98ns	3.53ns	0.02	1.58 KB
master	`EnrichedLog`	net472	4.15μs	1.36ns	5.26ns	0.311	1.96 KB
#4814	`EnrichedLog`	net6.0	2.45μs	0.936ns	3.5ns	0.022	1.53 KB
#4814	`EnrichedLog`	netcoreapp3.1	3.69μs	1.36ns	5.26ns	0.0203	1.58 KB
#4814	`EnrichedLog`	net472	4.03μs	1.29ns	5.01ns	0.31	1.96 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4814

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.126	393.59	443.07

Faster 🎉 in #4814

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.113	732.65	658.32

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	394ns	0.332ns	1.29ns	0.00753	536 B
master	`StartFinishSpan`	netcoreapp3.1	578ns	0.177ns	0.661ns	0.00724	536 B
master	`StartFinishSpan`	net472	733ns	0.242ns	0.936ns	0.0853	538 B
master	`StartFinishScope`	net6.0	522ns	0.28ns	1.08ns	0.00927	656 B
master	`StartFinishScope`	netcoreapp3.1	666ns	0.262ns	0.982ns	0.00889	656 B
master	`StartFinishScope`	net472	862ns	0.274ns	1.06ns	0.0979	618 B
#4814	`StartFinishSpan`	net6.0	443ns	0.173ns	0.671ns	0.00757	536 B
#4814	`StartFinishSpan`	netcoreapp3.1	581ns	0.0959ns	0.346ns	0.00736	536 B
#4814	`StartFinishSpan`	net472	658ns	0.106ns	0.41ns	0.0853	538 B
#4814	`StartFinishScope`	net6.0	472ns	0.132ns	0.494ns	0.00934	656 B
#4814	`StartFinishScope`	netcoreapp3.1	688ns	0.319ns	1.23ns	0.00899	656 B
#4814	`StartFinishScope`	net472	852ns	0.479ns	1.85ns	0.098	618 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	573ns	0.209ns	0.811ns	0.00914	656 B
master	`RunOnMethodBegin`	netcoreapp3.1	830ns	0.434ns	1.68ns	0.00885	656 B
master	`RunOnMethodBegin`	net472	992ns	0.399ns	1.55ns	0.0978	618 B
#4814	`RunOnMethodBegin`	net6.0	622ns	0.403ns	1.56ns	0.00917	656 B
#4814	`RunOnMethodBegin`	netcoreapp3.1	826ns	1.34ns	5.19ns	0.00907	656 B
#4814	`RunOnMethodBegin`	net472	971ns	0.276ns	1.07ns	0.0979	618 B

andrewlock · 2023-11-03T22:11:52Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4814) (10.781M)   : 0, 10780768
    master (11.022M)   : 0, 11021502
    benchmarks/2.38.0 (11.867M)   : 0, 11866956
    benchmarks/2.9.0 (10.921M)   : 0, 10920504

    section Automatic
    This PR (4814) (7.608M)   : 0, 7607940
    master (7.631M)   : 0, 7631258
    benchmarks/2.38.0 (8.176M)   : 0, 8175636
    benchmarks/2.9.0 (7.899M)   : 0, 7898906

    section Trace stats
    This PR (4814) (7.758M)   : 0, 7758299
    master (7.920M)   : 0, 7920297
    benchmarks/2.38.0 (8.450M)   : 0, 8450370

    section Manual
    This PR (4814) (9.466M)   : 0, 9466173
    master (9.471M)   : 0, 9470525
    benchmarks/2.38.0 (10.334M)   : 0, 10334368

    section Manual + Automatic
    This PR (4814) (7.146M)   : 0, 7146491
    master (7.041M)   : 0, 7040870
    benchmarks/2.38.0 (7.750M)   : 0, 7750484

    section Version Conflict
    This PR (4814) (6.537M)   : 0, 6537450
    master (6.494M)   : 0, 6494459
    benchmarks/2.38.0 (7.137M)   : 0, 7136691

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4814) (9.318M)   : 0, 9318011
    master (9.383M)   : 0, 9383006
    benchmarks/2.38.0 (9.548M)   : 0, 9548121
    benchmarks/2.9.0 (9.696M)   : 0, 9695973

    section Automatic
    This PR (4814) (6.715M)   : 0, 6714860
    master (6.724M)   : 0, 6723908
    benchmarks/2.38.0 (6.747M)   : 0, 6747227

    section Trace stats
    This PR (4814) (6.942M)   : 0, 6942243
    master (6.831M)   : 0, 6830837
    benchmarks/2.38.0 (6.815M)   : 0, 6814846

    section Manual
    This PR (4814) (8.468M)   : 0, 8468162
    master (8.449M)   : 0, 8448970
    benchmarks/2.38.0 (8.263M)   : 0, 8263131

    section Manual + Automatic
    This PR (4814) (6.285M)   : 0, 6285284
    master (6.324M)   : 0, 6323913
    benchmarks/2.38.0 (6.275M)   : 0, 6275411

    section Version Conflict
    This PR (4814) (5.925M)   : 0, 5925077
    master (5.887M)   : 0, 5887218
    benchmarks/2.38.0 (5.670M)   : 0, 5669744

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4814) (9.719M)   : 0, 9718830
    master (9.357M)   : 0, 9357106
    benchmarks/2.38.0 (9.007M)   : 0, 9007066
    benchmarks/2.9.0 (9.586M)   : 0, 9585650

    section Automatic
    This PR (4814) (6.764M)   : 0, 6764455
    master (6.772M)   : 0, 6772057
    benchmarks/2.38.0 (6.423M)   : 0, 6422756
    benchmarks/2.9.0 (7.033M)   : 0, 7033199

    section Trace stats
    This PR (4814) (7.071M)   : 0, 7070818
    master (6.990M)   : 0, 6989713
    benchmarks/2.38.0 (6.721M)   : 0, 6720838

    section Manual
    This PR (4814) (8.580M)   : 0, 8579848
    master (8.431M)   : 0, 8430779
    benchmarks/2.38.0 (7.977M)   : 0, 7977494

    section Manual + Automatic
    This PR (4814) (6.543M)   : 0, 6543409
    master (6.422M)   : 0, 6422429
    benchmarks/2.38.0 (6.349M)   : 0, 6349066

    section Version Conflict
    This PR (4814) (6.035M)   : 0, 6035215
    master (6.082M)   : 0, 6082367
    benchmarks/2.38.0 (5.682M)   : 0, 5681654

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4814) (7.664M)   : 0, 7663838
    master (7.554M)   : 0, 7554137
    benchmarks/2.38.0 (7.578M)   : 0, 7577544
    benchmarks/2.9.0 (8.040M)   : 0, 8039615

    section No attack
    This PR (4814) (2.183M)   : 0, 2182790
    master (2.165M)   : 0, 2165372
    benchmarks/2.38.0 (2.184M)   : 0, 2184279
    benchmarks/2.9.0 (3.290M)   : 0, 3289982

    section Attack
    This PR (4814) (1.732M)   : 0, 1731716
    master (1.737M)   : 0, 1736755
    benchmarks/2.38.0 (1.711M)   : 0, 1710542
    benchmarks/2.9.0 (2.581M)   : 0, 2581167

    section Blocking
    This PR (4814) (3.482M)   : 0, 3482148
    master (3.478M)   : 0, 3478004
    benchmarks/2.38.0 (3.486M)   : 0, 3486124

    section IAST default
    This PR (4814) (6.891M)   : 0, 6891064
    master (6.937M)   : 0, 6937357

    section IAST full
    This PR (4814) (6.167M)   : 0, 6166521
    master (6.255M)   : 0, 6255084

    section Base vuln
    This PR (4814) (0.966M)   : crit ,0, 965962
    master (1.024M)   : 0, 1023940

    section IAST vuln
    This PR (4814) (0.943M)   : 0, 942520
    master (0.910M)   : 0, 909507

tracer/test/Datadog.Trace.Security.IntegrationTests/AspNetBase.cs

tracer/test/Datadog.Trace.Security.IntegrationTests/IAST/AspNetCore5IastTests.cs

tracer/test/Datadog.Trace.Security.IntegrationTests/IAST/AspNetMvc5IastTests.cs

tracer/test/snapshots/Iast.ExecuteCommandFromHeader.AspNetMvc5.IastEnabled.verified.txt

…tCore5IastTests.cs Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Add tests

e4b4ba3

github-actions bot added the area:tests unit tests, integration tests label Nov 3, 2023

NachoEchevarria changed the title ~~Add tests~~ [ASM] Add header tainting integration tests Nov 3, 2023

NachoEchevarria added area:asm area:asm-iast labels Nov 3, 2023

NachoEchevarria changed the title ~~[ASM] Add header tainting integration tests~~ [ASM] Add IAST header tainting integration tests Nov 3, 2023

NachoEchevarria marked this pull request as ready for review November 6, 2023 10:24

NachoEchevarria requested a review from a team as a code owner November 6, 2023 10:24

daniel-romano-DD approved these changes Nov 6, 2023

View reviewed changes

andrewlock approved these changes Nov 6, 2023

View reviewed changes

NachoEchevarria and others added 3 commits November 6, 2023 15:53

Update tracer/test/Datadog.Trace.Security.IntegrationTests/IAST/AspNe…

4ff37e4

…tCore5IastTests.cs Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

Update tracer/test/Datadog.Trace.Security.IntegrationTests/AspNetBase.cs

e066f56

Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>

remove debug and fixes

4bd2b6d

robertpi approved these changes Nov 7, 2023

View reviewed changes

NachoEchevarria merged commit 8011ea1 into master Nov 7, 2023
50 of 54 checks passed

NachoEchevarria deleted the nacho/HeaderTaintingIntegrationTests branch November 7, 2023 15:42

github-actions bot added this to the vNext milestone Nov 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Add IAST header tainting integration tests #4814

[ASM] Add IAST header tainting integration tests #4814

NachoEchevarria commented Nov 3, 2023 •

edited

Loading

datadog-ddstaging bot commented Nov 3, 2023 •

edited

Loading

andrewlock commented Nov 3, 2023

andrewlock commented Nov 3, 2023 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #4814

Raw results

Slower ⚠️ in #4814

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4814

Faster 🎉 in #4814

Raw results

Raw results

andrewlock commented Nov 3, 2023

[ASM] Add IAST header tainting integration tests #4814

[ASM] Add IAST header tainting integration tests #4814

Conversation

NachoEchevarria commented Nov 3, 2023 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Nov 3, 2023 • edited Loading

Datadog Report

andrewlock commented Nov 3, 2023

Execution-Time Benchmarks Report ⏱️

andrewlock commented Nov 3, 2023 • edited Loading

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #4814

Raw results

Slower ⚠️ in #4814

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #4814

Faster 🎉 in #4814

Raw results

Raw results

andrewlock commented Nov 3, 2023

Throughput/Crank Report:zap:

NachoEchevarria commented Nov 3, 2023 •

edited

Loading

datadog-ddstaging bot commented Nov 3, 2023 •

edited

Loading

andrewlock commented Nov 3, 2023 •

edited

Loading