[ASM] Update WAF v1.21.0 Ruleset 1.13.3 #6287

NachoEchevarria · 2024-11-14T14:18:30Z

Summary of changes

This PR updates the WAF to its latest version:
https://github.com/DataDog/libddwaf/releases/tag/1.21.0

The default ruleset has been also updated to version 1.13.3
https://github.com/DataDog/appsec-event-rules/releases/tag/1.13.3

Reason for change

Implementation details

Test coverage

Other details

github-actions · 2024-11-14T14:19:25Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

1 occurrences of :

-        "_dd.appsec.event_rules.version": "1.13.2",
-        "_dd.appsec.waf.version": "1.20.1",
+        "_dd.appsec.event_rules.version": "1.13.3",
+        "_dd.appsec.waf.version": "1.21.0",

2 occurrences of :

-        "_dd.appsec.event_rules.loaded": 158.0,
+        "_dd.appsec.event_rules.loaded": 159.0,

1 occurrences of :

-        "_dd.appsec.waf.version": "1.20.1",
[...]
+        "_dd.appsec.waf.version": "1.21.0",

26 occurrences of :

-      _dd.appsec.event_rules.version: 1.13.2,
+      _dd.appsec.event_rules.version: 1.13.3,

1 occurrences of :

-      _dd.appsec.event_rules.version: 1.13.2,
+      _dd.appsec.event_rules.version: 1.13.3,
[...]
-      _dd.appsec.waf.version: 1.20.1,
+      _dd.appsec.waf.version: 1.21.0,

1 occurrences of :

-      _dd.appsec.event_rules.loaded: 158.0,
+      _dd.appsec.event_rules.loaded: 159.0,

1 occurrences of :

-      _dd.appsec.waf.version: 1.20.1,
+      _dd.appsec.waf.version: 1.21.0,

6 occurrences of :

-      _dd.appsec.event_rules.version: 1.13.2,
-      _dd.appsec.waf.version: 1.20.1,
+      _dd.appsec.event_rules.version: 1.13.3,
+      _dd.appsec.waf.version: 1.21.0,
[...]
-      _dd.appsec.event_rules.loaded: 158.0,
+      _dd.appsec.event_rules.loaded: 159.0,

datadog-ddstaging · 2024-11-14T14:42:08Z

Datadog Report

Branch report: nacho/UpdateWAFv1.21.0
Commit report: 4f0aa4d
Test service: dd-trace-dotnet

❌ 6 Failed (0 Known Flaky), 449598 Passed, 2749 Skipped, 19h 41m 44.32s Total Time

❌ Failed Tests (6)

This report shows up to 5 failed tests.

AllCycleMoreComplexBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark - Details
AllCycleMoreComplexBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark - Details
AllCycleMoreComplexBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark - Details
AllCycleSimpleBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark - Details
AllCycleSimpleBody - Benchmarks.Trace.Asm.AppSecBodyBenchmark - Details

andrewlock · 2024-11-14T14:49:48Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (72ms)  : 64, 79
     .   : milestone, 72,
    master - mean (72ms)  : 63, 81
     .   : milestone, 72,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (1,106ms)  : 1086, 1125
     .   : milestone, 1106,
    master - mean (1,107ms)  : 1082, 1132
     .   : milestone, 1107,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (108ms)  : 106, 110
     .   : milestone, 108,
    master - mean (108ms)  : 105, 111
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (767ms)  : 748, 786
     .   : milestone, 767,
    master - mean (770ms)  : 753, 786
     .   : milestone, 770,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (92ms)  : 90, 93
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (719ms)  : 704, 734
     .   : milestone, 719,
    master - mean (725ms)  : 709, 741
     .   : milestone, 725,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (191ms)  : 187, 195
     .   : milestone, 191,
    master - mean (191ms)  : 185, 196
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (1,210ms)  : 1186, 1235
     .   : milestone, 1210,
    master - mean (1,213ms)  : 1190, 1237
     .   : milestone, 1213,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (276ms)  : 270, 281
     .   : milestone, 276,
    master - mean (276ms)  : 271, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (939ms)  : 924, 954
     .   : milestone, 939,
    master - mean (945ms)  : 929, 960
     .   : milestone, 945,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6287) - mean (265ms)  : 261, 269
     .   : milestone, 265,
    master - mean (265ms)  : 260, 270
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6287) - mean (922ms)  : 904, 940
     .   : milestone, 922,
    master - mean (930ms)  : 907, 952
     .   : milestone, 930,

andrewlock · 2024-11-14T20:24:07Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6287) (11.127M)   : 0, 11126944
    master (11.207M)   : 0, 11207454
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6287) (7.213M)   : 0, 7212815
    master (7.213M)   : 0, 7213054
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.646M)   : 0, 7646289

    section Manual
    master (11.210M)   : 0, 11209987

    section Manual + Automatic
    This PR (6287) (6.622M)   : 0, 6622432
    master (6.787M)   : 0, 6786812

    section DD_TRACE_ENABLED=0
    master (10.192M)   : 0, 10192322

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6287) (9.807M)   : 0, 9806912
    master (9.557M)   : 0, 9557430
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6287) (6.444M)   : 0, 6443828
    master (6.335M)   : 0, 6335118

    section Trace stats
    master (6.713M)   : 0, 6713489

    section Manual
    master (9.597M)   : 0, 9597235

    section Manual + Automatic
    This PR (6287) (6.102M)   : 0, 6102029
    master (5.989M)   : 0, 5988836

    section DD_TRACE_ENABLED=0
    master (8.804M)   : 0, 8803524

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6287) (9.514M)   : 0, 9513869
    master (9.893M)   : 0, 9892862
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6287) (6.196M)   : 0, 6195930
    master (6.418M)   : 0, 6418372
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.033M)   : 0, 7032568

    section Manual
    master (9.706M)   : 0, 9706262

    section Manual + Automatic
    This PR (6287) (5.755M)   : 0, 5754844
    master (5.752M)   : 0, 5752126

    section DD_TRACE_ENABLED=0
    master (9.107M)   : 0, 9107420

NachoEchevarria added 2 commits November 14, 2024 12:54

Update WAF version

b1c4112

Remove not needed

272aff4

github-actions bot added the area:asm label Nov 14, 2024

Update ruleset

43098bd

NachoEchevarria changed the title ~~[ASM] Update WAF v1.21.0~~ [ASM] Update WAF v1.21.0 Ruleset 1.13.3 Nov 15, 2024

NachoEchevarria added 3 commits November 15, 2024 15:31

Fix rules number in smoke tests. Remove not needed test.

1a0a0ae

Fix unit test

21e2390

Update rule number in snapshots

4f0aa4d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Update WAF v1.21.0 Ruleset 1.13.3 #6287

[ASM] Update WAF v1.21.0 Ruleset 1.13.3 #6287

NachoEchevarria commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

datadog-ddstaging bot commented Nov 14, 2024 •

edited

Loading

andrewlock commented Nov 14, 2024 •

edited

Loading

andrewlock commented Nov 14, 2024 •

edited

Loading

[ASM] Update WAF v1.21.0 Ruleset 1.13.3 #6287

Are you sure you want to change the base?

[ASM] Update WAF v1.21.0 Ruleset 1.13.3 #6287

Conversation

NachoEchevarria commented Nov 14, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

github-actions bot commented Nov 14, 2024 • edited Loading

Snapshots difference summary

datadog-ddstaging bot commented Nov 14, 2024 • edited Loading

Datadog Report

❌ Failed Tests (6)

andrewlock commented Nov 14, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Nov 14, 2024 • edited Loading

Throughput/Crank Report ⚡

NachoEchevarria commented Nov 14, 2024 •

edited

Loading

github-actions bot commented Nov 14, 2024 •

edited

Loading

datadog-ddstaging bot commented Nov 14, 2024 •

edited

Loading

andrewlock commented Nov 14, 2024 •

edited

Loading

andrewlock commented Nov 14, 2024 •

edited

Loading