Fixes/v3 #5816

victorjulien · 2021-02-03T14:30:07Z

#5815, with format fixup.

Fix a 'skipped' transaction early in the list leading to all further transactions getting skipped, even if they were fully processed and ready to be cleaned up.

Sleep 250 microseconds instead of 100 as running in KVM cause the old value to use 100% CPU for these threads. Perf testing suggests no measurable impact for the non-KVM case. Ticket: OISF#4096

codecov · 2021-02-03T14:44:45Z

Codecov Report

Merging #5816 (cd20711) into master (62e665c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #5816   +/-   ##
=======================================
  Coverage   72.38%   72.38%           
=======================================
  Files         604      604           
  Lines      179369   179370    +1     
=======================================
+ Hits       129837   129845    +8     
+ Misses      49532    49525    -7

Flag	Coverage Δ
suricata-verify	`49.15% <100.00%> (+<0.01%)`	⬆️
unittests	`63.07% <33.33%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Ticket OISF#5816

Related to Task OISF#5816

Task OISF#5816

7a044a9 removed the lines that incremented these defrag counters, but kept the entities themselves. This commit removes counters that we judge too complex to maintain, given the current state of the code, and re-adds incrementing max_hit (memcap related). Related to Task OISF#5816

Task OISF#5816

Ticket OISF#5816

Related to Task OISF#5816

Task OISF#5816

7a044a9 removed the lines that incremented these defrag counters, but kept the entities themselves. This commit removes counters that we judge too complex to maintain, given the current state of the code, and re-adds incrementing max_hit (memcap related). Related to Task OISF#5816

Task OISF#5816

Exception policy wouldn't be applied if we were in the context of a simulated flow memcap hit. Related to Task OISF#5816

We will register stats counters for all policies, even though for now Suri only uses one possible configuration policy at a time. The idea is that this could change in the near future, so we want to have this ready. Task OISF#5816

Add stats counters for exception policies applied in case of memcap hit during stream reassembly. Task OISF#5816

Counters for exception policies applied in case a stream session memcap is hit. Task OISF#5816

Decode file needed ExceptionPolicy types and exception-policy file needed Decode types, rendering some works quite difficult to work around. ExceptionPolicyToStr is useful for registering exception policy counters, so make that public. Part of Task OISF#5816 (cherry picked from commit c2c8cdb)

We will register stats counters for all policies, even though for now Suri only uses one possible configuration policy at a time. The idea is that this could change in the near future, so we want to have this ready. Task OISF#5816 (cherry picked from commit 657419b)

Add defrag memcap stats counter. Task OISF#5816 (cherry picked from commit 485c0e1)

Add stats counters for exception policy are applied for app-layer errors Part of Task OISF#5816 (cherry picked from commit a71ace8)

Add stats counters for exception policies applied in case a stream session memcap is hit. Task OISF#5816 (cherry picked from commit 2dee377)

Add stats counters for exception policies applied in case of memcap hit during stream reassembly. Task OISF#5816 (cherry picked from commit fd9a20f)

Add stats counters for when there is an exception policy applied in case of a session picked up midstream. Task OISF#5816 (cherry picked from commit caf590d)

Some exception policies can only be applied to entire flows or individual packets, for some exception scenarios. Make this easier to read, in the documentation. Related to Task OISF#5816 (cherry picked from commit 94b1112)

Configuration options and defaults, existing counters etc. Related to Task OISF#5816 (cherry picked from commit 514e8b8)

While our documentation indicated what were the possible configuration settings for exception policies, our yaml only explicitly mentioned exception policy for the master switch. Clearly indicate which config settings are about exception policies. Related to Task OISF#5816 (cherry picked from commit 8defee9)

With the addition of exception policy stats counters, the human readable version of the sats log was mis-aligned, when counters for per-app-proto were enabled. Width change made large enough to accomodate a counter as long as "app_layer.error.bittorrent-dht.exception_policy.pass_packet" which could be valid. Task OISF#5816 (cherry picked from commit 172b55c)