Skip to content
Ivan Koveshnikov edited this page Oct 27, 2022 · 6 revisions

System under test

CPU: 2x Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
NIC: 2x Intel(R) E810-2cqda2 (100G 2 ports)
PCIe: 4th gen
Total network bandwidth: 400 Gbps
Distro: Ubuntu 20.04 LTS
Kernel: 5.17
Driver: ice-1.9.11

Traffic generators

CPU: 2x Intel(R) Xeon(R) Gold 6242R CPU @ 3.10GHz
NIC: 2x Intel(R) E810 (100G 1 of 2 ports connected)
PCIe: 3rd gen
Total network bandwidth: 200 Gbps
Distro: Ubuntu 20.04 LTS
Kernel: 5.17
Generator: Cisco T-Rex

We use 2 servers for the generation.

Traffic patterns

  • Line rate - maximum practical throughput
  • Base - base performance of XDP action, no regular expressions attached
  • Literal - lookup for a constant sub-string inside of payload. Regex: /private/s; Corpus: printable characters
  • Full payload scan - search with access to the whole payload. Regex /pri.*ate/sH; Corpus: printable characters
  • Real-life - Our real-life regular expressions. 10 regexes in parallel, backtracking, search from the payload beginning.

We have evaluated XDP_DROP and XDP_TX actions as primerally used in our network.

Results

XDP_DROP

Mpps

pkt size linerate base literal full payload scan real-life
1500 32.0 32.0 32.0 32.0 32.0
1000 48.0 48.0 48.0 48.0 48.0
512 88.0 88.0 88.0 88.0 88.0
256 176.0 162.0 165.0 108.0 161.0
128 328.0 307.0 244.0 108.0 208.0
64 568.0 430.0 257.0 211.0 220.0

Gbps

pkt size linerate base literal full payload scan real-life
1500 384.0 384.0 384.0 384.0 384.0
1000 379.0 379.0 379.0 379.0 379.0
512 368.0 349.5 349.5 349.5 349.5
256 356.0 340.0 338.0 220.0 330.0
128 346.0 314.0 250.0 110.0 213.0
64 304.0 221.0 128.0 108.0 112.0

CPU usage

pkt size linerate base literal full payload scan real-life
1500 0% 6% 14% 33% 15%
1000 0% 8% 20% 48% 22%
512 0% 16% 34% 80% 38%
256 0% 28% 68% 98% 74%
128 0% 51% 98% 98% 98%
64 0% 75% 98% 98% 99%

image HFWEU1 image BCELU1

XDP_TX

Mpps

pkt size linerate base literal full payload scan real-life
1500 32.0 32.0 32.0 32.0 32.0
1000 48.0 48.0 48.0 48.0 48.0
512 88.0 88.0 88.0 80.0 88.0
256 176.0 151.0 146.0 95.0 146.0
128 328.0 220.0 195.0 98.0 172.0
64 568.0 286.0 199.0 176.0 181.0

Gbps

pkt size linerate base literal full payload scan real-life
1500 384.0 384.0 384.0 384.0 384.0
1000 379.0 379.0 379.0 379.0 379.0
512 368.0 349.5 349.5 323.0 349.5
256 356.0 310.0 299.0 196.0 300.0
128 346.0 226.0 200.0 100.0 176.0
64 304.0 146.0 102.0 89.0 101.0

CPU usage

pkt size linerate base literal full payload scan real-life
1500 0% 10% 19% 40% 18%
1000 0% 12% 28% 56% 26%
512 0% 22% 48% 98% 45%
256 0% 36% 75% 98% 86%
128 0% 53% 98% 98% 99%
64 0% 83% 98% 98% 99%

image ETWWU1 image S1IDU1

Flame Graphs

XDP_DROP

image BQLAU1

BPF prog: 75%
Hyperscan: 28%
FPU load/store: 2.1%

XDP_TX

image MO9PU1

BPF prog: 78%
Hyperscan: 40%
FPU load/store: 1.7%