Disable connection tracking for icmpv6 traffic #158
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
icmpv6 connection tracking can cause conntrack table in kernel to grow rapidly and lead to packets being dropped, making the device unresponsive to connection requests. The signature seen when this happens is the following: "kernel: nf_conntrack: nf_conntrack: table full, dropping packet” An investigation on this identified that most of the entries in conntrack table were due to unreplied icmpv6 requests which ended up in the dying list. Snippet of "conntrack -f ipv6 -L dying " on a device with the issue: icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:3de0:f090:e3e0 type=128 code=0 id=56516 [UNREPLIED] src=2603:10b0:105:1065:0:3de0:f090:e3e0 dst=2603:10b0:105:1065::1 type=129 code=0 id=56516 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:8:9e9a:fa90:e3d0 type=128 code=0 id=49585 [UNREPLIED] src=2603:10b0:105:1065:8:9e9a:fa90:e3d0 dst=2603:10b0:105:1065::1 type=129 code=0 id=49585 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:6af9:fd90:e3cc type=128 code=0 id=50650 [UNREPLIED] src=2603:10b0:105:1065:1:6af9:fd90:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=50650 mark=0 use=2 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:21fb:f090:e3c4 type=128 code=0 id=37425 [UNREPLIED] src=2603:10b0:105:1065:1:21fb:f090:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=37425 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:8715:f690:e3cc type=128 code=0 id=49153 [UNREPLIED] src=2603:10b0:105:1065:0:8715:f690:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=49153 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:4b95:f790:e3c4 type=128 code=0 id=12475 [UNREPLIED] src=2603:10b0:105:1065:0:4b95:f790:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=12475 mark=0 use=1 icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:453e:f090:e3d4 type=128 code=0 id=57081 [UNREPLIED] src=2603:10b0:105:1065:0:453e:f090:e3d4 dst=2603:10b0:105:1065::1 type=129 code=0 id=57081 mark=0 use=1 The fix here is to use the raw table PREROUTING and OUTPUT chains in ip6tables to disable CT for icmpv6 packets as these really don't need to be tracked. SONiC has other iptables rules below to accept icmpv6 traffic even without explicit tracking -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 128 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 129 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 1 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 3 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j ACCEPT -A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j ACCEPT The change has been tested manually using ipv6 neighbors in INCOMPLETE/FAILED states and verifying that the icmpv6 connection requests do not get added to the kernel conntrack table. The change has also been validated against arp/ndp/ cacl/nat sonic-mgmt test cases as well. A sonic-mgmt test gap is also opened to automate tests for this scenario. Signed-off-by: Prabhat Aravind <paravind@microsoft.com>
prsunny
reviewed
Sep 5, 2024
|
lgtm |
Signed-off-by: Prabhat Aravind <paravind@microsoft.com>
The newer version deleted some setup files (setup.cfg and setup.py) that easy_install looks for as shown below: 2024-09-06T18:14:01.1675873Z Searching for pycairo>=1.11.1 2024-09-06T18:14:01.1681723Z Reading https://pypi.org/simple/pycairo/ 2024-09-06T18:14:01.3478202Z Downloading https://files.pythonhosted.org/packages/07/4a/42b26390181a7517718600fa7d98b951da20be982a50cd4afb3d46c2e603/pycairo-1.27.0.tar.gz#sha256=5cb21e7a00a2afcafea7f14390235be33497a2cce53a98a19389492a60628430 2024-09-06T18:14:01.3865079Z Best match: pycairo 1.27.0 2024-09-06T18:14:01.3866098Z Processing pycairo-1.27.0.tar.gz 2024-09-06T18:14:01.4422111Z error: Couldn't find a setup script in /tmp/easy_install-2shnslr6/pycairo-1.27.0.tar.gz 2024-09-06T18:14:01.5173318Z 2024-09-06T18:14:01.6908243Z ##[error]Bash exited with code '1'. 2024-09-06T18:14:01.7023768Z ##[section]Finishing: Test Python 3 The workaround for now is to use pycairo v1.26.1. Signed-off-by: Prabhat Aravind <paravind@microsoft.com>
prabhataravind
force-pushed
the
master
branch
from
6ec3776
to
033d852
Compare
|
202311 PR - #160 @StormLiangMS FYI |
prsunny
approved these changes
Sep 9, 2024
siqbal1986
approved these changes
Sep 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
icmpv6 connection tracking can cause conntrack table in kernel to grow rapidly
and lead to packets being dropped, making the device unresponsive to connection
requests. The signature seen when this happens is the following:
"kernel: nf_conntrack: nf_conntrack: table full, dropping packet”
An investigation on this identified that most of the entries in conntrack table
were due to unreplied icmpv6 requests which ended up in the dying list.
Snippet of "conntrack -f ipv6 -L dying " on a device with the issue:
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:3de0:f090:e3e0 type=128 code=0 id=56516 [UNREPLIED] src=2603:10b0:105:1065:0:3de0:f090:e3e0 dst=2603:10b0:105:1065::1 type=129 code=0 id=56516 mark=0 use=1
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:8:9e9a:fa90:e3d0 type=128 code=0 id=49585 [UNREPLIED] src=2603:10b0:105:1065:8:9e9a:fa90:e3d0 dst=2603:10b0:105:1065::1 type=129 code=0 id=49585 mark=0 use=1
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:6af9:fd90:e3cc type=128 code=0 id=50650 [UNREPLIED] src=2603:10b0:105:1065:1:6af9:fd90:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=50650 mark=0 use=2
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:1:21fb:f090:e3c4 type=128 code=0 id=37425 [UNREPLIED] src=2603:10b0:105:1065:1:21fb:f090:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=37425 mark=0 use=1
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:8715:f690:e3cc type=128 code=0 id=49153 [UNREPLIED] src=2603:10b0:105:1065:0:8715:f690:e3cc dst=2603:10b0:105:1065::1 type=129 code=0 id=49153 mark=0 use=1
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:4b95:f790:e3c4 type=128 code=0 id=12475 [UNREPLIED] src=2603:10b0:105:1065:0:4b95:f790:e3c4 dst=2603:10b0:105:1065::1 type=129 code=0 id=12475 mark=0 use=1
icmpv6 58 0 src=2603:10b0:105:1065::1 dst=2603:10b0:105:1065:0:453e:f090:e3d4 type=128 code=0 id=57081 [UNREPLIED] src=2603:10b0:105:1065:0:453e:f090:e3d4 dst=2603:10b0:105:1065::1 type=129 code=0 id=57081 mark=0 use=1
The fix here is to use the raw table PREROUTING and OUTPUT chains in
ip6tables to disable CT for icmpv6 packets as these really don't need to be
tracked. SONiC has other iptables rules below to accept icmpv6 traffic even
without explicit tracking
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 128 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 129 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 1 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 3 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 133 -j ACCEPT
-A INPUT -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j ACCEPT
The change has been tested manually using ipv6 neighbors in INCOMPLETE/FAILED
states and verifying that the icmpv6 connection requests do not get added to
the kernel conntrack table. The change has also been validated against arp/ndp/
cacl/nat sonic-mgmt test cases as well. A sonic-mgmt test gap is also opened to
automate tests for this scenario.
sonic-net/sonic-mgmt#14350