Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable IPv6 on OVS internal port if needed in bridging mode #5409

Conversation

antoninbas
Copy link
Contributor

@antoninbas antoninbas commented Aug 19, 2023

Enable IPv6 on OVS internal port if needed in bridging mode

The uplink interface may have an IPv6 address, while the interface
created by OVS for the internal port may not support IPv6. For example,
such a situation has been observed in a Kind cluster, with IPv6 enabled
on the uplink but disabled by default on new interfaces:

root@kind-worker:/# sysctl net.ipv6.conf.all.disable_ipv6
net.ipv6.conf.all.disable_ipv6 = 1
root@kind-worker:/# sysctl net.ipv6.conf.default.disable_ipv6
net.ipv6.conf.default.disable_ipv6 = 1
root@kind-worker:/# sysctl net.ipv6.conf.eth0.disable_ipv6
net.ipv6.conf.eth0.disable_ipv6 = 0

When we detect that uplink addresses include an IPv6 address, we will
now ensure that IPv6 is enabled on the bridge port (using sysctl),
before attempting to move the addresses over. If it fails, we will
proceed with the rest of the initialization, but moving the IP addresses
to the bridge is very likely to be unsuccessful in that case.

We also make bridge cleanup more robust, by saving all uplink IP
addresses in the uplink config, and using the saved values to restore
the uplink interface. This ensures that cleanup can succeed, even if
bridge configuration failed half-way, as was the case in #5368.

Fixes #5368

Signed-off-by: Antonin Bas abas@vmware.com

@antoninbas antoninbas force-pushed the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch from cdef240 to 5c3d447 Compare August 19, 2023 00:51
@gran-vmv
Copy link
Contributor

Is it possible to check the "sysctl parameter" or some other flag and then move IPv6 addresses on success and log a warning on failure?
Although antrea doesn't support flexible-ipam + IPv6, we still support user to config IPv6 addresses on uplink, thus why antrea agent moves IPv6 addresses from uplink to bridge interface.

@antoninbas
Copy link
Contributor Author

@gran-vmv we can use sysctl, which is why I wrote in the comment:

Rather than try to enable IPv6 on the new interface, which requires setting a sysctl parameter, we simply skip moving the IPv6 addresses for now.

However, can you clarify why moving the IPv6 address is needed with the current code? I saw a couple of TODOs in the code for IPv6. For example, IPv6 routes are not currently handled (saved / restored):

// Skip IPv6 routes before we support IPv6 stack.
// TODO(gran): support IPv6
if route.Gw.To4() == nil {
klog.V(2).Infof("Skipped IPv6 host route: %+v", route)
continue
}

@gran-vmv
Copy link
Contributor

@gran-vmv we can use sysctl, which is why I wrote in the comment:

Rather than try to enable IPv6 on the new interface, which requires setting a sysctl parameter, we simply skip moving the IPv6 addresses for now.

However, can you clarify why moving the IPv6 address is needed with the current code? I saw a couple of TODOs in the code for IPv6. For example, IPv6 routes are not currently handled (saved / restored):

// Skip IPv6 routes before we support IPv6 stack.
// TODO(gran): support IPv6
if route.Gw.To4() == nil {
klog.V(2).Infof("Skipped IPv6 host route: %+v", route)
continue
}

@gran-vmv we can use sysctl, which is why I wrote in the comment:

Rather than try to enable IPv6 on the new interface, which requires setting a sysctl parameter, we simply skip moving the IPv6 addresses for now.

However, can you clarify why moving the IPv6 address is needed with the current code? I saw a couple of TODOs in the code for IPv6. For example, IPv6 routes are not currently handled (saved / restored):

// Skip IPv6 routes before we support IPv6 stack.
// TODO(gran): support IPv6
if route.Gw.To4() == nil {
klog.V(2).Infof("Skipped IPv6 host route: %+v", route)
continue
}

For current change, I think it is acceptable, but we might add this in release notes since this is a behavior change.

@antoninbas
Copy link
Contributor Author

antoninbas commented Aug 22, 2023

I spent more time thinking about this today. In the end, I think that it makes more sense to do the sysctl approach right away. Not moving the IPv6 address could mean an unexpected loss of IPv6 connectivity for users. I will update the PR.

@antoninbas antoninbas marked this pull request as draft August 22, 2023 05:42
@antoninbas antoninbas force-pushed the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch from 5c3d447 to 01de22d Compare August 22, 2023 18:28
@antoninbas antoninbas changed the title Do not move uplink IPv6 addresses to bridge in bridging mode Enable IPv6 on OVS internal port if needed in bridging mode Aug 22, 2023
@antoninbas antoninbas force-pushed the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch from 01de22d to fce3cbd Compare August 22, 2023 19:22
@antoninbas antoninbas marked this pull request as ready for review August 22, 2023 20:23
@antoninbas
Copy link
Contributor Author

/test-flexible-ipam-e2e

@antoninbas antoninbas force-pushed the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch from fce3cbd to d7c2d91 Compare August 22, 2023 21:06
@antoninbas
Copy link
Contributor Author

/test-flexible-ipam-e2e

1 similar comment
@gran-vmv
Copy link
Contributor

/test-flexible-ipam-e2e

@jianjuns
Copy link
Contributor

In commit message:

For example, such as situation

"such an situation"

pkg/agent/agent_linux.go Outdated Show resolved Hide resolved
The uplink interface may have an IPv6 address, while the interface
created by OVS for the internal port may not support IPv6. For example,
such a situation has been observed in a Kind cluster, with IPv6 enabled
on the uplink but disbled by default on new interfaces:

```
root@kind-worker:/# sysctl net.ipv6.conf.all.disable_ipv6
net.ipv6.conf.all.disable_ipv6 = 1
root@kind-worker:/# sysctl net.ipv6.conf.default.disable_ipv6
net.ipv6.conf.default.disable_ipv6 = 1
root@kind-worker:/# sysctl net.ipv6.conf.eth0.disable_ipv6
net.ipv6.conf.eth0.disable_ipv6 = 0
```

When we detect that uplink addresses include an IPv6 address, we will
now ensure that IPv6 is enabled on the bridge port (using sysctl),
before attempting to move the addresses over. If it fails, we will
proceed with the rest of the initialization, but moving the IP addresses
to the bridge is very likely to be unsuccessful in that case.

We also make bridge cleanup more robust, by saving all uplink IP
addresses in the uplink config, and using the saved values to restore
the uplink interface. This ensures that cleanup can succeed, even if
bridge configuration failed half-way, as was the case in antrea-io#5368.

Fixes antrea-io#5368

Signed-off-by: Antonin Bas <abas@vmware.com>
@antoninbas antoninbas force-pushed the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch from d7c2d91 to 0aa1261 Compare August 23, 2023 22:05
@antoninbas
Copy link
Contributor Author

/test-all
/test-flexible-ipam-e2e

Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@antoninbas antoninbas added area/ipam Issues or PRs related to IP address management (IPAM). kind/bug Categorizes issue or PR as related to a bug. action/backport Indicates a PR that requires backports. action/release-note Indicates a PR that should be included in release notes. labels Aug 24, 2023
@antoninbas
Copy link
Contributor Author

I will backport to 1.13

@antoninbas antoninbas merged commit d8dac45 into antrea-io:main Aug 24, 2023
@antoninbas antoninbas deleted the do-not-move-uplink-ipv6-addresses-in-bridging-mode branch August 24, 2023 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/backport Indicates a PR that requires backports. action/release-note Indicates a PR that should be included in release notes. area/ipam Issues or PRs related to IP address management (IPAM). kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Antrea agent pod restarts when IPAM is enabled on kind
4 participants