Support bridging and VLAN network for Pod secondary interfaces

[jianjuns]

Describe what you are trying to solve
For Pod secondary interfaces, an important use case is to connect them to a separate network through a separate physical interface on the host, and even connect them to VLANs. Antrea now already has "alpha" support of SR-IOV secondary network, and naturally we can extend the secondary network support to bridging and VLAN networks.

Describe the solution you have in mind
As Antrea uses OVS for Pod networking, it will not be hard to connect secondary interfaces to a separate OVS bridge with one or multiple physical interfaces, and VLAN tagging can also be configured on the OVS ports to connect secondary interfaces to VLANs.

In antrea-agent.conf, we can add OVS bridge configuration for users to define the OVS bridge(s) for secondary networks. The OVS bridges will be automatically created and configured by antrea-agent. We may start from supporting only one bridge and one physical interface on the bridge, but for future extensions we can define a list of bridges each can be configured with a list of physical interfaces.

secondaryNetwork:
  ovsBridges: [{"bridgeName": "br2", "physicalInterfaces": ["eth1"]}]

Like SR-IOV network, we will still leverage network-attachment-definitions.k8s.cni.cncf.io for users to configure a bridging/VLAN network for Pods. For example:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: virtual-net1
spec:
  config: '{
      "cniVersion": "0.3.0",
      "type": "antrea",
      "networkType": "vlan",
      "mtu": 9000,
      "vlan": 100,
      "ipam": {
        "type": "antrea-ipam",
        "ippool": "vlan100-subnet"
      }
    }'

vlanID defaults to 0 if not specified. The first OVS bridge in the secondary network configuration will be used, if bridge is not specified.

[jianjuns]

@meibensteiner

[meibensteiner]

Amazing. That would absolutely fullfill my usecase.
Just one question since the edge nodes I plan to use this feature on only have one 2.5g interface:

...a separate network through a separate physical interface on the host, and...

Will two physical interfaces be required for this to work?

[jianjuns]

@meibensteiner Good to know the proposal works for your use case.

If the Node has only one NIC, we need to attach the NIC to secondary OVS bridge. It should work if we probably copy all IP/route settings from the NIC to the bridge. In your case, do you have any special network configuration for the NIC interface, or any network manager (e.g. RHEL network manager) manages its configuration?

[meibensteiner]

In the current phase its just networkd and netplan.