Allow to configure custom network adapter labels

EVE already offers adapter labels "uplink" and "freeuplink" to match
a specific group of ports to provide external connectivity for a network
instance.

We should allow users to define and assign their own adapter labels and
support the following new network instance use-cases:

1. Switch network instance with multiple ports: user should be able to
   select a group of ports, all of which will be bridged together under
   one switch NI. EVE should then automatically run the STP protocol for
   the bridge to avoid bridge loops and broadcast storms.
2. Local network instance with multiple ports: the routing table on the
   host (EVE) side will contain routes from all selected ports and will
   be used to select output port for the given flow.
   For the default route we should:
     a) allow the user to pick the "default output interface"
     b) periodically test port connectivity and failover between default
        gateways when currently used one has broken connectivity
   a) and b) could be combined together to allow the user to select a
   subset of NI ports for the default route and then let EVE to pick one
   of them based on the probing results.
   We should also allow to limit a port-forwarding rule to only a subset
   of ports used by a local network instance. This can be done by
   defining a new ACEMatch type "adapter" where value is adapter label
   to match.

Currently, EVE supports a variant of the second use-case for hard-coded
"uplink" and "freeuplink" labels. It would make sense to unify the behaviour
between these existing labels and the user-defined ones. This means that
we would make small change to the current implementation:
when "uplink" or "freeuplink" is selected for a local network instance,
the NI routing table will have IP routes from *all* matching (mgmt)
ports and the default route will be selected by connectivity testing.
This is different to the present behaviour, where EVE installs only
routes of the port selected by connectivity testing into the NI routing
table. But lets say that there are 2 mgmt ports: eth0 and eth1, and
connectivity testing selected eth0. Then even if app is accessing hosts
which are directly inside the eth1 subnet, traffic will be routed via eth0,
which can be confusing. Also, user-defined static routes for eth1 are
not present in the routing table and therefore inactive when eth0 is
selected. The idea is to apply connectivity testing to only default
route selection.

Signed-off-by: Milan Lenco <milan@zededa.com>

proto/config/devmodel.proto

@@ -75,6 +75,16 @@ message SystemAdapter {
   // Load spreading will apply when multiple adapters have the same cost.
   // Higher cost adapters are only tried when none of the lower cost ones work.
   uint32 cost = 9;
+
+  // A set of user-defined labels attached to the adapter.
+  // There are no restrictions on the format of an adapter label, it can be any
+  // non-empty string.
+  // Note that EVE automatically assigns "all" label to every port, "uplink" label to every
+  // management port and "freeuplink" label to every management port with zero cost.
+  // It can be used to designate ad group of network adapters to be used by a network
+  // instances for external connectivity, or to limit port-forwarding firewall rule
+  // to only a subset of network adapters.
+  repeated string labels = 10;
 }
 
 // Given additional details for EVE software to how to treat this

proto/config/fw.proto

@@ -9,7 +9,46 @@ option java_package = "org.lfedge.eve.config";
 
 
 message ACEMatch {
-  // FIXME: We should convert this to enum
+  // Supported ACE match types:
+  //   * "ip": value should be an IP address of a remote endpoint. The match is satisfied
+  //           for outbound and inbound flow if the destination and the source IP address
+  //           matches the given value, respectively. Can be combined with any other match
+  //           type to further narrow down the selection criteria.
+  //   * "host": value should be a domain name of a remote endpoint. It can be either a fully
+  //             qualified, or a partially qualified domain name (FQDN or PQDN). A packet is
+  //             matched if it is destined to or originated from an IP address that was obtained
+  //             by a DNS query for that exact domain or any of its subdomains. For example,
+  //             match of type "host" with value "domain.com" will also apply to the endpoint
+  //             "subdomain.domain.com". Can be combined with other match types except for "eidset".
+  //   * "eidset": special match type for the overlay network. Matches IPs of all applications
+  //               deployed in the same network as well as all IPs with statically configured
+  //               DNS entries (under the config field NetworkInstanceConfig.Dns). For this type,
+  //               value field is not used. Can be combined with other match types except for
+  //               "host".
+  //   * "protocol": value should specify the protocol to match. Protocol can be one of "tcp",
+  //                 "udp", "icmp", or "all", or it can be a numeric value, representing one
+  //                 of these protocols or a different one. A protocol name from /etc/protocols
+  //                 is also allowed. Protocol match can be combined with any other match type
+  //                 (often combined with port numbers).
+  //   * "lport": value should be an application local port number. For filtering actions,
+  //              this is the source port for outbound traffic and destination port for inbound
+  //              traffic. For PORTMAP action, this represents application port as exposed
+  //              to the external network (i.e., if <edge-node-ip>:2222 is mapped to <app-ip>:22,
+  //              lport refers to 2222). lport can be combined with any other match type.
+  //              It is actually required to combine "lport" and "protocol" inside the same ACE.
+  //              In other words, port without protocol is not valid.
+  //   * "fport": value should be a remote endpoint port number (foreign port). Used for filtering
+  //              actions, but not for PORTMAP (do not confuse with "lport", which is still used to
+  //              represent the forwarded port - the forwarded port is still considered as local).
+  //              "fport" can be combined with any other match type. It is actually required
+  //              to combine "fport" with "protocol" inside the same ACE. In other words, port
+  //              without protocol is not valid.
+  //  * "adapter": value should be an adapter label (SystemAdapter.Labels). It can be used for
+  //               an inbound ACE to apply the rule only to packets arriving via one of the
+  //               matched network adapters. Typically used to activate a given port-forwarding
+  //               rule (PORTMAP) for only a subset of network adapters.
+  //               Adapter label cannot be used for outbound ACE. This is because the EVE firewall
+  //               is applied before routing, and the output network adapter is not yet known.
   string type = 1;
   string value = 2;
 }
@@ -39,6 +78,7 @@ message ACE {
   //  for example
   //     1) host=www.example.com & port=http
   //     2) ip=8.8.8.8 & port=53 & proto=UDP
+  //     3) adapter=uplink && port=8080 && proto=TCP
   repeated ACEMatch matches = 1;
 
   // Expect only single action...repeated here is

proto/config/netconfig.proto

@@ -88,6 +88,7 @@ message CellularConfig {
   uint32 activated_sim_slot = 5;
 }
 
+// TODO: replace with ProbeMethod?
 // CellularConnectivityProbe is used to periodically check the connectivity status of a cellular network
 // by probing a remote endpoint.
 // Whenever the probe fails, the cellular connection is automatically restarted. If the probe keeps failing

proto/config/netinst.proto

@@ -85,6 +85,56 @@ message IPRoute {
   string gateway = 2;
 }
 
+// Method to use to determine the port connectivity status.
+enum ProbeMethod {
+  // Port connectivity probing is disabled.
+  PROBE_METHOD_DISABLED = 0;
+  // Use ICMP ping against the probed endpoint to determine the status of the port
+  // connectivity.
+  PROBE_METHOD_ICMP = 1;
+  // Try to establish TCP connection with the probed endpoint to determine the status
+  // of the port connectivity.
+  PROBE_METHOD_TCP = 2;
+}
+
+// Configuration for port connectivity probing.
+message PortProbe {
+  // Method to use to determine the port connectivity status.
+  ProbeMethod probe_method = 1;
+  // IP or FQDN to probe using the selected probing mechanism to determine the connectivity
+  // status of the port.
+  // For PROBE_METHOD_TCP the address should include port number in one of the formats:
+  //   ipv4:port
+  //   [ipv6]:port
+  //   hostname:port
+  string probe_address = 2;
+}
+
+// DefaultRouteSelection defines how local network instance picks the default route
+// when it has multiple ports assigned.
+message DefaultRouteSelection {
+  // Either a single port referenced by its name (SystemAdapter.Name) or an adapter label
+  // matching zero or more NI ports (NI ports containing this label in SystemAdapter.Labels).
+  // Empty default_port matches all NI ports.
+  // Currently, load-balancing is not supported and thus even if default_port matches multiple
+  // NI ports, EVE will pick only one of them for the default route. The selection is based on
+  // probing (if enabled, see below) or random (preferring the currently used one if any).
+  string default_port = 1;
+  // Probe remote endpoint to determine connectivity status of each port and pick one
+  // with a working connectivity (and known gateway IP) for the default route (preferring
+  // the currently used one if any).
+  // Provides automatic fail-over between ports.
+  // Probes NI ports, potentially further filtered down by default_port label (see above).
+  // If default_port references single port (e.g. directly by its name), probing is skipped
+  // (nothing to fail-over to anyway).
+  PortProbe probe = 2;
+  // TODO: load-balancing config for when multiple ports match the criteria for the default
+  //       route selection (all matching default_port and have working connectivity or probing
+  //       is disabled).
+  //       For now we will assume that load-balancing is disabled and EVE will always pick
+  //       one port for the default route.
+}
+
 message NetworkInstanceConfig {
   UUIDandVersion uuidandversion = 1;
   string displayname = 2;
@@ -97,14 +147,30 @@ message NetworkInstanceConfig {
   //    itself is not created on the device.
   bool activate = 5;
 
-  // port - Only a single port is supported.
-  //    This is used as the external connection for the network instance.
-  //    This can be a physical (eth0 ) or logical port (vlan 0).
-  //    The port name comes from DeviceConfig ( When it is supported in future).
-  //    If the user needs multiple physical ports, Device config should be
-  //    used to create a label for multiple physical ports.
+  // Port(s) providing external connectivity for the network instance.
+  // Either a single port referenced by its name (SystemAdapter.Name) or an adapter label
+  // matching zero or more network ports (ports containing this label in SystemAdapter.Labels).
+  // Note that apart from the user-defined adapter labels, EVE automatically assigns "all"
+  // label to every port, "uplink" label to every management port and "freeuplink" label
+  // to every management port with zero cost.
+  // Both physical (e.g. eth0) and logical (e.g. vlan0) ports are allowed.
+  // Network instance with no assigned port (empty port reference or label not matching
+  // any port) is air-gapped, i.e. completely isolated from external networks.
+  // With multiple ports assigned, the behaviour depends on the network instance type.
+  // Switch network instance will run Spanning Tree Protocol (STP) to avoid bridge loops
+  // and the broadcast storm that results from them.
+  // Local network instance will apply IP routes configured for the matched ports (either
+  // statically or via DHCP) to select the next hop and the output port for the given flow.
+  // Traffic not matching any link-local, statically-configured, or DHCP-received route,
+  // will be routed according to the default route, which is selected based on the
+  // defaultRouteSelection algorithm (see below).
   Adapter port = 20;
 
+  // DefaultRouteSelection defines how local network instance picks the default route
+  // when it has multiple ports assigned.
+  // Not used with switch network instance.
+  DefaultRouteSelection default_route_selection = 21;
+
   // cfg - Used to pass some feature-specific configuration to the
   //       network instance. For Ex: Lisp, StriongSwan etc
   NetworkInstanceOpaqueConfig cfg = 30;