Allow to configure custom network adapter labels

EVE already offers adapter labels "uplink" and "freeuplink" to match
a specific group of ports to provide external connectivity for a network
instance.

We should allow users to define and assign their own adapter labels and
support the following new network instance use-cases:

1. Switch network instance with multiple ports: user should be able to
   select a group of ports, all of which will be bridged together under
   one switch NI. EVE should then automatically run the STP protocol for
   the bridge to avoid bridge loops and broadcast storms.
2. Local network instance with multiple ports: the routing table on the
   host (EVE) side will contain routes from all selected ports and will
   be used to select output port for the given flow. For the default route
   we should allow to select between:
     a) periodically test port connectivity and failover between default
        gateways when currently used one has broken connectivity; or
     b) let the user to select the port to use by default using costs.
        Gateway of the lowest-cost port will be used by default.
        With multiple lowest-cost ports, EVE could load-balance flows
        between these ports.
   We should also allow to limit a port-forwarding rule to only a subset
   of ports used by a local network instance. This can be done by
   defining a new ACEMatch type "adapter" where value is adapter label
   to match.

Currently, EVE supports a variant of the second use-case for hard-coded
"uplink" and "freeuplink" labels. It would make sense to unify the behaviour
between these existing labels and the user-defined ones under the use-case
2a. This means that we would make small change to the current implementation:
when "uplink" or "freeuplink" is selected for a local network instance,
the NI routing table will have IP routes from *all* matching (mgmt)
ports and the default route will be selected by connectivity testing.
This is different to the present behaviour, where EVE installs only
routes of the port selected by connectivity testing into the NI routing
table. But lets say that there are 2 mgmt ports: eth0 and eth1, and
connectivity testing selected eth0. Then even if app is accessing hosts
which are directly inside the eth1 subnet, traffic will be routed via eth0,
which can be confusing. Also, user-defined static routes for eth1 are
not present in the routing table and therefore inactive when eth0 is
selected. The idea is to apply connectivity testing to only default
route selection.

Signed-off-by: Milan Lenco <milan@zededa.com>

proto/config/devmodel.proto

@@ -75,6 +75,16 @@ message SystemAdapter {
   // Load spreading will apply when multiple adapters have the same cost.
   // Higher cost adapters are only tried when none of the lower cost ones work.
   uint32 cost = 9;
+
+  // A set of user-defined labels attached to the adapter.
+  // There are no restrictions on the format of an adapter label, it can be any
+  // non-empty string.
+  // Note that EVE automatically assigns "all" label to every port, "uplink" label to every
+  // management port and "freeuplink" label to every management port with zero cost.
+  // It can be used to designate ad group of network adapters to be used by a network
+  // instances for external connectivity, or to limit port-forwarding firewall rule
+  // to only a subset of network adapters.
+  repeated string labels = 10;
 }
 
 // Given additional details for EVE software to how to treat this

proto/config/fw.proto

@@ -10,6 +10,46 @@ option java_package = "org.lfedge.eve.config";
 
 message ACEMatch {
   // FIXME: We should convert this to enum
+  // Supported ACE match types:
+  //   * "ip": value should be an IP address of a remote endpoint. The match is satisfied
+  //           for outbound and inbound flow if the destination and the source IP address
+  //           matches the given value, respectively. Can be combined with any other match
+  //           type to further narrow down the selection criteria.
+  //   * "host": value should be a domain name of a remote endpoint. It can be either a fully
+  //             qualified, or a partially qualified domain name (FQDN or PQDN). A packet is
+  //             matched if it is destined to or originated from an IP address that was obtained
+  //             by a DNS query for that exact domain or any of its subdomains. For example,
+  //             match of type "host" with value "domain.com" will also apply to the endpoint
+  //             "subdomain.domain.com". Can be combined with other match types except for "eidset".
+  //   * "eidset": special match type for the overlay network. Matches IPs of all applications
+  //               deployed in the same network as well as all IPs with statically configured
+  //               DNS entries (under the config field NetworkInstanceConfig.Dns). For this type,
+  //               value field is not used. Can be combined with other match types except for
+  //               "host".
+  //   * "protocol": value should specify the protocol to match. Protocol can be one of "tcp",
+  //                 "udp", "icmp", or "all", or it can be a numeric value, representing one
+  //                 of these protocols or a different one. A protocol name from /etc/protocols
+  //                 is also allowed. Protocol match can be combined with any other match type
+  //                 (often combined with port numbers).
+  //   * "lport": value should be an application local port number. For filtering actions,
+  //              this is the source port for outbound traffic and destination port for inbound
+  //              traffic. For PORTMAP action, this represents application port as exposed
+  //              to the external network (i.e., if <edge-node-ip>:2222 is mapped to <app-ip>:22,
+  //              lport refers to 2222). lport can be combined with any other match type.
+  //              It is actually required to combine "lport" and "protocol" inside the same ACE.
+  //              In other words, port without protocol is not valid.
+  //   * "fport": value should be a remote endpoint port number (foreign port). Used for filtering
+  //              actions, but not for PORTMAP (do not confuse with "lport", which is still used to
+  //              represent the forwarded port - the forwarded port is still considered as local).
+  //              "fport" can be combined with any other match type. It is actually required
+  //              to combine "fport" with "protocol" inside the same ACE. In other words, port
+  //              without protocol is not valid.
+  //  * "adapter": value should be an adapter label (SystemAdapter.Labels). It can be used for
+  //               an inbound ACE to apply the rule only to packets arriving via one of the
+  //               matched network adapters. Typically used to activate a given port-forwarding
+  //               rule (PORTMAP) for only a subset of network adapters.
+  //               Adapter label cannot be used for outbound ACE. This is because the EVE firewall
+  //               is applied before routing, and the output network adapter is not yet known.
   string type = 1;
   string value = 2;
 }
@@ -39,6 +79,7 @@ message ACE {
   //  for example
   //     1) host=www.example.com & port=http
   //     2) ip=8.8.8.8 & port=53 & proto=UDP
+  //     3) adapter=uplink && port=8080 && proto=TCP
   repeated ACEMatch matches = 1;
 
   // Expect only single action...repeated here is

proto/config/netinst.proto

@@ -85,6 +85,25 @@ message IPRoute {
   string gateway = 2;
 }
 
+// DefaultRouteSelection defines how local network instance picks the default route
+// when it has multiple ports assigned (using adapter labels).
+enum DefaultRouteSelection {
+  // User did not specify how to select the default route for network instance.
+  // EVE will use the default option, which is connectivity testing for "uplink"
+  // and "freeuplink" port labels, and cost-based selection otherwise.
+  DEFAULT_ROUTE_SELECTION_UNSPECIFIED = 0;
+  // Periodically run connectivity tests against the port gateway IP (if available)
+  // and the controller URL (if management port), to apply default route from
+  // a port with working connectivity.
+  // This selection algorithm thus provides automatic fail-over between ports.
+  DEFAULT_ROUTE_SELECTION_CONNECTIVITY_TESTING = 1;
+  // Apply default route of the port with the lowest cost.
+  // This is limited to management ports because app-shared ports do not have cost defined.
+  // If there are multiple ports with the lowest cost, EVE statelessly load balances
+  // the flows at the Layer 3 between these ports (or should we just pick one of the ports?).
+  DEFAULT_ROUTE_SELECTION_LOWEST_COST = 2;
+}
+
 message NetworkInstanceConfig {
   UUIDandVersion uuidandversion = 1;
   string displayname = 2;
@@ -97,14 +116,29 @@ message NetworkInstanceConfig {
   //    itself is not created on the device.
   bool activate = 5;
 
-  // port - Only a single port is supported.
-  //    This is used as the external connection for the network instance.
-  //    This can be a physical (eth0 ) or logical port (vlan 0).
-  //    The port name comes from DeviceConfig ( When it is supported in future).
-  //    If the user needs multiple physical ports, Device config should be
-  //    used to create a label for multiple physical ports.
+  // Port(s) providing external connectivity for the network instance.
+  // Either a single port referenced by its name (SystemAdapter.Name) or an adapter label
+  // matching multiple network ports (ports containing this label in SystemAdapter.Labels).
+  // Note that apart from the user-defined adapter labels, EVE automatically assigns "all"
+  // label to every port, "uplink" label to every management port and "freeuplink" label
+  // to every management port with zero cost.
+  // Both physical (e.g. eth0) and logical (e.g. vlan0) ports are allowed.
+  // Network instance with no assigned port (empty port reference or label not matching
+  // any port) is air-gapped, i.e. completely isolated from external networks.
+  // With multiple ports assigned, the behaviour depends on the network instance type.
+  // Switch network instance will run Spanning Tree Protocol (STP) to avoid bridge loops
+  // and the broadcast storm that results from them.
+  // Local network instance will apply IP routes configured for the matched ports (either
+  // statically or via DHCP) to select the next hop and the output port for the given flow.
+  // Traffic not matching any link-local, statically-configured, or DHCP-received route,
+  // will be routed according to the default route, which is selected based on the
+  // defaultRouteSelection algorithm (see below).
   Adapter port = 20;
 
+  // DefaultRouteSelection defines how local network instance picks the default route
+  // when it has multiple ports assigned.
+  DefaultRouteSelection default_route_selection = 21;
+
   // cfg - Used to pass some feature-specific configuration to the
   //       network instance. For Ex: Lisp, StriongSwan etc
   NetworkInstanceOpaqueConfig cfg = 30;