-
Notifications
You must be signed in to change notification settings - Fork 6
Network barclamp
See all, Barclamp List
This is a Creole document
The Network barclamp provides two functions for the system. The first is a common role to instantiate network interfaces on the crowbar managed systems. The other function is address pool management.
The network interfaces are controlled by the network role that is applied by the barclamp as a node transition to "installed". Based upon assigned addresses, the network recipe will create the appropriate single, dual, or team mode interface sets.
The network assignment function is handled by the creation of an API extension of the base barclamp. The barclamp adds the allocate_ip REST API call. This function allocates an IP address from a requested network and updates the node's attributes and the network's data bag. The available networks (and their parameters) are defined in the configuration for the barclamp.
This page also describes the format of the config file and parameters in the UI, but here is an Annotated Network JSON File.
The Network Barclamp provides a single role: network. This role provides the recipe access to instantiate the specified interfaces for the allocated IP addresses.
The barclamp provides a single network cookbook. The cookbook is used for interface manipulation.
The standard CLI script has been extended to allow for the allocation of IP addresses.
The barclamp has the following parameters:
Name | Default | Description |
start_up_delay | 30 | A number in seconds that the chef recipe should wait to let the networks settle. |
mode | single | A string value of either single, dual, or team. This specifies the default network interface construction model. |
teaming | map | A map of values specific to teaming |
interface_map | list | A list of map of values to help order network interfaces |
conduit_map | map | A map of values to define conduits into the box |
networks | map | A map of networks that this barclamp should manage |
The mode specifies a base line goal of the system. Single means to use the first interface (usually eth0) as the only interface for all networks. VLANs and bridges will be added on top of this single interface. Dual means that the first interface should be used as the admin interface and the second interface should be used for the other interfaces. Team means that the first two interfaces should be bonded together and all VLANs run over it.
With the addition of conduit and interface maps, the mode can be custom generated to meet the environments needs. More on this below.
The teaming sub-parameters are:
Name | Default | Description |
mode | 6 | The default teaming algorithm to use for the bonding driver in Linux |
NOTE: Do not use mode 6 with Nova. Mode 6 and bridges do not work well.
The interface_map is a list of maps that have a common format. These maps help define the order of the interfaces and will build a list of interfaces where each interface is normalized to 1g1, 1g2, .... The naming format is 1Gb network 1. One day the system will enumerate 10Gb networks with 10g1, 10g2, and so on.
Each interface_map entry has the following parameters:
Name | Default | Description |
pattern | None | The name of the dmidecode system product name name value. This can be a regex expression. |
bus_order | list | A list of bus ids that can have network devices. These are strings of the form: "0000:00/0000:00:1c" |
When the system needs to configure an interface, the interface_map is searched in list order to find a pattern that matches. If no pattern matches, the default enumeration is used. If a pattern patches, the devices are ordered according to the bus they are part of. This handles the case were the bios/system will enumerate devices differently based upon how many adapters are in the box. For most systems, the ordering will be:
1g1 | eth0 |
1g2 | eth1 |
1g3 | eth2 |
This can change. For example, a Dell PEC2100 with two 2-port 1Gb nics will have this enumeration.
1g1 | eth4 | This is the first port on the motherboard |
1g2 | eth5 | This is the second port on the motherboard |
1g3 | eth0 | The first port on the top adapter (top left) |
1g4 | eth1 | |
1g5 | eth2 | |
1g6 | eth3 | The last port on the bottom adapter (bottom right) |
Once the interfaces are mapped to their normalized name, the conduit_map defines the paths the box should have configured. This is the step that allows for generating bonds of interfaces to create a single logical interface (or conduit) for a network to be configured on.
The conduit_map is a list of maps that define the logical interfaces to expose to the networks map. Each entry in list defines the conduits for the matching pattern.
Each entry has the following parts:
Name | Default | Description |
pattern | None | A regex expression around the triple of mode, number of 1Gb networks, and node role |
conduit_list | map | A containing logical interfaces and who to map them to the physical interfaces |
The pattern looks like this: mode/number of 1Gb interfaces/role the node has Since this is a regex string, the following matches all nodes: .*/.*/.* The default configuration file contains basic mappings like described under the mode variable above. You can do all sorts of combinations.
Here are some examples:
Pattern | What it would match |
.*/4/nova-multi-controller | Regardless of mode, all nodes with 4 1Gb interfaces and the nova-multi-controller role |
team/2/.* | Regardless of the roles on the node, all nodes with 2 1Gb interfaces and the global mode is team |
single/.*/.* | When in single mode, all nodes. |
The pattern list is processed in order and the first match is applied.
The other field is the conduit_list. It is a map with the following fields.
Name | Default | Description |
if_list | list | List of normalized interface names for this conduit. |
team_mode | optional | The team mode value for this bond if it is more than one interface. This overrides the global team mode |
The networks map contains names of networks associated with a map of network parameters. Each name is unique and can be used by other barclamps or users when allocating addresses. The network contains a reference to the conduit it should run on.
Each network has the following parameters:
Name | Default | Description |
conduit | String | The conduit name from the conduit map to run this network on |
vlan | Integer | The vlan to use on the switch and interfaces for this network |
use_vlan | true | A value of true indicates that the vlan should applied to the interface. A value of false assumes that the node will receive untagged traffic for this network. |
add_bridge | false | indicates if the network should have a bridge built on top of it. The bridge will be br<VLAN ID>. This is mostly for Nova compute. |
subnet | IP Address | The subnet for this network |
netmask | Netmask | The netmask for this network |
router | IP Address | The default router for this network |
broadcast | IP Address | The default broadcast address for this network |
ranges | map | This contains a map of strings to start and stop values for network. This allows for sub-ranges with the network for specific uses. e.g. dhcp, admin, bmc, hosts. |
mtu | integer | specifies an optional MTU to set on the interface backing this network (on 1g/10g nics). The expected use is to support Jumbo frames on networks where performance can be helped by this (NOTE:this might not work correctly with bonded interfaces!) |
The range map has a string key that is the name and map defining the range.
Name | Type | Description |
start | IP Address | First address in the range, inclusive |
end | IP Address | Last address in the range, inclusive |
The system provides the following default networks.
Name | Usage | Notes |
admin | Private network for node to node communication | A router, if wanted, is external to the system. This network must be owned by the crowbar system to run DHCP on. |
bmc | Private network for bmc communication | This can be the same as the admin network by using the ranges to limit what IP goes where. A router, if wanted, is external to the system. |
bmc_vlan | Private network for admin nodes on the bmc network | This must be the same as the bmc network and have the same vlan. This will be used to generate a vlan tagged interface on the admin nodes that can access the bmc lan. |
storage | Private network for storage traffic | A router, if wanted, is external to the system |
public | Public network for crowbar and other components | A router, if wanted, is external to the system. |
nova_fixed | Public network for nova Virtual Machines | The nova-network node acts as a router. This must be completely owned by the nova system. |
nova_floating | Broken | deprecated - most likely to be replaced by nova config. |
The defaults for the barclamp assume independent networks for all of them except the bmc and admin networks. They are assumed to be the same subnet.
Operations
At create proposal time, the barclamp will create and initialize the IP address allocation data bags for each network from the default config package. This is WRONG and needs to move to the new pre-chef call. DE10.
The barclamp has a transition function that listens for the "installed" state. The function makes sure that the node has the network role associated with it.
The barclamp also uses the pre-chef apply role call out to make sure that all the allocated addresses match the appropriate mode per the configuration. This will reconfigure the node's interfaces if the mode changes.
The API extension is as follows:
Name | URL | Method | Description |
Allocate IP | /crowbar/network/<version>/allocate_ip/<barclamp-instance-name> | PUT | Putting a json document will allocate an IP address to the node |
Enable Interface | /crowbar/network/<version>/enable_interface/<barclamp-instance-name> | PUT | Putting a json document will enable the interface for that network without an IP (unless one is already assigned). |
Deallocate IP | /crowbar/network/<version>/deallocate_ip/<barclamp-instance-name> | PUT | Putting a json document will deallocate an IP address from the node |
Disable Interface | /crowbar/network/<version>/disable_interface/<barclamp-instance-name> | PUT | Putting a json document will disable the interface for that network without an IP (unless one is already assigned). |
The required JSON document for allocate_ip looks like this:
{ "name": "FQDN.node.net", "network": "name of network", "range": "range inside of network to allocate from", "suggestion": "Optional parameter providing a suggestion for the address as a string" }
The required JSON document for enable_interface, disable_interface, and deallocate_ip looks like this:
{ "name": "FQDN.node.net", "network": "name of network" }
Useful Commands and Information
The chef-server can be used to look at the network data bags to see allocated addresses and their owners.
The network barclamp uses a file-based lock to prevent multiple writers to the network data bags. This would have to change if multiple framework servers where to server barclamp transition and allocation requests.
Limitations and/or Futures
Nova will not use this for address allocation. Nova assumes a range and it seems silly to have the same range managed by two components. This will cause some headaches for the FlatManager and FlatDHCPmanager cases, but should be covered by documentation. Nova now uses the network barclamp to "reserve" addresses from the DHCP pool section of a network for use by Nova.
The Network Barclamp does not handle IPv6.
Prior running the install script the network json needs to be edited