Add support for tracking physical cable plants

[jeremystretch]

Currently, NetBox does not support any concept of physical cable plant installations. We can connect one interface to another, but there is no way too associate that connection with information about the physical path taken by the cabling.

Ideally, NetBox should support the construction of physical cable paths to be associated with data (and perhaps console) connections. This would entail the tracking of individual path panels and cables, as well as the ability to support bundles of cabling (e.g. MPO fiber trunks).

The database schema necessary to support this feature will take quite a bit of thought. However, once that schema has been devised and agreed upon, it should be relatively easy to incorporate in NetBox.

[shakalandy]

Nice proposal.
Please keep in mind that patch panels could be ordered horizontally or vertically. This would be a great enhancement.

[doon]

This is where I've always run into roadblocks in designing the schema, especially when looking at some of the cross connect panels we have here. Rack 1, 2U Fiber Shelf in pos41,42 6 Cassettes (Duplex LC) and each cassette goes to a different Rack / Room, etc. So the schema kept getting hairy. couple that with input / output / the ability to connect to either a interface, another patch panel, or say a circuit / provider it snowballs pretty quickly.

[puck]

With @doon's comment about things snowballing quickly. My thought would be to treat anything that a cable can terminate on as a port. Each socket on a patch panel has two ports, one on the back for structured cabling, and one on the front for patching (and yes, I'd treat a punch down block as a port). A cable can have one or more plugs on each end. Those plugs can go into ports. If you need to connect two cables together it might be able easier to have a coupler that has two (or more) ports on it.

This kind of model should also support octopus cables, with mismatching number of plugs (think MTP on one end and many LC on the other).

Cables could have metadata with them about the maximum speed they support, colour etc.

This model also extends to power infrastructure where PDUs can be treated as patch panels. DC bus bars etc.

[mdomore]

For a patch panel i create a device type and i insert 1 interface front and one interface rear for each FO patch in the patch panel.

i connect rear from patch 1 to rear from patch 2 and front to active devices.

the only big problem is the form factor.

it's a trick but if it can help others it's free

[isodude]

@mdomore Nice trick, I figured I would solve it that way as well, however, I loose the connection between server -> switch in that case since it is just a connection in terminology.

I would like a socket-port, possibly a hub, since that's exactly what it is :)
That would mean instead of add a 10G SFP+ interface I add a Hub instead where you can connect several interfaces. Maybe that will also shorten the timespan for designing the models since we don't need to invent that much around it.

[isodude]

I also noted that it's not possible to add new Interfaces, if we could focus on that part, maybe this is just a matter of the SysOp to define them and specify how many connections a interface can have.

Today there's not a XenPak connections i.e., also not 10GE Copper. That would be nice be able to add yourself.

[mdomore]

Yes and if we can add connector type like LC, SC, SCA, LCA ... this could be really usefull

[brianaddicks]

Something else that I think goes along with this feature is tracking the usage of a patch panel. It'd be great to see that on the Rack list next to the RU Utilization %. That way if I have a rack with lots of free physical space but no patch panel room I won't accidentally tell someone it's available to use.

[swickstrom]

My current bodge for this is to create a parent device which acts as a copper/fiber panel, and then create child devices that serve as copper/fiber cassettes. Each cassette has ports and there is a crosslink interface under the OOB interfaces that connects the cassette pairs. The cassettes are mirrors of eachother as far as port assignments go; if you wish to determine where a device on port-01 traces to you follow the crosslink port to the cassettes mate and see which device is on port-01.
The issue with this idea revolves solely around the binding action of interface types. I specifically ran into issues representing console/serial connections; due to how console interfaces work, I had to have the console ports on the cassette in the rack with the console server be OOB ports, and the console interaces in the rack with the servers act as console server ports in order for the ports to be assignable.
Enter the passthrough form-factor. A passthrough form-factor would be defined as:
• Having an in and an out interface assignement
• Having no interface type; allowing it to bind with any interface and not require pre-requisites such as being a console server port or being an OOB port

[amatamalas]

We are currently using openDCIM and one of the point to start using in the past was the ability to track cabling (but missing the IPAM feature like netbox has). Basically you have devices with data ports (network, console,...) and power ports (to connect to your PDU). The Patch panels are the only special object which have front and back port in order to be able to represent all the patching path. Then, each connection can have a label and you can represent the full path of a connection by searching by one of the labels of any cable in the path. I think this is really useful when you need to dive into the datacenter. The form factor, cable type (ethernet, fiber, coax,...) or even the color of the cable are very good attributes (optional and configurable) to have into account. I'm not a fan of reinventing the wheel and maybe it could be a good starting point for the cable plan design, as it's working very well for us. Of course there are quite a lot of usage cases and maybe this is too simple to cover all of them.

[iamdadmin]

One way you could accommodate single connections would be to allow edit the device types and devices forms to support "back-to-back connection" in the same way that you have "Power Ports" and "Interfaces" only this type of connection has an A side and a B side.

The A side can be connected to a fibre bundle such as Device: "48 CORE BUNDLE 4728" which in turn can have back to back connections. In this way you'd eventually describe the entire path.

[doon]

I've been toying with this on my own (in ruby/rails as I not so good with Parseltongue :)).

What I've come up with, and it seems to work so far...

ports belong to a PortGroup, which then belongs to a PortGroupOwner, these are basically abstractions to represent anything that can have a "port". So for example.

PortGroupOwner == ASR9K
PortGroup (index: 0) GigabitEthernet0/0/0
Ports(0) GigabitEthernet0/0/0/0
Ports(1) GigabitEthernet0/0/0/1

or

PortGroupOwner = RandomDellServer
PortGroup (index: 0) Mezzanine Ethernet Card
Ports(0) Eth0
Ports(1) Eth1

or even an abstract entity such as a hand off to another carrier,

PortGroupOwner = RandomCarrier
PortGroup(index:1) 80SS Handoffs
Port(0) TieCircuit XYZ

Then each circuit consists of port connects (cross connects).

this connection has a port_a and a port_b (Which are both FKs into ports) and both are required. the circuit_id is also required unless this connection is marked as permanent.

port_a and port_b have unique constraints on them. So each port can only be used once as port_a and once as a port_b.

Permanent port_connects are used between panels, and are generally created at panel creation time. I have scripts that build perm connections between 2 port_groups with the same number of ports to automate panel creation).

So when assigning to a circuit I just need to create connections between 2 ports.(service tries to create it as a,b, and then b,a, and both violate constraints it fails). if either ports belongs to a perm cross connect, then it updates the circuit_id for the existing port_connection for those ports, else it just creates the new one and assigns the circuit id.

Once we have all the ports mapped out in the circuit you can find the endpoints by taking the set of ports as A ports and the set of ports as B ports and look for which elements are exclusive.

To graph it can just dump it into dot/graphviz as the the series of cross connects.

to build a DLR or the like can just start at one of the ends and follow the port connects til you reach the other end.

So far it seems to solve most of the issues/requirements that I have.

Hope this helps, or feel free to tell me I Am crazy/misguided/etc..

[harsh-kotak]

[Suggestion] You can start with adding colums line this in the interface connections menu:
A-END -- Z-END
Row/Rack DeviceA Interface Media -- Row/Rack DeviceB Interface Media

where media is cable/connector type. e.g.: SM LC, MM LC, QSFP twinax, copper, etc.

This does not map the entire Layer 1 but helps to keep track of cabling type used.

[nward]

Something that would be useful is to treat fibre pairs as individual fibres, or bundle them together. Some providers track every strand, rather than pairs. This would also enable use of BiDi optics, etc.

MPO/MTP support would be very useful, also. I currently model MTP connected patch panels in Device42 by pretending that there are many panel-to-panel cables, which isn't right obviously!

Perhaps support for inline circulators and taps, too. A company I work with splices these inline, rather than going through a rack mounted device.

[RyanBreaker]

I'm glad to see this is on the roadmap to be implemented as this is the most-needed feature for us to switch from a spreadsheet for tracking all our cable paths. Is there an ETA or milestone for when this will be implemented?

[spencerryan]

It depends on the platform. Typically they can be configured for 4 x 10G mode or 1 x 40G mode (and similar for 25/100G)

As an example, on an Arista switch a QSFP will show up as something like Ethernet52/1, 52/2, 52/3, and 52/4. In 4 x 10G mode all 4 subinterfaces work, and in 40G mode subinterfaces 2-4 are in err-disable and /1 is the up/up interface.

[isodude]

Hi, QSFP+/QSFP28 can be split into 4x10G, 4x25G, 2x50G, allowing 400G interfaces we will probably see more splits. In the switch it's one 400G/200G/100G/40G optic, and in the other end you have an 10/25/50/100G optic that eventually decide what splitting there will be. In the end you will have several interfaces in the switch. If you don't break out the connector it will just be one interface. Will have my hand on the equipment in a month, but I assume that they are treated as separate interfaces in the switch. I will implement 14 ports in a 32p 100G as 25G ports, having them as 100G ports in one end and 25G in the other would be helpful. You will terminate them in a split-out cable or casette that will be an intermediate anyway. So the physical ports: MPO 100G SR4 -------- MPO->4x25G(LC-LC) ( https://www.fs.com/products/68402.html) --------- LC 25G SR4 Virtual would be TwentyFiveGigabitEthernet1/3 ------ eth0 So in this case there will be two physical entries, between the optic and the casette, the casette and the server. The virtual interface actually passes several physical ports. After doing a bit of research on the fiber cables there is a lot of information that should exist on the cables as well, like Single or Multi mode? Type A/B/C? Female-Female/Female-Male? OM4/OM3/OS/2? MPO-8 12 or 24? Copper cable is so much easier.. but still it must be important if it's Cat5 or 6. Do we have kind of detail on the cables right now? I could try out your changes to see if I can document my new setup properly?

…

On Wed, 18 Jul 2018, 20:14 Brian Candler, ***@***.***> wrote: I think I have a decent model designed now - but before I write it up, I'd like to check something about QSFP+ (since I don't have any such equipment). Does a QSFP+ port typically appear as four separate 10G interfaces in the device configuration? The reason I ask: I am wondering whether there is a need for separate models for "interface" (logical) and "connector" (physical), with an N:1 relationship between them. At the moment, Netbox combines the two concepts. Is this separation important? Or would it be acceptable to treat a QSFP+ port as a single "interface/connector", and then create four virtual interfaces representing the actual 10G circuits? If so, I think it would be necessary to be able to create a LAG between these - but Netbox does not currently allow LAGs between virtual interfaces. Also, it would be desirable to associate these interfaces with the physical port. At that point, it becomes much like a separate interface/port model. Alternatively, a QSFP+ connector might be considered as four separate connectors bound together. But then the model would probably require you to plug in the four "parts" separately. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACyOM9IblPEYHd3D1xDr0iXPpJ-pNHMgks5uH3r4gaJpZM4I-GO2> .

[ljb2of3]

On my Extreme 7100 series switch it shows up as 4x10g connections in the config, but that may be due to the optic I have plugged in, as I'm using a MPO to 4x LC breakout cable for separate 10g connections. I'm not sure what would happen if I put a single 40g LC optic in the slot. I just want to say thank you for your work on this. I have a need to map out our fiber usage on campus and am looking forward to using netbox for this. From a UI perspective, it would be nice to group things together, such as "single mode from building 1 to 2" and see a free vs used chart, perhaps like the % utilized for IP space in prefixes, or even a list of strands in the group.

…

-Landy Senior Network Engineer The University of Tulsa

On Wed, Jul 18, 2018, 13:14 Brian Candler ***@***.***> wrote: I think I have a decent model designed now - but before I write it up, I'd like to check something about QSFP+ (since I don't have any such equipment). Does a QSFP+ port typically appear as four separate 10G interfaces in the device configuration? The reason I ask: I am wondering whether there is a need for separate models for "interface" (logical) and "connector" (physical), with an N:1 relationship between them. At the moment, Netbox combines the two concepts. Is this separation important? Or would it be acceptable to treat a QSFP+ port as a single "interface/connector", and then create four virtual interfaces representing the actual 10G circuits? If so, I think it would be necessary to be able to create a LAG between these - but Netbox does not currently allow LAGs between virtual interfaces. Also, it would be desirable to associate these interfaces with the physical port. At that point, it becomes much like a separate interface/port model. Alternatively, a QSFP+ connector might be considered as four separate connectors bound together. But then the model would probably require you to plug in the four "parts" separately. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFh_T9TJpmIiRjnxrmRl6JkKFbQsLrOsks5uH3r5gaJpZM4I-GO2> .

[nward]

I have a couple hundred optics with >10G in production so can speak pretty authoritatively about this.

As @spencerryan the presentation to the OS can depend on the platform.

The connector can be:

• Simplex LC single mode (rare)
  • 8 different wavelengths, 4 per direction, 10G or 25G per wave
• Duplex LC single mode
  • 4 different wavelengths on each fibre, 10G or 25G per wave
• MTP/MPO 12 core multimode (6 pair)
  • Uses 8 cores, or 4 pairs, 10G or 25G per fibre
• MTP/MPO 24 core multimode (12 pair)
  • Uses 20 cores, or 10 pairs, 10G per fibre
• MTP/MPO 12 core single mode (6 pair)
  • Uses 8 cores, or 4 pairs, 10G or 25G per fibre

Presentation to the OS is typically as a single 100G or 40G link, however, it can be "channelised" in to the per-fibre or per-wavelength signalling rate. i.e. a 40G optic can be 1x40G or 4x10G. Not all OSes support this channelisation.

I have never seen an option for 100G being divided to 2x50 as @isodude mentions - I don't know how that would work, as that would be combining 2x50G services in the same manner as 4x25G for 100G - I don't think there are standards for that right now but no reason there couldn't be. I think we should allow for modelling whatever combination of these we can think of - even combining channels differently (so maybe a 4x25G running as 1x50G and 2x25G).

Note that when you have a multi-channel optic, running it in 40G or 100G mode does not behave like an 802.1AD "etherchannel"/"bond"/whatever. It is a single 40G or 100G service.

[isodude]

Regarding 50G, I've seen it on the switch specs, but searching for it reveals both NICs and DAC-cables at least. Not the same amount that exists for 25G though. Mellanox SN2100 can carry 32 50G ports i.e. Regarding conmectors I hear that some are using 24 core trunks to cabinets to just split out to raw LC, and replace LC harnesses that way. Just for upstream fiber patches or patches betweeen cabinets. Sounds like a smart thing :)

…

On Wed, 18 Jul 2018, 23:50 Nathan Ward, ***@***.***> wrote: I have a couple hundred optics with >10G in production so can speak pretty authoritatively about this. As @spencerryan <https://github.com/spencerryan> the presentation to the OS can depend on the platform. The connector can be: - Simplex LC single mode (rare) - 8 different wavelengths, 4 per direction, 10G or 25G per wave - Duplex LC single mode - 4 different wavelengths on each fibre, 10G or 25G per wave - MTP/MPO 12 core multimode (6 pair) - Uses 8 cores, or 4 pairs, 10G or 25G per fibre - MTP/MPO 24 core multimode (12 pair) - Uses 20 cores, or 10 pairs, 10G per fibre - MTP/MPO 12 core single mode (6 pair) - Uses 8 cores, or 4 pairs, 10G or 25G per fibre Presentation to the OS is typically as a single 100G or 40G link, however, it can be "channelised" in to the per-fibre or per-wavelength signalling rate. i.e. a 40G optic can be 1x40G or 4x10G. Not all OSes support this channelisation. I have never seen an option for 100G being divided to 2x50 as @isodude <https://github.com/isodude> mentions - I don't know how that would work, as that would be combining 2x50G services in the same manner as 4x25G for 100G - I don't think there are standards for that right now but no reason there couldn't be. I think we should allow for modelling whatever combination of these we can think of - even combining channels differently (so maybe a 4x25G running as 1x50G and 2x25G). Note that when you have a multi-channel optic, running it in 40G or 100G mode does *not* behave like an 802.1AD "etherchannel"/"bond"/whatever. It is a single 40G or 100G service. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACyOM2BmPn0qoWVCtvzGoUD4wnbuWUL-ks5uH62LgaJpZM4I-GO2> .

[tardoe]

I can backup the information provided by Nathan, having also deployed a large amount of 10/40/100G optics. Using the Extreme Exos platform as an example, the 32 QSFP28 switch interfaces on the X870 model are labelled 1,5,9,13... 125 with the option to carve these into either 1x100G, 2x50G, 4x25G, 1x40G or 4x10G. So 25G/10G interfaces would be 1,2,3,4 and 40G/100G interfaces would be 1,5,9,13... Leaving 50G interfaces to be 1,3,5,9 etc. This is a fairly stock broadcom platform which is quite popular so this kit might form a good reference. Cheers, Tim Raphael

…

On 19 Jul 2018, at 7:50 am, Nathan Ward ***@***.***> wrote: I have a couple hundred optics with >10G in production so can speak pretty authoritatively about this. As @spencerryan the presentation to the OS can depend on the platform. The connector can be: Simplex LC single mode (rare) 8 different wavelengths, 4 per direction, 10G or 25G per wave Duplex LC single mode 4 different wavelengths on each fibre, 10G or 25G per wave MTP/MPO 12 core multimode (6 pair) Uses 8 cores, or 4 pairs, 10G or 25G per fibre MTP/MPO 24 core multimode (12 pair) Uses 20 cores, or 10 pairs, 10G per fibre MTP/MPO 12 core single mode (6 pair) Uses 8 cores, or 4 pairs, 10G or 25G per fibre Presentation to the OS is typically as a single 100G or 40G link, however, it can be "channelised" in to the per-fibre or per-wavelength signalling rate. i.e. a 40G optic can be 1x40G or 4x10G. Not all OSes support this channelisation. I have never seen an option for 100G being divided to 2x50 as @isodude mentions - I don't know how that would work, as that would be combining 2x50G services in the same manner as 4x25G for 100G - I don't think there are standards for that right now but no reason there couldn't be. I think we should allow for modelling whatever combination of these we can think of - even combining channels differently (so maybe a 4x25G running as 1x50G and 2x25G). Note that when you have a multi-channel optic, running it in 40G or 100G mode does not behave like an 802.1AD "etherchannel"/"bond"/whatever. It is a single 40G or 100G service. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[nward]

Hey Tim :-)

Juniper boxes use a sort of sub-interface when channelising:
et-0/0/0 - 40G

• or -
  et-0/0/0:0 - 10G
  et-0/0/0:1 - 10G
  et-0/0/0:2 - 10G
  et-0/0/0:3 - 10G

I think the key is that this stuff is flexible. The QSFP channelisation is something I hadn't considered though. When tracking individual strands, breakout with same wavelength - i.e. MTP/MPO - is quite easy.
It's also possible to break out multi wavelength - i.e. take a 40G-LR4, put it in to a mux/demux, and then get 4x10G channels on a WDM system. I think this is covered though, as the mux/demux can be modelled as a "device" - maybe? Not sure.

[tardoe]

I would agree with this, any sort of mux should be modelled as a device as 1) it performs a unique network function and 2) it changes the physical / logical path. Regards, Tim Raphael

…

On 19 Jul 2018, at 9:38 am, Nathan Ward ***@***.***> wrote: Hey Tim :-) Juniper boxes use a sort of sub-interface when channelising: et-0/0/0 - 40G or - et-0/0/0:0 - 10G et-0/0/0:1 - 10G et-0/0/0:2 - 10G et-0/0/0:3 - 10G I think the key is that this stuff is flexible. The QSFP channelisation is something I hadn't considered though. When tracking individual strands, breakout with same wavelength - i.e. MTP/MPO - is quite easy. It's also possible to break out multi wavelength - i.e. take a 40G-LR4, put it in to a mux/demux, and then get 4x10G channels on a WDM system. I think this is covered though, as the mux/demux can be modelled as a "device" - maybe? Not sure. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[candlerb]

This is the more physically-oriented outline design I've been working on.

Connectors and ConnectorTypes

• A Connector has a ConnectorType
• A Connector is either male or female
• A male Connector mates with a female Connector
• A Connector belongs to either a Device or a Lead
• A Lead has two or more Connectors (discussed in more detail later)

Example: ConnectorTypes

ConnectorType

id	name	pins
4	RJ45	[1, 2, 3, 4, 5, 6, 7, 8]
5	IEC C14/IEC C13	["L", "N", "E"]
6	LC Duplex	["TX", "RX"]
7	ST	["pin"]

Normally the connector has one name, but in some cases the male and female are known by different names (e.g. C14 and C13)

Pins are internally numbered from 1, but can have user-defined pin names.

Example: mated RJ45 connectors

Suppose that there are devices 51 and 52. Each has an RJ45 socket. Then there is a lead with id 2000, which has an RJ45 plug on both ends.

Connector

id	connector_type	is_female	mated_id	device_id	lead_id
101	4	Y		51
102	4	Y		52
103	4	N	101		2000
104	4	N	102		2000

A connector must have either device_id or lead_id set (but not both).

A connector may have either is_female or mated_id set (but not both). That is: when a connector is connected, the mated_id of the male connector points to the id of the female connector. A male connector can have mated_id set to null, if it is unconnected.

Alternative proposal for connector model

This data model shown above allows a connector on one lead to plug directly into a connector on another lead - something that happens in real life - e.g. extension cables, and couplers/adapters like RJ45 to DB9 serial adapters.

I did consider a more compact approach, where the connector on the Device always points to the Lead:

id	connector_type	is_female	device_id	lead_id	lead_end
101	4	Y	51	2000	A
102	4	Y	52	2000	B

lead_id and lead_end are set to null when the device port has nothing plugged in.

This has the advantage that when you create a Lead you only need to create one object instead of three. A Device connector is 'mated' by means of being associated with a particular end on a lead. A unique constraint on (lead_id, lead_end) avoids the same lead end being used more than once.

However that model does not allow two Leads to be connected to each other, unless we introduce a new model for this case, e.g. Coupling.

id	a_lead_id	a_lead_end	b_lead_id	b_lead_end
...	2000	A	2001	B

This is also a reasonable approach, although it does complicate some logic. When tracing a path from a Lead, you would have to look for Devices which point to this lead, and also Coupling objects where a_lead_id or b_lead_id equals this lead. And the process for connecting/disconnecting two Lead ends to each other is different for connecting/disconnecting a lead into a Device.

How Connectors relate to Interfaces

I see Interfaces as a logical entity. In the existing model an interface on one device can be "connected to" an interface on another device, meaning they are direct neighbors, and I think this should remain, to be able to display neighbors without having to trace physical paths. (Or it can be simplified so that each interface has a "neighbor" attribute pointing directly at another Interface)

Furthermore: multiple interfaces may be presented on the same physical connector: e.g. there may be four 10G interfaces but they are on the same QSFP+ connector.

So I see there being a N:1 relationship between Interface and Connector. However most of the physical attributes of Interface would move on to Connector. An Interface would still have a MAC address and MTU, and be what IP addresses are attached to.

The ConnectorType model allows the user to add new types of connector, without having them hard-coded in Netbox (e.g. special stacking connectors).

If each Interface has a link to Connector, we also gain for free the ability to associate subinterfaces with the same physical port (e.g. Gi0/1.100, Gi0/1.200 etc) - #1519

Leads and LeadTypes

A Lead is somewhat like a device, in that it has two or more Connectors - but it is a free-standing entity. You are likely to have many identical or similar Leads in the network. The LeadType defines the set of connectors which a Lead should have, so that when you create a Lead the related connectors can be created automatically.

Example: RJ45 patch lead

LeadType

id	name	connectors	symmetrical	attribute_values	harness
14	RJ45 straight patch	{"A":[4,false], "B":[4,false]} #type,female	true	{"category": [null, "CAT5e", "CAT6", "CAT6A"], "length": [null, "0.3m", "0.5m", "1m", "1.5m", "2m"], "colour": [null, "red", "blue", "green"], "manufacturer": [null, "monoprice", "hiprice"]}	[[["A","1-8"], ["B","1-8"]]]

"Symmetrical" means that all the cable ends are equivalent and interchangable, so "A" and "B" are just arbitrary labels without any physical meaning. (Note to self: maybe the connectors on a lead should just be numbered from 1 anyway)

Lead

id	lead_type	serial	asset_tag	attributes
999	14			{"category": "CAT5e", "length": "0.5m", "colour": "red"}

In this model I've assumed that you don't want to repeat the definition of LeadType for minor variations of leads (e.g. lengths and colours), so we make them attributes of the Lead itself.

I've shown these fields as JSON, because it's compact and is how I'd do it; but it could be broken into a bunch of separate normalised tables in the way that CustomFields are.

I have included "serial" and "asset_tag" because some types of lead are expensive assets to be tracked in their own right - for example a copper SFP+ Direct Attach cable.

It seems reasonable to me to permit a lead to exist with no connections, i.e. unused lead. It might be worth having an optional location (maybe just site_id) on the Lead object for that case. This situation is comparable to a spare, unracked Device. I would never expect a Lead to span sites anyway.

Example: IEC Y cable

LeadType

id	name	connectors	symmetrical	attribute_values	harness
14	IEC Y cable	{"A":[5,false], "B1":[5,true], "B2":[5,true]}	false	{"length": [null, "0.3m", "0.5m"]}	[[["A","1-3"], ["B1","1-3"], ["B2","1-3"]]]

Example: duplex LC to 2 x ST

LeadType

id	name	connectors	symmetrical	attribute_values	harness
14	Duplex LC to dual ST	{"A":[6,false], "B1":[7,false], "B2":[7,false]}	false	{"length": [null, "0.3m", "0.5m"]}	[[["A","1"], ["B1","1"]],[["A","2"], ["B2","1"]]]

Harness

The "harness" indicates how the connector pins are wired together: in the RJ45 example lead end A pins 1-8 are connected to lead end B pins 1-8, and in IEC Y cable example all three pins of connector A are linked to the corresponding three pins of connectors B1 and B2. In other words, there are groups of linked pins. Again, this is shown as JSON for compactness.

A duplex LC to LC fibre patch lead would have the two strands swapped over (A pin 1 to B pin 2, and vice versa).

The harness is optional. If you just want to define a lead with N ends and N different connector types, but no harness, that's fine. The internal harness may simply not be worth modeling, or the lead may have active components (e.g. a USB to RS232 lead).

SFPs as Leads

We can usefully model an SFP as a "Lead" with an SFP (electrical) connector at one end, and another connector (e.g. duplex LC or RJ45) at the other. There would be no direct connection between the pins of the connectors (i.e. harness), because this is an active element. That's fine.

The LeadType can distinguish between different types of SFP, and the serial and asset_tag attributes of Lead helps to track them.

This is another reason why it's very useful to allow Leads to connect into Leads:

Cable plant

The above doesn't add all that much to the existing data model: a lead (or series of leads) connects a device port to one or more device ports.

The other side required is cable plant, which allows fixed infrastructure connections between patch panels and ODF ports to be modelled. This model is in some ways similar to Lead and harness, but represents permanent infrastructure rather than dynamically mated connectors.

CrossLink

A "CrossLink" represents one or more strands of pre-installed copper or fibre between two connectors, normally on two different Devices. These are internal connections, in the sense that they are at the "back" of the connector, not the mating face.

I could have just modelled individual Strands, but I didn't want to have to create 8 separate objects to represent the very common case of a connection between two RJ45 patch panel ports.

CrossLink

id	a_connector	a_pins	b_connector	b_pins
200	500	1-8	600	1-8
201	501	1-8	601	1-8

I had originally called this StrandSet, but it's useful to be able to model that there is some sort of connection between two device ports without having to model any strand-level detail; that is, leave a_pins and b_pins as null.

A CrossLink can also exist between two connectors on the same device: this allows internal cross-connects to be modelled, e.g. a patch panel with RJ45 sockets front and back, and various types of break-out device.

If you want to route different strands on one connector to different remote connectors, then you create separate CrossLinks carrying a subset of strands:

id	a_connector	a_pins	b_connector	b_pins
200	500	1	600	1
201	500	2	601	1

NOTE: a CrossLink represents an end-to-end connection between certain pins of a connector on one device, and certain pins of a connector on another device. At this level there are no splices, nor does it model which cable(s) or specific strands on each cable are used.

However, an electrical test or an optical test can confirm the accuracy of a CrossLink model.

Cables

Cables are the (optional) physical manifestation of the CrossLink model, and may be spliced. One cable may serve multiple CrossLinks, and one CrossLink may be spread over multiple cables.

CableType

id	name	strand_names
1	CAT5e T568B	["Ora/Wht","Ora","Grn/Wht","Blu","Blu/Wht","Grn","Brn/Wht","Brn"]
2	4-core multimode	["A1-Red", "A2-Blue", "A3-Green", "A4-White"]

Cable strands are internally numbered from 1, but can also have user-visible labels such as tube/core colour.

The cables themselves are simple objects that are just instances of a CableType:

Cable

id	cable_type	label
1	2	A001
2	2	A002

(But we could have additional attributes, such as length or dB loss)

Cable attributes

I think it would be useful to add attributes like manufacturer, colour, diameter on the CableType. (Alternatively we could have user-settable attributes on the Cable itself, but I don't think that creating extra CableTypes when required is a major problem)

Cable Segments

A Cable Segment links one or more strands in a CrossLink to the corresponding set of strands in a Cable.

Example 1: two ODFs with duplex LC connectors, connected via two four-strand fibre cables which are patched back-to-back 1:1 in the middle.

CableSegment

id	crosslink_id	crosslink_strands	cable_id	cable_strands
100	200	1-2	1	1-2
101	200	1-2	2	1-2
102	201	1-2	1	3-4
103	201	1-2	2	3-4

Example 2: one ODF with two duplex LC connectors, going to two LC ports on two different ODFs. There is a single 4-strand cable to start with, then it splits onto two separate 4-strand cables, with 2 strands in each of the second cables unused.

NOTE: there are no explicit splices as such. If one particular CrossLink strand uses cable A strand 5 and cable B strand 8, then it's implicit that cable A strand 5 must be spliced onto cable B strand 8.

A limitation of this is that it's not possible to record a splice, when the strand doesn't run all the way end-to-end between two device ports (i.e. it's not part of a CrossLink). In practice, I don't think this is a major problem, as you're unlikely to pay for splices which don't get used. I am open to suggestions for better ways to model this, but I think this keeps things simple.

The proposed model also allows you to change the cable strand assignments safely, without touching the end-to-end CrossLink connectivity (which as I said before, may have been verified physically e.g. by OTDR tests)

Splice locations

The above model does not allow you to verify that a physical path has been fully modelled; it just tells you that CrossLink A passes through particular strands on cables W, X and Y, but can't tell you if you forgot to include cable Z.

However, if we include the cable endpoint locations, we can do this. A cable endpoint may be at a Device, or at a JointLocation (which could represent e.g. a splice chamber; many cables can have have their ends in this location)

If the places where cables 1, 2 and 3 come together is called X, then it would look like this:

We can now trace the Cables which make up a SpliceSet. As long as every cable with an end at a particular JointLocation is paired with another cable that also has an end at the same JointLocation, we know that the whole path has been modelled.

This has the additional benefit that you can model the endpoints of a cable, even if they are not connected at all - e.g. are just coiled up in the JointLocation waiting to be connected.

A JointLocation could usefully have some location attributes, e.g. GPS coordinates or just a description of the location, and/or a site_id.

It should be stressed that this model builds up in layers:

• CrossLinks
• Cables / CableSegments
• JointLocations

You don't have to use all the levels if you don't want to model at that level of detail.

Cable plant alternatives

As described above, the CrossLink can only connect pin(s) on one connector to pin(s) on one other connector; it does not allow 3 or more pins to be electrically or optically connected together. I don't think this is an important limitation; if you want to model power distribution or passive optical networks, you can model PDUs or optical splitters as separate devices. But this makes it different to Lead, which can model such multi-way connections, e.g. the IEC Y connector example.

There is an argument to consider about whether to model CrossLink in a similar way, for consistency. I think this would end up with a join table, which I've called CrossLinkConnector here:

You can see that separate CrossLinks can be coalesced, and hence the granularity of CrossLink is up to the user - it could be at the level of normal cables (e.g. 8 strands for a CAT5e cable) but could be at the level of the entire "wiring loom" if you wanted: this would mean giving each strand in the loom a unique number though.

I don't think this an improvement though. Logically, it's helpful if a CrossLink represents a bundle of pin connections between exactly two connectors: it makes it much easier to trace through the network, in the common case where the path does not bifurcate. That is: if I have an RJ45 port on patch panel A, and another RJ45 port on patch panel B, and those two ports are linked by a single CrossLink (and no other ports share the same CrossLink) this makes the CrossLink a helpful logical model that these two things are connected, without having to trace individual strands. Indeed, you could have a CrossLink with zero strands and still use it to identify that there is some sort of path between those ports on A and B, without having to do any strand-level modelling or tracing at all.

Discussion of the above model

Managing connections

Instead of (or as well as) adding a "Connection" between two device interfaces, you would create a Lead and connect it to two (or possibly more) device connectors. The process for doing this would be similar to interface connections, but the lead itself has attributes (a lead type, and maybe optional attributes like colour, length, serial number etc), and therefore needs to be visible as an object in its own right.

It should be possible to unplug a single Lead connector, and reconnect it to a different connector on the same device or a different device (whereas Netbox currently requires you to destroy and recreate an InterfaceConnection)

The existing Interface Connection model can remain, representing the logical neighbor relationship between two Interfaces, alongside the physical model. When you are entering data, you may find that the Interface Connection model may either not have a corresponding Connector / Lead / CrossLink model, or the latter may be incomplete, or inconsistent with the former. There should be reports for highlighting those cases. Alternatively, whenever changing the Lead model, it can flag whether changes are inconsistent with the Interface Connection model, and whether you want to make the corresponding updates automatically. That is: people who create physical connection models will get the logical model updated automatically as a side effect, when possible. (It might not always be possible, e.g. when one physical connector related to multiple logical interfaces)

Even with explicit patching it may not always be possible to update the logical interface connections automatically. For example: a single QSFP+ port may be linked to four logical interfaces; the transceiver has an MTP/MPO connector with 8 pins; and then a break-out cable (with five connectors) connects to four duplex 10G interfaces on other devices. Extra modelling would be required to be able to determine which interface connects to which: e.g. those four logical interfaces might have to be associated with "channels" of the QSFP+ connector, which in turn pass through the transceiver as four "channels" (fibre pairs) on an MTP/MPO connector, which in turn are associated with four fibre pairs on the MTP/MPO break-out cable. But if this is not done, it would still be possible to create the logical interface connections manually.

Interfaces and Connectors

A Device will now have both Interfaces and Connectors. That is:

• A Device can have Interfaces
• A Device can have Connectors
• An Interface can optionally link to one Connector (as outlined earlier)

When adding an Interface to a Device, you would need to be able to associate it with an existing Connector, or create a new Connector. Or you might want to be able to create a Connector without creating an Interface. (A patch panel has Connectors, but no Interfaces)

DeviceType templates would need to gain connectors too. To be complete, DeviceTypes should also have CrossLinks representing internal cross-connections (e.g. patch panels with internal front-back connections)

Interface and Connector naming

Both Connectors and Interfaces need to be named. In simple cases they might be the same name (e.g. "Gi0/1"). However moving forward, this may not be true. Consider for example what happens when we extend this to a chassis switch model: on each line card the physical connector port names may be 0 to 7, and hence "slot 1 port 0", "slot 2 port 0" etc; but the corresponding logical interface names may be "Gi0/0", "Gi1/0" etc.

Therefore, long term, I think the set of Connectors (and their names) should become a fixed part of the DeviceType (and LineCardType). After all, in real life you do not modify a device by drilling it to add more Connectors! But on a particular Device instance, you can change the set of interfaces, and which connectors they are associated with. With modular Line Cards, a logical interface on the Device can be associated with a physical connector on a Line Card.

Moving the set of connectors up into DeviceType has some other advantages. It makes LeadType and DeviceType much more similar to each other; and the connections from Lead to Device and Lead to Lead could be represented by a single object, representing the mating, rather than the individual connectors. That is, you would not need to create unmated connectors on Devices and Leads.

This idea is interesting enough that I'll sketch it out separately.

Console and power

Netbox currently has completely separate models for Console ports and Power ports and connections into them. I think moving Console ports into the above model would make a huge amount of sense: console connections can be patched via RJ45 patch panels.

Whilst that is not true of power, I still think there is a strong argument for modelling power like this too; see the example of the Y IEC lead (which in Netbox today I have to model as an unracked "device" with one power inlet and two power outlets). I think the general solution here is to create port roles: network, console [in/out], power [in/out]. People may wish to extend this, e.g. to add USB, or if they want to consider Fibre Channel as a role distinct to "network". But having a hard-coded set of roles is probably good enough.

In the UI, ports can be grouped by role to keep the same logical separation that they have today.

Lifecycle

I have not addressed the lifecycle status of any of the entities above: e.g. whether they are "planned" or "in service" or "failed". Marking a CrossLink as Planned before it's build would most likely be useful; marking a Crosslink as Failed is definitely useful, as this does happen in practice.

[candlerb]

Updated model - Connectors fixed to DeviceType

Now taking the idea of applying physical connectors to DeviceType instead of Device, whilst Interfaces remain on Device.

Let me sketch out a sort of pseudo-ERD which includes some sample data. This is the connectors and leads part:

You can mate a lead to a device, or a lead to a lead, but not device to device. There are no explicit "connector" objects any more: rather, the pairing of (device_id, index) or (lead_id, index) refers to the Nth connector on the device type of the given device, or the lead type of the given lead.

A DeviceTypePort is flagged "passive" if it is a patch panel or ODF port which can be the target of a CrossLink (such ports cannot be linked to an Interface).

Not shown: InterfaceTemplate should gain a link to DeviceTypeConnector. (InterfaceTemplate gives the default set of interfaces when an instance of a Device are created from a DeviceType, and when those interfaces are non-virtual, we want them linked to the correct connector)

And this is the CrossLink part:

Notes

Comparison to current model

In Netbox today, "Interface" is linked to a hard-coded list of InterfaceTypes. We would instead be linking to a DeviceTypeConnector, which is a particular port (named instance of a ConnectorType) on a given DeviceType.

A "mating" looks at first glance rather similar to the existing Interface Connection. But the key difference is, instead of going Device to Device, it goes Device to Lead or Lead to Lead. That is, one (logical) connection is formed by at least two matings, for the two ends of the cables. Also: it goes between Connectors and not Interfaces (although the Connectors themselves are not explicit objects, but the Nth instance of DeviceTypeConnector or LeadTypeConnector)

Data migration

An Interface normally links to a DeviceTypeConnector, but it doesn't have to (if the link is null). This is how we deal with virtual interfaces, but it's also how we could deal with the data migration from old model to new. Existing DeviceTypes would have no Connectors on them, so existing Interfaces would all not be linked to any Connector. It would be up to users to add DeviceTypeConnectors to the relevant DeviceTypes, and then update their Interfaces to link to them, before making use of the physical model. This is a one-time change.

However it would also be desirable to retain existing interface physical characteristics, rather than lose them; maybe they should be retained for now.

Line cards

Netbox doesn't yet have the concept of line cards, and this model suffers without them, because a given DeviceType will have a fixed number of ports. Therefore, maybe line cards should be added at the same time.

Line Cards would be fairly simple I think:

• LineCardType
• LineCardTypeConnector (for this new model)
• LineCard (an instance of LineCardType, inserted into a Device)
• A join table of which LineCardTypes can fit into which DeviceTypes
• Interface then needs to be able to link to either DeviceTypeConnector or LineCardTypeConnector

I don't really like the duplication between DeviceTypeConnector and LineTypeConnector. The simplest way to deal with this is that DeviceTypeConnector just becomes Connector, and it can link to either DeviceType, LineCardType, or LeadType.

Note that this doesn't describe an individual connector, but the Nth connector on a give template (DeviceType, LineCardType or LeadType).

When a LineCard is added to a Device, it goes in a particular Slot. It would be good if default Interfaces could be created with a slot-based name, e.g. Gi%d/1, Gi%d/2 where %d is slot number. That is: LineCard needs a feature comparable to DeviceType's InterfaceTemplate, but with slot number insertion.

[nward]

I havne't read details of your proposal yet but I noted the line card thing. A "LineCard" should perhaps just be a "module". In this way, it could be a PCI card, or whatever.

Modules should be able to contain other modules - for example:

• Juniper MX chassis routers have MPCs which can have MICs inserted in them.
• Cisco ASR9000 have MODs which take ... something I've forgotten
• Nokia 7x50 has MDAs
• Cisco X2 ports can take TwinGig adapters which take a 1x10G X2 and give you 2x1G SFP

and on, and on.

In short - they should permit nesting.

[candlerb]

A "LineCard" should perhaps just be a "module"

I presume by "module" you mean "Inventory Item"?

Modules should be able to contain other modules ... In short - they should permit nesting.

Turns out they already do in the data model - it's just not exposed in the UI (#1692)

For the benefit of other people reading, Netbox currently has two types of "module-like" things:

• "Child devices" in "Device bays" are when one device physically fits inside another device. However, the child device is a completely independent device, with its own OS, management IP, and interfaces (which are not available to the parent device). This is intended for modelling blade servers, where an entire independent server fits inside a chassis. Some people try to misuse this for line cards, but that's wrong: the interfaces of the child do not appear on the parent, and you can't make a LAG between two child devices.
• "Inventory Items" (formerly called "modules") are simple objects which have a manufacturer / part number / serial number. They are intended for things like hard drives and DIMMs, and today this is the only way to model line cards, but there are limitations. When you add a line card, you must manually add interfaces to the device (Extend modules with types and interface associations #824). You cannot select from predefined "Inventory Item Types" (which is not great when you have lots of identical line cards). Inventory Items cannot exist outside a device (i.e. no uninstalled spares), nor be moved between devices (Ability to move inventory items between different devices #2233)

What I want is a thing which has a pre-defined fixed set of Connectors; but those connectors extend the device that it's plugged into, rather than being independent like a child device.

ISTM that making InventoryItem become more grown-up, by having InventoryItemType, and adding optional sets of Interfaces and Connectors defined on the InventoryItem, is the way to go. I think this is essentially what #824 is proposing. InventoryItemType and DeviceType would end up duplicating a large amount of functionality and it would be nice to share that somehow.

Now, regarding modules which plug into other modules, like WICs and possibly SFPs: I'm going to do some thinking aloud here.

Consider the following configuration:

Device <-- Line Card <-- WIC <-- SFP <-- ... (fibre cable)

In the cable plant model I've outlined, I suggested modelling the SFP transceiver as a "lead" which has two connectors: an SFP copper connector on one side, and an LC duplex connector on the other. This gives the benefit that you can support both direct-attach cables that plug directly into the SFP connector, and SFP transceivers.

So how does a line card differ from an SFP? The way I see it is that a line card adds new connectors to a device, while an SFP plugs into an existing connector on the device, and provides another connector on the other side.

Of course, in the real world, a line card slot has a very special custom connector at the back, so we could also model a device as having special line card connectors, and line cards are just "leads" that have one of these connectors on one side, and on the other side have more normal connectors (e.g. RJ45), or other custom connectors (e.g. WIC connectors).

But if we forget about all these fancy custom connectors, we can just say that a "line card" adds standard connectors to the underlying device - in which case a WIC is an example of a "line card", even though it is physically installed inside another line card. In terms of adding connectors, all "line cards" are associated directly with the device. But physically they may fit into a line card. That is: logically, the WIC adds interfaces to the Device; but physically, it sits inside a Line Card.

As I said before, the existing InventoryItems model already supports this:

                                  Table "public.dcim_inventoryitem"
     Column      |          Type          |                        Modifiers
-----------------+------------------------+----------------------------------------------------------
 id              | integer                | not null default nextval('dcim_module_id_seq'::regclass)
 name            | character varying(50)  | not null
 part_id         | character varying(50)  | not null
 serial          | character varying(50)  | not null
 discovered      | boolean                | not null
 device_id       | integer                | not null
 parent_id       | integer                |
 manufacturer_id | integer                |
 asset_tag       | character varying(50)  |
 description     | character varying(100) | not null

The device_id associates each inventory item with the device, but the nullable parent_id lets you say that the item is installed inside another inventory item.

[kaso17]

To throw in another use case for the LineCard/lead discussion.
Some newer devices support "compact SFPs" (two independent BiDi optics/interfaces in a single SFP module) or fanout cables (e.g. 1 x 100G QSFP28 to 4 x 25G SFP28). Depending on which transceiver you insert the switch will add the corresponding interfaces. This should be supported too.

[dowlingw]

Netflix folk mentioned they might consider opening up their CircuitDB project, might be worth considering whether integration and/or taking some lessons learnt from their model has value in the discussion: https://twitter.com/dtemkin/status/1024037852622012421

[jeremystretch]

Quick update. Work on this feature is underway, and anyone interested can follow along in the 20-physical-cabling branch. Hoping to have a v2.5 beta out in November.

[vink78]

LSH connector is common in my area. Is it possible to add it (check vink78@b22dd4b)?

[jeremystretch]

20-physical-cabling has been merged into develop-2.5. This completes the initial implementation of the feature. Additional minor development on this and related functionality will occur in the develop-2.5 branch as we work toward the beta release.

This feature request is now closed. 🎉

[Folg0re]

hi,
i'm on master branch v2.5.13 and i'm tring to add patch panel. i read all this discussion but i can't understand wich is the best practice.

i create a device with front port and rear port; the rear port are connected to rear port of other patch panel. when i connect front port, to interface of active device, apparently i have the right trace.

is it the best practice for patch panels??

thanks

[candlerb]

Please post support requests on the Google group.

Github tickets are for development issues / bug reports.