QinQ - Basic initial functionality (no vlan translation)

[ITJamie]

NetBox version

v3.5.7

Feature type

Data model extension

Proposed functionality

There would be two new sections under IPAM, svlan + svlan groups
When editing an interface there would be an additional option under 802.1 mode (Double tagged)
When selected two additional fields would appear (svlan group + svlan)

When viewing an svlan object it should list any interfaces that have that svlan assigned

on the API the interface model should include the svlan object

Use case

This would allow for basic documentation of basic QinQ uses

NOTE: im excluding vlan translation for the initial feature request:

• IMHO a larger PR needs to be had for general vlan translation options + ui features to allow cross-referencing objects)
• In a lot of cases the interface name will be a hint to the user if there needs to be vlan translation happening and could be handled in automation tooling (which is already the way many folks handle this right now)

Documenting that svlan's exist and showing that an svlan exists on an interface would be a major improvement.

Database changes

new svlan + svlan groups model (an exact clone of vlan + vlan groups models)

Interfaces model:

• Additional option under mode (double tagged)
• New fields:
  • Svlan (foreign key to svlan)

External dependencies

N/A

[sleepinggenius2]

This is something that I've definitely struggled with documenting in our network. In order to support properly modeling an EPL service, we would also need something like Double Tagged (All) to allow for associating all CVLANs with that SVLAN. Unfortunately, for EVPLs, we would also need the ability to map the CVLANs on an interface to multiple SVLANs, i.e. CVLAN 1 to SVLAN 1001 and CVLANs 2, 3 to SVLAN 1002. IMHO, there really needs to be a larger effort to support proper MEF modeling for Ethernet services to accurately document UNIs, ENNIs, EVCs, and OVCs. This FR could help on the ENNI side, but just ends up a bit confusing on the UNI or INNI side.

[ITJamie]

From my pov getting basic svlan models in will allow iteration of features as people use it and better describe their additional needs

Vlan translation is something thats needed but that will affect multiple areas (vxlan, vlans, svlan) and possibly new style of ui/forms. I would honnestly think that we would a new m2m relationship model to track translations and mappings (which should then make it possible to document all way way upto EVPLs.

Hence trying to get the basic svlan concept in first and then figure out a translation model + all of the additional ui complexity for translations

[ITJamie]

@sleepinggenius2 im on the netdev slack instance if you want to discuss alternatives directly or even work together on an alternative proposal that would cover more usecases

[sleepinggenius2]

@sleepinggenius2 im on the netdev slack instance if you want to discuss alternatives directly or even work together on an alternative proposal that would cover more usecases

I've never used Slack before, so it will take me a little bit to get that set up and up to speed on it. I would suggest getting @jsenecal involved in the discussion as well though, as this FR has a direct relation to #10558 and #13086.

IMHO, I would like to see full modeling for all the MEF models, especially EVC/OVC, L1VC, and IPVC, in addition to the various UNI and NNI models. That's the only place I've really seen QinQ in practice in our network.

[jsenecal]

@sleepinggenius2 This specific issue is aimed towards supporting the documentation of QinQ which on its own is not a small task.

I'm not sure how to tackle this but this issue could serve as a ground for discussing the implementation if any.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[github-actions]

This issue has been automatically closed due to lack of activity. In an effort to reduce noise, please do not comment any further. Note that the core maintainers may elect to reopen this issue at a later date if deemed necessary.

[jeffgdotorg]

Reviving as a starting point for work of this kind proposed for a future version's roadmap. There's some good conversation captured here. This issue will need additional detailing and breaking down into smaller chunks.

[nickper]

I do need this too, as we have it on multiple places in our network.

[ajackson79]

We have used custom fields to rudimentarily capture this data on a per interface or sub-interface level. It isn't ideal but works. A data model in netbox that supports TPID, Q-in-Q, and rewrite options would be ideal.

[jeremystretch]

Upon review of the initial proposal and discussion above, this is how I see the data model taking form:

• Introduce a new ServiceVLAN model. This essentially replicates the current VLAN model, but will be used to track SVLANs exclusively. (Alternatively, we could introduce a flag on the VLAN model to indicate SVLAN status.)

• I don't currently see a need to introduce a second VLANGroup model dedicated for SVLANs, but maybe we could add a boolean flag for this.

• Add an optional service_vlan ForeignKey to the current VLAN model. A VLAN which is assigned to an SVLAN will be treated as a CVLAN.

• Add a MODE_QINQ choice to InterfaceModeChoices, indicating the use of Q-in-Q encapsulation. (This could be used to ensure that only SVLANs are assigned to an interface, however my research suggests that mixed SVLAN & CVLAN assignment may be valid.)

Additionally, we need to give some thought to how this arrangement will interact with VLAN translation (FR #7336). My (possibly incorrect) assumption is that translation would apply to whatever is the "outer" VLAN: The CVLAN on a regular 802.1Q interface, or the SVLAN on a Q-in-Q interface.

Have I missed anything with regard to the data model?

[sleepinggenius2]

• Introduce a new ServiceVLAN model. This essentially replicates the current VLAN model, but will be used to track
  SVLANs exclusively. (Alternatively, we could introduce a flag on the VLAN model to indicate SVLAN status.)

I don't know that a new ServiceVLAN model is completely necessary, as it should be sufficient just to add a TPID field to the existing model to indicate 0x8100 for 802.1Q (C-TAG) or 0x88a8 for 802.1ad (S-TAG).

• I don't currently see a need to introduce a second VLANGroup model dedicated for SVLANs, but maybe we could add a boolean flag for this.

I can see a use case where you might have a VLANGroup to track VLAN utilization on an ENNI, in which case being able to restrict VLANs to only 802.1ad/TPID=0x88a8 would be nice. I agree that could just be a field on the existing model, maybe just a nullable choice (default null) with 802.1Q and 802.1ad as choices to indicate no restriction (null) or only VLANs with a specific TPID can be added.

• Add an optional service_vlan ForeignKey to the current VLAN model. A VLAN which is assigned to an SVLAN will be treated as a CVLAN.

I don't know of any examples of stacking S-TAGs, but it can certainly be done with C-TAGs, so the data model should support pushing both a 0x8100 and a 0x88a8 TPID on top of a 0x8100 TPID. My concern with this model is that it implies a given C-TAG is only associated with one S-TAG. I know in our network when delivering EVPL or EVP-LAN services (thus handing off a specific C-TAG on each UNI), sometimes only a portion of the path utilizes an S-TAG, or it's even possible that different S-TAGs are used on different rings. So, we would need at a very minimum a many-to-many relationship between a C-TAG and an S-TAG.

• Add a MODE_QINQ choice to InterfaceModeChoices, indicating the use of Q-in-Q encapsulation. (This could be used to ensure that only SVLANs are assigned to an interface, however my research suggests that mixed SVLAN & CVLAN assignment may be valid.)

I can absolutely confirm that mixed tags are possible on an interface. In fact, we have that quite often where we have both a C-TAG for our management traffic and one or more S-TAGs for customer traffic that exist on a given INNI. As with my suggestion about VLANGroups above, I could see where adding a new choice to InterfaceModeChoices to restrict VLAN assignment to a given TPID would have some use on an ENNI, where both the untagged and tagged VLANs should be restricted to 802.1ad/TPID=0x88a8.

Additionally, we need to give some thought to how this arrangement will interact with VLAN translation (FR #7336). My (possibly incorrect) assumption is that translation would apply to whatever is the "outer" VLAN: The CVLAN on a regular 802.1Q interface, or the SVLAN on a Q-in-Q interface.

VLAN tags are generally thought of as a stack, where your only operations are effectively push and pop, which would limit your interaction to the top/outermost tag in the stack. A translation/rewrite is equivalent to a pop + push, but often given its own action from a configuration standpoint. What technically happens at an interface is a set of ingress and egress actions. For example, at an access port, the ingress action is to allow untagged frames and push a tag on them, then the egress action is the opposite, pop the given tag from each frame, thus transmitting only untagged ones. At a trunk, the untagged/native VLAN is treated just like on an access port and tagged VLANs that match are allowed on ingress and the action is effectively a noop, similarly on egress. In addition to these basic actions, you can have things like translation/rewrite, which is a rewrite (pop + push) on ingress, and the same on egress, but with the opposite tag values. Then Q-in-Q would just be a push action on ingress and a pop action on egress. An ideal model to cover all scenarios would be the following, but would be a fundamental change from the existing model and might be a little more cumbersome to work with, though I believe the existing UX could be retained by automatically translating it to and from the new model.

Add new model InterfaceVLANAction

• interface = foreignkey(Interface)
• direction = choice(ingress, egress)
• all = boolean, used to indicate all VLANs, like tagged-all today, and also to push an S-TAG on top of all C-TAGs, like in the case of an EPL or EP-LAN UNI
• vlan = nullable foreignkey(VLAN), null = untagged or all
• action = choice(push, pop, rewrite, noop)
• action_vlan = nullable foreignkey(VLAN), only relevant to push or rewrite action
  • For action=rewrite, I believe it would make sense to restrict the vlan and action_vlan to have the same TPID
  • For action=push or action=pop, you can technically have multiple values, but I'm assuming that is fairly rare in the wild. To support that, this could be turned into a many-to-many relationship or a fixed number of fields could be added, like action_vlan1 and action_vlan2 to avoid that complexity. Similarly, the action choices could be changed to include pop1 and pop2, instead of just pop. I think 2 is a reasonable number, and I've never encountered a situation where there is more than 3 tags.

Alternatively, there could be a many-to-many relationship between Interface and InterfaceVLANAction, instead of a one-to-many one, where the interface field could be eliminated on the InterfaceVLANAction and thus actions could be defined once and reused on multiple interfaces. I believe this would be an unnecessarily complex design though, with seemingly little benefit.

A new choice could be added to InterfaceModeChoices, like MODE_ADVANCED to opt-in to the advanced UI/UX.

[sleepinggenius2]

Thank you for some additional configurations that I hadn't seen before!

all = boolean, used to indicate all VLANs, like tagged-all today, and also to push an S-TAG on top of all C-TAGs, like in the case of an EPL or EP-LAN UNI

For at least Cisco you also need a boolean for default. "encapsulation default" is used to catch all traffic not captured on another EVC or sub-interface on the parent interface. No rewrite or translation is allowed on these EVC.

You could potentially turn that field into a choice instead, since 'all' and 'default' would be mutually exclusive. I'm thinking change to field name to match, then the choices could be something like ('untagged', 'outer', 'exact', 'all', 'default'). The vlan field would then be null for everything but the outer and exact choices.

There also needs to be a way to capture "exact". This is used so the Vlan encapsulation of the traffic matches exactly what is configured. Keeps traffic from coming into the EVC with 3+ Vlan tags stacked on it.

I'm thinking that you could still accomplish that with this model by adding the 'exact' choice above, using the vlan field to match the outer tag, then the action_vlan field(s) to match each inner tag. You could also keep just the 'outer' choice for the match and add an 'exact' action choice instead, which doesn't sound quite right, but makes the use of those fields make more sense.

I think it would still all boil down to having N nullable VLAN foreign key fields for the match and action functions to use (the exact value for N should be a reasonable limit, I'm thinking 3, which is the most I've seen in practice, but open to suggestions). An alternative would be a single nullable VLAN foreign key for the match and an ArrayField for the additional VLANs, which could be used for match or action and arbitrarily long, but the application would need to enforce the referential integrity instead of the database, which is not ideal.

[jeremystretch]

It looks like we need to define discrete interface modes for S-UNI and NNI, but I'm not clear on the nuances of C-UNI configuration. It would seem that the configuration of a C-UNI is no different from a typical untagged or 802.1Q-encapsulated interface (one of the three modes which are already defined in NetBox). But at least some Cisco platforms appear to support explicit C-UNI configuration:

Router(config)# interface gig2/1
Router(config-if)# ethernet dot1ad uni c-port
Router(config-if)# switchport
Router(config-if)# switchport mode access
Router(config-if)# switchport access vlan 1000 

Do we need to declare a separate mode for C-UNI as well?

[sleepinggenius2]

Cisco 7600 series routers run plain old IOS and have been EoL for some time now, so I haven't had to look at that configuration in a while. I don't have a lot of experience with IOS-XE, but on IOS-XR the encapsulation options are:

• default (Packets unmatched by other service instances)
• dot1ad (IEEE 802.1ad VLAN-tagged packets)
  • WORD (List of VLAN Ranges in the form a-b,c,d,e-f,g)
    • [optional] dot1q (IEEE 802.1Q VLAN-tagged packets)
      • WORD (List of VLAN Ranges in the form a-b,c,d,e-f,g)
      • any (Match any VLAN id)
  • any (Match any VLAN id)
    • [optional] second-dot1q (IEEE 802.1Q VLAN-tagged packets)
      • any (Match any VLAN id)
• dot1q (IEEE 802.1Q VLAN-tagged packets)
  • WORD (List of VLAN Ranges in the form a-b,c,d,e-f,g)
    • [optional] second-dot1q (IEEE 802.1Q VLAN-tagged packets)
      • WORD (List of VLAN Ranges in the form a-b,c,d,e-f,g)
      • any (Match any VLAN id)
  • any (Match any VLAN id)
    • [optional] second-dot1q (IEEE 802.1Q VLAN-tagged packets)
      • any (Match any VLAN id)
• untagged (Packets with no explicit VLAN tag)
  • [optional] dot1q (IEEE 802.1Q VLAN-tagged packets)
    • WORD (List of VLAN Ranges in the form a-b,c,d,e-f,g)
    • any (Match any VLAN id)

Any of the dot1ad or dot1q forms can be followed by the exact parameter to not allow further inner tags. This is all related to matching the encapsulation. The following rewrite (rewrite ingress tag) options are available to manipulate the tags, and all are applied symmetrically.

• pop (Remove one or more tags)
  • 1 (Remove outer tag only)
  • 2 (Remove two outermost tags)
• push (Push one or more tags)
  • dot1ad (Push a Dot1ad tag)
    • <1-4094> (VLAN Id to push)
      • [optional] dot1q (Push an inner Dot1Q tag)
        • <1-4094> (VLAN Id to push)
  • dot1q (Push a Dot1Q tag)
    • <1-4094> (VLAN Id to push)
      • [optional] second-dot1q (Push another Dot1Q tag)
        • <1-4094> (VLAN Id to push)
• translate (Replace tags with other tags)
  • 1-to-1 (Replace the outermost tag with another tag)
    Equivalent to pop 1 + push dot1ad or dot1q with no inner tag option
  • 1-to-2 (Replace the outermost tag with two tags)
    Equivalent to pop 1 + push dot1ad or dot1q and inner tag option
  • 2-to-1 (Replace the outermost two tags with one tag)
    Equivalent to pop 2 + push dot1ad or dot1q with no inner tag option
  • 2-to-2 (Replace the outermost two tags with two other tags)
    Equivalent to pop 2 + push dot1ad or dot1q and inner tag

If you're talking about UNI and ENNI, then you're getting into the MEF Carrier Ethernet Service space, which would require EVC and OVC models and rules for mapping specific tags into specific VCs through their respective endpoints, which is certainly a use case for this kind of tag manipulation, but likely outside the scope of this FR.

[ajackson79]

I agree with @sleepinggenius2. MEF Services and models are great for providers when defining the end-to-end services they are offering, but we use the tools in the various platforms to create those services and models. What we need Netbox to model first is what is possible in the platforms. So we need the things mentioned above documented and tied to a specific interface. Usually a Sub-interface, sub-lag interface or a service instance. Most venders refer to it as an EFP. (Ethernet Flow Point)

Below is a Cisco ASR9k product slide to describe what we are trying to document and how it is used. The EFP is the part in bubble 1

[dan-dev14]

My use case would relate to Juniper specifically where we're trying to model QinQ. As an example.

set interfaces xe-2/2/3 unit 999 description "xxxx"
set interfaces xe-2/2/3 unit 999 vlan-tags outer 0x8100.1234
set interfaces xe-2/2/3 unit 999 vlan-tags inner 0x8100.5678

I think the idea of a service vlan works, as long as that can then be tracked back to the associated L3 VRF that consumes it?

[movedempackets]

I'd like to mention that VLAN QinQ/stacking (infinite depth) can be modelled today. I had raised #17046 to alter the VLAN model to include the tag protocol ID (TPID) field and the unique constraint. There is a need for custom fields where an existing model scope field choices can be expanded.

• Introduce a new ServiceVLAN model. This essentially replicates the current VLAN model, but will be used to track SVLANs exclusively. (Alternatively, we could introduce a flag on the VLAN model to indicate SVLAN status.)

Including the tag protocol ID (TPID) field on the VLAN model is the correct approach (#17046).

• I don't currently see a need to introduce a second VLANGroup model dedicated for SVLANs, but maybe we could add a boolean flag for this.

The VLANGroup scope can be expanded to include Interfaces. VLANs can be locally significant (scoped) to an Interface (NetBox VLANs resemble a bridge/flow). It is then useful to create the outermost VLANGroup for a physical interface, enforcing uniqueness.

The VLANGroup scope can be expanded to include VLANs. VLANs can be stacked to infinite depth (or defined max). It is then useful to create the inner VLANGroup for a VLAN, enforcing uniqueness.

It's important that the recommendation in #17046 to modify the unique constraint is made otherwise VLAN's of equal VID but of unequal TPID cannot be contained in the same group.

The final result is a tree that from tracing provides the VLAN stack. Child interfaces (subinterfaces) can be assigned available VLANs at a point in the tree, for example a child interface assigned a VLAN from the outermost VLANGroup is single tagged, whereas a child interface assigned a VLAN from an inner VLANGroup scoped to a VLAN in the outermost VLANGroup is double tagged, and so on.

Whilst the relations may seem unconstrained, I've implemented validators that enforce the correct Interface to (assigned, available) VLAN (untagged, tagged) and prevent VLANGroup duplication.

• Add an optional service_vlan ForeignKey to the current VLAN model. A VLAN which is assigned to an SVLAN will be treated as a CVLAN.

Whilst I do not believe that it is the right approach. If you did choose it I'd name the field outer_vlan and modify the unique constraint.

• Add a MODE_QINQ choice to InterfaceModeChoices, indicating the use of Q-in-Q encapsulation. (This could be used to ensure that only SVLANs are assigned to an interface, however my research suggests that mixed SVLAN & CVLAN assignment may be valid.)

The existing modes are sufficient. The form could benefit from better filters and an indicator of the TPID and stack when selecting a VLAN.

[jeremystretch]

Including the tag protocol ID (TPID) field on the VLAN model is the correct approach

I'm not convinced. The TPID is configured at the device or interface level; it may not make sense to define globally for a VLAN unless we explicitly accept that as an imposed constraint (which might be reasonable). I see two options for adding a field to the VLAN model:

1. Add a boolean svlan flag. This is set to true for any VLAN designated as an S-VLAN. It does not impose a specific TPID.
2. Add an svlan_tpid choice field. This is set to either 0x8100 or 0x88a8 (or maybe 0x9100) for an S-VLAN; otherwise it is left null.

The VLANGroup scope can be expanded to include Interfaces.

We won't be modifying the VLANGroup scope as part of this work. All we're after here is the ability to associate a set of customer VLANs with a service VLAN.

As @ITJamie points out above, our initial goal is to implement foundation support for modeling Q-in-Q topologies. This will likely not include all potential configurations, but rather serve as an initial model upon which we can iterate in future releases as needed.

[jeremystretch]

I've started work on a proof of concept in PR #17822. This is still a work in progress but the approach feels pretty solid so far. It does not tackle any complex push/pop logic; that may come later or it may be handled in conjunction with the work on #7336 (VLAN translation).

[NiclasSP]

To give some input for my usecase for this.

Our primary services with QinQ and Vlans include several layers of translation.

1. So when get double tagged traffic at the NNI.
2. Where we remove the outer-tag(S-VLAN) and put the traffic with into at L2VPN based that outer-tag.
3. At the L2VPN endpoint we repush a different Outer-tag(S-VLAN).
4. Then at the final node remove Outer-tag(S-VLAN) and then we translate inner-tag(T-VLAN to a C-VLAN)

I am not expecting this to be able to handle all this, especially not just in this FR.
But it would be amazing for the solution to allow us the document all of the translations.

From my view i see it as 3 parts.

• Documenting QinQ relation of vlans. (This FR)
• Translations of vlans between vlan-groups. ( VLAN Translation #7336 )
• Vlangroups relation for Interfaces and L2VPNs( Not required but would help requests )

For reference about the QinQ part i will mention the translation part.
The immediate solution i see is to have multiple vlans.
Where the translation creates a relationship between them.
So the Inner-tag vlan would potentially exist multiple times, once a NNI, once in L2VPN, once at UNI side + more potentially.
Some of this duplication of vlans can be avoided if Vlan Groups allow a "all" notion for Inner vlans where it just does 1-1.

This would allow the QinQ relation to be simple and just needing to document which Inner-tags belong to which S-VLAN.
All the complexity of relations between Vlan groups could be handled in Vlan translations.
This would mean a lot of Vlans for some of us, but would give all the flexibility that we have in the equipment.