ccie blog

802.1Q Tunnelling (QinQ)

LAB

In this lab we are going to join 2 sites of a customer network using QinQ through our Service Provider (SP).  The result will be that the two sites are logically trunked, meaning that they are able to send VLAN’s across to each other through the service providers dedicated QinQ VLAN.  In this example, the provider is going to be using VLAN34 as the QinQ VLAN.

Let’s start by configuring both customer & carrier switches

Cpe1.3560#
vlan 3
 name vlan3
!
interface GigabitEthernet0/1
 description Trunk to Virgin.Edge.Sw2
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet0/5
 description Link to PC1
 switchport access vlan 3

cpe2.3560#
interface GigabitEthernet0/1
 description Trunk to BT.Sw2
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
vlan 3
 name vlan3
!
interface GigabitEthernet0/5
 description PC2 is connected here
 switchport access vlan 3
 switchport mode access

Virgin.Edge.Sw2#
interface GigabitEthernet0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet0/2
 switchport trunk encapsulation dot1q
 witchport mode trunk

BT.Sw2#
interface GigabitEthernet0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet0/2
 description Trunk to edge.sw2.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk

Nothing fancy, we just trunk to the carrier, and make an access port to our PC’s.  On the carrier switches we just trunk both ends.

The config for our edge switches is shown below.

Edge.Sw1.3560#
vlan 34
 name vlan34
!
interface FastEthernet0/1
 description Trunk to Virgin.Edge.Sw2
 switchport access vlan 34
 switchport mode dot1q-tunnel
!
interface FastEthernet0/2
 description link to QinQ.Sw1.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk

Edge.Sw2.3560#
vlan 34
 name vlan34
!
interface FastEthernet0/1
 description feed to BT.Sw2
 switchport access vlan 34
 switchport mode dot1q-tunnel
!
interface FastEthernet0/2
 description feed to QInQ.Sw2.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk

When I put the configuration on the tunnel ports, the switch prompted me to change my MTU to 1504 & reload the switch.  If your switch doesn’t do that then I suggest you check your system MTU using #show system MTU, and ensure that it’s using 1504 bytes or more.  If not, you should manually set it using #system mtu 1504.  The reason for it is because QinQ adds 4 bytes when adding an extra Tag & EType field in the frame (for a more elaborated explanation see the bottom of this page – under “QinQ technical bits”).

The config for the QinQ.sw1.3560 & QinQ.sw2.3560 are shown below. Remember, you must check the system MTU on them, as they will also need to be 1504 to be able to process double tagged frames in the QinQ ring.

QinQ.Sw1.3560#
vlan 34
 name vlan34
!
interface FastEthernet0/1
 description Trunk to QinQ.Sw2.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface FastEthernet0/2
 description Trunk to Edge.Sw1.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
vlan 34
 name vlan34

QinQ.SW2.3560#
!
vlan 34
 name vlan34
!
interface GigabitEthernet0/1
 description Trunk to qinq.sw1.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet0/2
 description Trunk to edge.sw2.3560
 switchport trunk encapsulation dot1q
 switchport mode trunk

 

Let’s shove some PC’s on port 5 at both Cpe1.3560 and Cpe2.3560 to test this actually works!

And PC2

Notice that we can see the MAC of the PC’s at either side of the tunnel by checking the arp table (arp -a). The reason we can see these MACs is because we’ve formed a layer 2 network between both sites.

 

Tips/Considerations

  • Use one dot1q tunnel interface/VLAN per customer
  • Be sure to tag your customers native VLAN traffic into the QinQ VLAN (I forgot to do that in my lab).    If you don’t, and the customer decides he wants to use VLAN 100 as his management vlan (as a native VLAN), when he trys to SSH or manage his switch at his other site, his packets could get dropped.  This is because the native VLAN is left untagged, and when it hits the provider switchport in the dot1q-tunnel mode, it doesn’t double tag the packet (as there is no original VLAN tag to start with).  To avoid complications, just tag his native VLAN using #vlan dot1q tag native when inserting his traffic into the QinQ VLAN.
  • You can’t detect the received CoS value from frames in different customer side VLANs because you added a 2 byte Tag & Etype field in front of his Tag & Etype fields when you received the frame on the tunnel port.  So you can’t prioritise his traffic according to his VLANs.  You can, however, prioritise your QinQ VLAN if you needed to.
  • Encapsulating frames into a QinQ VLAN adds 4 bytes of overhead (from the Tag & Etype fields) to every frame.  Switches sending/receiving QinQ frames must  increase their system MTU to 1504 for VLAN tunneling work.  You can check your MTU using #show system mtu.  You can increase your MTU by issuing #system mtu 1504, however, this change requires a reboot.
  • You can also tunnel the customer’s CDP, STP, VTP, and CoS values by using the following commands:

#l2protocol-tunnel stp
#l2protocol-tunnel cdp
#l2protocol-tunnel vtp
#l2protocol-tunnel cos

 

QinQ Technical bits

When a frame arrives at a trunk port, it looks like the 802.1Q frame in the diagram below.  As you can see, it adds a Etype & Tag field (compared to the Original Ethernet Frame).  The Etype field (sometimes referred to as a Tag Protocol IDentifier: TPID) will have a value of 0x8100, indicating a VLAN-tagged frame (IEEE 802.1Q).    The Tag field (sometimes referred to as the Tag Control Information :TCI) will identify a VLAN ID & and any 802.1p priority (i.e. QoS) attached to the frame.

IEEE 802.1Q Frame (Cisco, 2006)

The last frame in the image above is a QinQ frame.  Now you can see why it’s double tagged!  A tunnel interface adds another Tag (sometimes referred to as an outer tag) and Etype field.

 

References

Cisco (2006) IEEE 802.1q Frame [Online] Available from:
http://www.cisco.com/en/US/tech/tk389/tk689/technologies_tech_note09186a0080094665.shtml
[Accessed 08/02/2011]

14 Comments

SchaefDecember 20th, 2012 at 2:48 pm

Good call with the “vlan dot1q tag native”. I was having issues until I had that configured.

GauravOctober 25th, 2013 at 2:01 pm

Hi , great post.

i would like to know what would be configuration if use ASR routers (9k or 903).

Appriciate your help.

StephenGarbettJune 11th, 2014 at 6:03 pm

I’ve never had the privilege of having a spare 9k to play with unfortunately, so I don’t know.

OzzieMarch 3rd, 2015 at 1:42 am

did you use real switches in this lab or GNS3

StephenGarbettJune 18th, 2015 at 11:59 am

I did this one on real switches. I had a 7200 to help me get it working at the time.

khizerJune 3rd, 2015 at 4:18 pm

Dear Stephen
I have never played with QinQ just jumped onto your blog and I have a couple of questions in mind.

1. Do we need to set system mtu 1504 on all Provider switches or Just on edge.switch1.3560 and edge.switch2.3560
2. According to my understanding the following commands should only be configured on Edge switches 1 and 2
#l2protocol-tunnel stp
#l2protocol-tunnel cdp
#l2protocol-tunnel vtp
#l2protocol-tunnel cos
#vlan dot1q tag native

3. The virgin Edge Switch and BT Edge switches ports connecting Edge Switch 1 and Edge Switch 2 will be configured as normal trunk ports ???

StephenGarbettJuly 20th, 2015 at 2:36 pm

Correct on all questions. You need 1504 MTU on all switches because you are “double tagging” VLAN traffic for the purpose of providing a layer 2 type tunnel for the customer. If you don’t have a 1504 MTU on the core switches, the frames would most likely get dropped. I don’t think they would get fragmented because fragmentation happens at layer 3, meaning there would be no “source IP” to send an “ICMP message: frame to large, fragmentation needed” message to. The frame doesn’t get any smaller once you put it into the dot1q tunnel. So yes, you need this MTU size on all your core switches.
Also for your question 3, yes it’s just a normal trunk port. The idea is to provide a logical trunk for the customer. The VPN should be transparent to the customer.

BernardSeptember 15th, 2015 at 10:25 am

Hi Stephen,
Very nice post.
Few questions:
1- on Virgin Edge-Sw2 and BT-Edge-Switch2 we should create vlan 3 and vlan 34 ?
2- lets say i have an additional customer with 2 sites, then we need to create lets say vlan 35 as the qinq vlan for this customer. should we create an additional physical connection between Edge-sw1 and Virgin Edge-Sw2 ?
if yes then we will have stp issues + scalability bcz for each customer we will need a switchport.
if no , can you please advise how to configure this? 🙂

Thank you in advance

StephenGarbettSeptember 23rd, 2015 at 7:37 am

1.No on the Bt and Virgin Edge switches you only need the customer VLAN. The reason why is because the traffic is tunnelled by the upstream switch (similar to having an upstream router sending the traffic over a VPN). The Edge.Sw1 and Edge.Sw2 will need both VLAN’s because they need to know how to encapsulate and decapsulate user data as well as tunnel the traffic.
2. The way I would do it is by just using the same ISP dot1q tunnel VLAN (34), and just assign the new customer a new vlan (say vlan 4). This would avoid the need to use an additional port & how I’ve seen it done in one particular ISP. It helped it scale significantly as you only need one core VLAN/STP instance for the dot1q tunnelling. VPLS would help this layer2 type tunnelling scale further, but tends to require more costly equipment.

BernardSeptember 24th, 2015 at 3:29 pm

Hi Stephen, Thank you for the clarification.

Are you sure that customer Vlan3 should be configured in edge-sw1 and edge-Sw2 ?
I don’t think so because this should be transparent.
The only switches with Vlan3 should be Cpe1, Cpe2 and Virgin edge and BT edge.

Outside this scope, do you have a configuration scenario example for vpls ?

Or any reference website or book with simplified and good explanation?

Thank you in advance.

Regards,

WendelDecember 21st, 2015 at 5:19 pm

Hello Stephen, Do you know if this is possible to be made in any lab software like Packet tracer or GNS3 ?
When i tried to set the command “switchport mode dot1q-tunnel” it was not recognized.
Thanks.

StephenGarbettJanuary 28th, 2016 at 8:24 am

Nah packet tracer doesn’t support it unfortunately.

MichaelDecember 23rd, 2016 at 5:50 pm

Hello Stephen :

I am seeing that the mtu of my PC is 1300, so when the package arrives at the switch that does the qinq only add 4 bytes, it is necessary in this case to change the mtu to 1504?

StephenGarbettApril 18th, 2017 at 7:20 am

Hi Michael,

In regards to your question below on my website (sorry for the late reply). If your PC’s mtu is 1300, that’s fine. However, you need to check PC’s at either end of the connection. Usually PC’s have an Ethernet MTU of 1500. But if you are sure that all PC’s are using an MTU of 1496 or less, then it would work just fine.

Leave a comment

Your comment