1 of 8

For Engineers

VxLAN Deployment (NVIDIA)

VxLAN is an encapsulation protocol that provides data center connectivity using tunnelling to stretch Layer 2 connections over an underlying Layer 3 network. In Datacenters, VxLAN is used to create overlay networks that sit on top of the physical network, enabling the use of virtual networks.

It uses a VLAN-like encapsulation technique to encapsulate OSI layer 2 Ethernet frames within layer 4 UDP datagrams. VxLAN endpoints, which terminate the VxLAN tunnel, may be either virtual or physical switch ports, which are known as VxLAN tunnel endpoints (VTEPs).

The following image shows a visual representation of the VxLAN deployment:

Configuration

1. Configure Network/Tool port(s) (DUT-1)

configure terminal
interface ethernet Ethernet64/1
forward-error-correction rs
type network
exit

The Tunnel port on both sides should be a tool port

configure terminal
interface ethernet Ethernet1/1
forward-error-correction none
type tool
exit

2. Configure the VxLAN tunnel(DUT-1)

configure terminal
tunnel tunnel1
interface Ethernet1/1
source-ip 1.1.1.1
destination-ip 1.1.1.2
vni 4099
vlan-tagging disable
exit

3. configure Flow to the Tunnel (DUT-1)

flow flow1
network-ports Ethernet64/1
tool-ports tunnel1
rule 1 permit match-all counters enable
end

4. Configure Network/Tool port(s) (DUT-2)

//Analyser port - Tool port
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type tool
exit

//VxLAN port - Tool port
configure terminal
interface ethernet Ethernet1/1
forward-error-correction none
type tool
exit

5. Configure Tunnel at far-end(DUT-2)

configure terminal
tunnel tunnel2
interface Ethernet1/1
source-ip 1.1.1.2
destination-ip 1.1.1.1
vni 4099
vlan-tagging disable
exit

6. Configure Flow to match traffic from the Tunnel (DUT-2)

flow flow01
network-ports tunnel1
tool-ports Ethernet12/1
rule 1 permit match-all counters enable
exit

7. Verify

pbnoscli# show vxlan tunnel all
===============================================
VXLAN Tunnel - tunnel1
===============================================
Tunnel Port       : Ethernet1/1       
Source IP         : 1.1.1.1         
Destination IP    : 1.1.1.2         
Source MAC        : 1c:34:da:23:77:00
VN-ID             : 4099            
Vlan Tagging      : false           
pbnoscli# 

pbnoscli# show vxlan tunnel all 
===============================================
VXLAN Tunnel - tunnel2
===============================================
Tunnel Port       : Ethernet1/1       
Source IP         : 1.1.1.2         
Destination IP    : 1.1.1.1         
Source MAC        : 1c:34:da:24:de:00
VN-ID             : 4099            
Vlan Tagging      : false            
pbnoscli#

GTP Deployment

GPRS Tunneling Protocol (GTP) is a suite of IP-based communication protocols defined by 3GPP, facilitating General Packet Radio Service (GPRS) across GSM, UMTS, LTE, and 5G networks.

The GTP deployment includes the following capabilities:

GTP Parsing: Supports both standard GTP packets and GTP packets with a extension header.
Rule Matching: Enables filtering based on inner headers, including IP addresses, Layer 4 ports, and TEID.
Symmetric Load Balancing: Supports load balancing on both inner and outer IP headers (IPv4/IPv6).

The following image shows a visual representation of the GTP deployment:

GTP Extension Header Support

Overview

Traditionally, GTP packets contain a standard header that encapsulates user payload or signaling data. However, advanced use cases require GTP extension headers to include additional metadata beyond the standard GTP header.

With this enhancement, the system now supports GTP packets with extension headers (specifically type 0x85) alongside standard GTP packets, ensuring compatibility with evolving network requirements.

Requirements;

Supported Hardware: Available only on NVIDIA Spectrum-2 and later platforms.
Configuration: GTP parsing must be enabled via CLI.
Supported Extension Header Type: 0x85

Configuration Details

No new CLI commands are introduced for this feature. Existing flow configurations for filtering based on inner headers will seamlessly support both standard GTP packets and GTP packets with extension headers.

TEID-based filtering is not yet supported for GTP packets with extension headers.

Configuration

1. Configure Network/Tool port(s)

//Network ports
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type network
exit

//Tool ports
configure terminal
interface ethernet Ethernet16/1
forward-error-correction rs
type tool
exit

2. Enable GTP filtering

configure terminal
gtp
exit

3. Configure flow with rules to filter GTP traffic

configure terminal
flow flow01
network-ports Ethernet12/1
tool-ports Ethernet16/1
rule 1 permit src-ip 1.1.1.1/32 dest-ip 2.2.2.2/32 gtp "inner-sip 10.0.0.1/24 inner-dip 20.0.0.1/24" counters enable
!

4. Verify the interfaces

pbnoscli# show interface status 
================================================================================================
Port              Name                            Oper    Admin   Vlan    Speed   MTU     AutoNeg 
================================================================================================
Ethernet1/1       etp1                            up      up              100000  9100    on      
Ethernet2/1       etp2                            up      up              100000  9100    on      
<..>
Ethernet63/1     etp63                           up      up              100000  9100    on      
Ethernet64/1     etp64                           up      up              100000  9100    on      
pbnoscli#

5. Verify the flow

pbnoscli# show flow summary 
Flow-Name       Rule-Id        Status      Counter-Value
=========================================================
flow01          1             Active          103511710               
pbnoscli#

Truncation and Load Balancing

Packet truncation, also called packet slicing, involves discarding bytes from a packet. It removes the payload and provides only the necessary metadata for analysis, and thereby reducing the disk storage requirement at the tool farms.

Port-channel by default provides symmetric hashing for IPv4 and IPv6 traffic, making the source-destination pair redirected to the same tool-connected port(s).

The following image shows a visual representation of truncation and load balancing:

Configuration:

1. Configure Network/Tool port(s)

//Network ports
configure terminal
interface ethernet Ethernet12
forward-error-correction rs
type network
end

//Tool ports
configure terminal
interface ethernet Ethernet64/72
forward-error-correction rs
type tool
end

2. Configure packet truncation

configure terminal
interface ethernet Ethernet12
truncate 64
end

3. Configure port-channel with Tool port(s)

configure terminal
port-channel 10 ports Ethernet64,Ethernet72
end

4. Configure flow

flow flow2
network-ports Ethernet12
tool-ports port-channel10
rule 1 permit src-ip 1.1.1.1/32 dest-ip 2.2.2.2/32 protocol tcp counters enable
rule 2 permit src-ip 2401::1 src-netmask f::f dest-ip 2401::2 dest-netmask f::f counters enable

5. Verify

// verify Truncation
pbnoscli# show interface npb Ethernet12/1
===================================
Interface : Ethernet12/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 14              
Egress-tagging : disable         
Truncate       : 64              
pbnoscli#

// verify port-channel status
pbnoscli# show port-channel information 
===================================
Name         : port-channel10
Member ports : Ethernet64/1[up] Ethernet63/1[up] 
===================================
pbnoscli#

// verify flow
pbnoscli# show flow summary 
Flow-Name       Rule-Id        Status      Counter-Value
=========================================================
flow2           2             Active          50414924        
flow2           1             Active          50410923        
pbnoscli#

// verify Loadbalancing 
pbnoscli# show interface counters
========================================================================================================================================================================================
Port                           InOctets       InUcastPackets       InMcastPackets       InBcastPackets            OutOctets      OutUcastPackets      OutMcastPackets      OutBcastPackets
========================================================================================================================================================================================
Ethernet1/1                           0                    0                    0                    0                    0                    0                    0                    0
Ethernet2/1                           0                    0                    0                    0                    0                    0                    0                    0
Ethernet3/1                           0                    0                    0                    0                    0                    0                    0                    0
Ethernet4/1                17597009152             68738285                    0                    0                  239                    0                    1                    0
<..>
Ethernet63/1                          0                    0                    0                    0                  240                    0                    1                    0
Ethernet64/1                          0                    0                    0                    0                    0                    0                    0                    0
pbnoscli#

Ingress VLAN and Egress Tagging

Ingress VLAN functionality allows customers to assign dedicated VLAN Tags to ingress traffic, creating a mapping between the Network port and tool ports.

Traffic received on network ports can be added with an additional VLAN tag and sent towards the tools for identifying the Network Port(with the assigned VLAN tag).

The following image shows a visual representation of Ingress VLAN and Egress Tagging:

Configuration:

1. Configure Network/Tool port(s)

//Network ports
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type network
end

//Tool ports
configure terminal
interface ethernet Ethernet16/1
forward-error-correction rs
type tool
end

2. Configure ingress VLAN

configure terminal
interface ethernet Ethernet12/1
ingress-vlan 512
end

3. Enable egress tagging on tool port(s)

configure terminal
interface ethernet Ethernet16/1
egress-tagging enable
end

4. Configure flow

configure terminal
flow flow1
network-ports Ethernet12/1
tool-ports Ethernet16/1
rule 1 permit match-all counters enable
end

5. Verify

// verify interface
pbnoscli# show interface npb Ethernet12/1
===================================
Interface : Ethernet12/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 512             
Egress-tagging : disable         

pbnoscli# show interface npb Ethernet16/1
===================================
Interface : Ethernet16/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 18              
Egress-tagging : enable

By default, the Interface will have a native VLAN of n+2 (i.e. Eth12 will have a tag of 14)

// verify flow
pbnoscli# show flow summary 
Flow-Name       Rule-Id        Status      Counter-Value
=========================================================
flow1           1             Active          1154               
pbnoscli#

Loopback-mode Port

Loopback-mode port means that a physical port can become a network port (ingress) and tool port (egress) to which map rules can be applied. A loopback-mode port is operated in loopback mode and avoids customers to connect a physical cable to make it operate in Loopback mode.

As soon as a loopback-mode port is configured, it is internally changed to loopback mode. This means that the link will always be UP with or without cables inserted. Traffic flows out of a loopback port (Tx direction) and loops back to it (Rx direction).

Loopback-mode ports can provide the following flexibility:

Support for multiple lookups on the same packet.
- For example, Decap the tunnel and look up based on the inner header
Multiple egress actions on the same traffic.
- For example, send to tool as-is and add a VLAN tag

The following image shows a visual representation of the loopback-mode port:

Configuration:

1. Configure Network/Tool port(s)

//Tool ports
configure terminal
interface ethernet Ethernet16/1
forward-error-correction rs
type tool
end

//Network ports
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type network
end

2. Configure the port as Loopback-mode

configure terminal
interface ethernet Ethernet20/1
loopback-mode
end

3. Configure Flow from Network to Loopback port

configure terminal
flow flow1
network-ports Ethernet12/1
tool-ports Ethernet20/1
rule 1 permit match-all counters enable
end

4. Configure Flow from Loopback to Tool port

configure terminal
flow flow2
network-ports Ethernet20/1
tool-ports Ethernet16/1
rule 1 permit src-ip 1.1.1.1/32 dest-ip 5.5.5.5/32 counters enable
end

5. Verify

// verify interface
pbnoscli# show interface status 
================================================================================================
Port              Name                            Oper    Admin   Vlan    Speed   MTU     AutoNeg 
================================================================================================
Ethernet1/1       etp1                            down    up              100000  9100    on      
Ethernet2/1       etp2                            down    up              100000  9100    on      
Ethernet3/1       etp3                            down    up              100000  9100    on      
Ethernet4/1       etp4                            up      up              25000   9100    on      
Ethernet5/1       etp5                            up      up              25000   9100    on      
Ethernet6/1       etp6                            up      up              100000  9100    on      
<..>
Ethernet63/1      etp21                           down    up              100000  9100    on      
Ethernet64/1      etp22                           down    up              100000  9100    on      
pbnoscli#

// verify loopback-mode port
pbnoscli# show interface npb Ethernet20/1
===================================
Interface : Ethernet20/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 22              
Egress-tagging : disable         
Loopback-mode  : enable          

pbnoscli#

Loopback-mode(i.e. Ethernet20/1) port will always be UP

// verify flow
pbnoscli# show flow summary 
Flow-Name       Rule-Id        Status      Counter-Value
=========================================================
flow1           1             Active          11592               
flow2           1             Active          590            
pbnoscli#

VXLAN Stripping

Sample Connectivity

pbnoscli(config)# flow swap
pbnoscli(config-flow-swap)# network-ports Ethernet1/1
pbnoscli(config-flow-swap)# tool-ports Ethernet50/1
pbnoscli(config-flow-swap)# rule 5 permit protocol 58 counters enable
pbnoscli(config-flow-swap)# rule 5 action override-to cpu
pbnoscli(config-flow-swap)# rule 6 permit match-all counters enable
pbnoscli(config-flow-swap)# rule 6 action overwrite dest-ip 10.10.10.1 dest-mac 1c:34:da:23:77:00 dest-port 4789
pbnoscli(config-flow-swap)# rule 7 permit match-all ipv6 counters enable
pbnoscli(config-flow-swap)# rule 7 action overwrite dest-mac 1c:34:da:23:77:00 dest-port 4789
pbnoscli(config-flow-swap)# end
pbnoscli#

A Physical loop is required from the tunnel1-egress_interface (Ethernet41/1) to Tool port to (Ethernet42/1) egress interface for Tagged inner Vlan .

Tunnel Configuration

Use the below command to configure the tunnel attributes:

Tunnel attributes cannot be modified directly. To make changes, delete the existing tunnel and configure a new one.

pbnoscli(config)# tunnel tunnel1
pbnoscli(config-tunnel-tunnel1)# ingress-interface Ethernet50/1
pbnoscli(config-tunnel-tunnel1)# strip-vxlan egress Ethernet41/1
pbnoscli(config-tunnel-tunnel1)# source-ip 10.10.10.1
pbnoscli(config-tunnel-tunnel1)# destination-ip 10.10.10.2
pbnoscli(config-tunnel-tunnel1)# vni 4098
pbnoscli(config-tunnel-tunnel1)# vlan-tagging disable
pbnoscli(config-tunnel-tunnel1)#

Flow-Based Egress Configuration

Use the below command to configure the flow to egress the stripped traffic

Example

pbnoscli(config)# flow egress
pbnoscli(config-flow-egress)# network-ports Ethernet42/1
pbnoscli(config-flow-egress)# tool-ports Ethernet64/1
pbnoscli(config-flow-egress)# rule 10 permit match-all counters enable
pbnoscli(config-flow-egress)# rule 20 permit match-all ipv6 counters enable
pbnoscli(config-flow-egress)# rule 20 action override-to Ethernet64/1
pbnoscli(config-flow-egress)# end
pbnoscli#

Show VXLAN Tunnel Command

You can display the vxlan tunnel configurations using this command.

Example

pbnoscli# show vxlan tunnel all

===============================================
VXLAN Tunnel - tunnel1
===============================================
Tunnel Port       : Ethernet50/1
Source IP         : 10.10.10.1
Destination IP    : 10.10.10.2
Source MAC        : 1c:34:da:23:77:00
VN-ID             : 4098
Vlan Tagging      : false
Tunnel Status     : UP
pbnoscli#

Show Running-Config on OPB

OPBNOS# show running-config
configure terminal
interface ethernet Ethernet41/1
mode vlan-aware
mtu 9100
speed 100000
forward-error-correction none
!
interface ethernet Ethernet42/1
mode vlan-aware
mtu 9100
speed 100000
forward-error-correction none
!
interface ethernet Ethernet1/1
mode vlan-aware
loopback-mode
mtu 9100
speed 100000
forward-error-correction none
!
interface ethernet Ethernet50/1
mode vlan-aware
mtu 9100
speed 100000
forward-error-correction none
!
interface ethernet Ethernet64/1
mode vlan-aware
mtu 9100
speed 100000
forward-error-correction none
!
hostname OPBNOS
tunnel tunnel_strip
ingress-interface Ethernet50/1
strip-vxlan egress Ethernet41/1
source-ip 10.10.10.1
destination-ip 10.10.10.2
vni 4098
vlan-tagging disable
!
flow in_vxlan
enable
network-ports Ethernet1/1
tool-ports Ethernet50/1
rule 5 permit protocol 58 counters enable
rule 5 action override-to cpu
rule 6 permit match-all counters enable
rule 6 action overwrite dest-ip 10.10.10.1 dest-mac 1c:34:da:23:77:00 dest-port 4789
rule 7 permit match-all ipv6 counters enable
rule 7 action overwrite dest-mac 1c:34:da:23:77:00 dest-port 4789
end
!
flow egress
enable
network-ports Ethernet42/1
tool-ports Ethernet64/1
rule 10 permit match-all counters enable
rule 20 permit match-all ipv6 counters enable
rule 20 action override-to Ethernet64/1
!
!
OPBNOS#

VLAN Mode/Tag Actions

VLAN-aware mode provides OPB administrators with the ability to match traffic based on the VLAN tag and redirect it to the tool port(s).

Push VLAN - Traffic replication to tool ports based on VLAN match and push new VLAN Tag

Pop VLAN - Traffic replication to tool ports based on VLAN match and pop VLAN Tag on egress traffic

The following image shows a visual representation of the VLAN Aware Mode and the VLAN Tag Actions:

Configuration:

Push VLAN:

Configure Network/Tool port(s)

//Tool ports
configure terminal
interface ethernet Ethernet16/1
forward-error-correction rs
type tool
end

//Network ports
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type network
end

2. Configure VLAN aware-mode

configure terminal
interface ethernet Ethernet12/1
type network
mode vlan-aware
end

configure terminal
interface ethernet Ethernet16/1
egress-tagging enable
type tool
end

3. Configure flow with Push VLAN

flow flow1
network-ports Ethernet12/1
tool-ports Ethernet16/1
push-vlan-tag 300
rule 1 permit match-all counters enable
end

4. Verify

// verify interface
pbnoscli# show interface npb Ethernet12/1
===================================
Interface : Ethernet12/1      
===================================
Type           : network         
Mode           : vlan-aware      
Ingress-vlan   : 14              
Egress-tagging : disable         

pbnoscli# show interface npb Ethernet16/1
===================================
Interface : Ethernet16/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 18              
Egress-tagging : enable          

pbnoscli#

// verify flow
pbnoscli# show flow all  
===================================
Flow : flow1 (CLI)
===================================
Status       : enable          
Network-Port : Ethernet12/1      
Tool-Port    : Ethernet16/1      
Push vlan    : 300             

Rule : 1               
++++++++++++++++++++++++++++++++++
Action                   : permit          
Counters                 : enable          
Match-all                : ipv4            
pbnoscli#

Pop VLAN:

Configure Network/Tool port(s)

//Tool ports
configure terminal
interface ethernet Ethernet16/1
forward-error-correction rs
type tool
end

//Network ports
configure terminal
interface ethernet Ethernet12/1
forward-error-correction rs
type network
end

2. Configure VLAN aware-mode

configure terminal
interface ethernet Ethernet12/1
type network
mode vlan-aware
end

configure terminal
interface ethernet Ethernet16/1
type tool
end

3. Configure flow with Pop VLAN

flow flow1
network-ports Ethernet12/1
tool-ports Ethernet16/1
pop-vlan enable
rule 1 permit match-all counters enable
end

4. Verify

// verify interface
pbnoscli# show interface npb Ethernet12/1
===================================
Interface : Ethernet12/1      
===================================
Type           : network         
Mode           : vlan-aware      
Ingress-vlan   : 14              
Egress-tagging : disable         

pbnoscli# show interface npb Ethernet16/1
===================================
Interface : Ethernet16/1      
===================================
Mode           : vlan-aware      
Ingress-vlan   : 18              
Egress-tagging : disable         

pbnoscli#

// verify flow
pbnoscli# show flow all

===================================
Flow : flow1 (CLI)
===================================
Status       : enable          
Network-Port : Ethernet12/1     
Tool-Port    : Ethernet16/1      
Pop vlan     : enable          

Rule : 1               
++++++++++++++++++++++++++++++++++
Action                   : permit          
Counters                 : enable          
Match-all                : ipv4            
pbnoscli#