Configuring Deduplication

Packet deduplication is a feature used to identify and eliminate duplicate network packets, improving network efficiency and reducing redundant data processing. It works by comparing incoming packets

To configure and manage packet deduplication of ASN via the FlowVision GUI, navigate to:

Configuration > ASN > Deduplication

Deduplication Configuration Options:

1. Packet Deduplication: Toggle this option to enable/disable deduplication.

2. Selecting the Packet Source, you can choose one of the following packet sources:

   1. Full Packet: Deduplication is applied to the entire packet.

   2. Routed Packet: In real time, the packets can be routed across multiple devices/hops. Hence there are chances the duplicate packets can be received to monitoring fabric with different Src MAC/TTL/Checksum fields.

3. Anchor : Determines the starting position for packet comparison during deduplication. Available options:

   1. Packet Start – Begins deduplication from the start of the packet.

   2. L3 Start – Uses the Layer 3 (network layer) header as the reference point.

   3. L4 Start – Anchors deduplication at the Layer 4 (transport layer) header.

   4. L4 Payload – Focuses on the Layer 4 payload for identifying duplicates.

4. Offset: Define the byte offset from the anchor point where deduplication begins (14 to 128 bytes).

5. Window Size: The time interval within which duplicate packets will be identified. Available options: 2,4,6 & 8ms.

6. Deduplication Interface: Select the network interface where deduplication should be applied.

Below image show packet deduplication for full packet

Deduplication For Full Packet

Below image show the Packet Deduplication for Routed Packet

Deduplication for Routed Packet

When the Routed Packet option is selected, users can exclude the following fields from packet deduplication:

• Ignore IPv4 TTL: Ignores changes in the TTL (Time-To-Live) field for IPv4 packets.

• Ignore L2 Header: Excludes Layer 2 (Ethernet) header fields from deduplication.

• Ignore IPv4 Checksum: Ignores differences in IPv4 checksums when comparing packets.

• Packet drop for a few milliseconds is expected during configuration changes as the deduplication module reinitializes.

• The deduplication feature operates in DC mode.

• The maximum supported speed per port is 50Gbps.