Endace DAG Packet Capture Cards: Part 2

In Part 1, we covered an overview of the Endace DAG packet capture cards. In this post, we will look at a simple example of using some of their features.

Working with the Endace DAG cards sometimes reminds me of coding in assembly language: since you are at a low level, you can optimize the code to be as efficient as possible, but you end up doing a lot more manual labor than you would in a high-level language. (I might add that as compilers have gotten better at optimizing their code, the performance differential between assembly and high-level languages has become far less.)

In this case, the various filtering or load balancing operations you set up are performed on the Endace DAG card, before the data ever reaches the PCIe bus, so if you are dealing with high-speed data, it greatly reduces the load on the bus. And the Endace software has a comprehensive configuration API, so you can automate setting up the card.

In this exercise, we want to create three receive streams:

  • Stream 0 will contain all received packets.
  • Stream 2 will contain UDP packets.
  • Stream 4 will contain TCP packets.

(Receive streams are even numbered. Transmit streams, which we will not be using, are odd numbered.)

This example selects packets based on protocol, but there are actually several selectors that we can use:

  • Source IP address
  • Destination IP address
  • Source port
  • Destination port
  • Interface on the card
  • Protocol (TCP, UDP, ICMP, IGMP)
  • TCP flags
  • MPLS top label
  • MPLS bottom label
  • Number of MPLS or VLAN labels

The first thing we want to do is make sure the ERF header for captured packets includes the Packet Signature header. This will help us see that our filter rules are working. We do this as follows:

sudo dagconfig ext_hdr hdr_packet_signature

We need to set up a file with filter rules for setting packet colors. The term color is left over from early Endace cards that had two streams, red and blue. Now it is just a number. We create a file, filterRules, containing the rules:

2 tcp
3 udp

This indicates that TCP packets should have a color number of 2 and UDP packets should have a color number of 3.

There are two banks for filter rules, one of which is active. We first write our rules into the inactive bank (in our case, 1), then make that bank active:

sudo dagfilter-loader -f filterRules
sudo dagconfig -S activate_bank=1

Now we can run Wireshark to see how the packets are being colored:

sudo wireshark -i dag0:0 -y ERF

The arguments tell wireshark to use stream 0 of the first Endace DAG card. The “-y ERF” is necessary for Wireshark to display the ERF headers.

Now, if you select a TCP packet in the Wireshark display, you will see that the filter color is 0x02.

If you select a UDP packet, it will be 0x03.

Next we need to set up our streams. We start by allocating buffer space for them. Receive streams have even numbers, in our case, 0, 2, and 4. We allocate 128 MiB to each one. Transmit streams have odd numbers. We are not using them, so they all have zero allocation.

sudo dagconfig mem=128:0:128:0:128

Next we create a file, catRules, to contain the Color Attribute Table (CAT) rules that will steer packets to the correct stream:

color 2 stream 0,2
color 3 stream 0,4
not color [2-3] stream 0

This will steer TCP packets to streams 0 and 2, UDP packets to streams 0 and 4, and everything else to stream 0. The result is that stream 0 will contain all the received packets, while streams 2 and 4 will just contain TCP and UCP packets, respectively.

We load the rules into the card like this:

sudo dagcat-setup -f catRules

Now, if you run Wireshark on stream 2 (“-i dag0:2“) you will see only TCP packets, and if you run it on stream 4 (“-i dag0:4“), you will see only UDP packets. If you run it on stream 0, you will see all the received packets.

The commands to set up the DAG card do not survive a reboot, so you will need to issue them each time you reboot. A good place for them would be in the script that runs dagload to load the DAG drivers.


Leave a Reply

Your email address will not be published. Required fields are marked *