One-way network links for small businesses: Part 2

In Part 1, we got our signals going over our one-way link to the receiving machine. Now how do we receive them?

On the receiving machine, the Network Interface Controllers, or NICs (NIC2 and NIC3 in this example) must be set to promiscuous mode, meaning that they see all packets, not just the ones addressed to them.

At the Ethernet level (Layer 2 in the OSI model), packets are addressed by MAC (Media Access Control) address, rather than IP address. So when NIC6 wants to send packet to NIC7 (10.10.10.7), it will first send an ARP (Address Resolution Protocol) request that says “Who has 10.10.10.7?”. NIC7 will send back a response giving its MAC address, and NIC6 will use that address in the Ethernet frame.

So even if we set up NIC2 and NIC3 to have the same IP addresses as NIC6 (10.10.10.6) and NIC7 (10.10.10.7), they would not see the packets addressed to NIC6 and NIC7, because they have different MAC addresses. (MAC addresses are unique to each NIC.) With the NICs set to promiscuous mode, they can receive whatever packets come down the wire.

In Unix-like systems, you set the NICs to promiscuous mode like this:
ifconfig nic2 promisc
ifconfig nic3 promisc

It is more difficult in Windows, and appears to depend on the specific driver for your NIC. A solution that will work for both Unix-like systems and Windows systems is libpcap/WinPcap, which is used by packet-capture tools like Wireshark and tcpdump. There are several bindings that let you use these libraries in the language of your choice. For Python, this includes PyPCAP, winpcapy, PyPl, PcapPy, and pylibpcap.

For our example, we will use a Windows system, so we need to download the WinPCap installer from the WinPcap  website and run it. Next we need a binding (interface) that will let us call WinPCap from Python. There are several available, but we will use WinPcapy. This can be easily installed by running the following command in an authorized command window:

pip install winpcapy

We will also use the DPKT package to parse the packets we receive. It can be installed by running the following command in an authorized command window:

pip install dpkt

WinPcap selects the device to monitor based on a wildcarded string that matches the description of the device. The description can be found by issuing an ipconfig /all command in a command window. Here is the output for the device I will be using.

Ethernet adapter Ethernet 2:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Intel(R) Ethernet Connection (2) I218-V
   Physical Address. . . . . . . . . : D0-50-99-8A-9D-AF
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::b8b4:1e1f:68db:d79a%9(Preferred)
   Autoconfiguration IPv4 Address. . : 169.254.215.154(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :
   DHCPv6 IAID . . . . . . . . . . . : 131092633
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-22-8B-E3-02-00-15-17-3A-54-62
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
                                       fec0:0:0:ffff::2%1
                                       fec0:0:0:ffff::3%1
   NetBIOS over Tcpip. . . . . . . . : Disabled

The description is “Intel(R) Ethernet Connection (2) I218-V”. Since none of the other devices have “I218” in their descriptions, I will use a wildcarded description of “*I218*”.

The easiest way to process incoming packets is to use the WinCapUtils.capture_on() function, with a callback routine that gets called for each packet received. Here is a sample program that captures messages from a single NIC. (We will deal with multiple NICs in a future installment.)

from PyCRC.CRC32 import CRC32
from winpcapy import WinPcapUtils
import dpkt
from dpkt.compat import compat_ord

# Author: Lynn Grant
# Date: 2018-05-31
# License: To the extent possible under law, Lynn Grant has waived all copyright and
# related or neighboring rights to this program. This work is published from: Belize. 

CAPTURE_DEVICE = "*Realtek*"

UDP_PORT = 5005

# XML <packet> tags wrapping the data. Note that the sequence number must
# immediately follow the XML_PREFIX1 string.
XML_PREFIX1 = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><packet seq=\""


# calcCrc: Calculate the CRC for a message
#
# message is the message for which the CRC is to be calculated.
#
# The CRC is returned.
 
def calcCrc(message): 
   return CRC32().calculate(message).to_bytes(4, 'big') 


# getSequence: Get sequence number from message
#
# We extract the text between the end of the 
# XML_PREFIX1, and the first double quote we find.
#
# If the message is too short to contain the prefix, 
# or if the prefix is not what we expect, a sequence of
# "*error*" is returned.
#
# Note: I generally do not kludgey manual parsing
#       of XML; there are are libraries to do this.
#       But this is a single field, so it does not
#       seem worth the overhead of a proper parser. 
#
# message is the message to be processed.

def getSequence(message): 
   sequence = "*error*" 
   if len(message) >= len(XML_PREFIX1):        # Is the message long enough for the prefix?
      if message[:len(XML_PREFIX1)] == XML_PREFIX1:   # Is the prefix what we expect?
         nopref = message[len(XML_PREFIX1):]   # Message without XML_PREFIX1
         j = nopref.index("\"")                # Index of trailing double quote on seq field
         sequence = nopref[:j]                 # Isolate the sequence number
   return sequence


# splitMsg: Split a received message into message data and CRC
#
# message is the received message
# 
# Message data and CRC are returned. 
 
def splitMsg(message):
   dataLen = len(message) - 4
   msgData = message[0:dataLen]
   crc = message[dataLen:]
   return msgData, crc


# Callback function to parse IP packets
def packet_callback(win_pcap, param, header, pkt_data):
    eth = dpkt.ethernet.Ethernet(pkt_data)
    if isinstance(eth.data, dpkt.ip.IP) | isinstance(eth.data, dpkt.ip6.IP6):   # IPv4 or IPv6 is OK
        ip = eth.data
        if isinstance(ip.data, dpkt.udp.UDP):   # Protocol must be UDP
            udp = ip.data
            if udp.dport == UDP_PORT:   # Must be our port

           # Split the encoded message and the CRC        
               messageEncode, rcvdCrc = splitMsg(udp.data)
               
           # Make sure the CRC is right
               if (rcvdCrc != calcCrc(messageEncode)):
                  print( "CRC error" )

           # Decode the message
               message = messageEncode.decode('UTF-8')

           # Get the sequence number
               sequence = getSequence(message)
               print( "Received message " + sequence + ": " + message )

#
# Mainline program
#

# Start capturing packets
WinPcapUtils.capture_on( CAPTURE_DEVICE, packet_callback)

Note that the program makes sure the packet is an Ethernet frame containing an IP packet, containing a UDP datagram addressed to port 5005 (the port we used on the sending side) before processing it. This is good engineering practice, but it is also necessary, because there is more traffic between the two interfaces on the sending machine than just the messages our program generates. There may be ARP messages asking “Who owns this IP address?”, and other sorts of housekeeping messages that we don’t usually notice.

Here is the output of the program, showing the messages received from the program in Part 1:

Received message 114538: <?xml version="1.0" encoding="UTF-8"?><packet seq="114538"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>
Received message 114539: <?xml version="1.0" encoding="UTF-8"?><packet seq="114539"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>
Received message 114540: <?xml version="1.0" encoding="UTF-8"?><packet seq="114540"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>
Received message 114541: <?xml version="1.0" encoding="UTF-8"?><packet seq="114541"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>
Received message 114542: <?xml version="1.0" encoding="UTF-8"?><packet seq="114542"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>
Received message 114543: <?xml version="1.0" encoding="UTF-8"?><packet seq="114543"><data><report1>Data for report 1</report1><report2>Data for report 2</report2></data></packet>

Also note that this is a sample. There several things you need to consider for a production program:

  • You should have a watchdog timer to detect if the transmission interrupted, as well as a way to re-synchronize if the transmission is interrupted only temporarily.
  • Makie sure that the sequence number of each packet is one greater than that of the previous one, in order to detect missing packets.
  • If you have a CRC error, than you may not be able to trust the sequence number; it could be where the error is. In that case, you might assume it is one greater than the last good packet, and expect the next one to be two higher.
  • If you have a CRC error, the sequence field might not be valid at all, so make sure you validate it before trying to convert it to a number.

This sample program listens to a single network interface. If you are using both cables from the Ethernet TAP for redundancy, you will need to listen to both incoming interfaces. This can be done with the appropriate calls, but is a bit trickier, as evidenced by this discussion regarding libpcap, the version for Unix-like operating systems. An easier way, where it is possible, is to bond the two interfaces together, so they appear to be one to the application. The program will be able to determine the direction for the message by looking at the direction prefix, and can match up the messages to make sure it gets an A copy and a B copy of each. If it does not, that probably indicates a problem with one of the two cables from the Ethernet TAP.

In FreeBSD Unix, it is done like this (where vr1 and vr2 are the two interfaces to be bonded):

ifconfig vr1 up
ifconfig vr2 up
ifconfig bridge0 create
ifconfig bridge0 addm vr1 addm vr2 monitor up

On Linux, you would do something like this (where etho and eth1 are the two interfaces to be bonded):

/sbin/ifconfig bond0 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 up
/sbin/ifenslave bond0 eth0
/sbin/ifenslave bond0 eth1

In Windows, how you bond interfaces depends on the driver for your particular network card. For example, here is how Intel supports bonding (or teaming, as they call it), on their network cards.

Recovering from errors

In Part 1 of this article, we discussed dealing with major outages, like cut cables, by ringing a bell, or displaying a message on a scrolling LED sign. But what about errors in individual packets?

First, the frequency of packet errors is highly dependent on your environment. A short run of cable between the ICS and office rooms in an office environment is going to have a lot less noise to deal with, resulting in fewer errors, than a long run that goes through a factory where spot welding is being done. When you get an error, you cannot just ask for a retransmission (or let the protocol software do so, like you could if you could use TCP), because we have a one-way link. Here are a few ways you might deal with error packets: 

  • If your data is just periodic data to update reports, you can just wait for the next transmission, perhaps putting an indication on the report that the update is delayed.
  • If your bandwidth is not maxed out, you can transmit each packet multiple times, and the receiving end can discard duplicates.
  • You can use Forward Error Correction codes, which make the packets larger, but make it possible to recover from many errors.

This article on the SANS website details an innovative way of assuring reliable transmission when sending syslog messages through a one-way link.

In Part 3, we will discuss some simple use cases for one-way links, as well as some commercial solutions.

Forward Error Correction References

Using Forward Error Correction is definitely a “some assembly required” option, but there are a lot of references and libraries on the Internet to help. Here are a few you might find useful:


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *