tshark one-liners

Since most of the hits on this blog seem to come from tshark filter related searches, and since I spend a good part of my day either running or analysing packet captures, I thought it might be useful to create a series of "tshark one-liners" in homage to the brilliant "sed one-liners" collection compiled by Eric Pement.

These are capture filters, not display filters, and are equally applicable to Wireshark, tshark and tcpdump, since they all use the same pcap filter syntax. In wireshark the capture filter options are now hidden away and you have to double click on the interface under capture options to set or adjust the filter string.

The filters are broadly grouped by purpose and I will try to add more as I think of them. Please comment if there is something you think I have missed or would like added.

Note: if you want to strip off VLAN, MPLS, PPPoE or GRE headers from an existing pcap file, please see this post: Removing VLAN/MPLS/PPPoE/GRE Encapsulation

Ethernet

Match 802.1D spanning tree:
"ether dst 01:00:c2:00:00:00" (manpages say "ether proto stp" but I've had trouble with that)

Match Cisco PVST+:
"ether dst 01:00:0c:cc:cc:cd"

Match Cisco CDP / VTP / DTP / PAgP / UDLD:
"ether dst 01:00:0c:cc:cc:cc"

Match LLDP:
"ether proto 0x88cc"


Match LACP (slow protocols):
"ether dst 01:80:c2:00:00:02"

General IP
Match host A (10.0.0.1) communicating with host B (192.168.0.1):
"host 10.0.0.1 && host 192.168.0.1"

Match host A (10.0.0.1) communicating with anything on network B (192.168.0.0/24):
"host 10.0.0.1 && net 192.168.0.0/24"
or, if you don't like CIDR notation:
"host 10.0.0.1 && net 192.168.0.0 mask 255.255.255.0"

Match ARP:
"ether proto 0x0806"

Match DHCP:"udp port 67 || udp port 68"

VLANs

Match any traffic with at least one VLAN tag:
"vlan"

Match traffic with exactly one VLAN tag:
"vlan && not vlan"

Match traffic with an SVLAN of 100 and any CVLAN:
"vlan 100 && vlan"

Match traffic where the first VLAN tag has an 802.1p marking of:
0: "vlan && ether[14] & 224 == 0"
1: "vlan && ether[14] & 224 == 32"
2: "vlan && ether[14] & 224 == 64"
3: "vlan && ether[14] & 224 == 96"
4: "vlan && ether[14] & 224 == 128"
5: "vlan && ether[14] & 224 == 160"
6: "vlan && ether[14] & 224 == 192"
7: "vlan && ether[14] & 224 == 224"

Note: to match the second VLAN tag use "vlan && vlan && ether[18] & 224" on the left hand side of the equality.

MPLS

Match traffic with at least one MPLS label:
"mpls"

Match traffic with exactly one MPLS label (match S bit of first label):
"mpls && ether[16] & 1 == 1"

Match traffic with a first or single label of 12345:
"mpls 12345"

Match traffic with an inner (e.g. service) label of 67890:
"mpls && mpls 67890"

Match traffic with exactly three MPLS labels (e.g. traffic on facility bypass FRR):
"mpls && mpls && mpls && ether[24] & 1 == 1"

Match 6PE traffic:
With transport label: "mpls && mpls 2"
Without transport label (after PHP): "mpls 2"

Match traffic with an EXP marking (on the first label) of:
0: "mpls && ether[16] & 14 == 0"
1: "mpls && ether[16] & 14 == 2"
2: "mpls && ether[16] & 14 == 4"
3: "mpls && ether[16] & 14 == 6"
4: "mpls && ether[16] & 14 == 8"
5: "mpls && ether[16] & 14 == 10"
6: "mpls && ether[16] & 14 == 12"
7: "mpls && ether[16] & 14 == 14"

Note: to match the EXP marking of the second label, use "mpls && mpls && ether[20] & 14" on the left hand side of the equality.

Multicast

Match any Ethernet multicast:
"ether multicast"

Match IP multicast traffic:
"ip multicast"

Match IGMP traffic:
"ip proto 2" (the manpages say "ip proto igmp" but I've had trouble with that)

Match PIM traffic:
"ip proto 0x67" (the manpages say "ip proto pim" but I've had trouble with that)

OSPFv2

Match all OSPF:
"ip proto 89"

Match specific OSPF packet types:
Hello: "ip proto 89 && ip[20:2] == 0x0201"
DBD: "ip proto 89 && ip[20:2] == 0x0202"
LSR: "ip proto 89 && ip[20:2] == 0x0203"
LSU: "ip proto 89 && ip[20:2] == 0x0204"
LSA: "ip proto 89 && ip[20:2] == 0x0205"

IS-IS

Match all IS-IS traffic:
"isis"

Match specific IS-IS PDU types:
"l1", "l2", "iih", "lsp", "snp", "csnp" or "psnp"

BGP

Note: These rules do not handle multi-segment messages very well but they are good enough for most purposes.

Match only BGP OPEN messages:
"tcp port 179 && tcp[50] == 1"

Match only BGP UPDATE messages:
"tcp port 179 && tcp[50] & 5 != 0"

Match only BGP NOTIFICATION messages:
"tcp port 179 && tcp[50] == 3"

Match only BGP KEEPALIVE messages:
"tcp port 179 && tcp[50] == 4"

 L2TP

Match only L2TP control messages:
"udp port 1701 && udp[8:2] & 0x80ff == 0x8002"

Match L2TP control messages for tunnel ID 1234:
"udp port 1701 && udp[8:2] & 0x80ff == 0x8002 && udp[12:2] == 1234"

Match L2TP data messages for tunnel ID 1234:
"udp port 1701 && udp[8:2] & 0x80ff == 0x0002 && udp[10:2] == 1234"

Match L2TP control messages for session ID 5678:
"udp port 1701 && udp[8:2] & 0x80ff == 0x8002 && udp[14:2] == 5678"

Match L2TP data messages for session ID 5678:
"udp port 1701 && udp[8:2] & 0x80ff == 0x0002 && udp[12:2] == 5678"

PPPoE

Note: Offsets will need to be manually increased by 4 bytes  for each VLAN tag or MPLS label present.

Match PPPoE discovery phase (PADI / PADO / PADR / PADS / PADT):
"pppoed"

Match PPPoE session phase (i.e. PPP traffic):
"pppoes"

Match PPPoE LCP messages:
"pppoes && ether[20:2] == 0xc021"

Match PPPoE CHAP authentication messages:
"pppoes && ether[20:2] == 0xc223"

Using Capture Filters to Match Higher Layer Protocols

In my previous post I went through some of the tricks that can be used to match MPLS and / or 802.1Q tagged traffic in packet filters. That's a great benefit when analysing traffic on carrier networks or large corporate networks but it only goes up to the transport layer (i.e. TCP and UDP port numbers).

Sometimes it's very desirable to filter on upper layer protocol information which has no corresponding parameters in the pcap-filter syntax. Take for example a situation where you are monitoring a busy BGP route reflector where you only want to see NOTIFICATION messages without all the KEEPALIVEs  and UPDATEs cluttering things up. It's possible to match these cases quite easily using a display filter, however your capture files could get quite large in relation to the amount of useful data. Once again it would be nice to be able to restrict at source, using a capture filter.

The following method can be used reliably for some protocols, somewhat reliably for a few and is completely inapplicable to others. In general if your protocol uses a fixed packet format or you want to match part of a fixed-format header then you're in luck.

Many protocols such as RADIUS, encode their parameters using attribute / value pairs (AVPs) or type / length / value (TLV) format, which can present parameters in an arbitrary order. If the parameter you want to match is in an AVP or TLV, your results are likely to be variable at best. Remember that capture filters work on fixed offsets and cannot cycle through parameters until the right one is found. If you're lucky the particular implementation you're looking at may put the AVPs / TLVs into the same order every time and your value may be early enough in the list not to get 'bumped' by other parameters inserted before it. In general, though, this technique is unlikely to work well.

Method

If at all possible, the best approach is to get a few sample packets of the data you want to capture. The captures should be taken from the same point in the network where you intend to run the real mirror to avoid any differences in encapsulation that would throw out the offsets. Generally it's possible to 'seed' such packets by, for example, manually clearing sessions.

In our example, we want to just see the BGP packets which contain a NOTIFICATION message. We start by obtaining a sample capture, obtained by shutting down a BGP session at one end while sniffing at the point where we intend to monitor. Below is the capture we get, with the interesting packet selected:

Here we can see there are two VLAN headers, beyond which we can see the IP details and the expanded decode of the BGP message. Logically, if we want to catch all the NOTIFICATION messages, we need to do the following:

• Parse and discard the VLAN tags so that IP can be decoded correctly
• Match only TCP traffic using either source or destination port 179
• Of this, match only those packets of type NOTIFICATION

Starting at the first line, we can begin to write our capture filter. Assuming that we don't care which VLAN IDs are being used, just that they are present, the following will match traffic with any two VLAN tags:

"vlan && vlan"

Note that this not only matches traffic which has two VLAN headers - it also adjusts the decoding offset. This is critical to the success of the filter as in a normal, untagged frame the IP header would start directly after the Ethernet header at offset 14 (decimal). With two VLAN tags, the IP header will actually be at offset 20 (decimal). By matching the VLAN tags in this way, the capture filter knows that the IP will start further into the frame. The same happens with the "mpls" and "pppoes" keywords, so if you have these headers make sure you match them.

So next we want to make sure that only BGP packets are matched - this is as simple as you would expect using "tcp port 179" - this will match either a source or a destination port of 179 so you don't have to worry about which end initiated the BGP session. Let's add it to the expression:

"vlan && vlan && tcp port 179"

Now this rule will match any double-tagged BGP traffic. The tricky part is that there are no capture filter keywords for matching BGP packet types and we want to do precisely that. The only option remaining for us is to match bytes at a given offset. Eek!

It takes a little getting used to but for fixed format headers it can be very reliable. I find the easiest way to do this is to:

• Select the field you want to match in Wireshark
• Find the offset in the packet where that value is stored
• Set a filter to match the required value at the required offset

So in our example, I have selected the BGP message type. This is at offset 005C hex / 92 decimal and a type of NOTIFICATION is encoded as a byte of value 3. A simple filter to match this would be "ether[92] == 3". Matching this on its own would get all the BGP NOTIFICATIONs, but also a load of other junk so let's combine it with the rest of our filter:

"vlan && vlan && tcp port 179 && ether[92] == 3"

OK, we can be pretty sure now that this will only match genuine NOTIFICATIONs. The BGP header ip to and including the type field is fixed length, so it is not going to move, and the value of 3 always means NOTIFICATION. Now let's test it out with a real capture on the same conversation as above:

root@sniff:~# tshark -i eth1 "vlan && vlan && tcp port 179 && ether[92] == 3"

Running as user "root" and group "root". This could be dangerous.
Capturing on eth1
  0.000000      1.1.1.2 -> 1.1.1.1      BGP NOTIFICATION Message
^C1 packet captured
root@sniff:~#

Perfect. Exactly what we wanted to capture!

Note: It's easy to see the offset from the start of the frame by just looking at the packet capture, but where possible you should consider using offsets from IP or TCP. That way, if you want to re-use your filter with more or less encap, you can just add or remove VLANs, MPLS, etc, without having to re-calculate the offsets.

You will have to use your imagination and ingenuity to work out whether this technique can be used to match your interesting traffic reliably. There are many aspects I have not covered which may prove essential, depending on what you are trying to do, for example:

• It is possible to match multiple byte fields using [offset:size] notation in place of the simple [offset] used in this example
• It is possible to bitmask values using the normal bitwise operators, so for example to check if the least significant bit of byte 80 is set, the expression "ether[80] & 1 == 1" can be used
• Offsets within a protocol can be used, i.e. ip[12]. Offsets like this start from the beginning of the layer being referenced.
• It is possible to put together some very complex filter statements using AND (&&), OR (||) and NOT (!) operators in conjunction with parentheses.

See the pcap-filter manpage for further details. With experimentation you can almost certainly filter out most of the junk even if it is not possible to cut it out altogether.

Final Tip

While you are practising with these filters you will probably find that you make mistakes with offsets and generally defining the filter correctly. One good way to learn and also to prove your filters work before deploying them is to take a live capture at the point where you plan to sniff, then, on a non-production box, use tcpreplay to pass the traffic while capturing with your filter applied.

References

RFC 4271 - A Border Gateway Protocol 4 (BGP-4) -  http://tools.ietf.org/html/rfc4271
pcap-filter manpage - http://manpages.ubuntu.com/manpages/lucid/man7/pcap-filter.7.html

Simulating a broken LNS

A common requirement when testing a LAC is to confirm its reaction when various failure codes are returned by the LNS. In theory you would expect the LAC to react to an LNS failure in the same way (i.e. try another) irrespective of the error type or code returned, but as we all know theory and practice don't always align and that is why we test.

I recently had to prove exactly this area of functionality and found that, while it is relatively easy to put an LNS together which will terminate sessions, it's actually quite hard to get a real LNS to return error messages. Would you believe that they appear to be designed not to fail?

So the aim was:

• To have an 'LNS' which could be configured to reject incoming start control connection requests (SCCRQs)
• To be able to configure the result code, error code and, to make the packet captures easier to read and more authentic, the error message contained within the StopCCN message
• Ideally, to be able to service requests arriving on multiple IP addresses

As usual, the answer to this problem turned out to be scapy.

Important:

The script shown below does exactly what I needed but doesn't exactly work how you might expect. In order to reduce reconfiguration between test cases I have made it respond to queries arriving on any IP address - it does this by inspecting the incoming SCCRQ's source and destination MAC and IP addresses, then flipping them around on the response. That means that it does not attempt to bind to port 1701 on the host, therefore if the LAC sends an SCCRQ to the host's real IP it will get an ICMP unreachable and a StopCCN back. This is almost certainly not what you want.

The intended use case for this script is to have the LAC attempt to connect to an LNS which is "behind" the host running scapy, i.e. the last hop router should have a static route directing traffic for the LNS via the scapy host, in effect creating the following topology:

Alternatively, you could use a static ARP entry on the gateway router to direct traffic for an address on the attached LAN to the scapy host.

Usage

Usage is simple - firstly run scapy, then call 'execfile("BrokenLNS.py")' to load the script. You must create an instance of "LNS" and then, if the defaults to not suit, set the following member values:
 interface (default "eth1")

• resultcode (default 0)
• errorcode (default 4)
• errormessage (default "Internal error")

The script will sit there and close as many sessions as you care to offer it. Press control-C to stop.

Example

root@scapyhost:~/Projects/BrokenLNS# scapy
WARNING: No route found for IPv6 destination :: (no default route?)
Welcome to Scapy (2.0.1)
>>> execfile('BrokenLNS.py')
>>> lns = LNS()
>>> lns.resultcode = 1
>>> lns.errorcode = 6
>>> lns.errormessage = "Oh, no!"
>>> lns.run()
Received L2TP packet from 1.2.3.4
Got an SCCRQ
Sending spoofed StopCCN from 172.16.0.20  to 1.2.3.4.
.
Sent 1 packets.
Received L2TP packet from 1.2.3.4
^C>>>

Code

import os
# Flags
MANDATORY = 32768
HIDDEN = 16384
CONTROL = 32768
L = 16384
S = 2048
# Types
CONTROLMESSAGE = 0
ERRORMESSAGE = 1
PROTOCOLVERSION = 2
HOSTNAME = 7
RECVWIN = 10
FRAMING = 3
BEARER = 4
FIRMWARE = 6
TUNNELID = 9
CHALLENGE = 11
# Control Message Types
SCCRQ = '\x00\x01'
SCCRP = '\x00\x02'
StopCCN = '\x00\x04'
def word(value):
# Generates a two byte representation of the provided number
  return(chr((value/256)%256)+chr(value%256))
def AVP(bitmask, vendor, attribute_type, data):
# Generates an L2TP AVP using the given attribute number and payload
  length = len(data) + 6
  return(word(bitmask + (length % 1024)) + word(0) + word(attribute_type) + data)
def genL2TP(flags, tunid, sessid, ns, nr, payload):
# Generates an L2TP payload with the given parameters and AVP payload
  length = len(payload) + 12
  return(word(flags | 2) + word(length) + tunid + sessid + word(ns) + word(nr) + payload)
def getAVP(avp, payload):
  loc = 0
  while(loc < len(payload)):
    avp_type = payload[loc+2:loc+6]
    avp_len = ((ord(payload[loc:loc+1]) & 3) * 256) + ord(payload[loc+1:loc+2])
    # Uncomment the following line if you want to see info on every AVP checked
#    print "Got AVP " + str(ord(avp_type[0:1])).zfill(2) + str(ord(avp_type[1:2])).zfill(2)  + str(ord(avp_type[2:3])).zfill(2) + str(ord(avp_type[3:4])).zfill(2) + " of length " + str(avp_len) + " value " + payload[loc+6:loc+avp_len]
    if avp_type == avp:
      return(payload[loc+6:loc+avp_len])
    loc = loc + avp_len
class LNS(Automaton):
  interface = "eth1"
  resultcode = 0
  errorcode = 4
  errormessage = "Internal error"
# Define possible states
# Since this is so simple we only need one state :)
  @ATMT.state(initial=1)
  def WAIT(self):
    pass
# Define transitions
# Transitions from WAIT
  @ATMT.receive_condition(WAIT)
  def receive_sccrq(self,pkt):
    if (UDP in pkt) and pkt.dport==1701:
      print "Received L2TP packet from " + pkt[IP].src
      # scapy's built in L2TP handling doesn't deal well with control messages so
      # we just grab the raw data from beyond the UDP header
      payload = pkt[UDP].build_payload()
      # Check what type of L2TP message arrived by chopping off the header and passing
      # the rest to getAVP
      packet_type = getAVP(word(0) + word(CONTROLMESSAGE), payload[12:])
      if(packet_type == SCCRQ):
        # If we get an SCCRQ, generate a StopCCN in response.
        print "Got an SCCRQ"
        client_ip = pkt[IP].src
        server_ip = pkt[IP].dst
        client_mac = pkt[Ether].src
        server_mac = pkt[Ether].dst
        tun_id = getAVP(word(0) + word(TUNNELID), payload[12:])
        print "Sending spoofed StopCCN from " + server_ip + "  to " + client_ip + "."
        sendp(Ether(src=server_mac, dst=client_mac)/IP(src=server_ip, dst=client_ip)/UDP(sport=1701, dport=1701)/Raw(load=genL2TP(CONTROL | L | S, tun_id, word(0), 0, 1, AVP(MANDATORY, 0, CONTROLMESSAGE, StopCCN) + AVP(MANDATORY, 0, ERRORMESSAGE, word(self.resultcode) + word(self.errorcode) + self.errormessage) + AVP(MANDATORY, 0, TUNNELID, word(12345)))), iface=self.interface)
        raise self.WAIT()
      elif(packet_type == SCCRP):
        print "is an SCCRP"
      elif(packet_type == StopCCN):
        print "is a StopCCN"
      else:
        print "is a ZLB or non-control message"

Using Scapy to test PPPoE AC-Cookie validation

AC-Cookies are a mechanism designed to help mitigate certain denial of service attacks against PPPoE access concentrators. To understand the function it is important to first understand the normal flow of the PPPoE discovery process, which is as follows:

1. The PPPoE client sends a broadcast PADI (initiate) message
2. Any PPPoE access concentrators willing to service the client respond with a unicast PADO (offer) message
3. The client selects which access concentrator to use and unicasts a PADR (request) message asking for a session to be established
4. The access concentrator unicasts a PADS (session) message to the client to indicate that the session has been established

If an attacker is able to spoof PADI and PADR messages from a number of MAC addresses, a large amount of PPPoE state can be created in the access concentrator. An AC-Cookie is an unpredictable (to the client) value which is attached to the PADO message which must be echoed back in the PADR in order for it to be accepted by the access concentrator. Since the AC-Cookie cannot be predicted by the client, if the correct value is echoed back to the concentrator then it is extremely unlikely to have been spoofed and it is therefore safe for the access concentrator to allocate resources to the session.

This is all well and good but what if you need to prove the mechanism works or to show what error messages that are generated on the receipt of invalid AC-Cookies? As usual with my blog posts I have had to do this so I thought I would share the code. It's not going to win any awards but it works, all you need is scapy (I use 2.0.1, later should be fine).

Usage is pretty straightforward; simply run scapy, instantiate an object of type PPPoESession, override options as appropriate and then instruct it to "run()".

For example, to verify that a valid PPPoE session will come up:

root@client-pc:~/Projects/PPPoED# scapy
Welcome to Scapy (2.0.1)
>>> execfile("PPPoED.py")
>>> p=PPPoESession()
>>> p.outif="eth0"
>>> p.run()
[ debugging messages removed ]
Received PADS
>>>

Once the PADS is received, the process is complete and control returns to the console.

To verify that the access concentrator checks the value of AC-Cookies returned in PADR messages, we can set the script to reply using garbage values for the AC-Cookie tag as follows:

root@client-pc:~/Projects/PPPoED# scapy
Welcome to Scapy (2.0.1)
>>> execfile("PPPoED.py")
>>> p=PPPoESession()
>>> p.randomcookie=True
>>> p.retries=200
>>> p.run()
This will send a normal PADI and wait for a PADO before sending, up to the configured number of retries, PADR messages with randomised AC-Cookie tag values. When a PADS is received or the number of retries is exceeded, control returns to the console.

References

RFC 2516, Section 9 - http://tools.ietf.org/html/rfc2516

Code

import os
class PPPoESession(Automaton):
  randomcookie = False
  retries = 100
  outif="eth1"
  mac="00:10:20:30:40:50"
  hu="\x7a\x0e\x00\x00"
  ac_cookie=""
  ac_mac="ff:ff:ff:ff:ff:ff"
  our_magic="\x01\x23\x45\x67"
  their_magic="\x00\x00\x00\x00"
  sess_id = 0
# Method to recover an AC-Cookie from the tags
  def getcookie(self, payload):
    loc = 0
    while(loc < len(payload)):
      att_type = payload[loc:loc+2]
      att_len = (256 * ord(payload[loc+2:loc+3])) + ord(payload[loc+3:loc+4])
      print "Got attribute " + str(ord(att_type[:1])).zfill(2) + str(ord(att_type[1:])).zfill(2)  + " of length " + str(att_len) + " value " + payload[loc+4:loc+4+att_len]
      if att_type == "\x01\x04":
        self.ac_cookie = payload[loc+4:loc+4+att_len]
        print "Got AC-Cookie of " + self.ac_cookie
        break
      loc = loc + att_len + 4
# Define possible states
  @ATMT.state(initial=1)
  def START(self):
    pass
  @ATMT.state()
  def WAIT_PADO(self):
    pass
  @ATMT.state()
  def GOT_PADO(self):
    pass
  @ATMT.state()
  def WAIT_PADS(self):
    pass
  @ATMT.state(error=1)
  def ERROR(self):
    pass
  @ATMT.state(final=1)
  def END(self):
    pass
# Define transitions
# Transitions from START
  @ATMT.condition(START)
  def send_padi(self):
    print "Send PADI"
    sendp(Ether(src=self.mac, dst="ff:ff:ff:ff:ff:ff")/PPPoED()/Raw(load='\x01\x01\x00\x00'+'\x01\x03\x00\x04'+self.hu),iface=self.outif)
    raise self.WAIT_PADO()
# Transitions from WAIT_PADO
  @ATMT.timeout(WAIT_PADO, 3)
  def timeout_pado(self):
    print "Timed out waiting for PADO"
    self.retries -= 1
    if(self.retries < 0):
      print "Too many retries, aborting."
      raise self.ERROR()
    raise self.START()
  @ATMT.receive_condition(WAIT_PADO)
  def receive_pado(self,pkt):
    if (PPPoED in pkt) and (pkt[PPPoED].code==7):
      print "Received PADO"
      self.ac_mac=pkt[Ether].src
      self.getcookie(pkt[Raw].load)
      raise self.GOT_PADO()
#
# Transitions from GOT_PADO
  @ATMT.condition(GOT_PADO)
  def send_padr(self):
    print "Send PADR"
    if(self.randomcookie):
      print "Random cookie being used"
      self.ac_cookie=os.urandom(16)
    sendp(Ether(src=self.mac, dst=self.ac_mac)/PPPoED(code=25)/Raw(load='\x01\x01\x00\x00'+'\x01\x03\x00\x04'+self.hu+'\x01\x04\x00'+chr(len(self.ac_cookie))+self.ac_cookie),iface=self.outif)
    raise self.WAIT_PADS()
#
# Transitions from WAIT_PADS
  @ATMT.timeout(WAIT_PADS, 1)
  def timeout_pads(self):
    print "Timed out waiting for PADS"
    self.retries -= 1
    if(self.retries < 0):
      print "Too many retries, aborting."
      raise self.ERROR()
    raise self.GOT_PADO()
  @ATMT.receive_condition(WAIT_PADS)
  def receive_pads(self,pkt):
    if (PPPoED in pkt) and (pkt[PPPoED].code==101):
      print "Received PADS"
      self.sess_id = pkt[PPPoED].sessionid
      raise self.END()
  @ATMT.receive_condition(WAIT_PADS)
  def receive_padt(self,pkt):
    if (PPPoED in pkt) and (pkt[PPPoED].code==167):
      print "Received PADT"
      raise self.ERROR()

A quickie about logging in PuTTY

As a rule, I log every CLI session I ever run. It has saved my neck on countless occasions and just this morning saved me a couple of hours of effort tallying serial numbers. It's surprising how often I use these logs to find some forgotten bit of info or to prove that "it really was all working when I left it"!

Every saved session I create in PuTTY is configured to log into a common directory using a file name in the format year-month-day-time-hostname.log, which gives me a nice, searchable, date-sorted record of what I typed and saw for every session I've ever opened.

The best way to do this is by just configuring Session -> Logging to "All session output" and the file name to something appropriate (the string I use is "&Y-&M-&D-&T-&H.log"). If you like, while you're there, go into Window and set the scrollback buffer to 2 million lines. Finally, return to the Session screen and save as "Default Settings".

From then on all your sessions you create, including ones where you just enter the IP and connect, will be logged. Note that any pre-existing saved sessions will need to be edited if they were not originally set to log.

Do it now, you know it makes sense!

Using Capture Filters with Encapsulated Packets

One of the most annoying things I found when I started working on carrier networks was that while Wireshark's display filters worked perfectly, the capture filters frequently did not. I would regularly set up a capture filter only to find that no packets at all were saved - that's a real pain if you want to pull a few easily described packets out of a 50 Mbps stream across a period of 20 minutes.

After a while I realised that my problem was related to encapsulation. Unlike the hierarchical and detailed display filters, capture filters have to be really fast - that basically means using bit masks and comparing values at fixed offsets. With plain old untagged Ethernet frames the filters work fine, however as soon as you add 802.1Q tags, PPP or MPLS suddenly all the offsets are no longer valid and anything you match will be purely coincidental.

Luckily there are filter keywords to handle that situation. All of the following adjust the offsets for you each time they are used:

vlan [x] - matches a single VLAN tag, the ID of which may optionally be specified by the user
pppoes - matches a PPPoE session header
mpls [x] - matches a single MPLS label, the number of which may optionally be specified by the user

These are very flexible - for example if you are capturing QinQ traffic, you could match all the SMTP packets using:

vlan && vlan && tcp port 25

If you know the VLAN IDs (or MPLS labels) in use, you can narrow the selection based on those. To show all the IGMP passing over a particular MPLS pseudowire with VLAN ID 200, you could use:

mpls 131066 && mpls 131068 && vlan 200 && pppoes && ip proto 2

For a long time I was using makeshift capture filters along the lines of "ether[39] = 2" to match pertinent bytes in the packet (see my next blog post for info on that) however you will probably agree this is much simpler. These filters are equally applicable to Wireshark, Tshark and tcpdump so they may be useful even when forced to capture using some really obscure UNIX box. For Tshark and tcpdump don't forget to put quotes around any expressions that use the ampersand (&).

IGMP Testing, part 1

Maybe it's just my famous inability to find things that are right in front of me but I've needed some tools over the last week that would let me 'play' with IGMP and I've drawn (almost) a total blank.

Firstly, I wanted to generate a good old-fashioned flood of reports to test processing performance and rate limiting.

Plan A was to use the tester for this - despite not having a specific tool for flood testing it does let you create streams of, more or less arbitrary, hand crafted packets. That's the theory, anyway. After carefully putting together a stream profile that should have given me join after join for cycling group numbers I put it to the test - only to find that it had other ideas and was generating complete garbage. By garbage I mean not even the IP headers were correct - the protocol was coming out set to 0xfd (unknown) rather than 0x02 for IGMP and, strangely, the source and destination IPs were populated with the group ID and source that should have been in the report payload. Based on bitter past experiences I didn't waste my time trying to fix that.

OK, time for plan B - back to the packet crafting on a PC. I thought I'd be spoiled for choice but, for Linux anyway, the only option for generating arbitrary IGMP seemed to be nemesis. Nemesis seems to be exactly what I want but it is no longer maintained and won't compile on a modem distro - at least *I* couldn't get it to compile.

My favourite scapy knows what IGMP is from its protocol ID but doesn't have a stack for it, so there was no straightforward way to use that.

Dead end. I couldn't find anything to build me one packet let alone throw 1000 out per second.

Then it occurred that, actually, in normal use the tester can generate valid joins at a civil pace... So I mirrored the tester port and sniffed a genuine join off the wire, whittled the capture file down to the single frame I wanted and fed it to tcpreplay. Yay.

One small problem - it could only manage 100pps and I needed 1000. I noticed it was generating a message every time it sent a packet saying it had re-opened the file, which gave me a hunch that the file operations and CLI might be a bottleneck. I solved that problem the same way as the first - by sniffing the 100pps output for a while and then replaying *that* at full tilt. 960pps... Not quite 1000pps but close enough!

At the last moment it occurred to me that it would be a more convincing test if I cycled the group IDs rather than always reporting on one group. I went back to scapy and, with Wireshark in the other hand, started to play. I thought if I just loaded in the original join packet I could use a loop to tweak a byte or two for the group ID and dump it out to a file which I could then replay.

When I did that I noticed that my router still only showed one group as joined. Rubbish. I had obviously missed something. Looking at the generated file in Wireshark I could see that its checksum was incorrect.

Scapy could re-calculate the IP checksum for me but it didn't understand IGMP so that was going to be a programming exercise. The checksum is only 2 bytes in the payload so it wasn't too hard to adjust. I won't bore you with the maths, check out RFC 3376 if you're curious.

Finally, with that done, I had a pcap file full of valid joins over 100 groups and the ability to fire them out at (nearly) 1000pps. I can't help thinking it should have been easier, though!

Source code to follow - it's very scruffy and fairly fragile but might be useful to someone else... You never know!