Removing VLAN/MPLS/PPPoE/GRE/GTP/VXLAN Encapsulation Headers from pcap Files

Many years ago, when I worked in a school, I used to port mirror our proxy server to an old PC running driftnet and leave the screen where the kids could see it as a warning that staff could "see what you're doing on the Internet". I haven't played with driftnet since but certainly at the time it could only handle native frames (no VLAN tags, certainly no MPLS or PPPoE). I vaguely remember some other tools being similar, unfortunately I can't remember which ones.

Looking at the analytics for this blog, I can see I'm not the only one who's had the problem. It's certainly not the number one issue that people are searching for when they get here but there have been a few and the thought occurred that the packet processing engine I wrote for dechap would be really good for this task - it already stripped back VLANs and MPLS, plus it knows how to detect PPPoE and L2TP.

After a couple of hours it was working to the point of being able to strip VLANs and MPLS off, with a little more effort PPPoE also gave way. GRE came quite easily, too, as it has simple headers and uses the same etypes as Ethernet.

Anyway, here is "stripe" (from STRIP Encapsulation), a command line tool which takes a pcap file as input, re-assembles IP fragments and strips off all the encap it can (currently VLAN tags, MPLS shim headers, PPPoE, L2TP, GRE GTP and VXLAN) then outputs another pcap containing just payload over Ethernet.

**UPDATE** - Version 0.3b now adds support for VXLAN.

Download

Stripe is available from my github: https://github.com/theclam/stripe

Usage

The command line is pretty straightforward, as shown in the online help:

Harrys-MacBook-Air:stripe foeh$ ./stripe
stripe: a utility to remove VLAN tags, MPLS shims, PPPoE, L2TP headers,
etc. from the frames in a PCAP file and return untagged IP over Ethernet.
Version v0.1 alpha, November 2014

Usage:
./stripe -r inputcapfile -w outputcapfile

Where inputcapfile is a tcpdump-style .cap file containing encapsulated IP 
outputcapfile is the file where the decapsulated IP will be saved

Harrys-MacBook-Air:stripe foeh$ 

Simply specify the files you want to read encapsulated packets from (-r) and write the cleaned up packets to (-w). Stripe will remove as many layers of encap as it can until you are left with straight payload over Ethernet.

How it Works

The majority of stripe's work is done by the "decap" function. This function takes in a block of memory, a length parameter, a data type hint and a frame template. The process runs as follows:

1. If the type is Ethernet, populate the source / destination MACs of the frame template
2. If the type has an Ethertype or protocol type field, use this to populate the ethertype of the frame template
3. If the next protocol is possibly or definitely payload, set the payload pointer of the frame template to the address of the next protocol and return
4. If the next protocol is possibly or definitely encapsulation, call decap against the remainder of the packet

So essentially it eats up encap, recording MACs and protocol types as it goes, until there is no more encap left. By the end there is a fully populated frame template with source and destination MAC (the innermost copy if there are multiple as in the case of MPLS pseudowires), the etherype of the payload and the payload itself. Piecing these together gives a minimally encapsulated frame, i.e. one with just an Ethernet header and payload.

Here is a worked example for a frame with VLAN, MPLS, GRE over IP and an IP payload:

Step 1 - The "decap" function is called on the entire frame. Since the first header is Ethernet, the frame template gets populated with the source / destination MACs and the etype from the Ethernet header. The frame template's length field gets populated with the size of the frame minus the Ethernet header and the payload pointer is adjusted to point at the next header. The decap function then calls itself on the remainder of the frame, hinting that the type is VLAN tag based on the current header's etype.

Step 2 - The decap function now considers the partial frame starting at the VLAN tag. Since the VLAN tag has an etype associated, the frame template's etype is overwritten with the one from the VLAN header. The length is overwritten with the length of the payload after the VLAN header and the pointer adjusted to point at the next header. The decap function then calls itself again with a hint of MPLS, based on the etype in the VLAN header.

Step 3 - The decap function now considers the partial frame starting at the MPLS label. Since the MPLS label is bottom of stack, we know there are no more MPLS labels left . Unfortunately there is no protocol type in an MPLS header (these are signaled on the control plane) so we have to take a peek at the byte immediately following the label. If we find a "4" or a "6" in the high order nibble then we have to guess that the next protocol is IPv4 or IPv6, respectively. If the following four bytes are all zeroes then we assume Ethernet over MPLS with control word, otherwise we assume Ethernet over MPLS without control word. In this case we find a 4 in the low nibble, so call decap with an "IP" hint.

Step 4 - The IP header tells us that GRE is the next protocol so for now nothing changes in the frame template (the remainder could be decodable or not). We just call decap again on the GRE part...

Step 5 - The GRE header is decoded and the etype is copied into the frame header. The length of the remaining payload is updated in the frame template and the pointer is adjusted. Decap is called on the next header, which is IP. When the decap function inspects the IP payload it can go no further and just returns the frame template.

In essence, the process has started with a deeply encapsulated frame and ended with IP over Ethernet. The source and destination MACs are taken from the innermost ones found (which in this case is the outermost Ethernet header) but with the etype changed to match the payload, which is the first non-encapsulating payload found in the frame, in this case the second IP.

References

https://tools.ietf.org/html/rfc2784
https://tools.ietf.org/html/rfc1701
http://www.ieee802.org/1/pages/802.1Q.html
http://www.3gpp.org/DynaReport/29060.htm

BGP support added to dechap

Hot on the heels of adding the ability to attack OSPF MD5 authentication, I've added BGP support to dechap. It is now possible to feed a pcap file with PPPoE, L2TP, RADIUS, OSPF and BGP packets to the same tool and perform offline dictionary attacks on the authentications within.

As usual, if you're not interested in the theory just skip right to the end for the download link.

TCP MD5 Signatures

BGP authentication uses the MD5 Signature TCP option field, which is defined in RFC 2385. Personally, I found this RFC very vague and it took a lot of iterations to get the technique right. It's particularly fuzzy about what is included in the hash, what isn't and how to present values correctly. I'm hoping to document the process a little more clearly for the next poor guy who tries to implement it as I couldn't find a sufficiently detailed reference anywhere.

RFC 2385 states that the hash must be calculated over the following:

1. the TCP pseudo-header (in the order: source IP address,
   destination IP address, zero-padded protocol number, and
   segment length)
2. the TCP header, excluding options, and assuming a checksum of
   zero
3. the TCP segment data (if any)
4. an independently-specified key or password, known to both TCPs
   and presumably connection-specific

Now, maybe it's just me, but this raised a lot of questions in my mind. Zero padding usually means to fill the trailing space with zeros, but padding the second byte would effectively multiply the protocol number by 256 so should it be a leading zero? Which headers and options are included in the "segment length"? Should the pad bytes be copied with the TCP header?

Through a lot of trial and error I found that:

• The zero padding goes before the protocol number
• The "segment length" includes the TCP header, the TCP options (including room for the MD5 signature option being calculated) and the actual payload data
• The copied TCP header should be 20 bytes long, i.e. includes two padding bytes after the (zeroed out) checksum. The header length remains as-is, including the length of the options.
• The TCP segment data starts immediately after the TCP options and runs to the last byte indicated by the IP length field
• The null byte terminating the password is not passed to the hash algorithm

 The resulting hash value is then stored inside the MD5 signature option (kind 19, length 18).

Checking / Attacking BGP Packets

Using the above method it is straightforward to run a dictionary attack as follows:

• Start with a sniffed BGP packet (see the original dechap blog post for info on how this is extracted).
• Extract and store the authentication hash (look for option kind 19) for later comparison
• Put together the "pseudoheader" as described above
• Append the TCP header without options
• Append the TCP payload
• Append the candidate password
• Calculate the MD5 hash over the complete data set and compare to the value seen in the sniffed packet. A matching hash indicates a matching password.

As of v0.4a, dechap can now be used to automate this process.

Obtaining the Tool

The C source code may be downloaded from: https://github.com/theclam/dechap

Provided the OpenSSL dev libraries are installed it should be possible to simply extract the source code, cd into the directory then run "make". I've only tested this under Ubuntu Linux but there are very few dependancies so I would imagine it will work on most distributions.

Using the Tool

As usual - this is for legitimate audit and recovery purposes and must not be used for any kind of malicious activity.

The usage is pretty straightforward - there are only two parameters and both are mandatory. Specify your capture file (original pcap format) with the -c flag and your word list with the -w flag. Here's an example:

lab@lab:~/dechap$ ./dechap -w mywords.txt -c bgp.cap
Found password "password1" for TCP from 10.0.0.2 to 10.0.0.1.
Found password "password1" for TCP from 10.0.0.1 to 10.0.0.2.
Found password "password1" for TCP from 10.0.0.2 to 10.0.0.1.lab@lab:~/dechap$
I'm not sure how quickly it runs but it doesn't seem quite as quick as the OSPF version. I suppose BGP packets tend to be a little bigger than OSPF so there's more to hash. You can improve the speed by only including one packet for each source / destination pair in each capture as, at present, it doesn't check for multiple packets between pairs and attacks each instance individually.

If you try this out, please leave a comment on this post with your experiences - good or bad. Any suggestions would also be welcome, particularly for other protocols to attack.

References

RFC2385 - Protection of BGP Sessions via the TCP MD5 Signature Option
RFC1321 - The MD5 Message-Digest Algorithm

Bringing Up a PPPoE Session - The Theory

In a previous post, I shared a Scapy script that implements the PPPoE discovery stage and stops once the session stage is reached. As handy as that script is for testing AC Cookie validation, it is not particularly useful for anything else. It would be much better if the script could bring the PPP session all the way up.

Luckily, the PPPoE discovery script is a cut-down version of another script that I wrote a long time back which goes all the way from PPPoED, through LCP and CHAP authentication and stops at IPCP. At the time, the script was far too messy to share but I've tidied it up and it is now in a state that it could be useable by others. I've also added IPCP negotiation and a couple of methods for sending and receiving IP traffic over the resulting session.

Before I present the script, I'll cover the theory involved, step by step. The impatient may want to just go to the next post (when it is available) for the script itself and instructions on how to run it.

PPPoE Discovery

PPP is (a) point-to-point protocol, designed to run over a dedicated link between two devices. Ethernet is a multi-access network, so if we want to run PPP over Ethernet then we need a mechanism to discover peers and establish a point-to-point relationship between two devices over the shared medium.

PPPoE provides this service and operates in two distinct stages:

1. Discovery: The discovery stage is responsible for locating PPPoE peers and negotiating session parameters so that, ultimately, a PPPoE session can be created.
2. Session: Once the discovery stage is complete the protocol enters the session stage, at which time the two peers have a tunneled connection between them over which to start passing PPP.

Once the session stage is reached, the peers bring up and operate their PPP session exactly as they would over a dedicated link.

The diagram below summarises the PPPoE "Discovery" stage:

PPPoE State Transitions

The first step in the journey is to find a PPPoE access concentrator which is willing to terminate our session. To do this, we must broadcast a PPPoE Active Discovery Initiation (PADI) message. It is possible to specify a service name in the PADI - this is just a string that identifies a particular type of service in which the client is interested. The access concentrator may use this to decide whether or not to offer to terminate the session, though in most cases it is just ignored. For this reason clients generally use an empty service name.

Any access concentrators listening on the segment will receive the PADI message, inspect its contents and then make a decision whether or not to make an offer to terminate the client's session. If the access concentrator is willing to terminate the session, it signals this to the client by sending a unicast offer (PADO) message. Typically, the PADO has an AC-Cookie attached to it - essentially the AC-Cookie is an "unpredictable" string, derived from the client's MAC address, which the access concentrator uses to mitigate against certain kinds of resource exhaustion attacks. When AC-Cookies are used, a PADO is generated 'mechanically' from the incoming PADI and no state is created on the access concentrator at this point.

When the client has received at least one PADO, it must select a favourite. It is common to just use the first offer received, but other selection criteria may be used. The client then sends a unicast request (PADR) to the chosen access concentrator, indicating that it would like to access its offer. If an AC-Cookie was contained in the PADO message then is echoed back in the PADR. The requirement to echo the cookie back to the access concentrator is designed to validate that the client really exists and is available on the MAC address where the PADO was sent.

Finally, it is up to the access concentrator to confirm that it the session has been created. If AC-Cookies are in use then the incoming PADR is examined to check whether the AC would have generated the provided cookie given the source MAC - in the case of a mismatch the PADR is silently dropped, otherwise the session state is created in the AC and a session (PADS) message is unicast to the client to confirm that the session has been created and the "Session" stage has begun. The PADS always contains a PPPoE session ID number, which is used to discriminate between multiple PPPoE sessions on the same LAN. The session ID is used to differentiate between multiple PPPoE sessions on the same LAN and must be present in the header of every PPPoE frame exchanged with the AC during the "Session" stage.

The fifth type of PPPoE discovery message is the terminate (PADT) which, as its name suggests, is used to terminate (i.e. end) a session which has been established. Either end may send a PADT message to close the session and once a PADT has been received, no further traffic may be sent for that session.

PPP

PPP itself consists of a number of sub-protocols. There are:

• Link Control Protocol (LCP) which is responsible for negotiating overall link parameters
• PAP and CHAP which are used for authentication
• A family of Network Control Protocols (NCPs) used to negotiate the transport of each upper layer protocol

PPP also defines that once a higher layer protocol has been negotiated by its corresponding NCP, that protocol's traffic will be encapsulated with header indicating that particular protocol's protocol number.

Link Control Protocol (LCP)

RFC 1661 defines LCP as the protocol that is responsible for "establishing, configuring,
and testing the data-link connection." Essentially this means that LCP is used to bring up and take down PPP links, negotiate the configuration parameters and check that the link is still alive. There are a range of LCP codes which are used to fulfil these aims, discussed below.

Configuration Type Codes

In order to bring up a PPP session both peers must agree on certain parameters, for example the maximum size of frame that may be passed, whether to use compression and so on. Both peers propose the settings they would like to use - the opposite peer will then either acknowledge (accept), nak (i.e. suggest alternative) or reject (outright refuse) the proposed options. The aim is to reach a state where the opposite peer has acknowledged the locally proposed parameters.

The following LCP codes are standard and must be implemented:

Configure-Request - Used to propose a set of parameters that we would like to use for the session. The peer will then respond to the proposed parameters with one of the next three responses.

Configure-Ack - Used to advise the peer that their proposed parameters are acceptable. The accepted parameters are echoed back in the ack message.

Configure-Nak - Used to advise the peer that their proposed parameters are not acceptable and that the alternative values should be used. The proposed changes are attached to the nak message.

Configure-Reject - Used to advise the peer that their proposed parameters are not supported and cannot be used. The unacceptable parameters are echoed back in the reject message.

Termination Type Codes

Either peer may request to terminate the session at any point and the opposite peer must honour that request. There are two termination related codes in LCP:

Terminate-Request - Generated by a peer to initiate the tear-down of the link. A Terminate-Request should be re-sent if no Terminate-Ack is received in response.

Terminate-Ack - Generated to confirm receipt of a Terminate-Request. A Terminate-Ack must be generated in response to a Terminate-Request.

Liveness Check Codes

LCP includes a ping-like echo mechanism to verify that the opposite peer is still available, with LCP in an open state and is responding. The same mechanism is used to detect a looped interface - due to the symmetric nature of PPP it's quite possible to negotiate a connection to yourself without necessarily realising or for a connection to be looped mid-session. The following codes are used for liveness checks:

Echo-Request - Sent to the remote peer to solicit an Echo-Reply message. There is no requirement to negotiate the use of LCP echoes and an Echo-Request may be generated at any time while LCP is open. If the Magic-Number option was negotiated during LCP, the Echo-Request must contain the "random" 4 octet magic number decided at that time.

Echo-Reply - Sent in response to an Echo-Request message. When LCP is open, an Echo-Reply message must be sent whenever an Echo-Request is received. The magic number contained within the incoming Echo-Request must be copied into the outgoing Echo-Reply. If the incoming packet has our magic number then the connection has become looped.

Other Codes

There are other codes such as Code-Reject, Protocol-Reject and Discard-Request which do pretty much what you would expect. You don't get to see them very often so I will not discuss them here. I suggest referring to RFC 1661 for more detail on these.

LCP State Diagram

Below is a simplified state diagram showing how LCP makes its way from the "Starting" state into an "Opened" state. Most parts of PPP are referred to as "open" when they are up and running. I have omitted a number of transitions that deal with strange corner cases (like if the peer acks something we never sent, etc) and also transitions related to closing the connection (the Term commands discussed above). RFC 1661 contains a complete state transition table which is far more complex. If you bear in mind that at any stage either peer may terminate the session then this minimal version will cover 95% of "normal" cases.

LCP State Transitions

Authentication

Once LCP is open, the next stage is typically to start authentication. Authentication may be done by either, neither or both the peers as negotiated by LCP and can be done using plaintext PAP or MD5 hashed CHAP. If no authentication was negotiated by LCP, an implicit pass is assumed.

PAP is hardly ever used these days, is strongly discouraged and in any case is pretty simple, so I will not discuss it here. Please refer to RFC 1334 if you require details on PAP.

CHAP, though not immune to attack, offers reasonable security. The password itself is never sent "over the wire" and there is good protection against replay attacks via the use of random challenges. Here is how CHAP operates:

CHAP Authentication

Essentially, security is provided in two ways:

1. The password is never exchanged in the clear but instead is passed through a one-way cryptographic hash function. It is computationally infeasible to recover the password from the hash function's output, so it is quite safe to pass this output over the wire.
2. If the client just hashed the password, then it would be possible for an attacker to capture the hashed value and authenticate with the server at a later time by simply replaying the same response. CHAP requires the server to generate a random challenge string, which is also fed into the hash function and affects its output. Provided the server never re-uses a challenge value, an attacker cannot simply replay a previous authentication response to gain access.

When the CHAP response comes in, the server compares the received hash value with the output of a local calculation using the same method to determine whether the authentication attempt was successful. While this is precisely true when the server has a local copy of the password, typically this is not desirable and in practice the authentication check is deferred to an external RADIUS server. In order for the RADIUS to validate the attempt, the server must pass it a copy of the ID and challenge sent, plus the response received. The RADIUS can then use the ID, its own copy of the plaintext password and the challenge value to compute the expected response. If the expected and actual responses match then the RADIUS will return an "Accept" response, otherwise it will return a "Reject" response.

Network Control Protocols (NCPs)

Before any higher layer protocol can be passed through a PPP tunnel, it must be negotiated by a corresponding NCP. For example before you can pass IP through a PPP tunnel, IPCP must be open, indicating that all the required IP parameters have been successfully negotiated. To pass OSI traffic, OSICP must be open. For IPv6, IPV6CP is used.

The operation of each NCP is different but they all essentially follow the same model as LCP - parameters are proposed by each peer and ack'd, nak'd or rejected by the opposite peer. - and the state transition diagram pretty much looks the same.

IPCP

I'll go into a little more detail on IPCP since that is the most commonly used (for now) with a worked example of a DSL subscriber connecting to his ISP, starting immediately after authentication succeeds.

Client Side

The client generally does not know anything when it first connects and relies on the server to provide it with everything it needs. The client sends a Configure-Request proposing an IP address, primary and secondary DNS of 0.0.0.0. Proposing 0.0.0.0 for these is actually an  explicit request for the server to provide legitimate values for the client to use.

The server will then respond with a Configure-Nak message containing the IP address and DNS servers that the client should use.

The client will then send another Configure-Request with the newly acquired details, to which the server responds with a Configure-Ack.

Server Side

The server will typically send out a Configure-Request containing only its own IP address. There is no reason to argue over this so the client should just respond with a Configure-Ack. If the client tries to push a different address to the server using a Configure-Nak, it is typically ignored and after a few retries the session gets pulled down.

Passing Traffic

Once the two peers are agreed and IPCP is open, IP packets may be passed through the PPP tunnel by attaching a header - in most cases, for PPPoE connectivity, the PPP header consists of only a two byte protocol number (0x0021 for IP). The protocol number is analogous to the EtherType field of an Ethernet frame and indicates to the receiver how to interpret the payload. Alternative encapsulations exist - refer to RFC 1662 for more details on HDLC style framing which is often seen in L2TP.

Further Reading

That about covers the protocols involved in bringing up a PPPoE session at a high level. If you require more information I would suggest turning to the following RFCs:

RFC 2516 - PPPoE - http://tools.ietf.org/html/rfc2516
RFC 1661 - PPP - http://tools.ietf.org/html/rfc1661
RFC 1994 - CHAP - http://tools.ietf.org/html/rfc1994
RFC 1332 - IPCP - http://tools.ietf.org/html/rfc1332
RFC 1877 - IPCP extensions for DNS - http://tools.ietf.org/html/rfc1877