Bending the MPLS Security Model part 1 (introduction)

A lot of people think MPLS is quite a secure technology - perhaps it's all the talk of VPNs or the fact that various government security standards permit the use of VRFs to segregate data at different security levels while VLANs are not considered secure enough. I've always thought that was a bit strange, because fundamentally when on the wire VLANs and MPLS both use a 4 byte tag to determine what can talk to what.

In this mini-series of blog posts I will briefly go through the building blocks of layer 2 and layer 3 MPLS VPNs, the controls these provide and a few gotchas before finally exploring some ways to use and abuse the mechanisms to cause mischief on the network.

Introduction to the MPLS Security Model

In addition to its unrivaled flexibility and scalability, the most significant benefit of MPLS is its very simple (and therefore fast) data plane:

• As frame enter at the MPLS edge, the PE adds (pushes / imposes) labels to the frame header.
• At each hop inside the MPLS core a simple table lookup is done on the outer label number. The lookup returns a replacement label, outgoing interface and encap to be written onto the outgoing frame (a label swap).
• As frames reach the edge of the MPLS core, the PE removes (pops) the outer label and acts on (i.e. routes, switches or processes) whatever it finds inside.

For "vanilla" MPLS networks, two labels are used. The first, or inner, label is used to indicate what layer 2 or layer 3 service should handle the frame. The second, or outer, label is known as the transport label and is used to get the frame to the correct endpoint. Service labels need only be understood by the ingress and egress devices, meaning that intermediate devices only need to be able to interpret and act on the transport label. So the transport label is to get where you're going, the service label is to do what you're doing.

Labelled traffic can be queued / scheduled at the interface for QoS purposes but once traffic is labelled there's very little that can be done with regards to manipulation. ACLs can't be applied to filter labelled traffic and, since there is no "source" field in an MPLS header, uRPF checking is not possible.

The whole security model of MPLS is premised on the fact that traffic cannot pass without the right labels and route / label learning is restricted by the control plane. For anyone not familiar with MPLS VPNs, I'll provide a very high-level introduction to how this happens in practice.

Layer 3 VPNs

In order to maintain several, possibly overlapping, routing tables, an MPLS router uses the concept of VRFs. A VRF is a distinct routing instance, a bit like a virtual router, with its own interfaces, routing table and routing protocols. Internally the MPLS router uses a route distinguisher, or RD, to keep one VRF's routes separate from another's.

Separation is achieved by basically bolting the RD onto the beginning of every IP prefix known within the VRF. So for instance, if VRF "A" uses RD 100:100, when it learns a route to 10.10.10.0/24 that will be known internally as 100:100:10.10.10.0/24. If an identical prefix is learned in VRF "B" with an RD of 200:200, that will be known internally as 200:200:10.10.10.0/24. Since the extended versions of the two prefixes are different, there is no conflict and both can be used independently.

Layer 3 MPLS VPNs use MP-BGP to share routing information between devices. MP-BGP works with the extended prefixes described above and any VPN route that will be advertised to other devices must have a route target community attached (defined as an export route target for the VRF). That route can only be learned by another VRF, either locally or remotely, which is configured to import routes with that target.

In simple VPNs the import and export route targets match, meaning that every member VRF learns the routes that are present in every other member VRF. This produces a basic any-any connectivity model.

Any-Any VRF Topology

Clearly, since each VRF learns routes from every other VRF there is a full mesh of connectivity.

The import and export can be controlled independently, so a variety of topologies can be created quite simply. For example, to create a hub and spoke setup, simply configure the hub to export the hub route target while importing the spoke route target and configure the spokes to export the spoke route target while importing the hub route target.

Hub and Spoke VRF Topology

Spokes do not learn from each other because they only import routes with the hub target. Without knowledge of each other's routes, no traffic can pass between the spoke VRFs.

Another common application of multiple route targets is in setting up "grey management" VRFs - that is a single VRF containing centralised management platforms which are used for operational support and maintenance of devices in multiple customer VRFs.

Grey Management VRF Topology

The grey management routes are leaked into each customer's VRF, providing reachability to the NMS. The management VRF in turn imports each customer VRF's routes to provide reachability to the managed CPEs. Typically the managed CPEs will each have a loopback address assigned from a known range and the management VRF will have an import filter allowing only the routes to those loopbacks to be imported. Public address space is usually used to avoid any addressing conflicts within the customer VRFs.

Gotchas

In both the hub-and-spoke and grey management scenarios there is a centralised point which has all the routing information and can communicate with everything, while the peripheral VRFs only learn a reduced set of routes and therefore can only communicate with the networks within their limited view. Spokes do not import the spoke route-target and therefore cannot "see" each other. Customer VRFs do not import each others' route targets and so separation is maintained.

One big mistake that can easily be made is for the spoke to accidentally advertise a summary or default route. The diagram below shows what happens in this case:

Hub and Spoke Topology Broken by Default Route

The spokes still cannot learn each others' routes, however they do learn the default route from the hub. If PC B tries to send traffic to PC C's IP address, the traffic will follow the default route towards the hub. The hub knows all the spoke routes and so sends the traffic onwards towards PC C. So despite VRF B not containing any routes from VRF C and vice-versa, traffic can still flow between them.

The workaround: don't summarise or default route within overlapping or partial mesh VRFs, particularly at hub sites.

Layer 2 VPNs (Pseudowires)

MPLS pseudowires are essentially point-to-point tunnels that transport layer 2 frames using MPLS transport. A pseudowire simply takes a frame in one end and moves it, as is, to the other end. Pseudowires are typically signalled using targeted LDP - that is a Link Distribution Protocol connection directly between the PEs on each end of the pseudowire. The two endpoints communicate directly with each other to signal the labels to be used for a particular pseudowire, as differentiated by a virtual circuit identifier (VCID).

Since the necessary service labels are signalled directly between the two interested PEs, they are not known elsewhere in the network and no other devices can participate in the pseudowire, i.e.  it is point-to-point.

Gotchas

Due to the simple nature of pseudowires, the gotchas are pretty minor. Provided the two peers are configured with matching VC IDs, most problems are going to be down to "schoolboy errors":

• Mismatched VC IDs
• Mismatched VC types (VLAN vs. Ethernet)
• Broken label-switched path between peers
• TLDP session unable to come up (control plane filters, mismatched LDP keys)
• Mismatched MTU (each vendor seems to count in a different way)
• Port MTU smaller than the configured VC MTU

Summary

This post really only reviewed at a very high level how MPLS VPNs are set up and how one is kept distinct from another. In the next post I will start to look at some weaknesses and opportunities for mischief.

IOS XR Gotchas

I've recently started doing some work with IOS XR - I have to say I'm quickly getting to love XR and, if I'm honest, completely going off IOS as far as MPLS and BGP are concerned. I've found a couple of gotchas around the way RPL and BGP works as compared to normal IOS. Aside from the obvious need to type "commit" every time you make changes there are a few quirks that have caught me out - I expect I'll continue to add to this post as and when I get caught out by new and wonderful things.

No Policy = No Routes

One of the big, fun changes in IOS-XR is that if you don't apply either an inbound or an outbound route policy to a BGP peer then it assumes you've made a mistake and does not advertise or accept any routes at all. You can see that this is the case because the peer is marked with an excalmation mark (!) in the BGP summary.

The fix: apply either an inbound policy, an outbound policy or both.

Soft Reconfiguration Inbound

Another surprise when migrating to IOS-XR from IOS is that the rules for soft-reconfiguration have changed. Originally, before route-refresh capability came along, soft-reconfiguration was primarily there so that you could edit an inbound route-map and apply it without doing a hard reset on the BGP session. IOS-XR is a bit of a smarty-pants so if you configure "soft-reconfiguration inbound" and the peer turns out to support the refresh capability, it decides you don't need that local copy of the received routes - if it needs to apply an updated policy it can just request a refresh from the peer.

Most people who do a lot with BGP will be familiar with how useful it is to have a copy of all the routes received from a peer, including the ones dropped by policy, for troubleshooting purposes. Luckily, it is possible to force the router's hand by configuring "soft-reconfiguration inbound always".

Empty Prefix Sets Don't Work

IOS-XR has a new CLI construct called a set - mathematicians will already be familiar with these as an un-ordered group of items. In RPL sets can be used for communities or prefixes, but beware - if you create a prefix set but leave it empty, the CLI will accept it but any policies that reference the set will fail. If you're seeing a message like this:

% The policy [policy-name] uses an invalid argument to the [pfxmatch] condition on the [destination] field. Internal error:  no parameters

... then you've got an empty prefix set somewhere. Curiously it doesn't seem to mind about empty community sets. The workaround for this is to put a dummy prefix into any sets that need to be there but don't currently have anything in them.

Applying a Route Policy (or maybe destroying it!)

One nasty bit of CLI I've found is this: to apply a route-policy to a BGP peer you would type "route-policy policyname in" or "route-policy policyname out" under the configure -> router bgp -> neighbor context. If, however, you omit the "in" or "out" keyword, XR kindly assumes you want to blow away the policy and build it from scratch! Luckily you can just type "abort" to escape disaster... but you lose whatever you changed since your last commit.

Unexpected LDP Support in dechap

About a week ago I made a couple of updates to dechap which allowed it to run dictionary attacks firstly against OSPF packets and then against BGP packets. Since then I've been thinking about adding other protocols such as TACACS+ (turns out to be hard, see my other post on this), LDP, HSRPv2, you name it.

This evening I decided to sit down and work on LDP support, only to discover that just like BGP, LDP uses the TCP MD5 signature option for authentication. After pausing for a moment to wonder if I really was that lucky I decided to knock together a quick test. Basically the answer is "yes, I am that lucky" and "yes, dechap v0.4a works against LDP". So I can add another protocol to the growing list without even modifying the code.

So to summarise, dechap can now attack:

• PPP / PPPoE CHAP authentication
• RADIUS CHAP authentication
• L2TP CHAP authentication
• OSPF MD5 authentication
• BGP MD5 authentication
• LDP MD5 authentication

...straight from the pcap file, even if the captured traffic has MPLS labels and / or VLAN tags.

Time to move onto the next target... any suggestions?

Meanwhile, dechap v0.4a can be downloaded at my github.

Be Careful where you use TACACS!

As part of my on-going work to add more and more protocols into my hobby project dechap, I started looking into the workings of TACACS+ today. I was looking to see whether TACACS+ would be a likely candidate as the next attackable protocol. I had in my mind a couple of events from my past that made me suspect that a TACACS+ server couldn't really tell when an incorrect key was in use, other than that the packet decoded to garbage that it was then not able to interpret. If that is the case, it would be very hard to attack the protocol as there is not a straightforward way to tell when you've hit the correct key. I'll let you know when I've figured that out because as I read the protocol spec something else derailed my train of thought.

Like most network engineers I was raised on Cisco's literature and read through plenty of their whitepapers such as this one comparing RADIUS and TACACS+. This document is full of useful facts such as "RADIUS does not allow users to control which commands can be executed on a router and which cannot" (no, Cisco, because for some reason you notched out the ability to do so in IOS) and that RFC compliance doesn't guarantee interoperability. One of the parts I always remembered and believed, in the olden days at least, was the part that says that far beyond the argument of TCP being better than UDP, TACACS+ is more secure than RADIUS due to the way it encrypts the entire message body. I suspect most people just swallow that without chewing, I know I did. Encryption is good, so more encryption must be better.

RADIUS, by comparison, uses CHAP for secure password authentication but makes no attempt to encrypt the parameters within requests and responses. To be fair, these can just be read in plain text straight off the wire. The shared key used when configuring RADIUS is purely an authentication measure - it guards against arbitrary spoofed requests and tampering but does not offer any kind of privacy.

In reality the biggest part of the decision when choosing one or the other will be a "horses for courses" argument. Want to authenticate PPP subscribers with a nice wholesale / retail proxy model? Use RADIUS. Want to authenticate logins to infrastructure devices and authorise specific commands down to the parameter level? Use TACACS+.

A Slightly More Balanced Comparison

Let's compare the relative merits, security wise of each protocol.

Message Encryption

TACACS+, as previously mentioned, encrypts the entire message body using a pre-shared key. It only leaves the header in the clear, so without the key it is only really possible to determine who is client and who is server, plus what kind of messages are being passed (authentication or authorisation, query or response).

RADIUS uses a pre-shared key to authenticate messages going back and forth, but the messages themselves are unencrypted and can easily be read straight off the wire.

Credentials

RADIUS relies on CHAP for user credential validation. The NAS sends a "random" challenge to the user, who produces a one-way hash of the challenge data and password (plus some other stuff) and returns that to the NAS. The NAS then sends the challenge and response off to the RADIUS, meaning that the credentials are never sent over the wire in any reversible way. In order to get the password an attacker must capture the challenge and response data then run a dictionary or brute force attack. On the down side, the RADIUS server itself must have a plaintext copy of the password available in order to verify that a response is correct given the challenge. Clearly if the RADIUS server's password database is compromised then things get quite sticky. For proxy RADIUS, the proxy does not need access to plaintext passwords. In summary, passwords are safe in flight but exposed at rest.

TACACS+ relies on the pre-shared key to encrypt everything, including password information. No form of CHAP or similar system is used, so credentials are passed in a reversible form over the wire. It's encrypted, though, so don't worry - unless an attacker knows the key it's all just gibberish. On the positive side of this, the TACACS+ server does not need to store plaintext passwords for the end users and can instead keep one-way hashes on disk meaning that a compromised database is arguably less of an issue. Safe at rest and safe in flight.

Or is it? Think about the typical use case again. TACACS+ is more-or-less always used to authenticate CLI users logging into routers and switches. The key used to encrypt the TACACS+ communications is stored in the device config, either completely in plain text or using (trivially) reversible type 7 encryption. Virtually all devices are left with the password recovery mechanism enabled. Most of the time the key is re-used across every device in the estate since it makes administration easy and, what's the risk anyway?

A Really Easy Attack

I'd like to point out I'm not suggesting or endorsing any kind of illegal or immoral behaviour. Even as a joke :)

The thought occurs that many TACACS+ managed devices are in remote locations - far flung or sparsely populated offices, in accessible wiring closets, even (*shudder*) customer sites. Given physical access to a device, it's very possible to make a terminal emulator script to perform a password recovery, dump out the config then reset the config register to its original value within only a couple of seconds more than it takes to double-reboot the device. I know this because I did it many moons ago (I don't have it any more - it was pretty easy to write, though).

If I were an evil adversary who wanted to get some credentials, perhaps a good way would be:

1. Feign a power cut, on-site work or some other convincing reason for a device to go down
2. Take the device off the network (to avoid it phoning home by syslog / SNMP) and perform a quick password recovery / config dump before putting it back to its original condition:

*break*
Readonly ROMMON initialized
program load complete, entry point: 0x8000f000, size: 0xcb80

monitor: command "boot" aborted due to user interrupt
rommon 1 > confreg 0x2142

You must reset or power cycle for new config to take effect
rommon 2 > reset
*snip snip*
Would you like to enter the initial configuration dialog? [yes/no]: no

Press RETURN to get started!
Router>enable
Router#show startup-config | include tacacs
aaa authentication login default group tacacs+ local
aaa authorization exec default group tacacs+ none
aaa authorization configuration default group tacacs+
tacacs-server host 10.4.4.10
tacacs-server key supersecret
Router#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
Router(config)#config-register 0x2102
Router(config)#^Z
Router#reload
System configuration has been modified. Save? [yes/no]: no
Proceed with reload? [confirm]


3. Stick a sniffer inline between the device and wherever its administrators are
4. Call in a fault saying that since the power cut / whatever nothing attached to that router / switch is able to see the network - perhaps leave the LAN side disconnected for authenticity
5. Capture TACACS+ packets as the administrators log in to investigate
6. Come up with some compelling reason for comms to go down again while the sniffer is taken out

Now with the config dump it is trivial to get the TACACS server key - it's either just there in the clear or can be decoded from the type 7 encrypted version using any number of free tools. If you put this key into the TACACS+ protocol settings of Wireshark (in the preferences screen expand the protocols area then scroll down to TACACS+), it will happily decrypt the captured packets from step 5:

Configuring Wireshark

Viewing the Decrypted Payload

Now you have the administrator username(s) and password(s) in plain text!

Yikes!

Are you sure you still want to run TACACS on that remote box?

BGP support added to dechap

Hot on the heels of adding the ability to attack OSPF MD5 authentication, I've added BGP support to dechap. It is now possible to feed a pcap file with PPPoE, L2TP, RADIUS, OSPF and BGP packets to the same tool and perform offline dictionary attacks on the authentications within.

As usual, if you're not interested in the theory just skip right to the end for the download link.

TCP MD5 Signatures

BGP authentication uses the MD5 Signature TCP option field, which is defined in RFC 2385. Personally, I found this RFC very vague and it took a lot of iterations to get the technique right. It's particularly fuzzy about what is included in the hash, what isn't and how to present values correctly. I'm hoping to document the process a little more clearly for the next poor guy who tries to implement it as I couldn't find a sufficiently detailed reference anywhere.

RFC 2385 states that the hash must be calculated over the following:

1. the TCP pseudo-header (in the order: source IP address,
   destination IP address, zero-padded protocol number, and
   segment length)
2. the TCP header, excluding options, and assuming a checksum of
   zero
3. the TCP segment data (if any)
4. an independently-specified key or password, known to both TCPs
   and presumably connection-specific

Now, maybe it's just me, but this raised a lot of questions in my mind. Zero padding usually means to fill the trailing space with zeros, but padding the second byte would effectively multiply the protocol number by 256 so should it be a leading zero? Which headers and options are included in the "segment length"? Should the pad bytes be copied with the TCP header?

Through a lot of trial and error I found that:

• The zero padding goes before the protocol number
• The "segment length" includes the TCP header, the TCP options (including room for the MD5 signature option being calculated) and the actual payload data
• The copied TCP header should be 20 bytes long, i.e. includes two padding bytes after the (zeroed out) checksum. The header length remains as-is, including the length of the options.
• The TCP segment data starts immediately after the TCP options and runs to the last byte indicated by the IP length field
• The null byte terminating the password is not passed to the hash algorithm

 The resulting hash value is then stored inside the MD5 signature option (kind 19, length 18).

Checking / Attacking BGP Packets

Using the above method it is straightforward to run a dictionary attack as follows:

• Start with a sniffed BGP packet (see the original dechap blog post for info on how this is extracted).
• Extract and store the authentication hash (look for option kind 19) for later comparison
• Put together the "pseudoheader" as described above
• Append the TCP header without options
• Append the TCP payload
• Append the candidate password
• Calculate the MD5 hash over the complete data set and compare to the value seen in the sniffed packet. A matching hash indicates a matching password.

As of v0.4a, dechap can now be used to automate this process.

Obtaining the Tool

The C source code may be downloaded from: https://github.com/theclam/dechap

Provided the OpenSSL dev libraries are installed it should be possible to simply extract the source code, cd into the directory then run "make". I've only tested this under Ubuntu Linux but there are very few dependancies so I would imagine it will work on most distributions.

Using the Tool

As usual - this is for legitimate audit and recovery purposes and must not be used for any kind of malicious activity.

The usage is pretty straightforward - there are only two parameters and both are mandatory. Specify your capture file (original pcap format) with the -c flag and your word list with the -w flag. Here's an example:

lab@lab:~/dechap$ ./dechap -w mywords.txt -c bgp.cap
Found password "password1" for TCP from 10.0.0.2 to 10.0.0.1.
Found password "password1" for TCP from 10.0.0.1 to 10.0.0.2.
Found password "password1" for TCP from 10.0.0.2 to 10.0.0.1.lab@lab:~/dechap$
I'm not sure how quickly it runs but it doesn't seem quite as quick as the OSPF version. I suppose BGP packets tend to be a little bigger than OSPF so there's more to hash. You can improve the speed by only including one packet for each source / destination pair in each capture as, at present, it doesn't check for multiple packets between pairs and attacks each instance individually.

If you try this out, please leave a comment on this post with your experiences - good or bad. Any suggestions would also be welcome, particularly for other protocols to attack.

References

RFC2385 - Protection of BGP Sessions via the TCP MD5 Signature Option
RFC1321 - The MD5 Message-Digest Algorithm

Offline Attack on MD5 keys in captured OSPF packets

A few months ago I released a tool called dechap which finds PPPoE, L2TP and RADIUS authentications in pcap files and performs dictionary attacks against them. Since writing dechap I've always thought it would be more useful if it were able to do a similar thing with OSPF packets.

Well, the good news is that I've finally got around to adding OSPF support to dechap! Woo and yay! If you just want the tool, scroll straight to the bottom. If you're interested in the theory, read on.

OSPF Authentication Basics

OSPF, or more accurately OSPFv2 as defined in RFC2328, has three options for authenticating incoming packets:

Null: no authentication is performed at all.

Password: a plaintext password is added in the clear to each OSPF packet. If the password contained in an incoming packet matches the one configured locally then the packet is considered valid and is processed, otherwise it is silently ignored.

Message Digest: an MD5 hash is calculated over a combination of the OSPF packet contents and the password. The hash output is then added to the OSPF packet before transmission. When a packet arrives, the receiving router computes an MD5 hash of the packet contents plus its locally stored password. If the calculated hash matches the one attached to the incoming packet then the check passes and the packet is processed; otherwise it is silently dropped.

Note that this is authentication only - in other words the password only serves to verify that the packet contents are authentic. It does not offer privacy, so all the information within the packet is visible  in the clear.

OSPF MD5 Authentication Detail

One thing I found unclear in RFC 2328 was exactly what data the MD5 hash was calculated over. The RFC states:

Input to the authentication algorithm consists of the OSPF packet and the secret key.

... and clarifies that:

(a) The 16 byte MD5 key is appended to the OSPF packet.

(b) Trailing pad and length fields are added, as
    specified in [Ref17].

(c) The MD5 authentication algorithm is run over the
    concatenation of the OSPF packet, secret key, pad
    and length fields, producing a 16 byte message
    digest (see [Ref17]).

(d) The MD5 digest is written over the OSPF key (i.e.,
    appended to the original OSPF packet). The digest is
    not counted in the OSPF packet's length field, but
    is included in the packet's IP length field. Any
    trailing pad or length fields beyond the digest are
    not counted or transmitted.

Confusingly, Ref17 refers to RFC1321, which defines the MD5 algorithm. MD5 defines a method to pad the input before the hash is calculated, so it's easy to assume that point (b) refers to that - it doesn't. I spent a couple of hours trying to work out why my hashes were coming out to the wrong value before finally figuring it out. To aid others, I've taken the liberty of rewriting the instructions so that they can be understood by thickos such as myself:

Calculating the MD5 Hash

In order to calculate the correct MD5 hash, the following method should be used:

(a) Build the OSPF packet as normal, ensuring that the key number and authentication sequence number are populated. The OSPF length field must contain the total number of bytes in the packet at this point. The checksum must be set to zero.

(b) The authentication key / password in plaintext must be adjusted to exactly 16 bytes, i.e. if the key is longer than 16 bytes then it must be truncated, shorter keys must be padded with null (0x00) bytes until 16 bytes long. The resulting 16 byte "modified authentication key" is then appended to the packet.

(c) The MD5 hash must be calculated over the entire result, i.e. the original OSPF packet plus the 16 byte modified authentication key.

(d) The resulting hash is then written over the modified authentication key in the last 16 bytes of the packet.

Testing / Attacking OSPF Packets

Using the above method it is straightforward to run a dictionary attack as follows:

• Start with a sniffed OSPF packet (see the original dechap blog post for info on how this is extracted).
• Extract the original OSPF packet (start immediately after the IP header and continue up to the length specified in the OSPF header)
• Extract and store the authentication hash (the 16 bytes following the packet) for later comparison
• Zero out the checksum
• For each candidate password, pad or truncate to 16 bytes and append to the original OSPF packet. 
• Calculate the MD5 hash as described above and compare to the value seen in the sniffed packet. A matching hash indicates a matching password.

As of v0.3a, dechap can now be used to automate this process.

Obtaining the Tool

The C source code may be downloaded from: https://github.com/theclam/dechap

Provided the OpenSSL dev libraries are installed it should be possible to simply extract the source code, cd into the directory then run "make".

Using the Tool

As usual - this is for legitimate audit and recovery purposes and must not be used for any kind of malicious activity.

The usage is pretty straightforward - there are only two parameters and both are mandatory. Specify your capture file (original pcap format) with the -c flag and your word list with the -w flag. Here's an example:

lab@lab:~/dechap$ ./dechap -w mywords.txt -c ospf-bcast.cap
Found password "password1" for user OSPF host 10.1.1.1 key 1.
Found password "password1" for user OSPF host 10.1.1.2 key 1.
Found password "password1" for user OSPF host 10.1.1.1 key 1.
lab@lab:~/dechap$

I haven't tried any serious benchmarks for this but it seems reasonably fast. In a worst case scenario (correct key not present) on my creaky old Athlon XP 2100 it can try 100k passwords in under 100ms.

If you try this out, please leave a comment on this post with your experiences - good or bad. Any suggestions would also be welcome (yes, I know BGP exists).

References

RFC2328 - OSPF Version 2
RFC1321 - The MD5 Message-Digest Algorithm

LACP miscellanea

According to the statistics, a few people have stumbled across this blog because they were searching for certain 7750-specific information relating to LACP. Here are a couple of answers that were missed out from my main LACP article:

What is the failover time for a LAG / etherchannel? 

The answer to this question varies considerably depending on the setup. If a device notices a bundled interface going physically down then it should unbundle it immediately, causing very low loss (50ms should be achievable).

In the event of an interface remaining physically up (i.e. where there is transmission equipment or EoMPLS between the two devices), also known as a silent failure, the failover will be up to 3 times the LACP timer. So the impact would be up to 3 seconds using fast timers or up to 90 seconds using slow timers. Most lower end Cisco kit only supports slow timers.

In the event of a single fibre fault or other asymmetric failure, you may see a combination of these effects where traffic in one direction heals faster than the other. There are other corner cases such as when administratively shutting down an interface - some devices send an out of sync LACPDU to inform the other end the link is about to go away which helps speed convergence. It is really best to lab test where possible to check different failure scenarios.

Be aware that when using load balanced LAGs, the impact to some streams may be zero. Typically traffic that is hashed onto one link in the bundle will not suffer loss when a different link in the bundle fails.

How can I transport, rather than terminate, LACP through epipe services on the Alcatel-Lucent 7750? 

The answer to this is pretty straightforward, but I know I looked in the wrong place when I first needed to use the feature.

All you need to do is to configure "lacp-tunnel" under the configure -> port -> ethernet context.

How can I transport, rather than terminate, LACP through a QinQ tunnel on a Cisco switch? 

Again, this is pretty straightforward. There are a load of different protocols that can optionally be tunneled on a dot1q-tunnel port, but we just need lacp enabled:


Simply configure "l2protocol-tunnel point-to-point lacp" under the dot1q-tunnel interface.

Normally it makes sense to tunnel everything (STP, CDP, VTP, LLDP, LACP, PAgP, ...) for consistency. Either be a tunnel or don't!

What is the valid range for LAG IDs on the 7750?

For IOM-based systems (i.e. SR-7, SR-12), the usable LAG ID range is 1 to 200. For integrated IOM systems such as SR-1 and ESS-1, the LAG ID range is 1 to 64.


Can you tell me something unusual about LACP on the 7750?

When  the Rx fibre for a LACP-speaking port loses light (i.e. fails), right before the port gets pulled down the 7750 sends an LACP out-of-sync message to inform the other end that it is going away. This is useful for single fibre faults and can drastically improve convergence times, particularly where transmission equipment between the two LACP peers does not forward link loss.

What does "mux: during state COLLECTING_DISTRIBUTING, got event 5(in_sync) (ignored)" mean in a Cisco debug?

As best I can tell it means that a LACPDU was received with the sync bit set, indicating that the far end is ready to use the link, but the link was already collecting / distributing (i.e. in use) so no change in state was required.

What does "lag number : partner oper state bits changed on member port : [expired false -> true]" mean on a 7750 debug?

This means that a particular port's state machine moved into the expired state due to missing three inbound LACPDUs from the peer. Once the port reaches the expired state it is removed from the bundle but the peer parameters are remembered for a further 3 intervals, after which point the peer information is flushed and the port enters the defaulted state.

Do the LACP keys need to match at both ends of a LAG?

No - the LACP key is locally significant and corresponds one-to-one with a LAG or etherchannel ID. It is used to check consistency (i.e. to catch crossed cables) so must be identical for all members within a LAG, however the devices at each end of the LAG can select any value they like for any particular LAG.

Why do ports get "suspended" from an etherchannel?

Basically a port gets suspended if its configuration is not in line with that of the port channel with which it is associated. The most common way to accidentally arrive in this state is for a member trunk port to have a different allowed VLAN list than its parent port-channel interface. While IOS allows member ports to be reconfigured, it is much more sensible to make the configuration changes to the port-channel interface - the changes are then pushed down to the member ports automatically, avoiding this kind of conflict.