Wednesday 3 April 2019

A "real" full Internet table / route server in the lab

A common requirement when lab / stress testing carrier kit is to have at least one copy of a full Internet routing table. While many people will use a network tester to generate, say, 700,000 routes programmatically, this is not really representative. Real Internet routes are not all the same size. They are not contiguous or predictable. They have different AS_PATHs of varying lengths. There are aggregates and specific prefixes, often both covering the same IP space. These factors may make no difference to your system under test, or they might make a huge difference - it's not unusual for TCAM to be partitioned or optimised by prefix length so surely it's best to test with the real thing, right?

This post shows how to build a fully populated and very fast BGP route server based on Ubuntu Bionic Beaver 18.04 LTS.

There are three main elements to this:
  • A daily dump of the RIPE RIB, which we will manipulate and shrink down to a single "view" ready for processing
  • A copy of RIPE's bgpdump and a (slightly tweaked) instance of the bgp_simple perl script which can be used to replay the processed dump file back to a listening BGP instance
  • An instance of BIRD, which is loaded up with routes by bgp_simple and can be used to re-advertise them (in a very fast and resilient fashion) to your systems under test.
This method overcomes a number of horrible issues - Trimming the RIB snapshot massively reduces the time to advertise the table, without compromising the number of prefixes. The bgp_simple instance is modified to reduce the amount of text output, which also increases speed. The BIRD instance sitting between bgp_simple and the system(s) under test increases robustness (BIRD will re-establish dropped BGP sessions whereas bgp_simple does not) and provides much faster update capability.

With these tweaks, a full table can be loaded into BIRD in around 2 minutes and, once fully loaded, BIRD can advertise the full table onward in a few seconds.

Topology


In this setup we will assume our route server will be 10.0.0.1 in AS 65001 and the system under test will be 10.0.0.2 in AS 65002.

Host Setup


This guide starts with a fresh Ubuntu 18.04 LTS instance. This can be a bare metal install, a VM or even a LXC container. You will need to set up this host's networking so that it can access the Internet, at least to begin with. Since this is a lab box, I'm being super sloppy and doing everything as root. You can mentally put "sudo" in front of everything if you'd rather.

First, update the packages and install a few necessary extras:

apt update && apt install bird git perl build-essential libz-dev libbz2-dev screen

Now we need to get some Perl libraries via CPAN. First open CPAN:

cpan

Accepting the auto config should normally do what you want. Within there, run the following:

install CPAN
reload CPAN
install Net::BGP


Exit out by pressing Ctrl-D

Now, edit /etc/bird/bird.conf as follows (replacing the existing "router id") config:

router id 10.0.0.1;
listen bgp address 10.0.0.1;

protocol bgp {
        local 10.0.0.1 as 65001;
        neighbor 10.0.0.2 as 65002;
        multihop;
        source address 10.0.0.1;
        next hop self;
        import all;
        export all;
}

protocol bgp {
        local 10.0.0.1 as 65001;
        neighbor 192.2.0.1 as 64999;
        multihop;
        source address 10.0.0.1;
        next hop self;
        passive;
        import all;
        export none;
}


Now, restart BIRD as follows:

service bird restart

At this point you should have BIRD running (you can check its status by running "birdc" and running some "show" commands - if your system under test is already configured you should see the peer come up). Next, we will set up bgp_simple to load the table into BIRD.

Before we can do that we need to download a few things:

git clone https://github.com/xdel/bgpsimple.git
wget ris.ripe.net/source/bgpdump/libbgpdump-1.6.0.tgz
wget data.ris.ripe.net/rrc00/latest-bview.gz


The daily dump file is pretty big and could take several minutes. Either wait for it or open another tab to build libbgpdump:

tar -xzf libbgpdump-1.6.0.tgz
cd libbgpdump-1.6.0/
./configure && make && make install
cd ..
rm -r libbgpdump-*


Now, the dump file contains many, many views of the full table. We just need one, so let's get a list of neighbours:

zcat latest-bview.gz | bgpdump -m - | head -50000 | cut -d '|' -f 4 | grep -v ':' | sort | uniq

Pick one of the IPs listed at random and filter down for just that neighbour. You can repeat this process using different addresses on different route servers if you'd like to simulate getting a table from two providers. In this example, we'll use 185.210.224.254, but any should be just as good.

zcat latest-bview.gz | bgpdump -m - | grep '185.210.224.254' > bgpsimple/routes.txt

This takes a looooong time, perhaps 25 minutes on a rubbish PC. Grab a cuppa.

Everything up until here will still be there after a reboot, the parts that follow will need to be re-run each time the route server gets rebooted.

Almost there, we need to add the unroutable test address onto the loopback interface:
ip address add 192.2.0.1/32 dev lo

Finally, we will make some minor tweaks to bgp_simple:

cd bgpsimple

Edit bgp_simple.pl and comment out lines 640-649 - this prevents the script echoing all 700,000 routes to the console as they are advertised (complete with their AS_PATH, MED, etc).

Now run an instance of "screen" to keep bgp_simple running after your CLI session ends:

screen -an

And execute the script:

./bgp_simple.pl -myas 64999 -myip 192.2.0.1 -peerip 10.0.0.1 -peeras 65001 -p routes.txt -holdtime 300 -keepalive 30

The script should connect and, after a short pause, tell you that it is advertising its routes to the peer. After a couple of minutes you should see a message to say the advertisement is complete.

At this point you can break out of screen using Ctrl-a, followed by "d" (reconnect later using "screen -r").

Finally, run "birdc" and execute "show route count" to confirm how many routes you are seeing. Once BIRD is loaded up it can blow a table down to a peer in seconds and can be configured for as many peers as you like.

Happy labbing!


3 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. What sort of resources to you give this box? I keep getting hold timer expired between the bird instance and bgpSimple.

    ReplyDelete
  3. AWS Cloud Consulting Services offers amazing benefits for businesses, including scalability, cost savings, a global infrastructure, robust security measures, and a platform for innovation and integration. By leveraging AWS services, businesses can optimize their infrastructure, drive growth, and deliver exceptional experiences to their customers. Whether you're a startup or an enterprise, AWS provides the tools and resources to help you succeed in today's digital landscape.

    ReplyDelete