Packet Actions – Python and Scapy
Hello and welcome to the “Packet Actions” series of blog posts. I’d like to spend a few posts talking through how you can programmatically integrate with a network dataplane. I had thrown around the idea of calling this series “Doing things with packets” but that seemed a bit long and also could mean just about anything. So what does Packet Actions mean? Well – its the shortest way I could come up with to say “Looking at packets on the wire and doing things based on what you see in the packet”. To discuss this further I’d like to talk about the often made analogy of network engineers being plumbers – an analogy that makes fairly good sense in most cases. For instance, network engineers create the paths for data to flow – plumbers make paths for water to flow. Additionally both need to make sure that there are no blockages or issues with handling the amount of data or water that needs to flow through the pipes. Going a step further – plumbers might use a diagnostic tool like a scope to physically look inside the pipes if theres a blockage or issue so they can see what’s going Continue reading
Working with Linux VRFs
The concept of VRFs is likely one that you’re familiar with. They are the de facto standard when we talk about isolating layer 3 networks. As we’ve talked about previously, they are used extensively in applications such as MPLS VPNs and really provide the foundation for layer 3 network isolation. They do this by allowing the creation of multiple routing tables. Any layer 3 construct can then be mapped into the VRF. For instance, I could assign an IP address to an interface and then map that interface into the VRF. Likewise, I could configure a static route and specify that the route is part of a given VRF. Going one step further I could establish a BGP session off of one of the VRF interfaces and receive remote BGP routes into the VRF. VRFs are to layer 3 like VLANs are to layer 2.
So while we’ve talked about how they are typically used and implemented on networking hardware like routers and switches – we haven’t talked about how they’re implemented in Linux. Actually – they’re fairly new to the Linux space. The functionality was actually written by Cumulus Networks and then contributed to the Linux kernel (kudos Continue reading
Python Pieces: Decorators
As some of you know – Im a big believer that we all learn differently. You may read something the first time and immediately grasp the topic whereas I may read it and miss the point entirely. For me, decorators have been one of those things that I felt like I was always close to understanding but still not quite getting it. Sure – some of the examples I read made sense but then I’d find another one that didn’t. In my quest to understand them, I spent a lot of time reviewing a lot of examples and asking a lot of very patient friends for help. At this point, I feel like I know enough to try and explain the topic in a manner that might hopefully help someone else who was having a hard time with the concept. With my learning philosophy out of the way, let’s jump right in….
I want to jump right into a real (albeit not super useful) example of decorators using the full decorator (or shorthand) syntax. Let’s start with this…
def a_decorator(a_function):
print("You've been decorated!")
return a_function
@a_decorator
def print_name_string(your_name):
name_string = "Your name is: " + your_name
return name_string
print(print_your_name("Jon"))
Python Pieces: PyEnv and Venvs
In my last post, we talked about PyEnv and how it can help manage your local Python environments. As it turns out it can also help you manage virtual environments as well! However – pursuing this functionality took me down a rabbit hole that was a bit deeper than expected. The way that PyEnv works causes some behaviors (and on my end assumptions) to change which made me start questioning some of the things that I’ve always just taken for granted. In other words – prepare yourself to go down the rabbit hole with me.
At first glance PyEnv promised the same sort of awesome automagically context switching craziness that we saw previously work with Python versions. However – the virtual environment management implementation with PyEnv felt rather foreign (and maybe a little clunky?) to me. Most notably, as I pointed out in my last post, the .zshrc alias provided to make the auto activation piece work slows down my terminal immensely which is why I omitted using it. A slow terminal is about the worst thing I can think of…
That said – I still think it’s worth reviewing what it can offer so you can Continue reading
Python Pieces: Using PyEnv
If you’re like me – one of the most frustrating things about Python is version management. You get a new Mac, the system default is 2.x something, you need 3.x something, and you’re wondering what the best (right) way to get the version you want installed. You install Python 3 but the default Python version stays the same until you do some symlink hack thing that you know is just creating a mess. So for awhile you just call python3 explicitly but then you realize that all of the packages you installed using pip are no longer available and you need to install them again using pip3.
Sound familiar? Maybe I’m the only one that struggles with this – but I tend to muddle my way through just making things work while in the back of my head I know that Im creating a complete disaster of the local Python installation. I shall muddle no longer thanks to PyEnv. I was recently introduced to the tool and it’s a total game changer. It allows you to seemlessly manage your local Python install, easily install different versions, easily switch versions, and even has the capability of automgically switching versions Continue reading
Working with TC on Linux systems
Hi folks! Long time no talk : ) Life has been incredibly busy for me over the last few months so I’ll apologize in advance for the lack of posts. However – I’m aiming to get back on the horse so please stay tuned!
With that out of the way – I wanted to spend some time in this post talking about the command line tool found on Linux systems called tc. We’ve talked about tc before when we discussed creating some network/traffic simulated topologies and it worked awesome for that use case. If you recall from that earlier post tc is short for Traffic Control and allows users to configure qdiscs. A qdisc is short for Queuing Discipline. I like to think of it as manipulating the Linux kernels packet scheduler.
Note: tc is traditionally part of the iproute2 toolset which Im pretty sure (but not positive) is included in most base Linux distros these days.
When tc comes up – it’s easy to immediately start thinking about QOS, queuing, and packet(traffic) control. And while some of the actions available to you when using tc seem obvious, or at least fit within the mindset of queue disciplines (the drop Continue reading
Carrier supporting Carrier with BGP-LU
In our last post we talked about the less used method of deploying CsC where we ran OSPF and LDP inside the CSC-PE routing-instance.
Note: I can’t help myself apparently so be aware that Carrier of Carriers (CoC) is the same as Carrier supporting Carrier (CsC)
This required some changes to be made to our default LDP export policy as well as how we moved routes between the inet.3 and inet.0 tables. That being said, if you’re a single org it might make good sense to run things that way. I liked how you were able to see all of the remote LDP domain loopbacks in your local inet.3 table which in my mind made it easier to imagine the LSP paths.
That being said, it is clearly not the preferred deployment methodology. Most examples you’ll find leverage BGP (BGP-LU specifically) for the CSC-CE to CSC-PE connections as well as within the local label domains. So in this example, we’ll do just that. Larges chunks of the base configuration will be the same as they were in the previous post but for the sake of clarity I’ll post our starting post the starting configurations and diagrams here Continue reading
Carrier Supporting Carrier with OSPF and LDP
I know we haven’t gotten to the point of actually discussing inter-as option B or option C but we did cover all of the mechanics required for them to work (at least briefly). Before we go there though – I did want to cover a different technology that can also help solve our end to end LSP problems. Carrier Supporting Carrier – or more commonly just referred to as CsC (or in Cisco parlance Carrier of Carriers or CoC (or maybe I have that backwards?)) is a means to nest MPLS VPN sessions. Think of it this way. If you’re a huge backbone provider (we’ll call you the “uber provider”) selling transport to customers (we’ll call them customer carriers) – there’s a good chance that those customer carriers will need a way to isolate their customer on their backbone. How do we do that? Well MPLS VPNs of course! However you, the uber provider, also need a means to keep the customer carriers isolated as they traverse your backbone. So what do we do? Sounds a lot like a carrier supporting a carrier huh? CsC to the rescue!
At this point you might be wondering how we got here. Continue reading
Fundamentals of MPLS LSPs
One of the items that often trips folks up with MPLS is the concept of label switched paths or LSPs. We’ve talked about them extensively before in many of the blog posts here and I’ve described them a couple of different ways. Many people look at an LSP as a sort of unidirectional tunnel. In fact, most network diagrams aiming to describe an LSP often show it just as that – a tunnel. It’s an easy thing to visualize especially when you start talking about nested tunnels or LSPs inside of LSPs, but I also think it can be rather confusing. This becomes even more confusing when people start talking about end to end LSPs or how a service label is the same end to end as traffic traverses an LSP. What does that mean? Where does an LSP start or stop? Is it really a tunnel? How far can an LSP reach? What if we run different label distribution protocols? In this post, and perhaps the next, I hope to address these questions as well as talk about how we can solve some of the common problems that are often encountered with LSPs.
So let’s dive right in and Continue reading
BGP-LU and MPLS VPNs
In my last post I showed you how you can use plain old BGP to distribute labels and create LSPs. While sort of interesting to see – it wasn’t super handy all by itself. In this post, we’re going to try and layer on MPLS VPNs to the same setup and show you how that might work. So let’s dig right in. I’m assuming that you’ve read the last post and that we’re picking up where things left off there.
Note: There are no pretty diagrams here so if you’re looking for some context on the lab we’re using go back and checkout the last post.
The first thing we want to do is put our client subnets into VRFs or routing-instances in Juniper parlance. Let’s do that on each tail router….
vMX1
set routing-instances customer1 instance-type vrf set routing-instances customer1 interface ge-0/0/0.0 set routing-instances customer1 route-distinguisher 1:1 set routing-instances customer1 vrf-target target:1:1 set routing-instances customer1 vrf-table-label
vMX7
set routing-instances customer1 instance-type vrf set routing-instances customer1 interface ge-0/0/1.0 set routing-instances customer1 route-distinguisher 1:1 set routing-instances customer1 vrf-target target:1:1 set routing-instances customer1 vrf-table-label
So nothing fancy here – we’re just creating a routing instance, assigning an RT/RD, Continue reading
Understanding BGP Labeled Unicast
We’ve talked in previous posts on how we can use LDP and RSVP as label distribution protocols. Without LDP and RSVP – we wouldn’t be able to easily create LSPs which means we’d have to do it manually as we did in my first post on MPLS. That being said – the discussion around MPLS label distribution usually focuses around these two protocols, but you might (or might not depending on how long you’ve been in networking) be surprised to learn that we can also use BGP to advertise labels. That is – we can build end to end LSPs without the use of LDP or RSVP. Using BGP for label distribution comes with it’s own set of requirements (and associated oddities) so in this post we’ll talk through the use case.
Advertising labels through BGP is something that we’ve seen before. Specifically, we saw it in the MPLS VPN use case where PE routers advertise a VPN label so that the remote PE knows what VRF/VPN the traffic belongs in. In that use case, we did a BGP peering with the inet-vpn address family. To do BGP labeled unicast (commonly called BGP-LU) we do a BGP peering with the Continue reading
Multipath routing with VPNv4
When we talk about VPNv4 prefixes – Route Distinguishers (RDs) play an incredibly important role in ensuring multipath routing. We talked about this a little bit in our last post, but I want to hit it home in this post as well as cover a couple of other items in a little great detail.
To start with, we’re going to use our same physical lab topology, but changes things up slightly. Namely….
- vMX7 will remain a BGP route reflector but will also now participate in the dataplane by having it’s interfaces enabled with LDP and MPLS
- vMX1 and vMX3 will act as remote PEs for our provider that are hosting an anycast service. Both will be advertising
140.10.20.0/24into a “Provider” VPN which we will then import into our customer VPN. - vMX2 will now act like a Customer Edge (CE) router peering to vMX5 which will act as the provider edge (PE) router. It will peer with the provider in AS65000 from AS12345.
So our diagram will now look something like this…
Alright. So for the sake of thoroughness, I’ll start by including our base configurations again since they did change ever so slightly, and I Continue reading
Understanding RTs and RDs
One of the items that continues to come up in my conversations with folks learning about about MPLS VPNs is defining what a Route Target (RT) and Route Distinguisher (RD) are. More specifically, most seem to understand their purpose – but often times they don’t quite understand the application. I (and many others – just google “Understanding RDs and RTs”) have written about this in the past but Im hoping to put a finer point on the topic in this post.
If someone were to ask me to summarize what route targets and route distinguishers were – I’d probably define them like this…
Route Distinguishers – serve to make routes unique
Route Targets – metadata used to make route import decisions
Now – I’ll grant that those definitions are awfully terse, but I also feel like this is a topic that is often over complicated. So let’s spend some time talking about RTs and RDs separately and then bring it all together in a lab so you can see what’s really happening.
Route Distinguishers
As I said, a route distinguisher serve to make routes look unique. So why do we care about making routes look unique? I’d argue one of Continue reading
ExaBGP and etcd – processing routes
In my last post – we took a look at how we could leverage etcd from Python. In this post, I want to propose a use for leveraging etcd as a sort of message bus for ExaBGP. We saw some pretty compelling features with etcd that I think can work nicely in our ExaBGP model. So without further blabbering – let’s start coding.
Note: I assume you have a local instance of etcd installed and it is currently empty. If it’s not empty – you’ll want to clear it all out using a command like this ETCDCTL_API=3 etcdctl del "" --from-key=true
If you recall – in our last post on ExaBGP we were at a point where the ExaBGP process was using two Python programs we wrote. One for processing received routes (exa_bgp_receive.py) and one for sending route updates (exa_bgp_send.py). My goal here it to remove a lot of the logic for static route processing from these two scripts and make them more about route processing. More specifically – I want to turn the two Python scripts that ExaBGP is running on our behalf into simple programs that read/write to to/from etcd. Once we Continue reading
Python Pieces – Working with etcd
Ah ha! Surprise – I’ve decided that in addition to the blog posts on MPLS and ExaBGP that I might as well start up a third series. Well – that’s not entirely true – but instead of trying to mix all sorts of details about Python into the blog posts, I thought I might split out some of the larger pieces. So Im starting a new series called “Python Pieces” where I’m going to pick one module, concept, or whatever else I decided warrants a post and talk about how to use it. Then – if/when I use that in one of my other posts – you’ve got a handy reference and starting point. I hope this makes the other posts less “all over the place” but we’ll see.
So – in my first edition of Python Pieces we’re going to talk about working with etcd from Python. For those of you that don’t know what etcd is – it’s a pretty popular key value store that’s used with lots of the more recent projects (Kubernetes comes to mind). What’s likely more important about etcd though is that it’s capable of being a distributed key value store which makes it Continue reading
ExaBGP – handling route withdrawal
In our last post we talked about how we can programmatically talk and listen to ExaBGP. By the end of the post, our Linux server was listening for BGP updates, processing them, and creating static routes based on the information it learned. We worked through some issues to get that far – but also recognized that we had a ways to go. In this post, we’ll start tackling some of the other issues that are lingering with this implementation. So let’s dive right in and start knocking these out!
The first issues I want to talk about isn’t actually an issue anymore – but it’s worth mentioning since we sort of solved it accidentally. At this point – we’ve only processed BGP update messages that have included a single router advertisement. Said more specifically – BGP update messages that included a single NLRI. A BGP update message can (and will) contain multiple NLRI’s so long as the path attributes are the same for all the prefixes. For folks not familiar with BGP – NLRI (Network layer reachability information) are basically the routes or prefixes that are being sent to us. If we go back and look at the BGP update Continue reading
Building static routes with ExaBGP
In our last post we covered the basic setup and configuration of ExaBGP. While we were able to make use of ExaBGP for dynamic route advertisement, it wasn’t able to help us when it came to actually programming the servers routing table. In this post, I want to show you how you can leverage ExaBGP from a more programatic perspective. We’ll start by handling route advertisement to our peer and then tackle reading and processing received route updates. We’ll also start using another Python module (pyroute2) to program the routing table of the bgp_server host so that it begins acting more like a normal router. Enough talk – let’s dive in!
Im going to assume you’re starting off at the end of the last post. So the first thing we need to do is clean up a couple of items. We’re not going to rely on the static route we provisioned so to clean that up we can simply reapply the netplan network configuration using the command sudo netplan apply…
user@bgp_peer:~$ ip route default via 192.168.127.100 dev ens3 proto static 10.10.10.0/30 dev ens7 proto kernel scope link src 10.10.10.1 10. Continue reading
Working with ExaBGP 4
One of my goals this year was to spend more time doing Python development so I thought I’d take a break from the MPLS deep dives (no worries – there are still lots more coming) and insert some development type things along the way. One of the opensource projects I’ve used in the past was ExaBGP by the folks over at Exa-Networks. Since then – they’ve released a new version (4) and I’ve been keen to play around with it some more.
The last time I played with ExaBGP was strictly from a testing perspective. This time – I want to focus on some use cases that are closer to real life. Things you might actually do – or at the very least – try. So to start things off – we’re going to once again start with a basic lab that looks like this…
So here we have our 2 friendly end users connected at the head to a small simple network. left_user is attached to an Ubuntu 18 server called ‘bgp_peer’ which is then connected to two other routers (Juniper vMX) and finally our friendly right_user. To start with – we’re going to configure the routers vMX1 and Continue reading
Understanding CSPF and the TED
In our last post, we talked about one of the major differences between LDP and RSVP – the ability to define EROs or explicit route objects. We demonstrated how we could configure LSP paths through our network by providing a set of loose or strict next hops for the LSP to take. This was a rather huge paradigm shift because it meant we could define paths that didn’t align with what the IGP thought to be the best path through the network. What we didn’t talk about was how the ingress router determined if these paths were feasible. In this post, we’ll deep dive on the traffic engineering database (TED) and how it works in conjunction with the constrained shortest path first (CSPF) algorithm to build RSVP LSPs through a network.
It’s important to remember that the ingress label switching router (LSR) is really the thing doing most of the work in regards to setting up RSVP LSPs. Well – to be fair – the egress LSR is the one that actally sends the RESV message back toward the ingress LSR with the label information which is what’s required for the LSP to work. However – the ingress LSR Continue reading
Understanding RSVP EROs
In our last post we covered the basics of getting an RSVP LSP setup. This was a tedious process at least when compared to what we saw with LDP setting up LSPs. So I think it’s worthwhile to spend some time talking about RSVP and what it offers that merit it’s consideration as a label distribution protocol. First off – let’s talk about naming. When talking about MPLS – RSVP is typically just called RSVP – but without the MPLS context – it might be a little confusing. That’s because RSVP itself initially had nothing to do with MPLS. RSVP was initially a means to reserve resources for flows across a network. The protocol was then extended to support setting up MPLS LSPs. In this use case, it is often referred to as “RSVP-TE” or “MPLS RSVP-TE”. For the sake of my sanity – when I reference RSVP going forward I’ll be referring to RSVP that’s used to setup MPLS LSPs.
So now let’s talk about some differences between LDP and RSVP. The first thing that everyone points out is that LDP is tightly bound to the underlying IGP. While this is an accurate statement, it doesn’t mean that RSVP Continue reading