Learning EVPN? Don’t skip Route Type 3

In this post

📚 The trouble with learning RT-3s.
🎥 EVPN-VXLAN Explainer 3 is finally out!

👹 An odd choice?

After many hours of writing, recording, re-recording, and rewriting; I was finally able to release the third video in my EVPN-VXLAN Explainer series (link below). This video deals with one of the aspects of EVPN that took me a while to grasp, but is fundamental to unlocking this protocol; that being Route Type 3.
After my initial decision to tackle RT-3 as the first Route Type that I cover for this series, I started to wonder whether this may seem like an odd choice. RT-2 being the update that most are interested in, thats your actual host update.
However, the more I progressed with the video writing process, the more natural it felt to tackle RT-3s first.
After all, this is the first update that you'll see in the EVPN table or your wireshark capture, with or without end hosts.
RT-3s are fundamental to the operation of EVPN networks, no Route Type 3s, no flooding; and without that, IP networks do not run so well.
But as fundamental as RT-3s are, they do seem to be a little over-looked in Continue reading

EVPN-VXLAN Video 2, Aruba Airheads and that bird app

In this post:

🎬 EVPN-VXLAN Explainer 2
🧡 My experience at the Aruba Airheads Netherlands event
🥀 Going off Twitter and joining Mastodon

🎬 EVPN-VXLAN Explainer 2

Hot of the presses, well, iMovie rendering, here is the latest EVPN-VXLAN Explainer video:

This one covers how BGP peers build an EVPN session, with the AFI/SAFI in a BGP OPEN. Plus there's a config demo and some packet captures in there, showing a BGP OPEN 'in the wild'.
Check it out and let me know what you think.

🇳🇱 Airheads Netherlands

A couple of weeks ago, I presented a couple of sessions at the Aruba Airheads event in NL, just outside Utrecht.
☝️ My first session was on Aruba Central NetConductor, Aruba's all-encompassing architecture; that unifies wired and wireless networking with a strong focus on security and management.
That went well, and my live demo, as much as I got to show in the timeslot, was a success.
The big challenge with that type of presentation, because it isn't a single product or service, but an architecture and design approach; is to answer the question 'what is it?'. I'm not a fan of marketing at all, so I always attempt to Continue reading

Null Zero Long Form Video Content 🎬

In this is post

🎬 EVPN-VXLAN Explainer - now on youtube.
📔 An update on what I've been up to, lots of EVPN.
🦀 MY other project - learning Rust

EVPN-VXLAN Explainer 1

My recent EVPN Explainer posts were essentially preparation for a video series, covering roughly the same content.
With the large amount of detail required, and as I refined my messaging, I felt that a blog was a better medium to use at first. Videos are much harder to edit and correct any mistakes. Thus I wanted to try out my content in writing, then move on to creating the videos.

After a couple of weeks preparing and recording, along with my main job, I'm happy to say that I've released the first video, and here it is:

This video is a gentle introduction to EVPN, focusing on the high-level operation and configuration, rather than going to deep. With the basics out of the way, I'll be going into the details, much like my blog posts, in later videos.

A Brief Update AKA What I've been up to

As the long, hot summer of 2022 draws to a close, I've put away the flip-flops and dived straight back Continue reading

EVPN-VXLAN: Symmetrical IRB versus Asymmetrical IRB

Now that we've covered the two flavours of IRB in depth, I want to share more of a discussion piece. Technical details are interesting, sometimes even fun, but what about real-world operational considerations?

"Everyone has a plan..."

Viewing the intimidating assortment of pikes, swords, sharp, and bashy objects in the Tower of London armoury during a recent visit, I was reminded of that Tyson quote "Everyone has a plan until they get punched in the mouth."
My train of thought was, "being a knight riding around on horseback would be fun and all until an encounter with a big stick with a pointy metal end."
Similarly (or maybe not similar at all, but I hope you get where I'm going here), playing around with the various types of IRB for EVPN has been enlightening and, at times, fun; but what about on a real-world network with its everyday concerns and risk of outages - the pokey, hurty things in my analogy.
What works on paper might not be feasible on a live network, when the focus is primarily on reliability and deploying networks that the NetOps team can realistically support.

Symmetrical IRB - it scales, but at Continue reading

EVPN-VXLAN Explainer 6 – Symmetrical IRB

Now let's continue our look at routing with EVPN-VXLAN as we focus on symmetrical IRB.

This post is essentially building upon a lot of what we covered in the previous post. So, if you haven't read that yet, please do, then meet me back here. This post will make a lot more sense if you do.

🔬 Symmetrical IRB in Detail

While symmetrical and asymmetrical IRB have the same functional outcome; to route inter-subnet traffic, there are a number of major differences in the requirements and configuration of each.
Most notably, symmetrical IRB frees us from the requirement to configure all VLANs & L2VNIs on all VTEPs.

Here's an overview of the features and components that we'll be covering:

Symmetrical IRB Architecture Notes

• Symmetrical IRB offers a more scalable approach to routing VXLAN traffic because VTEPs are not required to have knowledge of all destination clients, they do not need to hold an ARP cache entry for a destination, unlike asymmetrical IRB.
• VTEPs are only configured with the VLANs, subnets and VNIs that host locally connected clients.
• To ensure successful end-to-end connectivity for inter-subnet traffic, a number of new requirements and features are deployed with symmetrical IRB:
  • A L3VNI for Continue reading

EVPN-VXLAN Explainer 5 – Layer 3 with Asymmetrical IRB

Thus far, this series of posts have all been about Layer 2 over Layer 3 models; the customer ethernet frames encapsulated in UDP, traversing L3 networks. The routing has been confined underlay, the customer traffic has stayed within the same network.
No longer! In this post, things start getting a little more interesting, as we look at routing the customer traffic with an EVPN feature called Integrated Routing and Bridging, or IRB.

• First we look at the concept of routing in VXLAN networks.
• Then we have an in-depth look at asymmetrical IRB (I'll be dealing with symmetrical in the next post).

✅ L2 is intra-subnet, L3 is inter-subnet

📥 Intra-subnet

To define terms, when I say 'intra-subnet', that is L2 traffic transferred between nodes in the same subnet.

📤 Inter-subnet

'Inter-subnet' refers to a traffic flow that traverses subnet boundaries.

☎️ The Centralized IP L3 Gateways of Old

• With VXLAN networks in the past, inter-subnet communication was often performed by a centralized, IP only, gateway on behalf of the rest of the network.
• Traffic from customer-side networks would need to be sent to this central device for routing, which often created inefficient traffic flows, and possibly a bandwidth choke-point.
• Imagine Continue reading

EVPN-VXLAN Explainer 4 – Route Type Three and Auto-Discovery

In this post we will have a look at another EVPN Route Type, that being RT-3, which goes by the rather opaque name of 'Inclusive Multicast Ethernet Tag'; and look at how it is used to ensure EVPN peers flood traffic to those neighbours that need it.
Firstly, we'll look at EVPN packet forwarding to provide some context around why this Route Type is important, then we will dive into its details, with all the usual show commands, packet captures and plethora of RFC name drops.

EVPN Packet Forwarding

Let's start off by running through EVPN Packet forwarding, for that we need an example network.

Example Network

• For this post, we'll use a slightly larger network, this time with three switches.
• As shown in Figure 1 below, this consists of three Aruba 6300s, all configured for OSPF and EVPN.
• To emulate customer workloads, I have a physical linux server attached to each 6300. I have configured the interconnecting port as a trunk to generate traffic in different VLANs.
• Each node is configured with customer-faced VLAN 10, which is bound to VNI 1010.
• However, only two of the three peers, 6300-1 and 6300-2, are configured with a second VLAN and VNI, Continue reading

EVPN-VXLAN Explainer 3 – BGP UPDATE & Route Type 2

The story so far

• In post 1 we looked at VXLAN, an encapsulation type that enables L2 traffic to traverse L3 networks, the use case being virtual overlay networks to provide connectivity to VMs and containers.
• VXLAN alone uses data plane learning to build a view of the MAC addresses of the clients attached to the network. This can be suboptimal because traffic needs to hit a switch for it to learn a MAC, it is akin to basic L2 switching across the network.
• EVPN is used as a control plane for VXLAN networks, allowing peers to learn about MACs by advertising them in an extended form of BGP UPDATE.
• In post 2 we looked at the BGP EVPN session and configuration, now let's move on to advertising some routes.

Example Network - Two peers

The philosophy behind this series is start small and build. With that in mind, I'm going to start with just two EVPN peers, supporting a couple of customer VLANs.

EVPN Operations

In principle, L2VNI is very similar to the static VXLAN we saw in post 1 of this series. It consists of a Layer 2 segment that can be stretched Continue reading

EVPN-VXLAN Explainer 2 – The BGP Session

In this second post I will look at Ethernet VPN (EVPN), what is it and how to configure a BGP EVPN session on Aruba devices.

Please note, this post will focus on the establishment of the BGP EVPN session between peers, and thus will not present a fully functioning EVPN network. I aim to build the configuration up in stages to enable the reader to confidently understand the different pieces of EVPN-VXLAN as a technology.

Ethernet VPN - the VPLS successor seen about campus

Reading through EVPN RFCs one gets an impression of its convoluted development, or rather, the evolving area of focus for its application.
To summarize, it started life as a service-provider focused VPLS successor, then jumped over to the control plane for virtualized data centres, now gaining a foothold in campus networks.
I put together an overview of the various RFCs here.

EVPN - what is it used for?

In the first post in this series, I explained the VXLAN forwarding process, that relies upon flood and learn.

• VXLAN is a data plane technology, concerned with the forwarding of packets and what headers to add to data.
• EVPN is a control plane technology, concerned with building a Continue reading

EVPN-VXLAN Explainer 1 – Static VXLAN

In this series of blog posts I'm going to break down how to configure Aruba AOS-CX switches for VXLAN and EVPN, plus explain how to read the EVPN table and various 'show' commands.
In this first post I will look at VXLAN, its configuration and operation.

A note about EVPN-VXLAN

VXLAN: encapsulation type of 8 bytes, works at the data plane, concerned with the forwarding of packets.
Ethernet VPN (EVPN): Extension to BGP, works that the control plane, concerned with learning and advertising MAC/IP addresses.

Static VXLAN - Start Here

VXLAN configuration is one of the basic building blocks of a EVPN-VXLAN network, it is worth familiarising yourself with static VXLAN configuration, even if you are never going to use it.

Example Network

Figure 1 below shows a network comprised of three Aruba 6300 switches acting as VTEPs, with two customer VLANs, VLAN10 and VLAN 20, that are bound with VNI 1010 and VNI 1020 respectively across the VXLAN network.

VXLAN Configuration

Component parts of the configuration:

1. Customer-side VLAN - carries the traffic to be encapsulated.
2. An IP network between VTEPs (the underlay network)- there must be bi-directional connectivity between an IP address designated on the VTEP. Continue reading

Docker, Openvswitch & Aruba VXLAN Network Build

This blog provides details of how to build a static VXLAN network that connects physical hardware to a virtualised network, enabling communication from docker containers to external nodes.
The build is comprised of a hardware ArubaOS-Switch acting as a VTEP and an openvswitch VTEP running on an ubuntu server, which is the host for the docker containers.
This network also serves to prove interoperability between the ArubaOS-Switch VXLAN stack and that running on openvswitch.
The use of docker containers as target nodes enables rapid deploy and tear-down of network components, which is particularly useful in lab environments for testing.

Kit List

2 x ArubaOS-CX 6300 hardware switch (only 1 is required.)
1 x HP EliteDesk PC running Hyper-V hosting an ubuntu 21.04 VM
1 x HP EliteDesk PC running ubuntu 21.04 bare metal.

Notes:
I used a VM for the openvswitch / docker linux server to take advantage of snapshots while documenting this build. This server can be any linux server.

Network Diagram

Build Steps

Configure the ArubaOS-CX hardware switch

1. Configure the ArubaOS-CX switch and local server (172.18.1.1) so that they are in the same subnet and can ping each other.
2. Configure the ArubaOS-CX Continue reading

“What’s your IPv6 strategy?”

For many in enterprise networking, IPv6 is just a distant memory of a tedious mandatory training a few years back. Weird addresses, over-eager trainer, stories about v6 adoption that never came true. Why then for the last couple of years have I been presenting to Aruba audiences about IPv6 adoption? While many network engineers maybe unaware, IPv6 is very much upon us and numerous times this year I've heard from various sources, 'What's your IPv6 strategy?'

It's alive!

Turning back the clock a few years and IPv6 was for the specialist or for university campuses eager to deploy the latest technology. Live deployments of IPv6 outside of academia were largely unheard of. Then came the ISP deployments across the global, in roughly 2015-2017. Now IPv6 was out in the wild and in our homes. It was this transition of IPv6 from the textbook to the live networks around us that changed the nature of the protocol and breathed life into those 128 bits.
But this was a largely silent change. No big fanfare, hashtags or broadsheet ads. No proliferation of start-ups hunting VC money. No LinkedIn profiles being updated with 'IPv6 thought-leader'. For that reason I feel many network Continue reading

IPv6 SLAAC Host OS Address Allocation

While rebuilding my v6 lab with a variety different host OS, I found that there is no single approach to address generation in IPv6 SLAAC networks.

I've recorded my findings below for future reference, but also as a good way to delve deeper into the murky world of IPv6 address generation and shine a light on just what is all this stuff in our 'ifconfig/ip add' commands.

TL;DR

The table below summarizes my observations

| OS | Address Generation | Temporary Address |
|--------|---|---|---|---|
| macOS 10.14.6 | Stable-privacy | Yes |
| Ubuntu 18.04 | Stable-privacy | Yes |
| Debian 10 | EUI-64👈👀 | Yes |
| Fedora 30 | Stable-privacy | No 👈👀|
| Windows 10 1903 | Randomized IID | Yes |

Lab Setup

I'm running a basic LAN topology with a combination of hardware (Windows 10 and macOS), plus virtual machines for the Linux OS.

• Virtualization platform: Hyper-V on Windows 2019 Server
• Host OS:
  • Windows 10 running on an HP Z book.
  • macOS version 10.14.6 running on a MBP 2017
  • Ubuntu 18.04.3
  • Fedora 30
  • Debian 10 Stable
• Gateway: ArubaOS-Switch 2930F-8G-PoE+-2SFP+ running WC.16.07.0002

Gateway configuration:

ipv6  Continue reading

Why I’m telling network engineers to learn Python

During a wet autumnal walk, I was explaining to my girlfriend about my recent presentations. I've been doing my 'Getting Started with Python' talk at Aruba Airheads meet-ups. I recorded an early version of it, see below. One point I mentioned is that the reaction is always mixed. When I ask who is learning Python, about 5% of each audience put their hands up. Regularly people object to the idea of network engineers learning Python. The arguments are usually along the lines of 'network engineers already have enough to learn'.
The conversation continued as we walked through sodden leaves and we discussed why, if the other speakers were doing talks that the crowds want to hear, like product updates, I'm burdened with a subject the audience are less enthusiastic about. The assumption being that I was assigned this topic. My response: "Oh no, I choose to do this." The next question was, of course, "Why?"

My Why

There's something of the two-edged sword in the word 'why'. It can be used to undermine, casting doubt about the veracity of an argument, cutting through the rhetoric and leaving those with ill-reasoned ideas to flounder and stutter a response. But Continue reading

Android does not support DHCPv6 and Google ‘Won’t Fix’ that

Since 2012 there has been a ticket open on Google's public 'Issue Tracker' requesting Android support DHCPv6. On 6th November the status of the issue was changed to 'Won't Fix (Intended Behavior)'.

The fact that Android does not support DHCPv6 may come as something of a surprise to those network engineers more familiar with IPv4. DHCP is a keystone of IP networks, one piece of network automation that has been widely adopted for years, and an important source of auditing by storing the IP and MAC address combination at the server.

Autoconfig baked-in

But hang on you 32-bit wranglers, we are talking v6 here and that protocol has host address generation baked in, in the form of Stateless Address Autoconfiguration, or SLAAC. In the v6 world, the local gateway acts as vital source of information for the attached end clients, issuing periodic Router Advertisements, or RAs. These advertisements can contain the local prefix, and if that is a /64 with the 'A' flag set, the end client can generate its own address; using the /64 as the network portion and making up the other 64 bits itself. Moreover, it allows clients to generate multiple, temporary prefixes to help prevent attacks Continue reading

Windows for Network Automation?

The OS landscape is changing and, yes, you can learn and run network automation tools on Windows.

After a hot, dry June day in Richmond I boarded an empty train. It being just myself and a couple opposite we struck up a conversation, awaiting the train’s departure. Over the next twenty minutes, as the train slowly worked through a procession of sun-baked stations, I discovered that my companions worked in IT, but in a much earlier time. The white-haired gent confessed to be a C programmer back in the 60s. We talked about software, home lab specs, and genial conversation flowed from mutual interests. However, when I idly announced ‘Microsoft have really changed, Windows has greatly improved’ the C gent gave my comments short shrift and cut them down with:

‘They can’t change, they’re too big.’

No matter, it was a serendipitous meeting that enriched a dull train journey.

That encounter stuck with me, especially the rejection of Microsoft, and I was reminded of it last week in a brief exchange with David Bombal on twitter. The #netdevops guys put together a video about using Windows as a developer environment and David mentioned some negative comments they had received Continue reading

Workshop Epilogue 2

Networkers and Coding Q & A

In Part One of this blog I mentioned that I liked to start the second day of the workshop a little differently. The workshop itself was aimed very much at network engineers but the second day was all about using Python to interact with the ArubaOS-CX API. I know from experience that not everyone is comfortable with the notion of engineers diving into coding, that for many an API is just the latest ‘bright and shiny’ that will dull soon, and that network automation is just a marketing buzzword bubble. Regardless of all this, the exercises were all Python and the attendees were going to make API calls and pick through JSON. There was no exam, no compulsion to attend, no (ridiculous) participation certificate and no armed guards blocking the exits.

"Why are you here?"

With all this in mind I thought we might as well tackle the 'networker vs. dev' subject head on, so I put it to the attendees; "Today is about Python, you are network engineers, why are you here?" Rather than just have them listen to me provide my viewpoint, I wanted the group to interact and provide Continue reading