Dear Vendors, EVPN Route Attributes Matter
Another scary tale from the Archives of Sloppy Code: we can’t decide whether some attributes are mandatory or optional.
When I was fixing the errors in netlab SR-OS configuration templates, I couldn’t get the EBGP-based EVPN with overlapping leaf AS numbers to work. I could see the EVPN routes in the SR-OS BGP table, but the device refused to use them. I concluded (incorrectly) that there must be a quirk in the SR-OS EVPN code and moved on.
Public Videos: Whole IPv6 Curriculum
Based on the feedback I received on LinkedIn and in private messages, I made all my IPv6 content public; you can watch those videos without an ipSpace.net account.
Want to spend more time watching free ipSpace.net videos? The complete list is here.
netlab 25.06: Fixing Nokia SR-OS Configuration Templates
TL&DR: netlab release 25.06 was published last week.
Before discussing the new features, let’s walk the elephant out of the room: I changed the release versions to YY.MM scheme, so I will never again have to waste my time on the existential question of which number in the release specification to increase.
Now for the new features:
Finding Source Routing Paths
In the previous blog post, we discussed the generic steps that network devices (or a centralized controller) must take to discover paths across a network. Today, we’ll see how these principles are applied in source routing, one of the three main ways to move packets across a network.
Brief recap: In source routing, the sender has to specify the (loose or strict) path a packet should take across the network. The sender thus needs a mechanism to determine that path, and as always, there are numerous solutions to this challenge. We’ll explore a few of them, using the sample topology shown in the following diagram.
ArubaCX Cannot Count When Dealing with VXLAN
This blog post describes yet another bizarre example of how reliable digital twins are, but don’t worry; they all work great in PowerPoint.
After “fixing” the integration tests to deal with ArubaCX’s notion of VXLAN VNI having 16 bits, the bridging test worked, but the IRB tests kept failing.
In the IRB test, the lab has two layer-3 switches. Each of them should be able to bridge within a VLAN/VXLAN segment and route across the segments.
netlab 2.0: Routers, Hosts, Gateways and Bridges
In a previous blog post, I explained how you can use bridges in a netlab topology to create custom LAN segments. Netlab supports two other node roles (host and router), and we’ll eventually add gateways.
netlab assumes that most network devices are routers (it considers a firewall to be a router in disguise), apart from Linux hosts, but you can always change what a node is with the role node attribute:
Interesting: Juniper MX and Jumbo Frames
Did you know that there’s an Ethernet link between the Packet Forwarding Engine (PFE – data plane) and Routing Engine (RE – control plane) in every Juniper MX? That’s why you have to run two VMs to emulate it (sometimes conveniently packed into one larger VM, proving RFC 1925 rule 6a).
That Ethernet link happens to have the MTU fixed at 1500 bytes. Guess what happens in the world where everyone uses jumbo frames? Did you say fragmentation? Bingo! And what do you think happens when one of those fragments gets dropped due to control-plane policing, and the rest of them are stuck in the reassembly queue? You’ll find the gory details in a lengthy blog post by Nitzan Tzelniker.
Publishing Content as an Introvert
I got an interesting question from a reader. He listened to my podcast with Eric Chou and decided to try to learn in public:
Currently, I’m studying for the CCNP ENARSI exam, and would like to start posting my labs to LinkedIn, and perhaps even upload my lab topologies and configs to Git.
That’s a great idea. I would minimize the LinkedIn part1 and focus on Git:
Finding End-to-End Paths: Topology and Endpoints
We know there are three main ways to move packets across a network. However, before we can start forwarding packets, someone has to populate the forwarding tables in the intermediate devices or build the sequence of nodes to traverse in source routing.
Usually, whoever is responsible for the contents of the forwarding tables must first discover the network topology. Let’s start there, using the following network diagram to illustrate the discussion.
Weird: Ports on Linux Bridge Are Stuck
Just when you thought you got used to the weirdnesses in the networking implementations, you get a curveball like this one. Life is never dull if you test network devices.
Before releasing netlab release 2.0, I ran the full suite of integration tests for all devices for which I have the images. Interestingly, most VXLAN tests failed for Cumulus Linux 4.x even though we haven’t touched that code for ages.
Next step: trying to figure out what changed. The configuration changes were minimal. Even worse, the failure was non-deterministic. Somehow, we managed to transform a Cumulus Linux 4.x VM into a Heisenberg switch.
Where Are the NETCONF/YANG Tools?
Jo attempted to follow the vendor Kool-Aid recommendations and use NETCONF/YANG to configure network devices. Here’s what he found (slightly edited):
IMHO, the whole NETCONF ecosystem primarily suffers from a tooling problem. Or I haven’t found the right tools yet.
ncclient is (as you mentioned somewhere else) an underdocumented mess. And that undocumented part is not even up to date. The commit hash at the bottom of the docs page is from 2020… I am amazed how so many people got it working well enough to depend on it in their applications.
Interesting: Bootstrapping HTTPS
Jan Schaumann published an interesting blog post describing the circuitous journey a browser might take to figure out that it can use QUIC with a web server.
Now, if only there were a record in a distributed database telling the browser what the web server supports. Oh, wait… Not surprisingly, browser vendors don’t trust that data and have implemented a happy eyeballs-like protocol to decide between HTTPS over TCP and QUIC.
Cumulus Linux (As We Know It) Is Gone
A reader of my blog pointed out the following minutiae hidden at the very bottom of the Cumulus Linux 5.13 What’s New document:
NVIDIA no longer releases Cumulus VX as a standalone image. To simulate a Cumulus Linux switch, use NVIDIA AIR.
And what is NVIDIA AIR?
Multi-Layer Switching and Tunneling
When deep-diving into the confusing terminology of switching, routing, and bridging, I mentioned you could perform packet forwarding at different layers of a networking stack. In this blog post, we’ll explore what happens when we combine packet forwarding on multiple layers within a single network, resulting in multi-layer switching, where edge devices perform Layer n forwarding (usually Layer 3), and core devices perform Layer n-1 forwarding (typically Layer 2).
Each layer can use any forwarding paradigm you choose. However, since we generally use IP at Layer 3, edge devices typically perform hop-by-hop destination-based forwarding, while core devices can use alternative methods.
Dear ArubaCX, VXLAN VNI Has 24 Bits
I thought I’ve seen it all, but the networking vendors (and their lack of testing) never cease to amaze me. Today’s special: ArubaCX software VXLAN implementation.
We decided it’s a good idea to rewrite the VXLAN integration tests to use one target device and one FRR container to test inter-vendor VXLAN interoperability. After all, what could possibly go wrong with a simple encapsulation format that could be described on a single page?
Everything worked fine (as expected), except for the ArubaCX VM (running release Virtual.10.15.1005, build ID AOS-CX:Virtual.10.15.1005:9d92f5caa6b6:202502181604), which failed every single test.
Worth Reading: Practical Advice for Engineers
Sean Goedecke published an interesting compilation of practical advice for engineers. Not surprisingly, they include things like “focus on fundamentals” and “spend your working time doing things that are valuable to the company and your career” (OMG, does that really have to be said?).
Bonus point: a link to an article by Patrick McKenzie (of the Bits About Money fame) explaining why you SHOULD NOT call yourself a programmer (there goes the everyone should be a programmer gospel 😜).
ChatGPT Strikes Again: IS-IS on Unnumbered Interfaces 🤦♂️
In the last few days, I decided to check out how much better ChatGPT has gotten in the last year or two. I tried to be positive and was rewarded with some surprisingly good results. I even figured out I can use it to summarize my blog posts using prompts like this one:
Using solely the information from blog.ipspace.net, what can you tell me about running ospf over unnumbered interfaces
And then I asked it about unnumbered interfaces and IS-IS, and it all went sideways:
Repost: On the Advantages of XML
Continuing the discussion started by my Breaking APIs or Data Models Is a Cardinal Sin and Screen Scraping in 2025 blog posts, Dr. Tony Przygienda left another thoughtful comment worth reposting as a publicly visible blog post:
Having read your newest rant around my rant ;-} I can attest that you hit the nail on the very head in basically all you say:
- XML output big? yeah.
- JSON squishy syntax? yeah.
- SSH prioritization? You didn’t live it until you had a customer where a runaway python script generated 800+ XML netconf sessions pumping data ;-)
Video: netlab Introduction by Ethan Banks
Ethan Banks created an excellent introductory Netlab - Automate Your Network Labs With YAML! video showing how easy it is to start and configure a container- or VM-based lab, and explaining the basic netlab commands you need to get started.
Thanks a million!
P.S.: If you’ve done something similar and I haven’t noticed it, please send me the link.
Response: True Unnumbered Interfaces
Hendrik left an interesting comment on my Running IS-IS over Unnumbered Ethernet Interfaces blog post:
FRRouting (Linux) with pure IS-IS, the only way it currently (10.3) works is to copy the loopback IPv4 address to the interfaces that you need to do IPv4 routing on. The OpenFabric (IS-IS “extension” draft) does support true unnumbered interfaces and routes IPv6.
Let’s unpack this. There are (at least) four reasons a router needs an address associated with an interface1: