Rant: You Should Have Written a Book

I apologize for the rant; I have to vent my frustration with people whose quantity of opinions seems to be exceeding their experience (or maybe they’re coming from an alternate universe with different laws of physics, which would be way cool but also unlikely). You’ve been warned; please feel free to move on or skip the rant part of the blog post.

Rant mode: ON

This is the (unedited) gem I received after making some of my EVPN videos public:

netlab: Execute a Command on Multiple Devices

When I was updating the Network Migration with BGP Local-AS Feature blog post, I wanted to execute the same command (show ip bgp) on all routers in my network.

Not a problem: since Dan Partelly added the netlab exec command, it’s as simple as netlab exec * show ip bgp. Well, not exactly; there are still a few quirks.

Response: Peer-to-Peer Communication in IPv6 World

Daryll Swer posted a very informative response to my NAT Traversal Mess blog post, focusing on:

Punching holes through that firewall is equivalent to establishing NAT translations.

It would be a shame to let that response wither as small print at the bottom of a blog post; here it is:

Updated: Network Migration with BGP Local-AS

As I started describing the iBGP Local-AS Details, I discovered that anything I ever wrote about the BGP local-as feature was hosted on a now-extinct website. Fortunately, I found a copy of that article in the Internet Archive and migrated it to the ipSpace.net blog. Hope you’ll like it ;)

Response: NAT Traversal Mess

Let’s look at another part of the lengthy comment Bob left after listening to the Rise of NAT podcast. This one is focused on the NAT traversal mess:

You mentioned that only video-conferencing and BitTorrent use client-to-client connectivity (and they are indeed the main use cases), but hell, do they need to engineer complex systems to circumvent these NATs and firewalls: STUN, TURN, ICE, DHT…

Cleaning up the acronym list first: DHT is unlike the others and has nothing to do with NAT.

iBGP Local-AS Details

Did you know you could use the neighbor local-as BGP functionality to fake an iBGP session between different autonomous systems? I knew Cisco IOS supported that monstrosity for ages (supposedly “to merge two ISPs that have different AS numbers”) and added the appropriate tweaks¹ into netlab when I added the BGP local-as support in release 1.3.1. Someone couldn’t resist pushing us down that slippery slope, and we ended with IBGP local-as implemented on 18 platforms (almost a dozen network operating systems).

I even wrote a related integration test, and all our implementations passed it until I asked myself a simple question: “But does it work?” and the number of correct implementations that passed the test without warnings dropped to zero.

Public Videos: EVPN Fundamentals

The EVPN Fundamentals videos (part of the EVPN Technical Deep Dive webinar) are now public; you can watch them without an ipSpace.net account.

Want to spend more time watching free ipSpace.net videos? The complete list is here.

Transitioning into Networking, 2025 Edition

Elmer sent me the following question:

I’ve been working in systems engineering (Linux, virtualization, infrastructure ops) and am considering shifting toward network engineering or architecture. I got my CCNA years ago and started CCNP but didn’t continue.

I’d really appreciate any thoughts you might have on how someone with my background could best make that transition today, especially with how things are evolving around automation and the cloud.

I keep answering a variant of this question every other year or so (2019, 2021, 2023, 2024). I guess it’s time for another answer, so here we go.

ARP Challenges in EVPN/VXLAN Symmetric IRB

Whenever I claimed that EVPN is The SIP of Networking, vendor engineers quickly told me that “EVPN interoperability is a solved problem” and that they run regular multi-vendor interoperability labs to iron out the quirks. As it turns out, things aren’t as rosy in real life; it’s still helpful to have an EVPN equivalent of the DTMF tone generators handy.

I encountered a particularly nasty quirk when running the netlab EVPN integration test using symmetric IRB with an anycast gateway between Nokia SR Linux (or Juniper vSwitch) and FRR container.

Comparing IP and CLNP: Local (Node) Multihoming

Another area where CLNP is a clear winner when compared to the TCP/IP stack is multi-homed nodes (nodes with multiple interfaces, not site multi-homing, where whole networks are connected to two upstream providers).

Multi-homed TCP/IP nodes must have multiple IP addresses because IP uses address interfaces. There is no well-defined procedure in TCP/IP for how a multi-homed node should behave. In the early days of TCP/IP, they tried to address that in RFC 1122 (Host Requirements RFC), but even then, there were two ideas about dealing with multiple interfaces: the strong and weak end system models (more details).

Worth Reading: Why Quantum Crypto is Bollocks

Today is a perfect day to point out the fantastic Why Quantum Cryptanalysis is Bollocks presentation (HT: A blog post by George Michaelson)

It’s a must-read, even if you’re absolutely uninterested in the topic. Just replace “Quantum mumbo-jumbo” with AI or SDN ;) Have fun!

Passive BGP Sessions

The Dynamic BGP Peers lab exercise gave you the opportunity to build a large-scale environment in which routers having an approved source IP addresses (usually matching an ACL/prefix list) can connect to a BGP route reflector or route server.

In a more controlled environment, you’d want to define BGP neighbors on the BGP RR/RS but not waste CPU cycles trying to establish BGP sessions with unreachable neighbors. Welcome to the world of passive BGP sessions.

Click here to start the lab in your browser using GitHub Codespaces (or set up your own lab infrastructure). After starting the lab environment, change the directory to session/8-passive and execute netlab up.

Response: Any-to-Any Connectivity in the Internet

Bob left a lengthy comment arguing with the (somewhat black-and-white) claims I made in the Rise of NAT podcast. Let’s start with the any-to-any connectivity:

From my young millennial point of view, the logic is reversed: it is because of NATs and firewalls that the internet became so asymmetrical (client/server) just like the Minitel was designed (yes, I am French), whereas the Internet (and later the web, although a client/server protocol, was meant for everyone to be a client and a server) was designed to be more balanced.

Let’s start with the early Internet. It had no peer-to-peer applications. It connected a few large computers (mainframes) that could act as servers but also allowed terminal-based user access and thus ran per-user clients.

Worth Reading: On Writing

One of the most significant problems engineers face when trying to improve their online presence is the “How do I start writing?” roadblock (hint: publishing bland AI-generated slop won’t get you far unless you aim to become a Thought Leader).

Zvi Mowshowitz collected links to over a dozen different writing styles, starting with JRR Tolkien. I’m pretty sure you’ll find something useful in that vast collection.

netlab 1.9.6: Static Routes to Default Gateways

Last week, I had to push out netlab release 1.9.6 to address a particularly nasty Python dependency hell to make netlab work (again) on Ubuntu 24.04 (more details). The release also brought these goodies (and a bunch of bug fixes):

• Add default gateway (including anycast- and VRRP gateway )as a valid next-hop for static routes
• Rewrite the default gateway processing and add IPv6 default gateways on links without anycast or VRRP gateways
• Set libvirt MTU to 9500 on bridge-based networks to avoid the “transparent fragmentation” on Linux bridges.
• Use device- or node variables to specify the Juniper vMX license file.

IPv6 and the Revenge of the Stupid Bridges

This blog post describes another “OMG, this cannot possibly be true” scenario discovered during the netlab VRRP integration testing.

I wanted to test whether we got the nasty nuances of VRRPv3 IPv6 configuration right on all supported platforms and created a simple lab topology in which the device-under-test and an Arista cEOS container would be connected to two IPv6 networks (Arista EOS is a lovely device to use when testing a VRRP cluster because it produces JSON-formatted show vrrp printouts).

Most platforms worked as expected, but Aruba CX, Cumulus Linux with NVUE, and Dell OS10 consistently failed the tests. We were stumped until Jeroen van Bemmel discovered that the Arista container forwards IPv6 router advertisements between the two LAN segments.

Routed Interfaces on Layer-3 Switches and Internal VLANs

In the Router Interfaces and Switch Ports blog post, I described why we have switch ports and routed interfaces on layer-3 switches. Another blog post in the same series described the conceptual architecture of a layer-3 switch:

• All interfaces are connected to a VLAN-aware switch
• The switch interfaces could be access or trunk interfaces¹.
• Each VLAN in a VLAN-aware switch can be connected to an internal router through a VLAN interface.

However, that’s not how we configure layer-3 switches. There’s a significant gap between the conceptual configuration model and the internal architecture:

Comparing IP and CLNP: Network State Summarization

In the previous blog posts, we discussed how TCP/IP and CLNP reach adjacent nodes and build ARP/ND/ES caches and how they reach off-subnet nodes. Now, let’s move from the network edge into the network core and explore how the two protocol stacks reduce the amount of information they have to propagate in routing protocols.

While I’m not exactly an OSI fan, I must admit they got many things right (and IPv6 copied those ideas), but TCP/IP is a clear winner in this aspect.

Worth Reading: Standards for ANSI Escape Codes

I encountered the Escape sequences (named after the first character in the sequence) while programming stuff that would look nicely on the venerable VT100 terminals (not to mention writing one or two VT100 emulators myself).

In the meantime, those sequences got standardized and (par for the course) extended with “proprietary” stuff everyone uses now. Julia Evans did a great job documenting the state of the art. Thanks a million!

Arista EOS Spooky Action at a Distance

This blog post describes yet another bizarre behavior discovered during the netlab integration testing.

It started innocently enough: I was working on the VRRP integration test and wanted to use Arista EOS as the second (probe) device in the VRRP cluster because it produces nice JSON-formatted results that are easy to use in validation tests.

Everything looked great until I ran the test on all platforms on which netlab configures VRRP, and all of them passed apart from Arista EOS (that was before we figured out how Sturgeon’s Law applies to VRRPv3) – a “That’s funny” moment that was directly responsible for me wasting a few hours chasing white rabbits down this trail.