netlab 1.9.3: MLAG, Static Routes, Node Cloning

netlab release 1.9.3 brings these new features:

• Multi-chassis Link Aggregation (MLAG) on Arista EOS, Aruba CX, Cumulus NVUE, and Dell OS10
• VRF and VLAN groups
• Additional OSPF interface parameters (hello and dead timers, cleartext passwords, and DR priority) implemented on Arista EOS, Aruba CX, Cisco IOS/IOS-XE, Cisco Nexus OS, Cumulus Linux, Dell OS10, and FRRouting
• Static routes with direct or indirect next hops implemented on Arista EOS, Cisco IOS/IOS-XE, FRRouting, and Linux
• Node cloning plugin for users who want to build detailed digital twins of their networks.
• Consistent selection of default address pools based on the number of nodes attached to a link (this could change addressing in multi-provider topologies)
• Support for vjunos-router and Cisco NSO tool.

Other new features include:

Configuring IP Addresses Won’t Make You an Expert

A friend of mine recently wrote a nice post explaining how netlab helped him set up a large network topology in a reasonably short timeframe. As expected, his post attracted a wide variety of comments, from “netlab is a gamechanger” (thank you 😎) to “I prefer traditional labs.” Instead of writing a bunch of replies into a walled-garden ecosystem, I decided to address some of those concerns in a public place.

Let’s start with:

OSPFv3 on Bird Needs IPv6 LLA on the Loopback Interface

Wanted to share this “too weird to believe” SNAFU I found when running integration tests with the Bird routing daemon. It’s irrelevant unless you want Bird to advertise the IPv6 prefix configured on the main loopback interface (lo) with OSPFv3.

Late last year, I decided to run netlab integration tests with the Bird routing daemon. It passed most baseline netlab OSPFv3 integration tests but failed those that checked the loopback IPv6 prefix advertised by the tested device (test results).

Happy Holidays and All the Best in 2025!

Another year is almost gone, and it’s time for my traditional “I will disappear until mid-January” retreat (also, don’t expect me to read my email until I’m back).

I hope you’ll also be able to disconnect from the crazy pace of the networking world, forget the “AI will make networking engineers obsolete” shenanigans (hint: SDN did not), and focus on your loved ones. I would also like to wish you all the best in 2025!

I will probably get bored sometime in late December, so expect a few new netlab features in early January.

Worth Reading: Hard Truths about AI-assisted Coding

Addy Osmani published an excellent overview of the challenges of AI-assisted coding. They apply equally well to the “AI will generate device configurations for me” or “AI will troubleshoot my network” ideas (ignoring for the moment the impact of the orders-of-magnitude smaller training set), so it’s definitely worth reading.

I particularly liked the “‌AI is like having a very eager junior developer on your team” take, as well as the description of the “70% problem” (AI will get you 70% there, but the last 30% will be frustrating) – a phenomenon perfectly illustrated by the following diagram by Forrest Brazeal:

Use Disaggregated BGP Prefixes to Influence Inbound Internet Traffic

As much as I love explaining how to use BGP in an optimal way, sometimes we have to do what we know is bad to get the job done. For example, if you have to deal with clueless ISPs who cannot figure out how to use BGP communities, you might be forced to use the Big Hammer of disaggregated prefixes. You can practice how that works in the next BGP lab exercise.

Click here to start the lab in your browser using GitHub Codespaces (or set up your own lab infrastructure). After starting the lab environment, change the directory to policy/b-disaggregate and execute netlab up.

Video: Internet Routing Security (DEEP 2023)

My Internet Routing Security talk from last year’s DEEP conference (a shorter version of the Internet Routing Security webinar) is now available on YouTube.

Hope you’ll find it useful ;)

IBGP Source Interface Selection Still Requires Configuration

A fellow networking engineer recently remarked, “FRRouting automatically selects the correct [IBGP] source interface even when not configured explicitly.”

TL&DR: No, it does not. You were just lucky.

Basics first¹. BGP runs over TCP sessions. One of the first things a router does when establishing a BGP session with a configured neighbor is to open a TCP session with the configured neighbor’s IP address.

Netlab Is Four Years Old

On December 9th, 2020, I created a new GitHub repository and pushed the first commit of my “I hate creating Vagrantfiles by hand” tool. It could create Vagrantfile and Ansible inventory from a (very rudimentary) network topology and deploy handcrafted device configurations on Cisco IOS and Arista EOS.

Worth Reading: Codespaces for Network Engineers

When I discovered GitHub Codespaces (thanks to a pointer by Roman Dodin), I did the absolute minimum of research to get netlab up and running in a container to enable Codespaces-based labs (BGP, IS-IS) and netlab examples.

However, if you want to know the behind-the-scenes details, you MUST read the Codespaces for Network Engineers and Educators deep dive by Julio Perez.

Is BGP PIC Edge an Oxymoron?

This blog post discusses an old arcane question that has been nagging me from the bottom of my Inbox for almost exactly four years. Please skip it if it sounds like Latin to you, but if you happen to be one of those readers who know what I’m talking about, I’d appreciate your comments.

Terminology first:

• Prefix Independent Convergence allows entries in the forwarding table to point to shared next hops (or next-hop groups), reducing the FIB update bottleneck when changing the next hop for a large number of prefixes (for example, when dealing with a core link failure). More details in the initial blog post and PIC applicability to fast reroute.
• PIC Edge (as defined by vendor marketing) is the ability to switch to a backup CE route advertised to a backup PE router before the network convergence is complete.

Here’s (in a nutshell) how PIC Edge is supposed to work:

netlab: Sample Cisco ASAv Topology

A happy netlab user asked for a sample Cisco ASAv topology that would include an inside and an outside router.

We don’t have anything similar in the netlab examples yet, so let’s build a simple topology with two routers, a firewall, and a few hosts.

However, we have to start with a few caveats:

Example: Multi-AS netlab Topology

A few weeks ago, Urs Baumann posted a nice example illustrating the power of netlab: a 10-router topology running OSPF, IS-IS, and BGP:

He didn’t post the underlying topology file, so let’s create a simple topology to build something similar.

Lab: Dual-Stack IS-IS Routing

Contrary to the OSPF world, where we have to use two completely different routing protocols to route IPv4 and IPv6 (unless you believe in the IPv4 address family in OSPFv3), IS-IS provided multi-protocol support from the very early days of its embracement by IETF. Adding IPv6 support was only a matter of a few extra TLVs, but even there, IETF gave us two incompatible ways of making IPv6 work with IS-IS.

Want to know more? You’ll find the details in the Dual-Stack (IPv4+IPv6) IS-IS Routing lab exercise.

IPv6 Support for Multiple Routers and Multiple Interfaces

Fernando Gont published an Individual Internet Draft (meaning it hasn’t been adopted by any IETF WG yet) describing the Problem Statement about IPv6 Support for Multiple Routers and Multiple Interfaces. It’s so nice to see someone finally acknowledging the full scope of the problem and describing it succinctly. However, I cannot help but point out that:

• I was ranting about that problem in 2009 (15 years ago) and did a summary of older rants in 2015.
• It was evident to everyone but the religious zealots that the only solution we have at the moment is either NAT (because stuff simply does not work otherwise) or host-based solutions that never got implemented (apart from a few rare cases of multipath TCP).

Anyway, Fernando wraps up his draft with:

EVPN Designs: EVPN IBGP over IPv4 EBGP

We’ll conclude the EVPN designs saga with the “most creative” design promoted by some networking vendors: running an IBGP session (carrying EVPN address family) between loopbacks advertised with EBGP IPv4 address family.

There’s just a tiny gotcha in the above Works Best in PowerPoint diagram. IBGP assumes the BGP neighbors are in the same autonomous system while EBGP assumes they are in different autonomous systems. The usual way out of that OMG, I painted myself into a corner situation is to use BGP local AS functionality on the underlay EBGP session:

Dynamic BGP Peers

You might have an environment where a route reflector (or a route server) has dozens or hundreds of BGP peers. Configuring them by hand is a nightmare; you should either build a decent automation platform or use dynamic BGP neighbors – a feature you can practice in the next lab exercise.

Click here to start the lab in your browser using GitHub Codespaces (or set up your own lab infrastructure). After starting the lab environment, change the directory to session/9-dynamic and execute netlab up.

Public Videos: Enterprise MPLS/VPN

Over a decade ago, I created a webinar describing enterprise MPLS/VPN use cases. Surprisingly, some networking engineers still find it useful in the wonderful new world of SD-WAN duct tape, and starting today, you can access it without an ipSpace.net account.

Lab: Using IS-IS Metrics

It’s time for another “the vendor IS-IS defaults are all wrong” blog post. Wide IS-IS metrics were standardized in RFC 3784 in June 2004, yet most vendors still use the ancient narrow metrics as the default setting.

Want to know more? The Using IS-IS Metrics lab exercise provides all the gory details.

Latency Numbers Every Programmer Should Know

One of the key arguments against stretched clusters (and similar stupidities) I used in my Disaster Recovery Myths presentation was the SSD read latency versus cross-site round-trip time.

Thanks to Networking Notes, I found a great infographic I can use in my next presentation (bonus points: it also works great in a terminal when fetched with curl) and a site that checks the latency of your web site from various vantage points.