Ansible Set Operations Do Not Preserve List Order

Here’s another Ansible quirk, this time caused by Python set behavior.

When I created the initial device configuration deployment playbook in netlab, I wanted to:

• Be able to specify a list of modules to provision.¹
• Provision just the modules used in the topology and specified in the list of modules.

This allows you to use netlab initial to deploy all configuration modules used in a lab topology or netlab initial -m ospf to deploy just OSPF while surviving netlab initial -m foo (which would do nothing).

Worth Reading: Popular git config options

Another must-explore gem by Julia Evans: Popular git config options.

Side note: I keep collecting links to insightful Git articles in the Git and GitHub section of the Network Automation Tools webinar.

BGP Labs: Remove Private AS from AS-Path

In a previous BGP lab exercise, I described how an Internet Service Provider could run BGP with a customer without the customer having a public BGP AS number. The only drawback of that approach: the private BGP AS number gets into the AS path, and everyone else on the Internet starts giving you dirty looks (or drops your prefixes).

Let’s fix that. Most BGP implementations have some remove private AS functionality that scrubs AS paths during outgoing update processing. You can practice it in the Remove Private BGP AS Numbers from the AS Path lab exercise.

Implementing ‘Undo’ Functionality in Network Automation

Kurt Wauters sent me an interesting challenge: how do we do rollbacks based on customer requests? Here’s a typical scenario:

You might have deployed a change that works perfectly fine from a network perspective but broke a customer application (for example, due to undocumented usage), so you must be able to return to the previous state even if everything works. Everybody says you need to “roll forward” (improve your change so it works), but you don’t always have that luxury and might need to take a step back. So, change tracking is essential.

He’s right: the undo functionality we take for granted in consumer software (for example, Microsoft Word) has totally spoiled us.

Applying BGP Policy Templates

I got this question after publishing the BGP Session Templates lab exercise:

Would you apply BGP route maps with a peer/policy template or directly to a BGP neighbor? Of course, it depends; however, I believe in using a template for neighbors with the same general parameters and being more specific per neighbor when it comes to route manipulation.

As my reader already pointed out, the correct answer is It Depends, now let’s dig into the details ;)

Network Layer: Interface or Node Addresses

The fun question about network layer addresses is: are we addressing nodes or individual node interfaces? On the data link layer, we never had this issue because it was obvious that a data link layer endpoint is an interface, so each interface should have a unique data link layer address.

Interestingly, that’s not the case on transparent bridges. Even though they have multiple interfaces, the whole bridge has a single MAC address, so one could claim we’re addressing nodes connected to a single data link layer. The IEEE standard is unambiguous: in every relevant diagram, the MAC address sits on top of multiple interfaces because the MAC address belongs to the control plane.

Public Videos: 400 Gigabit Ethernet

The 400 Gigabit Ethernet videos with Mark Nowell and Lukas Krattiger are now public. Enjoy!

BGP Labs: Session Templates

Configuring an IBGP session on a route reflector takes a half-dozen parameters, starting with the remote BGP AS number (equal to the local one), remote IP address, and the source IP address or interface. You might have to specify the propagation of BGP communities and an MD5 password, and you will definitely have to specify that the BGP neighbor is a route reflector client.

Wouldn’t it be nice if you could group those parameters into a template and apply the template to a neighbor? Most BGP implementations have something along those lines. That feature could be called a session template or a peer group, and you can practice it in the next BGP lab exercise.

Can We Skip the Network Layer?

I mentioned that you don’t need node addresses when dealing with only two entities. Now and then, someone tries to extend this concept and suggests that the network layer addressing isn’t needed if the solution is local. For instance, if we have a solution that is supposed to run only on a single Ethernet segment, we don’t need network layer addressing because we already have data link layer addresses required for Ethernet to work (see also: ATAoE).

Too often in the past, an overly ingenious engineer or programmer got the idea to simplify everyone’s life and use the data link layer addresses as the ultimate addresses of individual nodes. They would then put the transport layer on top of that to get reliable packet transport. Finally, put whatever application on top of the transport layer. Problem solved.

Removing FRRouting Configuration Is Not Idempotent

One of the few beauties of most “industry standard CLI” implementations¹ is that they’re idempotent: nothing changes (apart from ACLs) if you configure the same stuff a dozen times. Most of these implementations allow you to deconfigure the same stuff multiple times; FRRouting is one of the unfortunate exceptions.

What Am I Talking About?

Imagine you have a bunch of IP prefixes you want to advertise with BGP. You could use network statements within the router bgp configuration to get that done:

BGP Labs: Use Multiple AS Numbers on the Same Router

Can you use BGP to connect to the global Internet without having a public BGP AS number? Of course, assuming your Internet Service Provider is willing to run BGP with a network using a private AS number. But what happens if you want to connect to two ISPs? It’s ridiculous to expect you’ll be able to persuade them to use the same private AS number.

That’s one of the many use cases for the local-as functionality available in most BGP implementations. You can practice it in the Use Multiple AS Numbers on the Same Router lab exercise.

Precedence of Ansible Extra Variables

I stay as far away from Ansible as possible these days and use it only as a workflow engine to generate device configurations from Jinja2 templates and push them to lab devices. Still, I manage to trigger unexpected behavior even in these simple scenarios.

Ansible has a complex system of variable (fact) precedence, which mostly makes sense considering the dozen places where a variable value might be specified (or overwritten). Ansible documentation also clearly states that the extra variables (specified on the command line with the -e keyword) have the highest precedence.

Now consider these simple playbooks. In the first one, we’ll set a fact (variable) and then print it out:

BGP Labs: Override Neighbor AS Number in AS Path

When I described the need to turn off the BGP AS-path loop prevention logic in scenarios where a Service Provider expects a customer to reuse the same AS number across multiple sites, someone quipped, “but that should be fixed by the Service Provider, not offloaded to the customer.”

Not surprisingly, there’s a nerd knob for that (AS override), and you can practice it in the next BGP lab exercise: Fix AS-Path in Environments Reusing BGP AS Numbers.

Public Videos: Kubernetes Container Network Interface (CNI)

All the Container Network Interface videos from the Kubernetes Networking Deep Dive webinar with Stuart Charlton are now public. Enjoy!

Availability Zones in VMware NSX-Based Cloud

One of the ipSpace.net subscribers sent me this question:

How could I use NSX to create a cloud-like software network layer enabling a VMware enterprise to create a public cloud-like availability zone concept within a data center (something like Oracle Cloud does)?

That’s easy: stop believing in VMware marketing shenanigans.

Podcast: Network Automation Source(s) of Truth

Figuring out how to describe your network (also known as “create a source of truth”) is one of the most challenging tasks you’ll face when building a network automation solution (more). As always, the devil is in the details, starting with “and what exactly is The Truth?”.

We discussed those details in a lively Packet Pushers podcast with Claudia de Luna, David Sinn, Dinesh Dutt, Drew Conry-Murray and Ethan Banks. Have fun!

BGP Labs: Work with FRR and Cumulus Linux

FRR or (pre-NVUE) Cumulus Linux are the best bets if you want to run BGP labs in a resource-constrained environment like your laptop or a small public cloud instance. However, they both behave a bit differently from what one might expect from a networking device, including:

• Interfaces are created through standard Linux tools;
• You have to start the FRR management CLI from the Linux shell;
• If you need a routing daemon (for example, the BGP daemon), you must enable it in the FRR configuration file and restart FRR.

A new lab exercise covers these intricate details and will help you get fluent in configuring BGP on FRR or Cumulus Linux virtual machines or containers.

BGP Graceful Restart Considered Harmful

A networking engineer with a picture-perfect implementation of a dual-homed enterprise site using BGP communities according to RFC 1998 to select primary- and backup uplinks contacted me because they experienced unacceptably long failover times.

They measured the failover times caused by the primary uplink loss and figured out it takes more than five minutes to reestablish Internet connectivity to their site.

Public Videos: Large Language Models (AI/ML)

All the Large Language Models videos from the AI/ML in Networking webinar with Javier Antich are now public. Enjoy!

Is It The End, Or Can You Do Something in 2024?

David Bombal invited me for another annual chat last December, focusing on (what else) networking careers in 2024. The results were published a few days ago, and I was amazed at how good it turned out. I always love chatting with David; this time, his editing team did a masterful job.