Modifying BGP Behavior with xBGP API

When I reposted a link to xBGP: Faster Innovation in Routing Protocols paper, someone immediately replied

Quite interesting, but it feels like this could become the proverbial 15th standard.

xBGP is an API that allows BGP users to implement routing policies (route selection, filtering, or propagation) that use attributes or mechanisms defined in newer IETF RFCs or drafts, so the proverbial 15th standard is not that far off the mark. However, we must remember that what we call BGP is more than just a set of competing standards.

Building a DMVPN Test Lab with netlab

I always love to hear about real-life netlab use cases, and try to make them even easier to implement with new netlab features – that’s how netlab got custom Vagrant configuration templates and per-node configuration templates.

When Anne Baretta sent me his initial DMVPN solution, we quickly figured out we could make it even cleaner if netlab supported tunnel interfaces; you can enjoy the results in release 1.5.2, and explore Anne’s solution on GitHub.

MUST READ: End-to-End Arguments in System Design

In case you ever wondered how old the “keep network simple and do complex stuff at the endpoints” approach is, read the End-to-End Arguments in System Design article from 1981.

For whatever reason (hint: profits), networking vendors keep ignoring those arguments, turning the network into a kitchen sink of complexity.

Fun tidbit: the article describes a variant of relying on layer-2 checksums will corrupt your data. Some things never change.

Worth Reading: IPv6 Deployment Status

RFC 9386 documenting IPv6 deployment status in late 2022 has been published a few weeks ago¹. It claims over a billion IPv6-capable users, and IPv6 deployment close to 50% in major countries.

Web content is a different story: while 40% of top-500 sites are IPv6-enabled, you can reach only ~20% of web sites over IPv6. Considering Cloudflare’s free proxying includes IPv6 that is enabled by default, that proves (once again) how slowly things change in IT.

Video: 400GbE Optics

When 400GbE was still an emerging technology, Mark Nowell explained its basics in an update session of the Data Center Fabric Architectures webinar, starting with 400GbE optics.

Silent Hosts in EVPN Fabrics

The Dynamic MAC Learning versus EVPN blog post triggered tons of interesting responses describing edge cases and vendor bugs implementation details, including an age-old case of silent hosts described by Nitzan:

Few years ago in EVPN network, I saw drops on the multicast queue (ingress replication goes to that queue). After analyzing it we found that the root cause is vMotion (the hosts in that VLAN are silent) which starts at a very high rate before the source leaf learns the destination MAC.

It turns out that the behavior they experienced was caused by a particularly slow EVPN implementation, so it’s not exactly the case of silent hosts, but let’s dig deeper into what could happen when you do have silent hosts attached to an EVPN fabric.

Small Site EBGP-Only Design

One of my subscribers found an unusual BGP specimen in the wild:

• It was a small site with two core switches and a WAN edge router
• The site had VPN concentrators running in virtual machines
• The WAN edge router was running BGP across WAN IPsec tunnels
• The VPN concentrators were running BGP with core switches.

So far so good, and kudos to whoever realized BGP is the only sane protocol to run between virtual machines and network core. However, the routing in the network core was implemented with EBGP sessions between the three core devices, and my subscriber thought the correct way to do it would be to use IBGP and OSPF.

netlab Release 1.5.2: Aruba CX, External Tools, Tunnel Interfaces

netlab release 1.5.2 brings another bunch of cool features, including:

• Aruba AOS-CX Support by Stefano Sasso
• External network management tools that you can start together with your lab
• Tunnel interfaces
• Reusable topology components

I’ll cover these features in separate blog posts; today I wanted to highlight a few minor additions:

Interesting: BGP Zombie Outbreak on Juniper Routers

BGP zombies are routes in the BGP table that refuse to disappear even though they should have been long gone. Recent measurements estimate between 0.5% and 1.5% of all routes in the global BGP table are zombies, which sounds crazy – after all, BGP is supposed to be pretty reliable.

Daryll Swer identified one potential source – Juniper routers do not revoke suppressed aggregated prefixes – and documented it in Navigating a BGP zombie outbreak on Juniper routers.

CloudFlare: From IP packets to HTTP

Want to know some details behind the CloudFlare SD-WAN implementation? You might find them in From IP packets to HTTP: the many faces of our Oxy framework.

I don’t know enough about Linux networking to figure out whether one could use those details to build something similar, but CloudFlare blog posts keep begin much better than Google’s Look How Awesome We Are recruitment drives.

Advantages and Drawbacks of EVPN-based Multihoming

Lukas Krattiger wrapped up his EVPN-versus-MLAG presentation (part of EVPN Deep Dive webinar) with an overview of the advantages and drawbacks of EVPN-based multihoming solutions:

• N-way multihoming
• Flexible connectivity (no need for a peer link)
• Fabric-wide scope (MAC multipathing required on ingress node)

Why Is OSPF (and BGP) More Complex than STP?

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough"):

Is Dynamic MAC Learning Better Than EVPN?

One of my readers worried about the control-plane-induced MAC learning lag in EVPN-based networks:

In all discussions about the advantages/disadvantages of VXLAN/EVPN, I can’t find any regarding the lag in learning new macs when you use the control plane for mac learning.

EVPN is definitely slower than data plane-based dynamic MAC learning (regardless of whether it’s done in hardware or software), but so is MLAG.

IPv6 Security in Layer-2 Firewalls

You can configure many firewalls to act as a router (layer-3 firewall) or as a switch bridge (layer-2 firewall). The oft-ignored detail: how does a layer-2 firewall handle ARP (or any layer-2 protocol)?

Unless you want to use static ARP tables it’s pretty obvious that a layer-2 firewall MUST propagate ARP. It would be ideal if the firewall would also enforce layer-2 security (ARP/DHCP inspection and IPv6 RA guard), but it looks like at least PAN-OS version 11.0 disagrees with that sentiment.

Straight from Layer 2 and Layer 3 Packets over a Virtual Wire:

Use Existing Device Configurations in netlab

Anne Baretta decided to use netlab to test a proposed DMVPN topology. As netlab doesn’t support DMVPN (and probably never will), he decided to use netlab capabilities to start the lab topology and perform initial configuration, adding DMVPN configuration commands as custom configurations. Here’s how he described the process:

In this case I used netlab as a quick way to get a topology up and running, and then add the DMVPN configuration by hand.

Systems Design: What We Hope We Know

Avery Pennarun published a lovely rambling on magic, science, engineering and a pinch of AI. You might enjoy reading it¹ with your Sunday morning coffee 😎.

180 Gbps Software-Only Linux Router

Pim van Pelt built an x86/Linux-based using Vector Packet Processor that can forwarding IP traffic at 150 Mpps/180 Gbps forwarding rates on a 2-CPU Dell server with E5-2660 (8 core) CPU.

He described the whole thing in a 8-part series of blog posts and a conference talk. Enjoy!

Video: SD-WAN CPE Architecture

Pradosh Mohapatra started the Typical SD-WAN Solution Architecture section of Software-Defined WAN (SD-WAN) Overview webinar with the backend architecture.

Next step: CPE architecture, the topic of today’s video.

Multi-Vendor EVPN Fabrics

Daniel left an interesting comment on my Studying EVPN to Prepare for a Job Interview blog post:

I also never build a VXLAN fabric with two vendors. So, is it possible now to build one large fabric consisting of multiple vendors?

TL&DR: Yes, but just because you could doesn’t mean that you should.

New: Network Infrastructure as Code Resources

While I was developing Network Automation Concepts webinar and the network automation online course, I wrote numerous blog posts on the Network Infrastructure as Code (NIaC) concepts, challenges, implementation details, tools, and sample solutions.

In March 2023 I collected these blog posts into a dedicated NIaC resources page that also includes links to webinars, sample network automation solutions, and relevant GitHub repositories.