MUST READ: End-to-End Arguments in System Design

In case you ever wondered how old the “keep network simple and do complex stuff at the endpoints” approach is, read the End-to-End Arguments in System Design article from 1981.

For whatever reason (hint: profits), networking vendors keep ignoring those arguments, turning the network into a kitchen sink of complexity.

Fun tidbit: the article describes a variant of relying on layer-2 checksums will corrupt your data. Some things never change.

Worth Reading: IPv6 Deployment Status

RFC 9386 documenting IPv6 deployment status in late 2022 has been published a few weeks ago¹. It claims over a billion IPv6-capable users, and IPv6 deployment close to 50% in major countries.

Web content is a different story: while 40% of top-500 sites are IPv6-enabled, you can reach only ~20% of web sites over IPv6. Considering Cloudflare’s free proxying includes IPv6 that is enabled by default, that proves (once again) how slowly things change in IT.

Worth Reading: IPv6 Deployment Status

RFC 9386 documenting IPv6 deployment status in late 2022 has been published a few weeks ago¹. It claims over a billion IPv6-capable users, and IPv6 deployment close to 50% in major countries.

Web content is a different story: while 40% of top-500 sites are IPv6-enabled, you can reach only ~20% of web sites over IPv6. Considering Cloudflare’s free proxying includes IPv6 that is enabled by default, that proves (once again) how slowly things change in IT.

Video: 400GbE Optics

When 400GbE was still an emerging technology, Mark Nowell explained its basics in an update session of the Data Center Fabric Architectures webinar, starting with 400GbE optics.

Video: 400GbE Optics

When 400GbE was still an emerging technology, Mark Nowell explained its basics in an update session of the Data Center Fabric Architectures webinar, starting with 400GbE optics.

Silent Hosts in EVPN Fabrics

The Dynamic MAC Learning versus EVPN blog post triggered tons of interesting responses describing edge cases and vendor bugs implementation details, including an age-old case of silent hosts described by Nitzan:

Few years ago in EVPN network, I saw drops on the multicast queue (ingress replication goes to that queue). After analyzing it we found that the root cause is vMotion (the hosts in that VLAN are silent) which starts at a very high rate before the source leaf learns the destination MAC.

It turns out that the behavior they experienced was caused by a particularly slow EVPN implementation, so it’s not exactly the case of silent hosts, but let’s dig deeper into what could happen when you do have silent hosts attached to an EVPN fabric.

Silent Hosts in EVPN Fabrics

The Dynamic MAC Learning versus EVPN blog post triggered tons of interesting responses describing edge cases and vendor bugs implementation details, including an age-old case of silent hosts described by Nitzan:

Few years ago in EVPN network, I saw drops on the multicast queue (ingress replication goes to that queue). After analyzing it we found that the root cause is vMotion (the hosts in that VLAN are silent) which starts at a very high rate before the source leaf learns the destination MAC.

It turns out that the behavior they experienced was caused by a particularly slow EVPN implementation, so it’s not exactly the case of silent hosts, but let’s dig deeper into what could happen when you do have silent hosts attached to an EVPN fabric.

Small Site EBGP-Only Design

One of my subscribers found an unusual BGP specimen in the wild:

• It was a small site with two core switches and a WAN edge router
• The site had VPN concentrators running in virtual machines
• The WAN edge router was running BGP across WAN IPsec tunnels
• The VPN concentrators were running BGP with core switches.

So far so good, and kudos to whoever realized BGP is the only sane protocol to run between virtual machines and network core. However, the routing in the network core was implemented with EBGP sessions between the three core devices, and my subscriber thought the correct way to do it would be to use IBGP and OSPF.

Small Site EBGP-Only Design

One of my subscribers found an unusual BGP specimen in the wild:

• It was a small site with two core switches and a WAN edge router
• The site had VPN concentrators running in virtual machines
• The WAN edge router was running BGP across WAN IPsec tunnels
• The VPN concentrators were running BGP with core switches.

So far so good, and kudos to whoever realized BGP is the only sane protocol to run between virtual machines and network core. However, the routing in the network core was implemented with EBGP sessions between the three core devices, and my subscriber thought the correct way to do it would be to use IBGP and OSPF.

netlab Release 1.5.2: Aruba CX, External Tools, Tunnel Interfaces

netlab release 1.5.2 brings another bunch of cool features, including:

• Aruba AOS-CX Support by Stefano Sasso
• External network management tools that you can start together with your lab
• Tunnel interfaces
• Reusable topology components

I’ll cover these features in separate blog posts; today I wanted to highlight a few minor additions:

netlab Release 1.5.2: Aruba CX, External Tools, Tunnel Interfaces

netlab release 1.5.2 brings another bunch of cool features, including:

• Aruba AOS-CX Support by Stefano Sasso
• External network management tools that you can start together with your lab
• Tunnel interfaces
• Reusable topology components

I’ll cover these features in separate blog posts; today I wanted to highlight a few minor additions:

Interesting: BGP Zombie Outbreak on Juniper Routers

BGP zombies are routes in the BGP table that refuse to disappear even though they should have been long gone. Recent measurements estimate between 0.5% and 1.5% of all routes in the global BGP table are zombies, which sounds crazy – after all, BGP is supposed to be pretty reliable.

Daryll Swer identified one potential source – Juniper routers do not revoke suppressed aggregated prefixes – and documented it in Navigating a BGP zombie outbreak on Juniper routers.

Interesting: BGP Zombie Outbreak on Juniper Routers

BGP zombies are routes in the BGP table that refuse to disappear even though they should have been long gone. Recent measurements estimate between 0.5% and 1.5% of all routes in the global BGP table are zombies, which sounds crazy – after all, BGP is supposed to be pretty reliable.

Daryll Swer identified one potential source – Juniper routers do not revoke suppressed aggregated prefixes – and documented it in Navigating a BGP zombie outbreak on Juniper routers.

CloudFlare: From IP packets to HTTP

Want to know some details behind the CloudFlare SD-WAN implementation? You might find them in From IP packets to HTTP: the many faces of our Oxy framework.

I don’t know enough about Linux networking to figure out whether one could use those details to build something similar, but CloudFlare blog posts keep begin much better than Google’s Look How Awesome We Are recruitment drives.

CloudFlare: From IP packets to HTTP

Want to know some details behind the CloudFlare SD-WAN implementation? You might find them in From IP packets to HTTP: the many faces of our Oxy framework.

I don’t know enough about Linux networking to figure out whether one could use those details to build something similar, but CloudFlare blog posts keep begin much better than Google’s Look How Awesome We Are recruitment drives.

Advantages and Drawbacks of EVPN-based Multihoming

Lukas Krattiger wrapped up his EVPN-versus-MLAG presentation (part of EVPN Deep Dive webinar) with an overview of the advantages and drawbacks of EVPN-based multihoming solutions:

• N-way multihoming
• Flexible connectivity (no need for a peer link)
• Fabric-wide scope (MAC multipathing required on ingress node)

Advantages and Drawbacks of EVPN-based Multihoming

Lukas Krattiger wrapped up his EVPN-versus-MLAG presentation (part of EVPN Deep Dive webinar) with an overview of the advantages and drawbacks of EVPN-based multihoming solutions:

• N-way multihoming
• Flexible connectivity (no need for a peer link)
• Fabric-wide scope (MAC multipathing required on ingress node)

Why Is OSPF (and BGP) More Complex than STP?

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough"):

Why Is OSPF (and BGP) More Complex than STP?

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough”):

Is Dynamic MAC Learning Better Than EVPN?

One of my readers worried about the control-plane-induced MAC learning lag in EVPN-based networks:

In all discussions about the advantages/disadvantages of VXLAN/EVPN, I can’t find any regarding the lag in learning new macs when you use the control plane for mac learning.

EVPN is definitely slower than data plane-based dynamic MAC learning (regardless of whether it’s done in hardware or software), but so is MLAG.