Path Failure Detection on Multi-Homed Servers

TL&DR: Installing an Ethernet NIC with two uplinks in a server is easy¹. Connecting those uplinks to two edge switches is common sense². Detecting physical link failure is trivial in Gigabit Ethernet world. Deciding between two independent uplinks or a link aggregation group is interesting. Detecting path failure and disabling the useless uplink that causes traffic blackholing is a living hell (more details in this Design Clinic question).

Want to know more? Let’s dive into the gory details.

Goodbye Twitter. It Was Fun While It Lasted

I joined Twitter in October 2008 (after noticing everyone else was using it during a Networking Field Day event), and eventually figured out how to automate posting the links to my blog posts in case someone uses Twitter as their primary source of news.

This week, I got a nice email from IFTTT (the solution I used) telling me they had to disable the post-to-Twitter applet. Twitter started charging for the API, and I was using their free service – obviously the math didn’t work out.

That left me with three options:

Worth Reading: Cargo Cult AI

Before we managed to recover from the automation cargo cults, a tsunami wave of cargo cult AI washed over us as Edlyn V. Levine explained in an ACM Queue article. Enjoy ;)

Also, a bit of a historical perspective is never a bad thing:

Impressive progress in AI, including the recent sensation of ChatGPT, has been dominated by the success of a single, decades-old machine-learning approach called a multilayer (or deep) neural network. This approach was invented in the 1940s, and essentially all of the foundational concepts of neural networks and associated methods—including convolutional neural networks and backpropagation—were in place by the 1980s.

Worth Reading: Building Trustworthy AI

Bruce Schneier wrote an excellent essay explaining why we need trustworthy AI and why we won’t get it as long the AI solutions are created by large tech companies with you are a product business model.

Network Security Vulnerabilities: the Root Causes

Sometime last autumn, I was asked to create a short “network security challenges” presentation. Eventually, I turned it into a webinar, resulting in almost four hours of content describing the interesting gotchas I encountered in the past (plus a few recent vulnerabilities like turning WiFi into a thick yellow cable).

Each webinar section started with a short “This is why we have to deal with these stupidities” introduction. You’ll find all of them collected in the Root Causes video starting the Network Security Fallacies part of the How Networks Really Work webinar.

Inter-VRF DHCP Relaying with Redundant DHCP Servers

Previous posts in this series covered numerous intricacies of DHCP relaying:

• DHCP relaying principles described the basics
• In Inter-VRFs relaying we figured out how a DHCP client reaches a DHCP server in another VRF without inter-VRF route leaking
• Relaying in VXLAN segments and relaying from EVPN VRF applied those lessons to VXLAN/EVPN environment.
• DHCP Relaying with Redundant DHCP Servers added relay- and server redundancy.

Now for the final bit of the puzzle: what if we want to do inter-VRF DHCP relaying with redundant DHCP servers?

Dealing with Cisco ACI Quirks

Sebastian described an interesting Cisco ACI quirk they had the privilege of chasing around:

We’ve encountered VM connectivity issues after VM movements from one vPC leaf pair to a different vPC leaf pair with ACI. The issue did not occur immediately (due to ACI’s bounce entries) and only sometimes, which made it very difficult to reproduce synthetically, but due to DRS and a large number of VMs it occurred frequently enough, that it was a serious problem for us.

Here’s what they figured out:

Simplify netlab Topologies with Link Groups

Last month I described how you can simplify your VLAN- or VRF lab topologies with VRF- and VLAN links, automatically setting vlan.access or vrf attribute on a set of links. Link groups allow you to do the same for any set of link attributes.

Sample Topology

Imagine you have a small network with three PE-routers connected to a central P-router:

Worth Reading: Trapped by Technology Fallacies

Michele Chubirka published a must-read article on technology fallacies including this gem:

Technologists often assume that all problems can be beaten into submission with a technology hammer.

As I’ve been saying for ages (not that anyone would listen): all the technology in the world won’t save you unless you change the mentality and rearchitect broken processes.

Why Is Source Address Validation Still a Problem?

I mentioned IP source address validation (SAV) as one of the MANRS-recommended actions in the Internet Routing Security webinar but did not go into any details (as the webinar deals with routing security, not data-plane security)… but I stumbled upon a wonderful companion article published by RIPE Labs: Why Is Source Address Validation Still a Problem?.

The article goes through the basics of SAV, best practices, and (most interesting) using free testing tools to detect non-compliant networks. Definitely worth reading!

Video: Chassis Switch Architectures

Pete Lumbis concluded his ASICs for Networking Engineers presentation with a brief overview of types of switching ASICs and a wrap-up.

You can watch his entire 90-minute presentation (sliced into shorter videos) with Free ipSpace.net Subscription.

Find the Optimal Level of Automation Abstraction

Tom Ammon sent me his thoughts on choosing the right level of abstraction in your network automation solution as a response to my What Is Intent-Based Networking blog post, and allowed me to publish them on ipspace.net.

I totally agree with your what vs how example with OSPF. I work on a NOS team where if we wanted, we could say, instead of “run OSPF on these links”, do this:

New: Disaster Recovery Resources

I wrote dozens of blog posts debunking disaster recovery fairy tales (mostly of the long-distance vMotion and stretched clusters variety) over the years. They are collected and sorted (and polished a bit) in the new Disaster Recovery Resources page. Hope you’ll find them useful.

ITNOG 7 Wrap-up

I attended ITNOG 7 last week, and thoroughly enjoyed a full day of interesting presentations, including how do you run Internet services in a war zone by Elena Lutsenko and Milko Ilari.

The morning was focused primarily on BGP:

netlab Release 1.5.3: libvirt Public Networks

containerlab release 0.41.0 that came out a few days ago changed a few topology attributes with no backward compatibility, breaking netlab for anyone doing a new installation. The only way out of that conundrum was to push out a new netlab release that uses the new attributes and requires containerlab release 0.41.0 (more about that in a minute).

On a more positive note, netlab release 1.5.3 brings a few interesting features, including:

• Support for public libvirt networks that can be used to connect your labs to the outside world, and reuse of existing libvirt networks
• ‘unknown’ device type that can be used to deploy devices not yet supported by netlab
• MPLS/VPN support on Nokia SR-OS

Worth Reading: Official Ansible Collection for SR Linux

Roman Dodin wrote an article describing Nokia’s Ansible collection for SR Linux. Although I don’t use SR Linux (even though it was the first container supported by netlab ;), it was still very interesting to read about the design tradeoffs they had to make:

Service Insertion with BGP FlowSpec

Nicola Modena had an interesting presentation describing how you can use BGP FlowSpec for traffic steering and service insertion during the recent ITNOG 7 event (more about the event in a few days).

One of the slides explained how to use three different aspects of BGP (FlowSpec, MPLS/VPN and multipathing), prompting me to claim the presentation title should be “BGP is the answer, what was the question?” 😉 Hope you’ll enjoy the PDF version of the presentation as much as we did the live one.

Video: Sample Kubernetes SDN Implementations

Read for more Kubernetes details? How about Container Networking Interface (CNI) described by Stuart Charlton as part of Kubernetes Networking Deep Dive webinar?

Notes:

• You REALLY SHOULD watch Kubernetes SDN architecture and Sample Kubernetes SDN Implementations videos first
• The video (and a large portion of Kubernetes Networking Deep Dive webinar) is available with Free ipSpace.net Subscription.

MLAG Clusters without a Physical Peer Link

With the widespread deployment of Ethernet-over-something technologies, it became possible to build MLAG clusters without a physical peer link, replacing it with a virtual link across the core fabric. Avaya was one of the first vendors to implement virtual peer links with Provider Backbone Bridging (PBB) transport, and some data center switching vendors (example: Cisco) offer similar functionality with VXLAN transport.

Is ChatGPT an Efficiency Multiplier?

I got this comment on one of my ChatGPT-related posts:

It does save time for things like converting output to YAML (I do not feed it proprietary information), or have it write scripts in various languages, converting configs from one vendor to another, although often they are not complete or correct they save time so regardless of what we think of it, it is an efficiency multiplier.

I received similar feedback several times, but found that the real answer (as is too often the case) is It Depends.