Archive

Category Archives for "ipSpace.net"

Private VLANs with VXLAN

Got this remark from a reader after he read the VXLAN and Q-in-Q blog post:

Another area where there is a feature gap with EVPN VXLAN is Private VLANs with VXLAN. They’re not supported on either Nexus or Juniper switches.

I have one word on using private VLANs in 2019: Don’t. They are messy and hard to maintain (not to mention it gets really interesting when you’re combining virtual and physical switches).

Read more ...

Cross-Data-Center L4-7 Services with Cisco ACI

Craig Weinhold sent me his thoughts on using Cisco ACI to implement cross-data-center L4-7 services. While we both believe this is not the way to do things (because you should start with proper application architecture), you might find his insights useful if you have to deal with legacy environments that believe in Santa Claus and solving application problems with networking infrastructure.


An “easy button” for multi-DC is like the quest for the holy grail. I explain to my clients that the answer is right in front of them – local IP addressing, L3 routing, and DNS. But they refuse to accept that, draw their swords, and engage in a fruitless war against common sense. Asymmetry, stateful inspection, ingress routing, split-brain, quorums, host mobility, cache coherency, non-RFC complaint ARP, etc.  

Read more ...

Last Week on ipSpace.net (2019W7)

Last Tuesday we continued the deep dive into new Ansible networking modules functionality introduced in recent software releases (up to 2.7), including a demonstration of a few simple playbooks that collect printouts from network devices and check software version or end-to-end connectivity.

In the second half of the live session we started digging into the intricacies of device configuration management, ending with the truly “fun part”: changing access control lists on Cisco IOS.

The Ansible for Networking Engineers webinar is part of standard ipSpace.net subscription and Building Network Automation Solutions online course.

Worth Reading: Blockchain and Trust

One of the rules of sane social media presence should be don’t ever engage with evangelists believing in a particular technology religion, more so if their funding depends on them spreading the gospel. I was called old-school networking guru from ivory tower when pointing out the drawbacks of TRILL, and clueless incompetent (in more polite words) when retweeting a tweet pointing out the realities of carbon footprint of proof-of-work technologies.

Interestingly, just a few days after that Bruce Schneier published a lengthy essay on blockchain and trust, and even the evangelists find it a bit hard to call him incompetent on security topics. Please read what he wrote every time someone comes along explaining how blockchains will save the world (or solve whatever networking problems like VTEP-to-MAC mappings).

Loop Avoidance in VXLAN Networks

Antonio Boj sent me this interesting challenge:

Is there any way to avoid, prevent or at least mitigate bridging loops when using VXLAN with EVPN? Spanning-tree is not supported when using VXLAN encapsulation so I was hoping to use EVPN duplicate MAC detection.

MAC move dampening (or anything similar) doesn’t help if you have a forwarding loop. You might be able to use it to identify there’s a loop, but that’s it… and while you’re doing that your network is melting down.

Read more ...

Video: Automating Simple Reports

Network automation is scary when you start using it in a brownfield environment. After all, it’s pretty easy to propagate an error to all devices in your network. However, there’s one thing you can do that’s usually pretty harmless: collect data from network devices and create summary reports or graphs.

I collected several interesting solutions created by attendees of our Building Network Automation Solutions online course and described them in a short video.

Want to create something similar? No time to procrastinate – the registration for the Spring 2019 course ends tomorrow.

Operating Cisco ACI the Right Way

This is a guest blog post by Andrea Dainese, senior network and security architect, and author of UNetLab (now EVE-NG) and  Route Reflector Labs. These days you’ll find him busy automating Cisco ACI deployments.


In this post we’ll focus on a simple question that arises in numerous chats I have with colleagues and customers: how should a network engineer operate Cisco ACI? A lot of them don’t use any sort of network automation and manage their Cisco ACI deployments using the Web Interface. Is that good or evil? As you’ll see we have a definite answer and it’s not “it depends”.

Read more ...

Last Week on ipSpace.net (2019W6)

Last week Howard Marks completed the Hyperconverged Infrastructure Deep Dive trilogy covering smaller HCI players (including Cisco’s Hyperflex) and explaining the intricacies of costing and licensing HCI solutions.

On Thursday I finally managed to start the long-overdue Data Center Interconnects update. The original webinar was recorded in 2011, and while the layer-3 technologies haven’t changed much (with LISP still being mostly a solution in search of a problem), most of the layer-2 technologies I described at that time vanished, with OTV being a notable exception. Keep that in mind the next time your favorite $vendor starts promoting another wonderful technology.

You can get access to both webinars with standard ipSpace.net subscription.

Tech Field Day Extra @ CLEUR19 Recap

I spent most of last week with a great team of fellow networking and security engineers in a windowless room listening to good, bad and plain boring presentations from (mostly) Cisco presenters describing new technologies and solutions – the yearly Tech Field Day Extra @ Cisco Live Europe event.

This year’s hit rate (the percentage of good presentations) was about 50% and these are the ones I found worth watching (in chronological order):

Read more ...

Worth Reading: Should I Write a Book?

Erik Dietrich (of the Expert Beginner fame) published another great blog post explaining when and why you should write a book. For the attention-challenged here’s my CliffNotes version:

  • Realize you have no idea what you’re doing (see also: Dunning-Kruger effect)
  • Figure out why you’d want to spend a significant amount of your time on a major project like book writing;
  • It will take longer (and will be more expensive) than you expect even when considering Hofstadter’s law.

SRv6: One Tool to Rule Them All

I got some interesting feedback from one of my readers on Segment Routing with IPv6 extension headers:

Some people position SRv6 as the universal underlay and overlay due to its capabilities for network programming by means of feature+locator SRH separation.

Stupid me replied “SRv6 is NOT an overlay solution but a source routing solution.

Read more ...

Not So Fast Ansible, Cisco IOS Can’t Keep Up…

Remember how earlier releases of Nexus-OS started dropping configuration commands if you were typing them too quickly (and how it was declared a feature ;)?

Mark Fergusson had a similar experience on Cisco IOS. All he wanted to do was to use Ansible to configure a VRF, an interface in the VRF, and OSPF routing process on Cisco CSR 1000v running software release 15.5(3).

Here’s what he was trying to deploy. Looks like a configuration straight out of an MPLS book, right?

Read more ...

Last Week on ipSpace.net (2019W4)

The crazy pace of webinar sessions continued last week. Howard Marks continued his deep dive into Hyper-Converged Infrastructure, this time focusing on go-to-market strategies, failure resiliency with replicas and local RAID, and the eternal debate (if you happen to be working for a certain $vendor) whether it’s better to run your HCI code in a VM and not in hypervisor kernel like your competitor does. He concluded with the description of what major players (VMware VSAN, Nutanix and HPE Simplivity) do.

On Thursday I started my Ansible 2.7 Updates saga, describing how network_cli plugin works, how they implemented generic CLI modules, how to use SSH keys or usernames and passwords for authentication (and how to make them secure), and how to execute commands on network devices (including an introduction into the gory details of parsing text outputs, JSON or XML).

The last thing I managed to cover was the cli_command module and how you can use it to execute any command on a network device… and then I ran out of time. We’ll continue with sample playbooks and network device configurations on February 12th.

You can get access to both webinars with Standard ipSpace.net subscription.

More on Leaky Abstractions

When I was writing the Back to Basics blog post I reread the Law of Leaky Abstractions masterpiece. You’ll love it – the first example Joel uses is TCP.

However, what really caught my eye was this bit:

The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying “learn how to do it manually first, then use the wizzy tool to save time.”

You should apply the same wisdom to shiny new gizmos launched by network virtualization vendors… oh wait, you can’t, they are mostly undocumented black boxes. Good luck ;)

Sadly, the Law of Leaky Abstractions blog post was written in 2002… and nothing changed in the meantime, at least not for the better.

Overview of Network Automation Mechanisms

I know many networking engineers who went into networking because they didn’t want to write code the rest of their lives. I also know a few awesome engineers who decided to keep coding while designing networks.

Andrea Dainese (author of UNetLab – the tool you might know as EVE-NG) is one of the latter and practiced network automation for years, dealing with all sorts of crappy device configuration and monitoring mechanisms, from screen- and web scraping to broken REST APIs.

He decided to write a series of articles describing individual mechanisms, starting with an overview and zero-touch provisioning.

Q-in-Q Support in Multi-Site EVPN

One of my subscribers sent me a question along these lines (heavily abridged):

My customer is running a colocation business, and has to provide L2 connectivity between racks, sometimes even across multiple data centers. They were using Q-in-Q to deliver that in a traditional fabric, and would like to replace that with multi-site EVPN fabric with ~100 ToR switches in each data center. However, Cisco doesn’t support Q-in-Q with multi-site EVPN. Any ideas?

As Lukas Krattiger explained in his part of Multi-Site Leaf-and-Spine Fabrics section of Leaf-and-Spine Fabric Architectures webinar, multi-site EVPN (VXLAN-to-VXLAN bridging) is hard. Don’t expect miracles like Q-in-Q over VNI any time soon ;)

Read more ...