Sample Solution: Automating L3VPN Deployments

A long while ago I published my solution for automated L3VPN provisioning… and I’m really glad I can point you to a much better one ;)

Håkon Rørvik Aune decided to tackle the same challenge as his hands-on assignment in the Building Network Automation Solutions course and created a nicely-structured and well-documented solution (after creating a playbook that creates network diagrams from OSPF neighbor information).

Want to be able to do something similar? You missed the Spring 2019 online course, but you can get the mentored self-paced version with Expert Subscription.

More Thoughts on Vendor Lock-In and Subscriptions

Albert Siersema sent me his thoughts on lock-in and the recent tendency to sell network device (or software) subscriptions instead of boxes. A few of my comments are inline.

Another trend in the industry is to convert support contracts into subscriptions. That is, the entrenched players seem to be focusing more on that business model (too). In the end, I feel the customer won't reap that many benefits, and you probably will end up paying more. But that's my old grumpy cynicism talking :)

While I agree with that, buying a subscription instead of owning a box (and deprecating it) also makes it easier to persuade the bean counters to switch the gear because there’s little residual value in existing boxes (and it’s easy to demonstrate total-cost-of-ownership). Like every decent sword this one has two blades ;)

Building the Network Automation Source of Truth

This is one of the “thinking out loud” blog posts as I’m preparing my presentation for the Building Network Automation Solutions online course. I’m probably missing a gazillion details - your feedback would be highly appreciated

One of the toughest challenges you’ll face when building a network automation solution is “where is my source of truth” (or: what data should I trust). As someone way smarter than me said once: “You could either have a single source of truth of many sources of lies”, and knowing how your devices should be configured and what mistakes have to be fixed becomes crucial as soon as you move from gathering data and creating reports to provisioning new devices or services.

Last Week on ipSpace.net (2019W8)

We started the Spring 2019 Building Network Automation Solutions course on Tuesday with building virtual labs presentation by one-and-only Matt Oswalt of the NRE Labs fame, and finished the AWS Networking Deep Dive saga on Thursday with an overview of AWS load balancing mechanisms, from elastic load balancing (CLB/NLB/ALB) to DNS-based load balancing, CloudFront and Global Accelerator… and figured out how Amazon reinvented VRFs and hub-and-spoke VPNs with Transit gateways.

The AWS Networking Deep Dive webinar is part of standard ipSpace.net subscription You can access Matt’s presentation and all other materials of the Building Network Automation Solutions online course with Expert Subscription (assuming you choose this course as part of your subscription).

Worth Reading: I Used To Think…

Ethan Banks joined the grumpy old networking engineer club - a must-read collection of lessons-learned and disappointments he encountered in his career (and I love to see how someone else says what I always wanted to say way more eloquently)

High-Speed IPsec on Snabb Switch on Software Gone Wild

In previous Software Gone Wild episodes we covered Snabb Switch and numerous applications running on it, from L2VPN to 4over6 gateway and integration with Juniper vMX code.

In Episode 98 we focused on another interesting application developed by Max Rottenkolber: high-speed VPN gateway using IPsec on top of Snabb Switch (details). Enjoy!

Private VLANs with VXLAN

Got this remark from a reader after he read the VXLAN and Q-in-Q blog post:

Another area where there is a feature gap with EVPN VXLAN is Private VLANs with VXLAN. They’re not supported on either Nexus or Juniper switches.

I have one word on using private VLANs in 2019: Don’t. They are messy and hard to maintain (not to mention it gets really interesting when you’re combining virtual and physical switches).

Worth Reading: MPLS and ExaBGP

Jon Langemak is on a writing spree: after completing his MPLS-on-Junos series he started a deep dive into ExaBGP. Well worth reading if you’re enjoying detailed technical blog posts.

Cross-Data-Center L4-7 Services with Cisco ACI

Craig Weinhold sent me his thoughts on using Cisco ACI to implement cross-data-center L4-7 services. While we both believe this is not the way to do things (because you should start with proper application architecture), you might find his insights useful if you have to deal with legacy environments that believe in Santa Claus and solving application problems with networking infrastructure.

An “easy button” for multi-DC is like the quest for the holy grail. I explain to my clients that the answer is right in front of them – local IP addressing, L3 routing, and DNS. But they refuse to accept that, draw their swords, and engage in a fruitless war against common sense. Asymmetry, stateful inspection, ingress routing, split-brain, quorums, host mobility, cache coherency, non-RFC complaint ARP, etc.  

Last Week on ipSpace.net (2019W7)

Last Tuesday we continued the deep dive into new Ansible networking modules functionality introduced in recent software releases (up to 2.7), including a demonstration of a few simple playbooks that collect printouts from network devices and check software version or end-to-end connectivity.

In the second half of the live session we started digging into the intricacies of device configuration management, ending with the truly “fun part”: changing access control lists on Cisco IOS.

The Ansible for Networking Engineers webinar is part of standard ipSpace.net subscription and Building Network Automation Solutions online course.

Worth Reading: Blockchain and Trust

One of the rules of sane social media presence should be don’t ever engage with evangelists believing in a particular technology religion, more so if their funding depends on them spreading the gospel. I was called old-school networking guru from ivory tower when pointing out the drawbacks of TRILL, and clueless incompetent (in more polite words) when retweeting a tweet pointing out the realities of carbon footprint of proof-of-work technologies.

Interestingly, just a few days after that Bruce Schneier published a lengthy essay on blockchain and trust, and even the evangelists find it a bit hard to call him incompetent on security topics. Please read what he wrote every time someone comes along explaining how blockchains will save the world (or solve whatever networking problems like VTEP-to-MAC mappings).

Loop Avoidance in VXLAN Networks

Antonio Boj sent me this interesting challenge:

Is there any way to avoid, prevent or at least mitigate bridging loops when using VXLAN with EVPN? Spanning-tree is not supported when using VXLAN encapsulation so I was hoping to use EVPN duplicate MAC detection.

MAC move dampening (or anything similar) doesn’t help if you have a forwarding loop. You might be able to use it to identify there’s a loop, but that’s it… and while you’re doing that your network is melting down.

Video: Automating Simple Reports

Network automation is scary when you start using it in a brownfield environment. After all, it’s pretty easy to propagate an error to all devices in your network. However, there’s one thing you can do that’s usually pretty harmless: collect data from network devices and create summary reports or graphs.

I collected several interesting solutions created by attendees of our Building Network Automation Solutions online course and described them in a short video.

Want to create something similar? No time to procrastinate – the registration for the Spring 2019 course ends tomorrow.

Operating Cisco ACI the Right Way

This is a guest blog post by Andrea Dainese, senior network and security architect, and author of UNetLab (now EVE-NG) and  Route Reflector Labs. These days you’ll find him busy automating Cisco ACI deployments.

In this post we’ll focus on a simple question that arises in numerous chats I have with colleagues and customers: how should a network engineer operate Cisco ACI? A lot of them don’t use any sort of network automation and manage their Cisco ACI deployments using the Web Interface. Is that good or evil? As you’ll see we have a definite answer and it’s not “it depends”.

Last Week on ipSpace.net (2019W6)

Last week Howard Marks completed the Hyperconverged Infrastructure Deep Dive trilogy covering smaller HCI players (including Cisco’s Hyperflex) and explaining the intricacies of costing and licensing HCI solutions.

On Thursday I finally managed to start the long-overdue Data Center Interconnects update. The original webinar was recorded in 2011, and while the layer-3 technologies haven’t changed much (with LISP still being mostly a solution in search of a problem), most of the layer-2 technologies I described at that time vanished, with OTV being a notable exception. Keep that in mind the next time your favorite $vendor starts promoting another wonderful technology.

You can get access to both webinars with standard ipSpace.net subscription.

SD-WAN Security Under the Hood

A while ago we published a guest blog post by Christoph Jaggi explaining the high-level security challenges of most SD-WAN solutions… but what about the low-level details?

Sergey Gordeychik dived deep into implementation details of SD-WAN security in his 35C3 talk (slides, video).

TL&DW: some of the SD-WAN boxes are as secure as $19.99 Chinese webcam you bought on eBay.

Automating Brownfield Device Configuration

Numerous network automation deployments happen in brownfield installations: you’re trying to automate parts of existing network deployment and operations processes. If you’re lucky you start automating deployment of new devices… but what if you have to automate parts of existing device configurations?

Tech Field Day Extra @ CLEUR19 Recap

I spent most of last week with a great team of fellow networking and security engineers in a windowless room listening to good, bad and plain boring presentations from (mostly) Cisco presenters describing new technologies and solutions – the yearly Tech Field Day Extra @ Cisco Live Europe event.

This year’s hit rate (the percentage of good presentations) was about 50% and these are the ones I found worth watching (in chronological order):

When You Have to Deal with **** at Work

Not that it helps that much but keep in mind: you're not the only one. Here's a wonderful blog post by Al Rasheed.

Oh, and if you still feel like a fraud after being in the industry for years (check out the Impostor Syndrome), you're not alone either.

Worth Reading: Should I Write a Book?

Erik Dietrich (of the Expert Beginner fame) published another great blog post explaining when and why you should write a book. For the attention-challenged here’s my CliffNotes version:

• Realize you have no idea what you’re doing (see also: Dunning-Kruger effect)
• Figure out why you’d want to spend a significant amount of your time on a major project like book writing;
• It will take longer (and will be more expensive) than you expect even when considering Hofstadter’s law.