Working with Filters in Ansible

This post is part of an ongoing series - see all the posts below.

Ansible Course

We've already used a filter a few times in this series without spending much time on it. In the loops part, we used dict2items to turn a dictionary into a list, so we could loop over it. We also ran into a couple of problems that we couldn't solve at the time, such as sharing the same list of syslog servers between Cisco and Arista, even though their modules expect the data in a different format.

Filters are what solve these kinds of problems. In this part, we'll look at what filters are, go through some of the most useful ones, and see how to chain them together. By the end of it, we'll come back to the syslog problem and fix it using filters.

Before we start, I rebuilt the lab once again, so all the config from the previous part is gone. The project files and inventory are the same as before.

If you're reading this part without reading the previous ones, here's a quick summary of the lab. We have five switches. core1, core2, and access1 are Cisco IOL Continue reading

Ansible – Adding Arista and Multi-Vendor Playbooks

This post is part of an ongoing series - see all the posts below.

Ansible Course

So far, every device in our lab has been a Cisco switch. But very few real networks are made up of a single vendor. You might have Cisco in the campus, Arista in the data centre, and a few firewalls from yet another vendor. One of the biggest benefits of Ansible is that you can manage all of them from the same project, using the same inventory, the same playbooks, and the same way of working.

In this part, we'll add Arista switches to our lab. Arista is a good first step into multi-vendor automation, because just like Cisco, Ansible connects to it over SSH using network_cli. So most of what we've learned so far applies straight away, and we can focus on the differences.

Lab Topology

For this part, I've rebuilt the lab from scratch, so all the config we added in the previous chapters, such as NTP, syslog, and interface descriptions, is gone. Every device starts with only the default config created by Netlab.

As always, your lab doesn’t need to match mine exactly. All you need is a mix of Cisco Continue reading

Palo Alto NAT Example

Most of the time, you'll be working with two types of NAT - Source NAT and Destination NAT. Source NAT is when you change the source IP address of your traffic (usually a private IP) to a public IP that is routable on the Internet. For example, if you have hundreds of people in your office, they are assigned private IPs. When their traffic goes out to the Internet, the Palo Alto firewall translates their private IP (source) to a public IP and the destination IP remains unchanged.

With Destination NAT, the process works the other way around. If you host publicly available resources in your network, like a web server, and someone on the Internet wants to access it, they will target a public IP address that is configured on the firewall. The firewall receives this request and then changes the destination IP to the server's real private IP and the source IP remains unchanged.

We'll also cover Bi-Directional NAT and U-Turn NAT. So, let's get started.

Ansible Loops, Conditionals, and Handlers

So far, our playbooks have been fairly simple. Each task runs once, on every device, every time we run the playbook. That's fine for basic tasks, but as your playbooks grow, you'll want more control over how and when tasks run.

In this post, we'll look at three features that give you that control. Loops let you run the same task multiple times, once for each item in a list. Conditionals let you run a task only when a certain condition is true, such as when a variable is defined or a device is running a specific software version. And handlers let you run a task only when another task has made a change, which is perfect for things like saving the config.

We've used VLANs a lot in the previous posts, so for this post, we'll move on to something different and configure NTP and syslog instead. These are things almost every network needs, and they're a good fit for showing how loops, conditionals, and handlers work.

Project Structure

Before we start, let's take a quick look at where our project is at. We've added and changed quite a few files over the last few posts, so it's easy Continue reading

Ansible Ad-hoc Commands, Facts, and Registering Output

So far, most of what we've done has been about making changes, such as creating VLANs and saving the config. But a lot of day-to-day network work can be also about getting information from devices such as checking software versions, finding serial numbers, looking at interface status, or just running a quick show command across a group of switches.

In this post, we'll look at three ways to do that. First, we'll cover ad-hoc commands, which let you run a single task against your devices straight from the command line, without writing a playbook. Then, we'll look at facts, which is how Ansible collects information about a device, such as its hostname, model, and software version. You may have noticed that we've been setting gather_facts: false in every playbook so far, and we'll finally see why.

Finally, we'll go back to register and debug, which we briefly used in our very first playbook, and look at how to save the output of a task and use it in a more useful way.

Ad-hoc Commands

So far, every time we wanted Ansible to do something, we wrote a playbook. But sometimes you just want to run a quick task, like Continue reading

Ansible Vault

Up until now, we've been storing the username and password for our switches in plain text. I mentioned a couple of times that this isn't a good idea, and now it's time to fix it.

Storing passwords in plain text might be fine in a lab, but in a real network, it's a big problem. Anyone who has access to your project directory can open the file and see the credentials for every device. And once you start storing your playbooks in Git, which you should, those passwords end up in your repository history as well, where they can stay long after you've removed them from the file.

Vault is a feature built into Ansible that lets you encrypt sensitive data, such as passwords, so it can't be read without a vault password. The best part is that Ansible decrypts it automatically when you run a playbook, so your playbooks don't need to change at all.

In this chapter, we'll encrypt the credentials for our switches, look at how to run playbooks with encrypted files, and cover a few different ways to work with Vault day to day.

Encrypting a File

We're continuing from where we left off in the Continue reading

Working with Ansible Inventory and Variables

In the previous chapter, we created inventory groups and moved the connection settings into the cisco group, so we only had to define them once. That works fine for now, but as your network grows, the inventory file can quickly get long and hard to manage, especially once you start adding more variables for each group and device.

We also have the VLANs hardcoded in the playbook. Right now, every switch gets the same four VLANs. But in a real network, you'll probably want different VLANs on the core switches and the access switches, and maybe even a specific VLAN on just one switch. Creating a separate playbook for each case isn't a good idea.

In this chapter, we'll look at how to organise the inventory and variables properly using group_vars and host_vars. We'll move the VLANs out of the playbook and into variables, so the same playbook can create different VLANs on different switches. We'll also look at how Ansible decides which variable to use when the same variable is defined in more than one place, and a few useful ways to target devices when running a playbook.

Moving Variables Out of the Inventory

At the moment, our connection Continue reading

Pushing Configuration with Ansible

So far, we've only used Ansible to run show commands, which is a great way to get started because there's no risk of breaking anything. But at some point, you'll want to use Ansible to actually make changes, and that's what this chapter is all about.

We'll start small by creating a single VLAN on a single switch. From there, we'll create multiple VLANs, then run the same playbook against more devices, and slowly build on it until we're managing different VLANs on different switches. Each step introduces something new, so by the end of the chapter, you'll have covered a lot of the core Ansible concepts without even noticing.

Since we're working with VLANs, you'll need switches in your lab. If you're using Cisco IOS like me, make sure you're using the Layer 2 image rather than the router image. To keep things simple, I'll start with a single switch called s1, and we'll add more devices later in the chapter.

As always, please make sure you're running this in a lab and not on production devices. We're about to start making changes, so it's a good time to be extra careful.

Lab Setup

Since we're working with Continue reading

Understanding YAML and Ansible Playbooks

YAML is a plain text format used to store and organize data. It's designed to be easy for humans to read and write, which is one of the main reasons Ansible uses it. The name originally stood for "Yet Another Markup Language", but it was later changed to "YAML Ain't Markup Language" to make it clear that it's meant for data, not documents.

In Ansible, almost everything you write is in YAML, including playbooks, inventory files, and variables. You've already seen two YAML files in the previous chapter, the inventory and the playbook.

If you've used Containerlab or Netlab, you've already worked with YAML, since their topology files are written in YAML too. It's also used by many other tools such as Docker Compose and Kubernetes, so learning it now will help you well beyond Ansible.

Here's a simple example of what YAML looks like.

hostname: r1
mgmt_ip: 192.168.202.101
ntp_servers:
  - 10.10.10.10
  - 10.10.10.11

Even if you've never seen YAML before, you can probably tell what this means. We have a device called r1, its management IP, and a list of two NTP servers.

YAML is simple, but it's also Continue reading

Why Network Automation and Why Ansible?

Before we install anything or write our first playbook, I want to spend a bit of time on why network automation matters and why Ansible is a good place to start. If you already know why you're here, feel free to skim this chapter, but I think it's worth understanding the problem before jumping into the tool.

The Way We've Always Done It

For most of us, managing a network looks something like this. You open a terminal, SSH into a device, run a few show commands, paste in some config from a text file, verify it, save it, and move on to the next device. Repeat that for as many devices as you need to change.

There's nothing wrong with this approach when you're dealing with a handful of devices. I've done it for years, and so have most network engineers. The problems start when the network grows.

Imagine you need to change the NTP servers on 200 switches or configure Radius authentication on hundreds of devices. Doing that manually means logging into each switch, removing the old servers, adding the new ones, and saving the config. Even if each device only takes two minutes, that's almost seven hours Continue reading

Python Generating Configs with Jinja2

If you have ever rolled out a batch of new sites or devices, I hope you can relate to the pain. The config is more or less the same each time, but the hostnames, IPs, and VLANs change. You copy the config from the last site, tweak the values, save it, and move on. It works, but it does not scale, and it is easy to make mistakes.

The way most of us solve this is with templates. You write the config once with placeholders where the values go, and you fill in the placeholders each time you need a new device config. Jinja2 is the tool we use for this in Python. It takes a template, some data, and gives you back a finished config.

In this post, I will walk you through how Jinja2 works, cover the syntax you actually need, and we will end up generating the same config for Cisco IOS, Juniper Junos, and Arista EOS from the same data. We are only going to generate the configs as text in this post. Pushing them to devices is something you already know how to do from the earlier posts.

Installing Jinja2

Jinja2 is not built into Continue reading

Python Working with REST APIs

Almost every modern piece of network gear has some form of REST API these days. Cisco DNAC, Meraki, Palo Alto Panorama, Juniper Mist, Arista CloudVision, NetBox, and countless others. If you want to automate anything beyond SSH, you will end up talking to an API sooner or later. In this part, we will look at how to do that with Python.

We will use the requests library, which is the most popular HTTP library in the Python world. It is not built into Python, but it is so widely used that most Python developers install it by default. requests handles all the messy details of HTTP for you, headers, cookies, authentication, encoding, so you can focus on the actual work.

To keep things easy at the start, we will use a service called https://mockhttp.org/, which is a testing sandbox for HTTP requests. It has endpoints that do exactly what their name suggests, /get returns a GET response, /post returns a POST response, /status/404 returns a 404, and so on. The responses are simple JSON that echo back the details of the request you sent, which is really helpful when you are learning what is happening under the hood.

Continue reading

Managing Python Projects with uv

If you have installed Python and worked on a few projects, your usual workflow probably looks something like this. Install Python, make sure pip and venv are available, create a virtual environment for each project, and use pip to install packages inside it. That is how we have been doing things throughout this series, and it works. It is what most of the Python world has been doing for years.

uv is a relatively new tool that bundles all of this into one command. It manages Python version installs, virtual environments, package installs, and dependency locking, all in a single binary. It is written in Rust, which makes it fast (like really fast). Most operations that take pip several seconds finish in milliseconds with uv.

In this part, we will start from a completely fresh Ubuntu machine (24.04) that only has the system Python installed. As you will see in a moment, we do not have pip or venv available yet, and normally we would need to install them separately before we could do anything.

A Fresh Ubuntu VM

Here is what is happening on the fresh Ubuntu VM. python --version fails because Ubuntu does not ship the python Continue reading

Python Regular Expressions

You have already used regular expressions once in this series when we told deepdiff to ignore uptime fields. I promised we would cover them properly later, and now is the time.

Regular expressions (regex for short) are a way to describe patterns in text. Instead of asking "does this string contain the exact word up?" you can ask "does this string contain any four-digit number followed by a slash and another number?" That kind of question is hard to answer with the string methods we have used so far, but regex handles it easily.

For a network engineer, regex comes up in a bunch of places. Extracting an IP address from a config line. Pulling interface names out of a log message. Parsing show output when there is no ready-made TextFSM template. Validating that a value looks like a MAC address before you use it.

Regex has a reputation for being cryptic, and honestly, it is cryptic. A complex regex pattern can look like someone leaned on the keyboard. But you do not need to master all of it to be useful. Learning even the basics unlocks a lot of what you will actually need day Continue reading

Strata Cloud Manager – Worth the Hype?

If you use Palo Alto firewalls, typically you manage them in one of two ways. You either log into each firewall directly and manage it on its own, or you use Panorama, which lets you manage multiple firewalls from a single place.

Most of us were used to this setup, myself included. If you had a handful of firewalls, logging into each one individually is okay. But once you started scaling up to tens or hundreds of firewalls across different sites, Panorama became the go-to option. It let you centrally manage the configurations and monitor logs centrally instead of touching every single firewall by hand.

Strata Cloud Manager (SCM)

However, Palo Alto released yet another option called Strata Cloud Manager (SCM) in 2023, which is hosted by Palo Alto themselves, unlike Panorama, which you manage internally. The idea at a very high level is that the firewalls connect to Strata Cloud Manager (outbound only) through the Internet, and you log into SCM to manage the configurations and push them to the firewalls.

From my experience and speaking to other people, Palo Alto is pushing SCM to customers and adding new features to it. So the unspoken assumption is that any Continue reading

Automating Let’s Encrypt Certificate Renewal for Palo Alto GlobalProtect

If you manage GlobalProtect VPN certificates on Palo Alto and use Let's Encrypt, at some point you will start thinking about automation. Let's Encrypt certificates have a short lifespan, so renewing them manually every few months isn't really practical. In this post, we will look at automating this process using Ansible, Cloudflare DNS, and the PAN-OS Ansible collection to renew certificates and push them to the firewall.

Prerequisite

Let's say your domain is mydomain.com and it's managed through Cloudflare. You want to generate a certificate for vpn.mydomain.com and have it automatically installed on your Palo Alto firewall as the GlobalProtect portal or gateway certificate, without logging in manually every time it's close to expiry. To do this, you will need:

• Python/Ansible installed
• A domain managed through Cloudflare, so certbot can complete the DNS-01 challenge. This doesn't have to be Cloudflare specifically, certbot supports several other DNS providers as well, but this post assumes Cloudflare since that's what we're using here.
• A Cloudflare API token with permission to edit DNS records for that zone
• Certbot installed along with the Cloudflare DNS plugin
• Ansible with the paloaltonetworks.panos collection installed
• An API key from your Palo Alto firewall so Continue reading

Testing Your Network with Netpicker

In my previous two posts, we covered what Pytest is, what it is traditionally used for, and how we can use it to test your network, checking things like BGP peer status, interface states, and so on. So far, we have looked at how to run these tests using the Pytest CLI command in the terminal.

In this post, we will look at Netpicker, a tool that takes the same Pytest tests and gives you a GUI to run and manage them, instead of relying on the command line.

What is Netpicker?

Netpicker is a tool that brings together config backups, security/compliance checks, and automation in one place. It supports 180+ network vendors, including Cisco, Juniper, Arista, Palo Alto, and Fortinet, and integrates well with tools like NetBox, Nautobot, and Infrahub for inventory, and Slurp'it for network discovery.

Under the hood, Netpicker uses pytest for its testing engine, but wraps it in a web GUI, so instead of running tests from the terminal, you run and manage them from a browser. This means you can run the same kind of pytest tests we covered in the previous two posts, things like checking BGP peer status or interface states, but Continue reading

Minisforum MS-A2 Review for Network Labs/Homelab

In my previous blog post, I covered setting up network labs on a budget using used mini PCs, and I saw good interest in that post. So, I thought I would write another one sharing my experience using the Minisforum MS-A2 for network labs. As of writing this post, the MS-A2 costs £1160 with an AMD Ryzen 9 9955HX, 32GB of DDR5 RAM, and 1TB of storage. You can buy the barebone version (without RAM and storage) for £727 on Amazon. I bought the 32GB version + 1TB of RAM.

I would have bought the 64GB RAM version (I'd recommend 64GB for anyone buying) if that was available on Amazon, but that option was only available on the Minisforum website, and it ships from their HK warehouse and takes a few days. So I went with Amazon, which came with a single 32GB stick, and then added another 32GB stick myself (RAM costs a fortune as of writing this post 😢)

• CPU options - AMD Ryzen 9 9955HX / 9 7945HX / 7 7745HX. I went with the 9955HX (16c/32t) for extra headroom running multiple labs at once.
• RAM - Dual channel DDR5-5600MHz, up to 96GB. Plenty for running Continue reading

UGREEN NASync DXP4800 GT Review and Initial Setup

Last year I bought a UGreen NASync DXP2800 for my homelab, along with two Seagate IronWolf 8TB HDDs. This was my first NAS, and I've been using it for just over a year now, and I'm very happy with it. I back up two of my Macs using Time Machine, store a lot of media, documents, photos, videos, etc, pretty much what a NAS is designed to do. I also run some Docker containers and Proxmox Backup Server as a VM. Even though it has an N100 CPU, so far I've never had a single issue in terms of resources. For £249, it was a bargain.

Fast forward to today, and UGreen released the NASync DXP4800 GT, a 4-bay NAS that comes with an AMD Ryzen Embedded R2514, dual 10GbE LAN, and dual M.2 NVMe SSD slots for £529 at the time of writing here in the UK. It looked like a great addition to my homelab setup, giving me a lot more power and flexibility compared to my current NAS. I was lucky enough to get my hands on a unit from UGreen to try out, so let's get into it.

Panorama Device Groups and Templates (with examples)

Overview

People who follow my blog may probably know that I'm a big fan of Cisco ASA firewalls and I worked quite extensively with them. When I was first introduced to the Palo Alto firewalls, I was amazed at how easy it is to use the web GUI compare to the ASDM which I absolutely hate.

In my early Palo Alto days, I used to work with just a single HA firewall pair where all the management and configuration were done directly on the firewall - similar to how you configure an ASA via ASDM or CLI.

When I first started working with Panorama, I was struggling to understand device group and template concepts. There were nested device groups, device group hierarchy, templates, templates stacks etc.

After reading various documents and speaking to my colleagues, I managed to get a grasp of the core concepts. At last, I've decided to write a blog post on the concepts I learnt to help fellow engineers. The purpose of this blog post is to simplify the concepts and explain them in plain English.