Getting Structured Output from Network Devices with Ansible

This post is part of an ongoing series on Ansible, and you can find all the other posts in the series here.

In the previous parts of this series, we built an OSPF network with Ansible, and then wrote a few checks to make sure it's healthy. One of those checks counted the OSPF neighbors on each router, by running show ip ospf neighbor and counting the lines that contain the word FULL. It works, but it's a bit broken. We're treating the output as plain text and searching through it, so we're relying on the router always formatting the output in exactly the same way.

The output of a show command is designed for humans to read, not for machines. What we really want is structured data, such as a list of neighbors, where each neighbor has its own neighbor ID, state, and interface that we can work with directly, just like the facts and variables we've been using throughout the series.

In this part, we'll look at how to parse command output into structured data using the cli_parse module and a library of ready-made templates called ntc-templates. We'll parse show ip ospf neighbor, compare the plain Continue reading

Pre and Post Checks with Ansible

This post is part of an ongoing series - see all the posts below.

Ansible Course

In the previous part, we wrote two checks for our OSPF network. One makes sure every router has all of its OSPF neighbours, and the other makes sure every router can reach every other router's loopback. They're useful on their own, but they're even more useful when you run them around a change.

The idea is that before making a change, you check that the network is healthy. If it isn't, you stop, since there's no point making a change on a network that's already broken. It also means that if something goes wrong later, you know it wasn't already broken before you started. Then, after the change, you run the same checks again to make sure everything is still healthy.

Most of us already do this manually. Before a change, we log in and save the output of a few show commands, and afterwards, we run them again and compare. In this part, we'll let Ansible do it for us, by reusing the playbooks we've already written, rather than copying them into every change playbook.

We're continuing with the same lab and Continue reading

Validating Your Network with Ansible Assert

This post is part of an ongoing series - see all the posts below.

Ansible Course

In the previous part, we built an OSPF network from scratch using Ansible, and then logged in to the routers to check that everything was working. We looked at the OSPF neighbors, checked the routing table, and ran a few pings. That's fine for six routers. But imagine doing it for fifty, before and after every change, at 2 AM during a maintenance window. By router number twelve, you're not really reading the output anymore, you're just looking for the word FULL and hoping for the best.

In this part, we'll let Ansible do the checking for us. We'll write a few simple tests that check whether every router has the right number of OSPF neighbors, and whether every router can reach every other router. If something is wrong, the playbook fails and tells us exactly where the problem is. We'll also break the network on purpose, so we can see what happens when a check fails.

We're continuing with the same lab and project from the previous part, so if you haven't read it yet, I recommend starting there.

What Is assert?

Continue reading

Building an OSPF Network from Scratch with Ansible

This post is part of an ongoing series - see all the posts below.

Ansible Course

So far in this series, we've been working with pretty much the same set of devices, core1, core2, access1, access2, and access3. We've used them to configure VLANs, NTP, syslog, and interface descriptions, and to learn the core Ansible concepts along the way.

For this part, we're going to start with a new topology and put everything we've learned into practice with a real-world example. The lab has six Cisco routers, split across three OSPF areas. I created it using Containerlab, and I've included the topology file below, so you should be able to build the same lab and follow along.

name: ospf_lab

mgmt:
  network: mgmt
  ipv4-subnet: 192.168.202.0/24

topology:
  kinds:
    cisco_iol:
      image: cisco-iol-17.12.01

  nodes:
    r1:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.101
    r2:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.102
    r3:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.103
    r4:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.104
    r5:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.105
    r6:
      kind: cisco_iol
      mgmt-ipv4: 192.168.202.106

  links:
    # Area 0
    - endpoints: ["r1:Ethernet0/1", "r2:Ethernet0/1"]   # 10.100.12. Continue reading

Useful Ansible Filters for Network Engineers

This post is part of an ongoing series - see all the posts below.

Ansible Course

In the previous part of this series, we looked at what filters are and covered some of the most commonly used ones, such as selectattr, map, and regex_replace. We also used filters to solve a real problem in the previous part by sharing the same syslog servers between Cisco and Arista.

Ansible has a huge number of filters, and it's impossible to cover all of them. So in this part, I want to focus on a handful that I find especially useful as a network engineer. We'll work with IP addresses and subnets, compare lists to check whether a device's config is what it should be, merge dictionaries together, and sort lists. Along the way, we'll also build a simple compliance check that you can use in your own network right away.

This post is part of a series on Ansible for network engineers. If you're new to the series, I recommend following the parts in order, since each one builds on the previous one. Here's a quick overview of the setup we're using. The lab has five switches, core1, core2 Continue reading

Palo Alto Global Protect VPN Configuration Example

In this blog post, we will cover how to configure Palo Alto Global Protect VPN. We'll go through setting up the portal, gateway, certificates, authentication profile, IP pools, split-tunnel, security policy, NAT policy and other necessary components. This is aimed at those who already have a basic understanding of Palo Alto firewalls, including interfaces, zones, and policies. We're just focusing on the Global Protect aspects here.

If you want to see the video version of this blog, please check out the video below.

Scenario

In this example, we have a Palo Alto firewall with a simple setup - one interface connects to the internal network (LAN), and another connects to the Internet (WAN). We're focusing on a remote user, someone working from the comfort of their home, who needs to connect to a server sitting at 192.168.10.10 in the LAN.

In a typical environment, the WAN interface on your firewall would typically have a public IP. But since we're running this setup in a Continue reading

Working with Filters in Ansible

This post is part of an ongoing series - see all the posts below.

Ansible Course

We've already used a filter a few times in this series without spending much time on it. In the loops part, we used dict2items to turn a dictionary into a list, so we could loop over it. We also ran into a couple of problems that we couldn't solve at the time, such as sharing the same list of syslog servers between Cisco and Arista, even though their modules expect the data in a different format.

Filters are what solve these kinds of problems. In this part, we'll look at what filters are, go through some of the most useful ones, and see how to chain them together. By the end of it, we'll come back to the syslog problem and fix it using filters.

Before we start, I rebuilt the lab once again, so all the config from the previous part is gone. The project files and inventory are the same as before.

If you're reading this part without reading the previous ones, here's a quick summary of the lab. We have five switches. core1, core2, and access1 are Cisco IOL Continue reading

Ansible – Adding Arista and Multi-Vendor Playbooks

This post is part of an ongoing series - see all the posts below.

Ansible Course

So far, every device in our lab has been a Cisco switch. But very few real networks are made up of a single vendor. You might have Cisco in the campus, Arista in the data centre, and a few firewalls from yet another vendor. One of the biggest benefits of Ansible is that you can manage all of them from the same project, using the same inventory, the same playbooks, and the same way of working.

In this part, we'll add Arista switches to our lab. Arista is a good first step into multi-vendor automation, because just like Cisco, Ansible connects to it over SSH using network_cli. So most of what we've learned so far applies straight away, and we can focus on the differences.

Lab Topology

For this part, I've rebuilt the lab from scratch, so all the config we added in the previous chapters, such as NTP, syslog, and interface descriptions, is gone. Every device starts with only the default config created by Netlab.

As always, your lab doesn’t need to match mine exactly. All you need is a mix of Cisco Continue reading

Palo Alto NAT Example

Most of the time, you'll be working with two types of NAT - Source NAT and Destination NAT. Source NAT is when you change the source IP address of your traffic (usually a private IP) to a public IP that is routable on the Internet. For example, if you have hundreds of people in your office, they are assigned private IPs. When their traffic goes out to the Internet, the Palo Alto firewall translates their private IP (source) to a public IP and the destination IP remains unchanged.

With Destination NAT, the process works the other way around. If you host publicly available resources in your network, like a web server, and someone on the Internet wants to access it, they will target a public IP address that is configured on the firewall. The firewall receives this request and then changes the destination IP to the server's real private IP and the source IP remains unchanged.

We'll also cover Bi-Directional NAT and U-Turn NAT. So, let's get started.

Ansible Loops, Conditionals, and Handlers

So far, our playbooks have been fairly simple. Each task runs once, on every device, every time we run the playbook. That's fine for basic tasks, but as your playbooks grow, you'll want more control over how and when tasks run.

In this post, we'll look at three features that give you that control. Loops let you run the same task multiple times, once for each item in a list. Conditionals let you run a task only when a certain condition is true, such as when a variable is defined or a device is running a specific software version. And handlers let you run a task only when another task has made a change, which is perfect for things like saving the config.

We've used VLANs a lot in the previous posts, so for this post, we'll move on to something different and configure NTP and syslog instead. These are things almost every network needs, and they're a good fit for showing how loops, conditionals, and handlers work.

Project Structure

Before we start, let's take a quick look at where our project is at. We've added and changed quite a few files over the last few posts, so it's easy Continue reading

Ansible Ad-hoc Commands, Facts, and Registering Output

So far, most of what we've done has been about making changes, such as creating VLANs and saving the config. But a lot of day-to-day network work can be also about getting information from devices such as checking software versions, finding serial numbers, looking at interface status, or just running a quick show command across a group of switches.

In this post, we'll look at three ways to do that. First, we'll cover ad-hoc commands, which let you run a single task against your devices straight from the command line, without writing a playbook. Then, we'll look at facts, which is how Ansible collects information about a device, such as its hostname, model, and software version. You may have noticed that we've been setting gather_facts: false in every playbook so far, and we'll finally see why.

Finally, we'll go back to register and debug, which we briefly used in our very first playbook, and look at how to save the output of a task and use it in a more useful way.

Ad-hoc Commands

So far, every time we wanted Ansible to do something, we wrote a playbook. But sometimes you just want to run a quick task, like Continue reading

Ansible Vault

Up until now, we've been storing the username and password for our switches in plain text. I mentioned a couple of times that this isn't a good idea, and now it's time to fix it.

Storing passwords in plain text might be fine in a lab, but in a real network, it's a big problem. Anyone who has access to your project directory can open the file and see the credentials for every device. And once you start storing your playbooks in Git, which you should, those passwords end up in your repository history as well, where they can stay long after you've removed them from the file.

Vault is a feature built into Ansible that lets you encrypt sensitive data, such as passwords, so it can't be read without a vault password. The best part is that Ansible decrypts it automatically when you run a playbook, so your playbooks don't need to change at all.

In this chapter, we'll encrypt the credentials for our switches, look at how to run playbooks with encrypted files, and cover a few different ways to work with Vault day to day.

Encrypting a File

We're continuing from where we left off in the Continue reading

Working with Ansible Inventory and Variables

In the previous chapter, we created inventory groups and moved the connection settings into the cisco group, so we only had to define them once. That works fine for now, but as your network grows, the inventory file can quickly get long and hard to manage, especially once you start adding more variables for each group and device.

We also have the VLANs hardcoded in the playbook. Right now, every switch gets the same four VLANs. But in a real network, you'll probably want different VLANs on the core switches and the access switches, and maybe even a specific VLAN on just one switch. Creating a separate playbook for each case isn't a good idea.

In this chapter, we'll look at how to organise the inventory and variables properly using group_vars and host_vars. We'll move the VLANs out of the playbook and into variables, so the same playbook can create different VLANs on different switches. We'll also look at how Ansible decides which variable to use when the same variable is defined in more than one place, and a few useful ways to target devices when running a playbook.

Moving Variables Out of the Inventory

At the moment, our connection Continue reading

Pushing Configuration with Ansible

So far, we've only used Ansible to run show commands, which is a great way to get started because there's no risk of breaking anything. But at some point, you'll want to use Ansible to actually make changes, and that's what this chapter is all about.

We'll start small by creating a single VLAN on a single switch. From there, we'll create multiple VLANs, then run the same playbook against more devices, and slowly build on it until we're managing different VLANs on different switches. Each step introduces something new, so by the end of the chapter, you'll have covered a lot of the core Ansible concepts without even noticing.

Since we're working with VLANs, you'll need switches in your lab. If you're using Cisco IOS like me, make sure you're using the Layer 2 image rather than the router image. To keep things simple, I'll start with a single switch called s1, and we'll add more devices later in the chapter.

As always, please make sure you're running this in a lab and not on production devices. We're about to start making changes, so it's a good time to be extra careful.

Lab Setup

Since we're working with Continue reading

Understanding YAML and Ansible Playbooks

YAML is a plain text format used to store and organize data. It's designed to be easy for humans to read and write, which is one of the main reasons Ansible uses it. The name originally stood for "Yet Another Markup Language", but it was later changed to "YAML Ain't Markup Language" to make it clear that it's meant for data, not documents.

In Ansible, almost everything you write is in YAML, including playbooks, inventory files, and variables. You've already seen two YAML files in the previous chapter, the inventory and the playbook.

If you've used Containerlab or Netlab, you've already worked with YAML, since their topology files are written in YAML too. It's also used by many other tools such as Docker Compose and Kubernetes, so learning it now will help you well beyond Ansible.

Here's a simple example of what YAML looks like.

hostname: r1
mgmt_ip: 192.168.202.101
ntp_servers:
  - 10.10.10.10
  - 10.10.10.11

Even if you've never seen YAML before, you can probably tell what this means. We have a device called r1, its management IP, and a list of two NTP servers.

YAML is simple, but it's also Continue reading

Why Network Automation and Why Ansible?

Before we install anything or write our first playbook, I want to spend a bit of time on why network automation matters and why Ansible is a good place to start. If you already know why you're here, feel free to skim this chapter, but I think it's worth understanding the problem before jumping into the tool.

The Way We've Always Done It

For most of us, managing a network looks something like this. You open a terminal, SSH into a device, run a few show commands, paste in some config from a text file, verify it, save it, and move on to the next device. Repeat that for as many devices as you need to change.

There's nothing wrong with this approach when you're dealing with a handful of devices. I've done it for years, and so have most network engineers. The problems start when the network grows.

Imagine you need to change the NTP servers on 200 switches or configure Radius authentication on hundreds of devices. Doing that manually means logging into each switch, removing the old servers, adding the new ones, and saving the config. Even if each device only takes two minutes, that's almost seven hours Continue reading

Python Generating Configs with Jinja2

If you have ever rolled out a batch of new sites or devices, I hope you can relate to the pain. The config is more or less the same each time, but the hostnames, IPs, and VLANs change. You copy the config from the last site, tweak the values, save it, and move on. It works, but it does not scale, and it is easy to make mistakes.

The way most of us solve this is with templates. You write the config once with placeholders where the values go, and you fill in the placeholders each time you need a new device config. Jinja2 is the tool we use for this in Python. It takes a template, some data, and gives you back a finished config.

In this post, I will walk you through how Jinja2 works, cover the syntax you actually need, and we will end up generating the same config for Cisco IOS, Juniper Junos, and Arista EOS from the same data. We are only going to generate the configs as text in this post. Pushing them to devices is something you already know how to do from the earlier posts.

Installing Jinja2

Jinja2 is not built into Continue reading

Python Working with REST APIs

Almost every modern piece of network gear has some form of REST API these days. Cisco DNAC, Meraki, Palo Alto Panorama, Juniper Mist, Arista CloudVision, NetBox, and countless others. If you want to automate anything beyond SSH, you will end up talking to an API sooner or later. In this part, we will look at how to do that with Python.

We will use the requests library, which is the most popular HTTP library in the Python world. It is not built into Python, but it is so widely used that most Python developers install it by default. requests handles all the messy details of HTTP for you, headers, cookies, authentication, encoding, so you can focus on the actual work.

To keep things easy at the start, we will use a service called https://mockhttp.org/, which is a testing sandbox for HTTP requests. It has endpoints that do exactly what their name suggests, /get returns a GET response, /post returns a POST response, /status/404 returns a 404, and so on. The responses are simple JSON that echo back the details of the request you sent, which is really helpful when you are learning what is happening under the hood.

Continue reading

Managing Python Projects with uv

If you have installed Python and worked on a few projects, your usual workflow probably looks something like this. Install Python, make sure pip and venv are available, create a virtual environment for each project, and use pip to install packages inside it. That is how we have been doing things throughout this series, and it works. It is what most of the Python world has been doing for years.

uv is a relatively new tool that bundles all of this into one command. It manages Python version installs, virtual environments, package installs, and dependency locking, all in a single binary. It is written in Rust, which makes it fast (like really fast). Most operations that take pip several seconds finish in milliseconds with uv.

In this part, we will start from a completely fresh Ubuntu machine (24.04) that only has the system Python installed. As you will see in a moment, we do not have pip or venv available yet, and normally we would need to install them separately before we could do anything.

A Fresh Ubuntu VM

Here is what is happening on the fresh Ubuntu VM. python --version fails because Ubuntu does not ship the python Continue reading

Python Regular Expressions

You have already used regular expressions once in this series when we told deepdiff to ignore uptime fields. I promised we would cover them properly later, and now is the time.

Regular expressions (regex for short) are a way to describe patterns in text. Instead of asking "does this string contain the exact word up?" you can ask "does this string contain any four-digit number followed by a slash and another number?" That kind of question is hard to answer with the string methods we have used so far, but regex handles it easily.

For a network engineer, regex comes up in a bunch of places. Extracting an IP address from a config line. Pulling interface names out of a log message. Parsing show output when there is no ready-made TextFSM template. Validating that a value looks like a MAC address before you use it.

Regex has a reputation for being cryptic, and honestly, it is cryptic. A complex regex pattern can look like someone leaned on the keyboard. But you do not need to master all of it to be useful. Learning even the basics unlocks a lot of what you will actually need day Continue reading