A Simple Network CI/CD Pipeline
In this blog post, let's look at a very simple Network CI/CD pipeline that manages my Containerlab network topology and configurations. We'll start with the benefits of using CI/CD, cover some basic terminology, and then go through an example.
To give an overview, I use Containerlab to deploy my network labs and Nornir to deploy the configurations. Before CI/CD, my typical workflow involves using containerlab commands to manage the topology. Once the lab is up and running, I use Python to run the Nornir script. This works well because I'm the only one using it. However, I ideally want to put all the configurations into a Git repo to track my changes over time. I also want to test my code (to ensure there are no syntax errors, for example) and automatically push the updates to the devices.
Here is the project repo if you want to clone it and follow along.
Suresh V / simple_bgp_lab · GitLab
GitLab.com
What Exactly is CI/CD?
CI/CD stands for Continuous Integration and Continuous Delivery. In simple terms, it means automatically testing and delivering your code. With Continuous Integration (CI), every time you make a change to your code, it's tested automatically Continue reading
Testing Network Automation Data Transformation
Every complex enough network automation solution has to introduce a high-level (user-manageable) data model that is eventually transformed into a low-level (device) data model.
High-level overview of the process
The transformation code (business logic) is one of the most complex pieces of a network automation solution, and there’s only one way to ensure it works properly: you test the heck out of it ;) Let me show you how we solved that challenge in netlab.
Testing Network Automation Data Transformation
Every complex enough network automation solution has to introduce a high-level (user-manageable) data model that is eventually transformed into a low-level (device) data model.
High-level overview of the process
The transformation code (business logic) is one of the most complex pieces of a network automation solution, and there’s only one way to ensure it works properly: you test the heck out of it ;) Let me show you how we solved that challenge in netlab.
Unlocking Ansible: Accessing host_vars and group_vars in a Python Scripts
Welcome to the world of Ansible magic! In this blog post, we're going to uncover the secrets of accessing host_vars and group_vars directly from Python scripts. These variables hold the keys to customizing your automation scripts, empowering you to unlock new levels of flexibility and efficiency in your infrastructure management.
Let’s dive in!
To do this, we’ll use the Ansible API. The Ansible API is a powerful tool that allows you to interact with Ansible programmatically. The documentation for the Ansible API can be found here.
Ansible Project Structure
My ansible project folder structure looks like this
.
├── ansible.cfg
├── ansible_pyapi.py
├── group_vars
│ ├── all.yaml
│ └── host1_2.yaml
├── host_vars
│ ├── host1.yml
│ ├── host2.yml
│ └── host3.yml
└── inventory.ini
folder structure
Inventory File
My inventory file looks like this:
host1
host2
host3
[host1_2]
host1
host2
[all:vars]
username= "username"
password= "password"
inventory.ini
Host Vars and Group Vars
Contents of host_vars and group_vars files are as follows:
host_vars_location: from host_vars/host1.ymlhost_vars/host1.yml
host_vars_location: from host_vars/host2.yml
host_vars/host2.yml
host_vars_location: from host_vars/host3.yml
host_vars/host3.yml
all_group_vars: from group_vars/all.yaml
group_vars/all.yml
group_vars_location: from group_vars/host1_2.yaml
group_vars/host1_2.yml
Accessing Ansible Variables Continue reading
Quantile Regression
The Median Isn’t the Message - Stephen Jay Gould
When we think of regression, the most common one, which we all know, is linear regression. It is a fairly popular and simple technique for estimating the mean of some variable conditional on the values of independent variables.
Now imagine if you are a grocery delivery or ride-hailing service and want to show the customer the estimated delivery or wait times. If the distance is smaller, there will be less variability in the waiting time, but if the distance is longer, many things can go wrong, and due to that there can be a lot of variability in the estimate time. If we have to create a model to predict that, we may not want to apply linear regression as that will only tell us the average time.
It’s important to note that one of the key assumptions for applying linear regression is a constant variance (Homoskedasticity). However, many times this is often not the case. The variability is not constant (Heteroscedastic), which violates the linear regression assumption (Linear Regression Notes).
Motivation
Let’s look at a running data for the distance vs. the time it takes to finish. We clearly know Continue reading
What is new in Calico 3.28
TL/DR
- A new Grafana dashboard that helps you monitor Calico Typha’s performance and troubleshoot issues.
- Calico eBPF dataplane IPv6 is now GA. It supports true IPv6-only clusters as well as dual-stack clusters.
- Optional Pod startup delay to ensure networking is up in high-churn scenarios.
- Tigera operator now supports multiple IP pools, IP pool modification, affinity for operator pods, priorityclassname, and more!
- Improved policy performance in both eBPF and iptables.
- Calico now ships with a pprof server. Activate the performance server for real-time views of Typha and Felix components and real-time debugging.
Important changes
Important changes Calico 3.28 now has enabled VXLAN checksum offload by default for environments with the kernel version of 5.8 or above. In the past, offloading was disabled due to kernel bugs.
Please keep in mind, if you are upgrading to 3.28 this change will take effect after node restarts.
If you encounter unexpected performance issues, you can use the following command to revert to the previous method by using the following command:
kubectl patch felixconfiguration default --type="merge" -p='{"spec":{"FeatureDetectOverride":"ChecksumOffloadBroken=true"}'
Please keep in mind that you can report any issues via GitHub tickets or Slack and include a detailed description of the environment (NIC hardware, kernel, distro, Continue reading
Hedge 226: Making Networking Cool Again
Alexis Bertholf joins Tom Ammon and Russ White to discuss how we can make network engineering cool again—and to talk about how we got into network engineering.
HN734: Russ White Hour Part 2: Snowflakes and Network Automation
Welcome to the second part of our interview with friend of the podcast, Russ White. We start our conversation with a listener question about VXLAN/EVPN which acts as a springboard for what Russ really thinks about network engineering these days. He defends network snowflakes, championing their power in business use cases. He questions the merit... Read more »Case Study: IPng’s mail servers
Intro
I have seen companies achieve great successes in the space of consumer internet and entertainment industry. I’ve been feeling less enthusiastic about the stronghold that these corporations have over my digital presence. I am the first to admit that using “free” services is convenient, but these companies are sometimes taking away my autonomy and exerting control over society. To each their own of course, but for the last few years, I’ve been more and more inclined to take back a little bit of responsibility for my online social presence, away from centrally hosted services and to privately operated ones.
First off - I love Google’s Workspace products. I started using GMail just after it launched, back in 2004. Its user interface is sleek, performant, and very intuitive. Its filtering, granted, could be a bit less … robotic, but that’s made up by labels and an incredibly comprehensive search function. I would dare say that between GMail and Photos, those are my absolute favorite products on the internet.
That said, I have been running e-mail servers since well before Google existed as a company. I started off at M.C.G.V. Stack, the computer club of the University of Continue reading
Public Videos: Kubernetes Service Mesh
All the Kubernetes Service Mesh videos from the Kubernetes Networking Deep Dive webinar with Stuart Charlton are now public. Enjoy!
A Transport Protocol’s View of Starlink
Let's look at the Starlink at a protocol level, and how TCP, the workhorse transport protocol of the Internet, interacts with the somewhat unique characteristics of Starlink's service.IPB151: What Changed Alexandra Huides’ Mind about IPv6
Alexandra Huides didn’t like IPv6 on her first encounter with it. Today she is globally renowned for spreading the IPv6 gospel and helping AWS customers adopt it. Alexandra joins the show today to share what changed her mind about IPv6 and what she sees change the minds of network engineers every day: Greater client traffic... Read more »Amazon EKS networking options
When setting up a Kubernetes environment with Amazon Elastic Kubernetes Service (EKS), it is crucial to understand your available networking options. EKS offers a range of networking choices that allow you to build a highly available and scalable cloud environment for your workloads.
In this blog post, we will explore the networking and policy enforcement options provided by AWS for Amazon EKS. By the end, you will have a clear understanding of the different networking options and network policy enforcement engines, and other features that can help you create a functional and secure platform for your Kubernetes workloads and services.
Amazon Elastic Kubernetes Service (EKS)
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service that simplifies routine operations, such as cluster deployment and maintenance, by automating tasks such as patching and updating operating systems and their underlying components. EKS enhances scalability through AWS Auto Scaling groups and other AWS service integrations and offers a highly available control plane to manage your cluster.
Amazon EKS in the cloud has two options:
- Managed
- Self-managed
Managed clusters rely on the AWS control plane node, which AWS hosts and controls separately from your cluster. This node operates in isolation and cannot be directly Continue reading
Must Read: OSPF Protocol Analysis (RFC 1245)
Daniel Dib found the ancient OSPF Protocol Analysis (RFC 1245) that includes the Router CPU section. Please keep in mind the RFC was published in 1991 (35 years ago):
Steve Deering presented results for the Dijkstra calculation in the “MOSPF meeting report” in [3]. Steve’s calculation was done on a DEC 5000 (10 mips processor), using the Stanford internet as a model. His graphs are based on numbers of networks, not number of routers. However, if we extrapolate that the ratio of routers to networks remains the same, the time to run Dijkstra for 200 routers in Steve’s implementation was around 15 milliseconds.
Must Read: OSPF Protocol Analysis (RFC 1245)
Daniel Dib found the ancient OSPF Protocol Analysis (RFC 1245) that includes the Router CPU section. Please keep in mind the RFC was published in 1991 (35 years ago):
Steve Deering presented results for the Dijkstra calculation in the “MOSPF meeting report” in [3]. Steve’s calculation was done on a DEC 5000 (10 mips processor), using the Stanford internet as a model. His graphs are based on numbers of networks, not number of routers. However, if we extrapolate that the ratio of routers to networks remains the same, the time to run Dijkstra for 200 routers in Steve’s implementation was around 15 milliseconds.
Rule 11 Academy is (Somewhat) Live
The Academy does not replace this blog, the Hedge, etc. Instead, it’s a place for me to recreate all the training materials I’ve taught in the past, put them in one place, and adding new training material besides. It’s light right now, but I plan to post about once or twice a week.
Note this is a subscription site with paid content and two memberships–six months and yearly.
Get six months free using the coupon code BEAG2DRUP0TORNSKUT.