Understanding CNI (Container Networking Interface)
If you’ve been paying attention to the discussions around container networking you’ve likely heard the acronym CNI being used. CNI stands for Container Networking Interface and it’s goal is to create a generic plugin-based networking solution for containers. CNI is defined by a spec (read it now, its not very long) that has some interesting language in it. Here are a couple of points I found interesting during my first read through…
- The spec defines a container as being a Linux network namespace. We should be comfortable with that definition as container runtimes like Docker create a new network namespace for each container.
- Network definitions for CNI are stored as JSON files.
- The network definitions are streamed to the plugin through STDIN. That is – there are no configuration files sitting on the host for the network configuration.
- Other arguments are passed to the plugin via environmental variables
- A CNI plugin is implemented as an executable.
- The CNI plugin is responsible wiring up the container. That is – it needs to do all the work to get the container on the network. In Docker, this would include connecting the container network namespace back to the host somehow.
- The CNI plugin is responsible for Continue reading
Getting started with TMUX
This is one of those ‘I must be living under a rock’ things. I’m not sure how I’ve never heard of TMUX before but it’s really pretty awesome. I initially came across it when searching for a way to share a terminal session with another user. It does that quite well but it’s also a great terminal session manager allowing for pane, window, and session management. Let’s take a look at a quick example to show you what I mean.
Here we have a server called ‘tmuxtest’. The server already has TMUX on it by default but if it’s not there you can easily install it (sudo apt-get install tmux, etc). So let’s say I want to start a new session. The easiest way to do this is to just type ‘tmux’..
Now we’re in TMUX. We are in what’s called a ‘session’. The session can contain multiple panes and multiple windows. For instance, if I wanted to create a second pane I could do by pressing ‘Ctrl-b + %’…
Notice that the screen on the right, the new one, has a green boarder around it. That’s my active screen. Now if I want to split this screen horizontally Continue reading
Creating your own Ansible filter plugins
I’ve been playing around with Ansible quite a bit lately. One of the issues I’ve started to run into is that Ansible can’t always do what you want by default. Put more plainly, Ansible can’t always easily do what you want. Many times I found myself writing tasks to manipulate variables and thinking to myself, “Man – if I could just run some Python code on this it would be way easier”. As luck would have it, you can! Ansible supports a whole slew of plugins but the type I want to talk about today are called filter plugins. Filter plugins, in my opinion, are one of the easiest ways to manipulate your variables with native Python. And once you know how to do it you’ll see that it opens up a whole realm of possibilities in your playbooks. Some of the most popular filters that exist today were once custom filters that someone wrote and then contributed to Ansible. The IP address (ipaddr) filter set is a great example of filters that can be used to manipulate IP related information.
When I first looked into writing a custom filter I found the Ansible documentation not very helpful. It essentially Continue reading
Docker Networking Cookbook is now available!
As many of you have noticed I’ve been neglecting the blog for past few months. The main reason for this is that the majority of my free time was being spent generating content for a new book. I’m pleased to announce that the book, Docker Networking Cookbook, has now been released!
Here’s a brief description of the book…
“Networking functionality in Docker has changed considerably since its first release, evolving to offer a rich set of built-in networking features, as well as an extensible plugin model allowing for a wide variety of networking functionality. This book explores Docker networking capabilities from end to end. Begin by examining the building blocks used by Docker to implement fundamental containing networking before learning how to consume built-in networking constructs as well as custom networks you create on your own. Next, explore common third-party networking plugins, including detailed information on how these plugins inter-operate with the Docker engine. Consider available options for securing container networks, as well as a process for troubleshooting container connectivity. Finally, examine advanced Docker networking functions and their relevant use cases, tying together everything you need to succeed with your own projects.”
The book is Continue reading
A new adventure…
As the title says, I’ll soon be starting off on a new adventure. After almost 6 years working in various networking roles at United Health Group I’ve decided to move on. The decision to leave wasn’t an easy one for me to make. I’ve built lots of great relationships and had the opportunity to work with some truly gifted individuals. Many of the people I worked with have had a profound impact on my development as a network engineer and architect. While I can’t possibly list all of the names, I want to sincerely thank all the people that have motivated and supported me during my tenure with UHG. There are countless people that gave me opportunities I couldn’t have had anywhere else and for that I am truly grateful.
So while it’s hard for me to close out this chapter of my career, I’m anxiously looking forward to starting the next one. In late August I’ll be starting in my new role as a network engineer at IBM within the Watson business unit. I’m excited about the opportunity and literally can’t wait to dig in!
In other news, some of Continue reading
Redefining the WAN
One of the more interesting recent trends in the network space has been around software defined WAN (SDWAN). While I’ll admit I first didn’t give it much attention, I’ve since given it a harder look and see quite a bit of promise in the technology. The WAN is a part of the network that, until recently, hasn’t received much attention particularly as it relates to SDN. SDN in the enterprise space seems mostly focused on the data center since that’s where the network always seems to be the most complicated. The unfortunate outcome of that mindset is that while we focus on the data center network, technologies like SDWAN appear and don’t always get the attention they deserve. I think the primary reason for this is that many of us have WANs that we think are ‘working just fine’. And while that may be the case, I think SDWAN has the potential to significantly reduce costs, improve WAN performance, and increase network agility.
One of the vendors in this market that I’ve recently had the chance to hear about is Silver Peak. Silver Peak has been around for quite some time and is Continue reading
Kubernetes Authentication plugins and kubeconfig
Kubernetes offers several different authentication mechanisms or plugins. The goal of this post is to review each of them and provide a brief example of how they work. In addition, we’ll talk about the ‘kubeconfig’ file and how it’s used in association with authentication plugins.
Note: In theory there’s no requirement to use any of these authentication plugins. With the proper configuration, the API server can accept requests over HTTP on any given insecure port you like. However – doing so is insecure and somewhat limiting because some features of Kubernetes rely on using authentication so it’s recommended to use one or more of the following plugins.
Kubernetes offers 3 default authentication plugins as of version 1.0. These plugins are used to authenticate requests against the API server. Since they’re used for communication to the API, that means that they apply to both the Kubelet and Kube-Proxy running on your server nodes as well as any requests or commands you issue through the kubectl CLI tool. Let’s take a look at each option…
Client Certificate Authentication
This is the most common method of authentication and is widely used to authentication node back Continue reading
Kubernetes with SaltStack revisited
I thought it would be a good idea to revisit my last Kubernetes build in which I was using Salt to automate the deployment. The setup worked well at the time, but much has changed with Kubernetes since I initially wrote those state files. That being said, I wanted to update them to make sure they worked with Kubernetes 1.0 and above. You can find my Salt config for this build over at Github…
https://github.com/jonlangemak/saltstackv2
A couple of quick notes before we walk through how to use the repo…
-While I used the last version of this repo as a starting point, I’ve stripped this down to basics (AKA – Some of the auxiliary pods aren’t here (yet)). I’ll be adding to this constantly and I do intend to add a lot more functionality to the defined state files.
-All of the Kubernetes related communication is unsecured. That is – it’s all over HTTP. I already started work on adding an option to do SSL if you so choose.
That being said, let’s jump right into how to use this. My lab looks like this…
Here we have 3 Continue reading
Docker networking 101 – User defined networks
In this post, I’d like to cover some of the new Docker network features. Docker 1.9 saw the release of user defined networks and the most recent version 1.10 added some additional features. In this post, we’ll cover the basics of the new network model as well as show some examples of what these new features provide.
So what’s new? Well – lots. To start with, let’s take a look at a Docker host running the newest version of Docker (1.10).
Note: I’m running this demo on CentOS 7 boxes. The default repository had version 1.8 so I had to update to the latest by using the update method shown in a previous post here. Before you continue, verify that ‘docker version’ shows you on the correct release.
You’ll notice that the Docker CLI now provide a new option to interact with the network through the ‘docker network’ command…
Alright – so let’s start with the basics and see what’s already defined…
By default a base Docker installation has these three networks defined. The networks are permanent and can not be modified. Taking a closer look, Continue reading
Ansible Roles and Variables
While automation is great, we have to be careful not to recreate past problems. What I mean is that playbooks should be written in a generic fashion that can be applied to more than one host. If we’re writing playbooks that only work on one single host, we aren’t much further ahead than we were before.
Two of the key components of making playbooks reusable are Ansible variables and roles. Let’s try and define each of them individually and while showing some examples along the way.
Roles
Roles allow you to call a set of variables, tasks, and handlers by simply specifying a defined role. Roles require the use of a defined file structure in order to work. Per the Ansible documentation, that structure looks like this…
Roles are really just a way to split up your playbook into smaller reusable parts. For instance, let’s consider we added another host to our lab…
Now look at this playbook…
---
- hosts: linuxservers
tasks:
- name: Install Apache Web Server
yum: name=httpd state=latest
notify:
- openport
- startwebserver
handlers:
- name: openport
service: name=httpd state=started
- name: startwebserver
firewalld: port=80/tcp permanent=true state=enabled immediate=yes
- hosts: Continue reading
Ansible up and running
After much delay – I’ve finally found time to take a look at Ansible. I’ve spent some time looking at possible platforms to automate network deployment and Ansible seems to be a favorite in this arena. One of the primary reasons for this is that Ansible is ‘clientless’ (I’m putting that in quotes for a reason, more on that in a later post). So unlike Chef, Puppet, and Salt (Yes – there are proxy modes available in some products) Ansible does not require an installed client on the remote endpoints. So let’s get right into a basic lab setup.
While the end goal will be to use Ansible to automate network appliances, we’re going to start with the a more standard use case – Linux servers. The base lab we will start with is two servers, one acting as the Ansible server and the second being a Ansible client or remote server. Both hosts are CentOS 7 based Linux hosts. So our base lab looks like this…
Pretty exciting right? I know, it’s not, but I want to start with the basics and build from there…
Note: I’ll refer to ansibleserver as Continue reading
Building an OpenStack home lab – Installing OpenStack
If you’ve made it this far, hopefully you’ve already completed steps similar to those outlined in my previous two posts…
If you have, we’re now ready to start installing OpenStack itself. To do this, I’ve built a set of installation scripts. All of the files are out on Github…
https://github.com/jonlangemak/openstackbuild
I suggest you pull them from there into a local directory you can work off of. There is a folder for each VM that needs to be built and each folder has a file called ‘install’. This file contains all of the steps required to build each on one of the three nodes. The remaining files are all of the configuration files that need to change in order for OpenStack to work in our build. We’ll be copying these files over to the VMs as part of the install.
A couple of notes before we start…
-The beginning of each each install file lists all of the packages that need to be installed for this to work. I suggest you start the package install on each VM at the same time as it can take some time Continue reading
Skyport Systems – Moving the edge
The traditional security model has put significant emphasis on what’s typically called the ‘external edge’. That is, the connection between your network and any third party network. This is also where we create a delineation between ‘trusted’ and ‘untrusted’ networks. Regardless of how you define this boundary, it becomes the focal point for any security related tooling. This creates some interesting challenges…
Scale – Applying security tooling at the external edge introduces some possible scale concerns. You now have a single point in the network has to scale to provide connectivity and security services to all of the users and applications. While this might make sense in smaller networks, aggregating everything in one place on larger networks can be challenging. Considering that many security tools can handle significantly lower amounts of traffic than routers and switches, you may find that doing this all in one place introduces a bottleneck in the network. Scaling security appliances is often a much larger task than scaling network links.
Network magic – I often joke that network engineers have to perform network magic to get all of the security tools all of the traffic they Continue reading
Building an OpenStack home lab – Prepping the VMs
In my last post, I talked about some of the more physical aspects of my virtual home lab. We talked about the need for nested virtualization as well as what the physical and virtual network would look like. In this post, we’re going to look at building the VMs as well as getting the operating systems ready for the OpenStack install. As a quick reminder, let’s take a look at what the logical lab looks like…
The lab will consists of 3 VMs (to start with), a controller, and two compute nodes. Wile OpenStack can be installed on a variety of Linux operating systems, this series will be focusing on Ubuntu version 14.04. The first thing we need to do is create a base image. Without a base image, we’re going to be forced to install Ubuntu individually on each server which is not ideal. So the first thing you’ll want to do is download the correct ISO and upload it to your ProxMox server.
Note: Getting around in ProxMox is out of scope for this series. HOWEVER – ProxMox has a rather large following on the internet which Continue reading
What is open networking?
Saying ‘open networking’ is a little like saying ‘SDN’. Without context, it can mean almost anything. Some argue it’s more around options on platforms while others believe it’s more to do with software. When I think about open networking, I think about these main points…
Generic Platforms – White box switches are all the rage these days and for good reason. A white box switch gives you the option to run a variety of different software platforms on generic hardware. This means you don’t need to buy a piece of proprietary hardware to run your proprietary software on. The net result here is that vendor lock in goes away. It also means that you don’t need to wait years and years to buy new hardware to get new software.
Linux – Linux is used EVERYWHERE. As it turns out, it’s already used quite extensively in networking platforms, but not how you might imagine. Most networking vendors use a highly customized version of Linux and the Linux kernel. The reason for this is simple – Linux wasn’t built for networking. Long story short, traditional network vendors had to modify the Continue reading
Building an OpenStack home lab – The Lab
I’ve recently started to play around with OpenStack and decided the best way to do so would be in my home lab. During my first attempt, I ran into quite a couple of hiccups that I thought were worth documenting. In this post, I want to talk about the prep work I needed to do before I began the OpenStack install.
For the initial build, I wanted something simple so I opted for a 3 node build. The logical topology looks like this…
The physical topology looks like this…
It’s one of my home lab boxes. A 1u Supermicro with 8 gigs of RAM and a 4 core Intel Xeon (X3210) processor. The hard drive is relatively tiny as well coming in at 200 gig. To run all of the OpenStack nodes on 1 server, I needed a virtualization layer so I chose ProxMox (KVM) for this.
However, running a virtualized OpenStack environment presented some interesting challenges that I didn’t fully appreciate until I was almost done with the first build…
Nested Virtualization
You’re running a virtualization platform on a virtualized platform. While this doesn’t seem like a huge deal in a home Continue reading
2015 in review and 2016 goals
Last year I started writing down my goals for each year. My thought was that writing them down would be more meaningful than just thinking “I should try and do X this year”. So I want to take a quick look at how I did and talk about 2016 as well.
2015 Goals
Run a marathon
I signed up for, and attempted to run, the Twin Cities Marathon this fall. I knew going into it that I wouldn’t run the whole thing. My training schedule fell apart after the first month. While I could list all of the reasons why (Our 2nd kid was on the way, work was crazy, etc) there really are no excuses. In the month before the marathon I seriously picked up training again but it wasn’t enough. My new goal became to run half which I was able to do and I considered it a huge win for me. Having never run a race that big, I was a little hesitant about it but the experience was AWESOME and I’m definitely keeping this one on the goal list for 2016.
Start work on finishing my basement
Continue reading
Networking Field Day 10 – Big Switch
Last night I finally finished watching all of the Big Switch Networking field day 10 videos. If you haven’t seen them yet, I’d recommend taking a look out at them out on YouTube…
Big Switch Networks – Overview
Big Switch Networks – Why SDN Fabrics?
Big Switch Networks – Big Cloud Fabrics
Big Switch Networks – Big Cloud Fabric GUI demo
Big Switch Networks – Big Cloud Fabric for VMWare
Big Switch Networks – Monitoring Fabric
All of the presentations were awesome and well worth your time especially if you’re new to their products.
If you haven’t looked at Big Switch before, their name sort of says it all. Their base concept is disaggregating a standard chassis switch into individual components. The breakdown would look something like this…
As you can see, each component of a standard data center chassis switch has a similar component in the Big Cloud Fabric. Leaf switches are the new line cards, spine switches the fabric modules or backplane, and the Big Cloud controller is the supervisor. Big switch then uses a standard IP management network to connect all of their components together. This isn’t a very big Continue reading
An introduction to network namespaces
Network namespaces allow you to provide unique views of the network to different processes running on a Linux host. If you’re coming from a traditional networking background, the closest relative to network namespaces would be VRF (Virtual Routing and Forwarding) instances. In both cases the constructs allow us to provide a different network experience to different processes or interfaces. For the sake of starting the conversation, let’s quickly look at an example of both VRFs and network namespaces so you get an idea of how they work.
The easiest scenario to illustrate either of these technologies is out of band management. Take for instance this very simple network diagram…
Note: I’m being purposefully vague here about the network layout and addressing. Bear with me for a moment while I get to the point.
As you can see, we have two users that live on the same segment (forgive me for not drawing an Ethernet segment connecting the two). Let’s assume that the user on the left has to traverse northbound to get to resources that hang off the top network cloud. Let’s also assume the user on the right has to Continue reading
Python – Lists and Dict(ionarie)s
One of the fist things you’ll most likely encounter with Python are the datatypes lists and dicts. While they initially seem quite simple, things can get awfully complex, awfully fast when you start intermingling the two datatypes. So we’ll start with the basics, then dive into some more complex examples.
Lists
Lists are defined as ‘a collection of different pieces of information as a sequence under a single variable name’. So that’s a fancy way of saying it’s just a list. In Python, lists are defined by using the ‘[]’ brackets. So for example…
# A list with one item list = ["Jon"] # A list with multiple items list = ["Jon", "Matt", "Bill"] # An empty list list = []
Items in lists can be accessed by index. For example…
# A list with multiple items list = ["Jon", "Matt", "Bill"] print "The second name in the list is " + list[1] # Result The second name in the list is Matt
We can also iterate through the list with a simple loop…
# A list with multiple items
list = ["Jon", "Matt", "Bill"]
for name in list:
print name
# Result
Continue reading