Looking Ahead: My 2018 Projects

For the last six years or so, I’ve been publishing a list of projects/goals for the upcoming year (followed by a year-end review of how I did with those projects/goals). For example, here are my goals for 2017, and here’s my year-end review of my progress in 2017. In this post, I’m going to share with you my list of projects/goals for 2018.

As I’ve done in previous years, I’ll list the projects/goals, along with an optional stretch goal (where it makes sense).

1. Become extremely fluent with Kubernetes. I’m focusing all my technical skills on Kubernetes this year, with the goal of becoming extremely fluent with the project in all its aspects. There are some aspects—like networking, for example—where some specialization/additional focus will be needed (focusing on particular network architectures/plugins). That means “leaving behind” other technologies, like OpenStack, in order to more fully focus on Kubernetes. (Stretch goal: Pass the Certified Kubernetes Administrator [CKA] exam.)

2. Learn to code/develop in Go. Given that Kubernetes is written in Go and that Go seems to be the language of choice for many new projects, tools, and utilities, I’m going to learn to code/develop in Go in 2018. Because I learned Continue reading

Technology Short Take 95

Welcome to Technology Short Take 95! This Short Take was a bit more challenging than normal to compile, given that I spent the week leading up to its publication visiting customers in Europe. (My travel schedule in Europe is also why it didn’t get published until Saturday instead of the typical Friday.) Nevertheless, I have persevered in order to deliver you this list of links and articles. I hope it proves useful!

Networking

• Larry Smith Jr. has a nice write-up on Cisco XR stemming from a presentation at NFD 17.
• VMware recently released a reference design guide for NSX-T; see here for more details.
• The engineering team at Lyft recently discussed a new overlay-free networking approach they’ve been working on for Kubernetes: IPVLAN-based CNI stack for running within VPCs on AWS. This is pretty cool, but does introduce some potential design considerations for deploying Kubernetes on AWS. (For those that may be unfamiliar: CNI, or Container Network Interface, is the means whereby network mechanisms “plug into” Kubernetes. IPVLAN is a low-latency means of providing IP connectivity to containers. VPCs, or Virtual Private Clouds, are Amazon’s software-defined networking mechanism for workloads running on AWS.)
• Viktor van den Berg writes Continue reading

Some Tools to Help Learn Kubernetes

Kubernetes is emerging as the clear leader in the container orchestration space. This makes it an important technology to know and understand. However, like other distributed systems, learning something like Kubernetes can be challenging due to the effort involved in getting Kubernetes up and running. It’s not about learning to set up Kubernetes (although that comes in time); at first, it’s about understanding what Kubernetes does and how to use Kubernetes. In this post, I’ll share some tools to help learn what Kubernetes does and how to use Kubernetes.

Note that this post is not intended to be a comprehensive list of learning resources for Kubernetes. Also, this post is not focused on providing resources to help you learn to deploy Kubernetes. Instead, I’m focusing here on tools and services that let you get Kubernetes up and running quickly and easily so that you can focus on using Kubernetes (deploying applications and workloads onto Kubernetes). I’m sure there are many more tools/options than what I have listed here; these are just some that I have used and feel might be useful for others.

I’ll briefly cover the following tools and services:

• Minikube
• Kops
• Kube-aws
• Azure Container Service (ACS/AKS)

You’ll note Continue reading

Technology Short Take 94

Welcome to Technology Short Take 94! Ready for another round of links, articles, and thoughts on data center technologies? (Who knows, maybe I’ll throw a rant or two in there.) OK, enough rambling…here’s the good stuff!

Networking

• Amit Aneja has a two-part series (so far) explaining the routing architecture in NSX-T (which brings multi-hypervisor and multi-cloud support to the NSX platform). This is some good content and reminds me of the the old NVP/NSX content I generated back in the day. Ah, good times…anyway, check out Amit’s stuff here and here.
• Sam McGeown has a nice diagram of the communications channels between the various VMware NSX components.
• Roie Ben Haim has a post providing an introduction to NSX and Kubernetes.
• Matt Oswalt tackles the idea of “intent-driven” or “intent-based” networking—all the rage right now—and outlines how something like this must interact with domains outside of networking in order to be effective. I particularly liked his (mini-)rant about how network automation can’t be only about making the network engineer’s life easier. Oh, snap!
• I’m not really sure if this belongs in networking or not (how does one classify OS kernel-level work on networking and security?), but we’ll stick it Continue reading

Running OVS on Fedora Atomic Host

In this post, I’d like to share the results of some testing I’ve been doing to run Open vSwitch (OVS) in containers on a container-optimized Linux distribution such as Atomic Host (Fedora Atomic Host, specifically). I’m still relatively early in my exploration of this topic, but I felt like sharing what I’ve found so far might be helpful to others, and might help spark conversations within the relevant communities about how this experience might be improved.

The reason for the use of Docker containers in this approach is twofold:

1. Many of the newer container-optimized Linux distributions—CoreOS Container Linux (soon to be part of Red Hat in some fashion), Project Atomic, etc.—eschew “traditional” package management solutions in favor of containers.
2. Part of the reason behind my testing was to help the OVS community better understand what it would look like to run OVS in containers so as to help make OVS a better citizen on container-optimized Linux distributions.

In this post, I’ll be using Fedora 27 Atomic Host (via Vagrant with VirtualBox). If you use a different version or release of Atomic Host, your results may differ somewhat. For the OVS containers, I’m using the excellent keldaio/ovs Docker containers.

Continue reading

Using Docker Machine with Azure

I’ve written about using Docker Machine with a number of different providers, such as with AWS, with OpenStack, and even with a local KVM/Libvirt daemon. In this post, I’ll expand that series to show using Docker Machine with Azure. (This is a follow-up to my earlier post on experimenting with Azure.)

As with most of the other Docker Machine providers, using Docker Machine with Azure is reasonably straightforward. Run docker-machine create -d azure --help to get an idea of some of the parameters you can use when creating VMs on Azure using Docker Machine. A full list of the various parameters and options for the Azure drive is also available.

The only required parameter is --azure-subscription-id, which specifies your Azure subscription ID. If you don’t know this, or want to obtain it programmatically, you can use this Azure CLI command:

az account show --query "id" -o tsv

If you have more than one subscription, you’ll probably need to modify this command to filter it down to the specific subscription you want to use.

Additional parameters that you can supply include (but aren’t limited to):

• Use the --azure-image parameter to specify the VM image you’d like to Continue reading

An Update on Using Docker Machine with Vagrant

As part of a project on which I’m working, I’ve been spending some time working with Docker Machine and Vagrant over the last few days. You may recall that I first wrote about using these two tools together back in August 2015. As a result of spending some additional time with these tools—which I chose because I felt like they streamlined some work around this project—I’ve uncovered some additional information that I wanted to share with readers.

As a brief recap to the original article, I showed how you could use Vagrant to quickly and easily spin up a VM, then use Docker Machine’s generic driver to add it to Docker Machine, like this:

docker-machine create -d generic \
--generic-ssh-user vagrant \
--generic-ssh-key ~/.vagrant.d/insecure_private_key \
--generic-ip-address <IP address of VM> \
<name of VM>

This approach works fine if the Vagrant-created VM is reachable without port forwarding. What do I mean? In the past, the VMware provider for Vagrant used functionality in VMware Fusion or VMware Workstation to provide an RFC 1918-addressed network that had external access via network address translation (NAT). In Fusion, for example, this was the default “Share with my Mac” network. Thus, when Continue reading

Technology Short Take 93

Welcome to Technology Short Take 93! Today I have another collection of data center technology links, articles, thoughts, and rants. Here’s hoping you find something useful!

Networking

• Matt Klein breaks down service mesh components, using terms networking professionals know well, like “data plane” and “control plane.” Those familiar with networking concepts will identify well with the ideas in this article, I think.
• This is pretty cool: a WYSIWYG topology designer for VMware NSX. Neat!
• Daniel Hertzberg provides an overview of Salt integration in Arista EOS.
• Barry Peddycord shares an update on Ansible support for the Cumulus Linux NCLU.

Servers/Hardware

Nothing this time around. Feel free to hit me up on Twitter if you have links you think I should include next time!

Security

• Detectify has a nice, in-depth write-up on AWS S3 access controls. Given the recent plethora of data breaches that have been attributed to misconfigured S3 buckets, now might be a good time to read this post.

Cloud Computing/Cloud Management

• The VMware Cloud-Native Apps group has a breakdown of some of the new features/functionality found in the 1.9 release of Kubernetes.
• Via Maish Saidel-Keesing, I saw this post about 10 open source Kubernetes tools for Continue reading

Experimenting with Azure

I’ve been experimenting with Microsoft Azure recently, and I thought it might be useful to share a quick post on using some of my favorite tools with Azure. I’ve found it useful to try to leverage existing tools whenever I can, and so as I’ve been experimenting with Azure I’ve been leveraging familiar tools like Docker Machine and Vagrant.

The information here isn’t revolutionary or unique, but hopefully it will still be useful to others, even if only as a “quick reference”-type of post.

Launching an Instance on Azure Using Docker Machine

To launch an instance on Azure and provision it with Docker using docker-machine:

docker-machine create -d azure \
--azure-subscription-id $(az account show --query "id" -o tsv) \
--azure-ssh-user azureuser \
--azure-size "Standard_B1ms" azure-test

The first time you run this you’ll probably need to allow Docker Machine access to your Azure subscription (you’ll get prompted to log in via a browser and allow access). This will create a service principal that is visible via az ad sp list. Note that you may be prompted for authentication for future uses, although it will re-use the existing service principal once it is created.

Launching an Instance Using the Azure Provider Continue reading

Issue with VMware-Formatted Cumulus VX Vagrant Box

I recently had a need to revisit the use of Cumulus VX (the Cumulus Networks virtual appliance running Cumulus Linux) in a Vagrant environment, and I wanted to be sure to test what I was doing on multiple virtualization platforms. Via Vagrant Cloud, Cumulus distributes VirtualBox and Libvirt versions of Cumulus VX, and there is a slightly older version that also provides a VMware-formatted box. Unfortunately, there’s a simple error in the VMware-formatted box that prevents it from working. Here’s the fix.

The latest version (as of this writing) of Cumulus VX was 3.5.0, and for this version both VirtualBox-formatted and Libvirt-formatted boxes are provided. For a VMware-formatted box, the latest version is 3.2.0, which you can install with this command:

vagrant box add CumulusCommunity/cumulus-vx --box-version 3.2.0

When this Vagrant box is installed using the above command, what actually happens is something like this (at a high level):

1. The *.box file for the specific box, platform, and version is downloaded. This .box file is nothing more than a TAR archive with specific files included (see here for more details).

2. The *.box file is expanded into the ~/.vagrant.d/boxes directory Continue reading

Technology Short Take 92

Welcome to Technology Short Take 92, the first Technology Short Take of 2018. This one was supposed to be the last Tech Short Take of 2017, but I didn’t get it published in time (I decided to spend time with my family instead—some things are just more important). In any case, hopefully the delay of one additional week hasn’t caused any undue stress—let’s jump right in!

Networking

• Lindsay Hill walks through using Telegraf, InfluxDB, and Grafana to monitor network statistics.
• Via Ivan Pepelnjak, I found this article by Diane Patton at Cumulus Networks talking about container network designs. The article is a bit heavy on pushing the Host Pack (a Cumulus thing), but otherwise provides a good overview of several different possible container network designs, along with some of the criteria that might lead to each design.
• Erik Hinderer takes a stab (based on his field experience) at estimating how long it takes to upgrade VMware NSX. Erik’s figures are just estimates, of course; actual values will be determined based on each customer’s specific environment.
• This post is a bit older, but covers a challenge faced by cloud-native darling Netflix—how does one, exactly, identify which application used which IP address Continue reading

Looking Back: 2017 Project Report Card

As has become my custom for the past several years, I wanted to take a look at how well I fared on my 2017 project list. Normally I’d publish this before the end of 2017, but during this past holiday season I decided to more fully “unplug” and focus on the truly important things in life (like my family). So, here’s a look back at my 2017 projects and a report card on my progress (or lack thereof, in some cases).

For reference, here’s the list of projects I set out for myself in 2017:

1. Finish the network automation book.
2. Launch an open source book project.
3. Produce some video content.
4. Get the Full Stack Journey podcast back on track.
5. Complete a “wildcard project.”

So, how did I do with each of these projects?

1. Finish the network automation book: I’m happy to report that all the content for the network automation book I’ve been writing with Jason Edelman and Matt Oswalt is done, and the book is currently in production (and should be available to order from O’Reilly very soon). I had hoped to get the content done in time for the book to be available for order before the Continue reading

Installing XMind 8 on Fedora 27

XMind is a well-known cross-platform mind mapping application. Installing the latest version of XMind (version 8) on Linux is, unfortunately, more complicated than it should be. In this post, I’ll show how to get XMind 8 running on Fedora 27.

So why is installing XMind more complicated than it should be? For reasons unknown, the makers of XMind stopped using well-known Linux package mechanisms with this version of the software, providing only a ZIP archive to download and extract. (Previous versions at least provided a Debian package.) While the ZIP archive includes a very simplistic “setup script”, the script does nothing more than install a few packages and install some fonts, and was written expressly for Debian-based systems. If you extract the archive and place the files outside of your home directory (as would be typical for installing an application on most desktop Linux distributions), you’ll run into problems with permissions. Finally, the application itself is extraordinarily brittle with regards to file locations and such; it’s easy to break it by simply moving the wrong file.

Through some research and some trial-and-error, I finally arrived at a configuration for XMind 8 on Fedora 27 that satisfies a couple criteria:

1. Continue reading

Installing the VMware Horizon Client on Fedora 27

In this post, I’ll outline the steps necessary to install the VMware Horizon client for Linux on Fedora 27. Although VMware provides an “install bundle,” the bundle does not, unfortunately, address any of the prerequisites that are necessary in order for the Horizon client to work. Fortunately, some other folks shared their experiences, and building on their knowledge I was able to make it work. I hope that this post will, in turn, help others who may find themselves in the same situation.

Based on information found here and here, I took the following steps before attempting to install the VMware Horizon client for Linux:

1. First, I installed the libpng12 package using sudo dnf install libpng12.

2. I then created a symbolic link for the libudev.so.0 library that the Horizon client requires:

   sudo ln -s /usr/lib64/libudev.so.1 /usr/lib64/libudev.so.0
   

3. I created a symbolic link for the libffi.so.5 library the Horizon client expects to have available:

   sudo ln -s /usr/lib64/libffi.so.6 /usr/lib64/libffi.so.5
   

With these packages and symbolic links in place, I proceeded to install the VMware Horizon client using the install bundle downloaded from the public VMware web site (for version 4. Continue reading

Using Vagrant with Azure

In this post, I’ll describe how to use Vagrant with Azure. You can consider this article an extension of some of my earlier Vagrant articles; namely, the posts on using Vagrant with AWS and using Vagrant with OpenStack. The theme across all these posts is examining how one might use Vagrant to simplify/streamline the consumption of resources from a provider using the familiar Vagrant workflow.

If you aren’t already familiar with Vagrant, I’d highly recommend first taking a look at my introduction to Vagrant, which provides an overview of the tool and how it’s used.

Prerequisites

Naturally, you’ll need to first ensure that you have Vagrant installed. This is really well-documented already, so I won’t go over it here. Next, you’ll need to install the Azure provider for Vagrant, which you can handle using this command:

vagrant plugin install vagrant-azure

You’ll also (generally) want to have the Azure CLI installed. (You’ll need it for a one-time configuration task I’ll mention shortly.) I’ve published a couple posts on installing the Azure CLI; see here or here.

Once you’ve installed the vagrant-azure plugin and the Azure CLI, you’ll next need to install a box that Vagrant can use. Here, the Continue reading

Technology Short Take 91

Welcome to Technology Short Take 91! It’s been a bit longer than usual since the last Tech Short Take (partly due to the US Thanksgiving holiday, partly due to vacation time, and partly due to business travel), so apologies for that. Still, there’s a great collection of links and articles here for you, so dig in and enjoy.

Networking

• Amanpreet Singh has a two-part series on Kubernetes networking (part 1, part 2).
• Anthony Spiteri has a brief look at NSX-T 2.1, which recently launched with support for Pivotal Container Service (PKS) and Pivotal Cloud Foundry, further extending the reach of NSX into new areas.
• Jon Benedict has a brief article on OVN and its integration into Red Hat Virtualization; if you’re unfamiliar with OVN, it might be worth having a look.
• sFlow is a networking technology that I find quite interesting, but I never seem to have the time to really dig into it. For example, I was recently browsing the sFlow blog and came across two really neat articles. The first was on RESTful control of Cumulus Linux ACLs (this one isn’t actually sFlow-related); the second was on combining sFlow telemetry and RESTful APIs Continue reading

Installing the Azure CLI on Fedora 27

This post is a follow-up to a post from earlier this year on manually installing the Azure CLI on Fedora 25. I encourage you to refer back to that post for a bit of background. I’m writing this post because the procedure for manually installing the Azure CLI on Fedora 27 is slightly different than the procedure for Fedora 25.

Here are the steps to install the Azure CLI into a Python virtual environment on Fedora 27. Even though they are almost identical to the Fedora 25 instructions (one additional package is required), I’m including all the information here for the sake of completeness.

1. Make sure that the “gcc”, “libffi-devel”, “python-devel”, “openssl-devel”, “python-pip”, and “redhat-rpm-config” packages are installed (you can use dnf to take care of this). Some of these packages may already be installed; during my testing with a Fedora 27 Cloud Base Vagrant image, these needed to be installed. (The change from Fedora 25 is the addition of the “redhat-rpm-config” package.)

2. Install virtualenv either with pip install virtualenv or dnf install python2-virtualenv. I used dnf, but I don’t think the method you use here will have any material effects.

3. Create a new Python virtual environment with Continue reading

Using Vagrant with Libvirt on Fedora 27

In this post, I’m going to show you how to use Vagrant with Libvirt via the vagrant-libvirt provider when running on Fedora 27. Both Vagrant and Libvirt are topics I’ve covered more than a few times here on this site, but this is the first time I’ve discussed combining the two projects.

If you’re unfamiliar with Vagrant, I recommend you start first with my quick introduction to Vagrant, after which you can browse all the “Vagrant”-tagged articles on my site for a bit more information. If you’re unfamiliar with Libvirt, you can browse all my “Libvirt”-tagged articles; I don’t have an introductory post for Libvirt.

Background

I first experimented with the Libvirt provider for Vagrant quite some time ago, but at that time I was using the Libvirt provider to communicate with a remote Libvirt daemon (the use case was using Vagrant to create and destroy KVM guest domains via Libvirt on a remote Linux host). I found this setup to be problematic and error-prone, and discarded it after only a short while.

Recently, I revisited using the Libvirt provider for Vagrant on my Fedora laptop (which I rebuilt with Fedora 27). As I mentioned in this post Continue reading

AWS re:Invent 2017 Keynote with Andy Jassy

This is a liveblog of the re:Invent 2017 keynote with Andy Jassy, taking place on Wednesday at the Venetian. As fully expected given the long queues and massive crowds, even arriving an hour early to the keynote isn’t soon enough; there’s already a huge crowd gathered to make it into the venue. Fortunately, I did make it in and scored a reasonable seat from which to write this liveblog.

The pre-keynote time is filled with catchy dance music arranged by a live DJ (same live DJ as last year, if I’m not mistaken). There’s already been quite a few announcements made this year even before today’s keynote: Amazon Sumerian (AR/VR service), new regions and availability zones (AZs), and new bare metal instances, just to name a few of the big ones. There’s been a great deal of speculation regarding what will be announced in today’s keynote, but there’s no doubt there will be a ton of announcements around service enhancements and new services. Rumors are flying about a managed Kubernetes offering; we shall see.

Promptly at 8am, the keynote starts with a brief video, and Andy Jassy, CEO of AWS, takes the stage. Jassy welcomes attendees to the sixth annual Continue reading

Liveblog: Deep Dive on Amazon Elastic File System

This is a liveblog of the AWS re:Invent 2017 session titled “Deep Dive on Amazon Elastic File System (EFS).” The presenters are Edward Naim and Darryl Osborne, both with AWS. This is my last session of day 2 of re:Invent; thus far, most of my time has been spent in hands-on workshops with only a few breakout sessions today. EFS is a topic I’ve watched, but haven’t had time to really dig into, so I’m looking forward to this session.

Naim kicks off the session with looking at the four phases users go through when they are choosing/adopting a storage solution:

1. Choosing the right storage solution
2. Testing and optimizing
3. Ingest (loading data)
4. Running it (operating it in production)

Starting with Phase 1, Naim outlines the three main things that people think about. The first item is storage type. The second is features and performance, and the third item is economics (how much does it cost). Diving into each of these items in a bit more detail, Naim talks about file storage, block storage, and object storage, and the characteristics of each of these approaches. Having covered these approaches, Naim returns to file storage (naturally) and talks about why file Continue reading