Scott Lowe

Author Archives: Scott Lowe

Additive Loops with Ansible and Jinja2

I don’t know if “additive” is the right word, but it was the best word I could come up with to describe the sort of configuration I recently needed to address in Ansible. In retrospect, the solution seems pretty straightforward, but I’ll include it here just in case it proves useful to someone else. If nothing else, it will at least show some interesting things that can be done with Ansible and Jinja2 templates.

First, allow me to explain the problem I was trying to solve. As you may know, Kubernetes 1.11 was recently released, and along with it a new version of kubeadm, the tool for bootstrapping Kubernetes clusters. As part of the new release, the Kubernetes community released a new setup guide for using kubeadm to create a highly available cluster. This setup guide uses new functionality in kubeadm to allow you to create “stacked masters” (control plane nodes running both the Kubernetes components as well as the etcd key-value store). Because of the way etcd clusters work, and because of the way you create HA control plane members, the process requires that you start with a single etcd node, then add the second node, and Continue reading

Technology Short Take 102

Welcome to Technology Short Take 102! I normally try to get these things published biweekly (every other Friday), but this one has taken quite a bit longer to get published. It’s no one’s fault but my own! In any event, I hope that you’re able to find something useful among the links below.

Networking

Security

More Handy CLI Tools for JSON

In late 2015 I wrote a post about a command-line tool named jq, which is used for parsing JSON data. Since that time I’ve referenced jq in a number of different blog posts (like this one). However, jq is not the only game in town for parsing JSON data at the command line. In this post, I’ll share a couple more handy CLI tools for working with JSON data.

(By the way, if you’re new to JSON, check out this post for a gentle introduction.)

JMESPath and jp

JMESPath is used by both Amazon Web Services (AWS) in their AWS CLI as well as by Microsoft in the Azure CLI. For examples of JMESPath in action, see the AWS CLI documentation on the --query functionality, which makes use of server-side JMESPath queries to reduce the amount of data returned by an AWS CLI command (as opposed to filtering on the client side).

However, you can also use JMESPath on the client-side through the jp command-line utility. As a client-side parsing tool, jp is similar in behavior to jq, but I find the JMESPath query language to be a bit easier to use than jq in Continue reading

A Quick Intro to the AWS CLI

This post provides a (very) basic introduction to the AWS CLI (command-line interface) tool. It’s not intended to be a deep dive, nor is it intended to serve as a comprehensive reference guide (the AWS CLI docs nicely fill that need). I also assume that you already have a basic understanding of the key AWS concepts and terminology, so I won’t bore you with defining an instance, VPC, subnet, or security group.

For the purposes of this introduction, I’ll structure it around launching an EC2 instance. As it turns out, there’s a fair amount of information you need before you can launch an AWS instance using the AWS CLI. So, let’s look at how you would use the AWS CLI to help get the information you need in order to launch an instance using the AWS CLI. (Tool inception!)

To launch an instance, you need five pieces of information:

  1. The ID of an Amazon Machine Image (AMI)
  2. The type of instance you’re going to launch
  3. The name of the SSH keypair you’d like to inject into the instance
  4. The ID of the security group to which this instance should be added
  5. The ID of the subnet on which this Continue reading

Examining X.509 Certificates Embedded in Kubeconfig Files

While exploring some of the intricacies around the use of X.509v3 certificates in Kubernetes, I found myself wanting to be able to view the details of a certificate embedded in a kubeconfig file. (See this page if you’re unfamiliar with what a kubeconfig file is.) In this post, I’ll share with you the commands I used to accomplish this task.

First, you’ll want to extract the certificate data from the kubeconfig file. For the purposes of this post, I’ll use a kubeconfig file named config and found in the .kube subdirectory of your home directory. Assuming there’s only a single certificate embedded in the file, you can use a simple grep statement to isolate this information:

grep 'client-certificate-data' $HOME/.kube/config

Combine that with awk to isolate only the certificate data:

grep 'client-certificate-data' $HOME/.kube/config | awk '{print $2}'

This data is Base64-encoded, so we decode it (I’ll wrap the command using backslashes for readability now that it has grown a bit longer):

grep 'client-certificate-data' $HOME/.kube/config | \
awk '{print $2}' | base64 -d

You could, at this stage, redirect the output into a file (like certificate.crt) if so desired; the data you have is Continue reading

Using Variables in AWS Tags with Terraform

I’ve been working to deepen my Terraform skills recently, and one avenue I’ve been using to help in this area is expanding my use of Terraform modules. If you’re unfamiliar with the idea of Terraform modules, you can liken them to Ansible roles: a re-usable abstraction/function that is heavily parameterized and can be called/invoked as needed. Recently I wanted to add support for tagging AWS instances in a module I was building, and I found out that you can’t use variable interpolation in the normal way for AWS tags. Here’s a workaround I found in my research and testing.

Normally, variable interpolation in Terraform would allow one to do something like this (this is taken from the aws_instance resource):

tags {
    Name = "${var.name}-${count.index}"
    role = "${var.role}"
}

This approach works, creating tags whose keys are “Name” and “role” and whose values are the interpolated variables. (I am, in fact, using this exact snippet of code in some of my Terraform modules.) Given that this works, I decided to extend it in a way that would allow the code calling the module to supply both the key as well as the value, thus providing more flexibility Continue reading

A Quadruple-Provider Vagrant Environment

In October 2016 I wrote about a triple-provider Vagrant environment I’d created that worked with VirtualBox, AWS, and the VMware provider (tested with VMware Fusion). Since that time, I’ve incorporated Linux (Fedora, specifically) into my computing landscape, and I started using the Libvirt provider for Vagrant (see my write-up here). With that in mind, I updated the triple-provider environment to add support for Libvirt and make it a quadruple-provider environment.

To set expectations, I’ll start out by saying there isn’t a whole lot here that is dramatically different than the triple-provider setup that I shared back in October 2016. Obviously, it supports more providers, and I’ve improved the setup so that no changes to the Vagrantfile are needed (everything is parameterized).

With that in mind, let’s take a closer look. First, let’s look at the Vagrantfile itself:

# Specify minimum Vagrant version and Vagrant API version
Vagrant.require_version '>= 1.6.0'
VAGRANTFILE_API_VERSION = '2'

# Require 'yaml' module
require 'yaml'

# Read YAML file with VM details (box, CPU, and RAM)
machines = YAML.load_file(File.join(File.dirname(__FILE__), 'machines.yml'))

# Create and configure the VMs
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Always use Vagrant's  Continue reading

Technology Short Take 101

Welcome to Technology Short Take #101! I have (hopefully) crafted an interesting and varied collection of links for you today, spanning all the major areas of modern data center technology. Now you have some reading material for this weekend!

Networking

Servers/Hardware

  • AWS adds local NVMe storage to the M5 instance family; more details here. What I found interesting Continue reading

Exploring Kubernetes with Kubeadm, Part 1: Introduction

I recently started using kubeadm more extensively than I had in the past to serve as the primary tool by which I stand up Kubernetes clusters. As part of this process, I also discovered the kubeadm alpha phase subcommand, which exposes different sections (phases) of the process that kubeadm init follows when bootstrapping a cluster. In this blog post, I’d like to kick off a series of posts that explore how one could use the kubeadm alpha phase command to better understand the different components within Kubernetes, the relationships between components, and some of the configuration items involved.

Before I go any further, I’d like to point readers to this URL that provides an overview of kubeadm and using it to bootstrap a cluster. If you’re new to kubeadm, go read that before continuing on here.

<aside>Quick side note: it’s my understanding that at some point the intent is to move kubeadm alpha phase out of alpha, at which point the command might look more like kubeadm phase or similar (that hasn’t been fully determined yet as far as I know). If you’re reading this at some point in the future, just make note that this was written back Continue reading

Book Review: Infrastructure as Code

As part of my 2018 projects, I committed to reading and reviewing more technical books this year. As part of that effort, I recently finished reading Infrastructure as Code, authored by Kief Morris and published in September 2015 by O’Reilly (more details here). Infrastructure as code is very relevant to my current job function and is an area of great personal interest, and I’d been half-heartedly working my way through the book for some time. Now that I’ve completed it, here are my thoughts.

Overall, Morris does a great job of crisply defining infrastructure as code (a somewhat vague and amorphous term at times) and outlining the key principles that are involved. Morris also does a really good job of staying high-level as he works through the various aspects of infrastructure as code and discusses some of the considerations, patterns (and anti-patterns), and recommended practices in each aspect.

The book’s high-level focus is, however, both its greatest strength as well as its greatest weakness. Because infrastructure as code can be implemented in a variety of ways with a variety of tools, the book must necessarily be high-level and somewhat abstract. As I mentioned, Morris does a really Continue reading

Technology Short Take 100

Wow! This marks 100 posts in the Technology Short Take series! For almost eight years (Technology Short Take #1 was published in August 2010), I’ve been collecting and sharing links and articles from around the web related to major data center technologies. Time really flies when you’re having fun! Anyway, here is Technology Short Take 100…I hope you enjoy!

Also, a quick note that I removed the “Servers/Hardware” and “Storage” sections this time around, as I didn’t have any useful content to share. I’ll continue to evaluate whether I will/should include those sections moving forward (your feedback is welcome; hit me up on Twitter).

Networking

Security

  • Container wizard Jessie Frazelle shares a proposal for hard multi-tenancy in Kubernetes. Along the way, she also provides some additional (useful) information about existing isolation mechanisms.

Cloud Computing/Cloud Management

Quick Post: Parsing AWS Instance Data with JQ

I recently had a need to get a specific subset of information about some AWS instances. Naturally, I turned to the CLI and some CLI tools to help. In this post, I’ll share the command I used to parse the AWS instance data down using the ever-so-handy jq tool.

What I needed, specifically, was the public IP address and the private IP address for each instance. That information is readily accessible using the aws ec2 describe-instances command, but that command provides a ton more information than I needed. So, I decided to try to use jq to parse the JSON output from the AWS CLI. If you’re not familiar with jq, I recommend you take a look at this brief introductory post I wrote back in 2015.

After some trial and error, here’s the final command I used:

aws ec2 describe-instances | jq '.Reservations[] | .Instances[] | \
{Id: .InstanceId, PublicAddress: .PublicIpAddress, \
PrivateAddress: .PrivateIpAddress}'

I’ll refer you to the jq manual for details on breaking down how this filter works. I’ll also point out that there’s nothing terribly groundbreaking or revolutionary about this command; I wanted to share it here just in case it may save someone Continue reading

Posts from the Past, May 2018

This month—May 2018—marks thirteen years that I’ve been generating content here on this site. It’s been a phenomenal 13 years, and I’ve enjoyed the opportunity to share information with readers around the world. To celebrate, I thought I’d do a quick “Posts from the Past” and highlight some content from previous years. Enjoy!

May 2017

A year ago, I touched on the topic of using a Makefile with Markdown documents to help streamline the process of generating various output formats.

I also explored the use of custom SSH configurations with SSH bastion hosts and uncovered a very basic (but important) error I’d previously overlooked.

May 2016

Two years ago in May I was using Terraform to build an etcd v2 cluster on OpenStack.

May 2015

Three years ago, I was doing a lot of work in my home lab, automating the setup of physical hosts. That led to a post on a fully automated Ubuntu install, which was also related to this post on using an Apt proxy (via apt-cacher-ng).

May 2014

Four years ago, I shared some useful Markdown tools for OS X. Of those tools, I still use pandoc pretty extensively.

May 2013

Five years ago, Continue reading

DockerCon SF 18 and Spousetivities

DockerCon SF 18 is set to kick off in San Francisco at the Moscone Center from June 12 to June 15. This marks the return of DockerCon to San Francisco after being held in other venues for the last couple of years. Also returning to San Francisco is Spousetivities, which has organized activities for spouses, significant others/domestic partners, friends, and family members traveling with conference attendees!

Registration is open right now, so hurry on over and sign up for one or more activities. What’s that—you’re wondering what’s been planned? Here’s a quick overview:

  • The activities kick off on Tuesday, June 12, with a tour of Monterey and Carmel. I must admit I’m a bit jealous; I’d love to have the opportunity to visit Cannery Row and see the canneries that inspired John Steinbeck. Join this tour and you’ll have that opportunity! This is a full-day activity, leaving the InterContinental at 8pm and returning around 7pm that evening. Lunch is included, of course.
  • On Wednesday, June 13, Spousetivities has arranged for private transportation to the Sonoma Valley for some wine tastings! You’ll visit a couple different wineries and get to enjoy lunch in Sonoma. This event will wrap up back Continue reading

Manually Installing Firefox 60 on Fedora 27

Mozilla recently released version 60 of Firefox, which contains a number of pretty important enhancements (as outlined here). However, the Fedora repositories don’t (yet) contain Firefox 60 (at least not for Fedora 27), so you can’t just do a dnf update to get the latest release. With that in mind, here are some instructions for manually installing Firefox 60 on Fedora 27.

These instructions assume you have a dnf-installed version of Firefox (typically Firefox 59) already installed on your Fedora system. These steps should allow you to upgrade your Fedora system to Firefox 60:

  1. Download the Firefox 60 archive (typically named firefox-60.0.tar.bz2 or similar) onto your Fedora system. You can do this with your already-installed version of Firefox, but be sure to close/quit Firefox before proceeding with the rest of the instructions.
  2. Make a copy of /usr/share/applications/firefox.desktop; you’ll use this later.
  3. Remove the version of Firefox installed from the Fedora repositories with dnf remove firefox. This will remove the firefox.desktop file you copied in the previous step (which is why you copied it somewhere else).

  4. Use bunzip2 to decompress the downloaded Firefox 60 archive. This will leave you with a plain . Continue reading

One Week Until Spousetivities in Vancouver

Only one week remains until Spousetivities kicks off in Vancouver at the OpenStack Summit! If you are traveling to the Summit with a spouse, significant other, family member, or friend, I’d encourage you to take a look at the great activities Crystal has arranged during the Summit.

Here’s a quick sneak peek at what’s planned:

  • On Monday, May 21, Spousetivities attendees will enjoy a tour of the highlights of Vancouver (including things like Stanley Park, Gastown, Chinatown, and Granville Island Public Market), followed by fun at the Capilano Suspension Bridge Park. (If for no other reason, you’ll want to attend this to see Crystal face her fear of heights and suspension bridges!)
  • On Tuesday, May 22, Spousetivities is off to Whistler Village. Along the way, see wonderful sights like Howe Sound, Britannia Beach, and Shannon Falls. Then you’ll get to ride the Sea to Sky Gondola up to Squamish for an adventure-filled time.
  • On Wednesday, May 23, the activities wrap up with a wine tour. This will include tastings at three beautiful wineries and a lovely picnic lunch at one of the venues.

All of these tours includes private transportation, and the pricing for each of the events is Continue reading

Technology Short Take 99

Welcome to Technology Short Take 99! What follows below is a collection of various links and articles about (mostly) data center-related technologies. Hopefully something I’ve included will be useful. Here goes!

Networking

  • David Gee makes the connection between coffee and network automation. No, really. It’s worth reading.
  • Matt Oswalt, one of the co-authors of our recently-released network automation book from O’Reilly, recently tackled the topic of running Kubernetes with Tungsten Fabric (formerly known as OpenContrail). A network engineer using AWS and CloudFormation? Yep, get used to it folks—it’s where the industry is headed.
  • Vince Power provides a high-level overview of some of the key principles underlying Kubernetes networking.

Servers/Hardware

Sorry, I don’t have anything for you. Feel free to send me links you’d like me to consider for inclusion in the next Tech Short Take!

Security

Installing GitKraken on Fedora 27

GitKraken is a full-featured graphical Git client with support for multiple platforms. Given that I’m trying to live a multi-platform life, it made sense for me to give this a try and see whether it is worth making part of my (evolving and updated) multi-platform toolbelt. Along the way, though, I found that GitKraken doesn’t provide an RPM package for Fedora, and that the installation isn’t as straightforward as one might hope. I’m documenting the procedure here in the hope of helping others.

First, download the latest release of GitKraken. You can do this via the terminal with this command:

curl -LO https://release.gitkraken.com/linux/gitkraken-amd64.tar.gz

Extract the contents of the GitKraken download into its own directory under /opt using this command (you can use a different directory if you like, but I prefer to install third-party applications like this under /opt):

sudo tar -C /opt -xvf gitkraken-amd64.tar.gz

This will extract everything into /opt/gitkraken.

Next, you’ll create a symbolic link to an existing library to fix an error with GitKraken when running on Fedora (this is documented here):

sudo ln -s /usr/lib64/libcurl.so.4 /usr/lib64/libcurl-gnutls.so.4

Once this is done, you could just run Continue reading

An Updated Look at My Multi-Platform Toolbelt

In early 2017 I posted about my (evolving) multi-platform toolbelt, describing some of the applications, standards, and services that I use across my Linux and macOS systems. In this post, I’d like to provide an updated review of that toolbelt.

  • Visual Studio Code: I switched from Sublime Text to Visual Studio Code during my latest migration to Fedora 27 on a Lenovo ThinkPad X1 Carbon. Since I’m also planning on expanding my coding skills with Golang, I felt that Visual Studio Code would be a better choice than Sublime Text. I’m still generating the majority of my content in Markdown (MultiMarkdown is the flavor that I generally use), and I’ve found Visual Studio Code to be pretty decent as a Markdown editor.

  • IMAP/SMTP: I’ve standardized on using IMAP/SMTP for all my e-mail accounts, which gives me quite a bit of flexibility in clients and OSes. It’s very likely I’ve pretty much standardized on Thunderbird (which supports OS X, Linux, and Windows).

  • Unison: This cross-platform file synchronization tool helps keep my files in sync across my macOS and Linux systems.

  • Dropbox: Dropbox gives me access to non-confidential files from any of my devices or platforms (macOS, iOS, and Linux).

  • jrnl: Continue reading

Technology Short Take 98

Welcome to Technology Short Take #98! Now that I’m starting to get settled into my new role at Heptio, I’ve managed to find some time to pull together another collection of links and articles pertaining to various data center technologies. Feedback is always welcome!

Networking

  • VMware has released a PowerCLI preview/fling for NSX-T; Kyle Ruddy has a write-up here. Looks like this preview provides some high-level cmdlets for NSX-T that weren’t available before.
  • Cilium, the open source project working to bring eBPF-powered networking and security to Kubernetes environments, has hit the 1.0 release. I will freely admit that I am a fan of what the Cilium folks are doing.
  • What’s that? Don’t know what eBPF is? Or XDP? Not to worry, the nice folks over at Netronome have a post that explains it all.
  • People do all kinds of interesting things with Raspberry Pis; here’s an article by Scott Helme on using a Pi to secure DNS traffic.

Servers/Hardware

  • This is more of a follow-up to one of my own articles than a pointer to someone else’s article. After continued use (including on business trips) of my Lenovo ThinkPad X1 Carbon running Fedora 27, I continue to be impressed Continue reading
1 … 13 14 15 16 17 … 20