Scott Lowe

Author Archives: Scott Lowe

Technology Short Take 145

Welcome to Technology Short Take #145! What will you find in this Tech Short Take? Well, let’s see…stuff on Envoy, network automation, network designs, M1 chips (and potential open source variants!), a bevy of security articles (including a couple on very severe vulnerabilities), Kubernetes, AWS IAM, and so much more! I hope that you find something useful here. Enjoy!

Networking

Servers/Hardware

  • Howard Oakley of The Eclectic Light Company discusses some details on Apple’s M1 chip and what it does differently than other chips. Also included in this post are links to other articles with even more details—very helpful.
  • Are open source M1-style chips a possibility? This article Continue reading

Technology Short Take 144

Welcome to Technology Short Take #144! I have a fairly diverse set of links for readers this time around, covering topics from microchips to improving your writing, with stops along the way in topics like Kubernetes, virtualization, Linux, and the popular JSON-parsing tool jq along the way. I hope you find something useful!

Networking

  • A short while ago I was helping someone (an acquaintance of a friend) with some odd DNS issues. I never found the root cause, but we did find a workaround; however, along the way, someone shared this article with me. I thought it was useful, so now I’m sharing it with you.
  • Michael Kashin shares the journey of containerizing NVIDIA Cumulus Linux.

Servers/Hardware

Security

Establishing VPC Peering with Pulumi and Go

I use Pulumi to manage my lab infrastructure on AWS (I shared some of the details in this April 2020 blog post published on the Pulumi site). Originally I started with TypeScript, but later switched to Go. Recently I had a need to add some VPC peering relationships to my lab configuration. I was concerned that this may pose some problems—due entirely to the way I structure my Pulumi projects and stacks—but as it turned out it was more straightforward than I expected. In this post, I’ll share some example code and explain what I learned in the process of writing it.

Some Background

First, let me share some background on how I structure my Pulumi projects and stacks.

It all starts with a Pulumi project that manages my base AWS infrastructure—VPC, subnets, route tables and routes, Internet gateways, NAT gateways, etc. I use a separate stack in this project for each region where I need base infrastructure.

All other projects build on “top” of this base project, referencing the resources created by the base project in order to create their own resources. Referencing the resources created by the base project is accomplished via a Pulumi StackReference.

In my Continue reading

Using the AWS CLI to Tag Groups of AWS Resources

To conduct some testing, I recently needed to spin up a group of Kubernetes clusters on AWS. Generally speaking, my “weapon of choice” for something like this is Cluster API (CAPI) with the AWS provider. Normally this would be enormously simple. In this particular case—for reasons that I won’t bother going into here—I needed to spin up all these clusters in a single VPC. This presents a problem for the Cluster API Provider for AWS (CAPA), as it currently doesn’t add some required tags to existing AWS infrastructure (see this issue). The fix is to add the tags manually, so in this post I’ll share how I used the AWS CLI to add the necessary tags.

Without the necessary tags, the AWS cloud provider—which is responsible for the integration that creates Elastic Load Balancers (ELBs) in response to the creation of a Service of type LoadBalancer, for example— won’t work properly. Specifically, the following tags are needed:

kubernetes.io/cluster/<cluster-name>
kubernetes.io/role/elb
kubernetes.io/role/internal-elb

The latter two tags are mutually exclusive: the former should be assigned to public subnets to tell the AWS cloud provider where to place public-facing ELBs, while the latter is assigned to private subnets Continue reading

Technology Short Take 143

Welcome to Technology Short Take #143! I have what I think is an interesting list of links to share with you this time around. Since taking my new job at Kong, I’ve been spending more time with Envoy, so you’ll see some Envoy-related content showing up in this Technology Short Take. I hope this collection of links has something useful for you!

Networking

Servers/Hardware

Security

  • I saw this blog post about Curiefense, an open source Envoy extension to add WAF (web application firewall) functionality to Envoy.
  • This post on using SPIFFE/SPIRE, Kubernetes, and Envoy together shows how to implement mutual TLS (mTLS) for a simple application. As a learning resource, I thought this post was helpful. However, Continue reading

Starting WireGuard Interfaces Automatically with Launchd on macOS

In late June of this year, I wrote a piece on using WireGuard on macOS via the CLI, where I walked readers using macOS through how to configure and use the WireGuard VPN from the terminal (as opposed to using the GUI client, which I discussed here). In that post, I briefly mentioned that I was planning to explore how to have macOS' launchd automatically start WireGuard interfaces. In this post, I’ll show you how to do exactly that.

These instructions borrow heavily from this post showing how to use macOS as a WireGuard VPN server. These instructions also assume that you’ve already walked through installing the necessary WireGuard components, and that you’ve already created the configuration file(s) for your WireGuard interface(s). Finally, I wrote this using my M1-based MacBook Pro, so my example files and instructions will be referencing the default Homebrew prefix of /opt/homebrew. If you’re on an Intel-based Mac, change this to /usr/local instead.

The first step is to create a launchd job definition. This file should be named <label>.plist, and it will need to be placed in a specific location. The <label> value is taken from the name given to the job Continue reading

An Alternate Approach to etcd Certificate Generation with Kubeadm

I’ve written a fair amount about kubeadm, which was my preferred way of bootstrapping Kubernetes clusters until Cluster API arrived. Along the way, I’ve also discussed using kubeadm to assist with setting up etcd, the distributed key-value store leveraged by the Kubernetes control plane (see here, here, and here). In this post, I’d like to revisit the topic of using kubeadm to set up an etcd cluster once again, this time taking a look at an alternate approach to generating the necessary TLS certificates than what the official documentation describes.

There is absolutely nothing wrong with the process the official documentation describes (I’m referring to this page, by the way); this process just creates slightly “cleaner” certificates. What do I mean by “cleaner” certificates? The official documentation uses a series of kubeadm configuration files, one for each etcd cluster member, to control how the utility creates the necessary certificates and configuration files. The user is instructed to use these configuration files on a single system to generate the certificates for all the cluster members. This works fine, with one caveat: each of the certificates will have an extra hostname—the hostname of the system being used Continue reading

Technology Short Take 142

Welcome to Technology Short Take #142! This time around, the Networking section is a bit light, but I’ve got plenty of cloud computing links and articles for you to enjoy, along with some stuff on OSes and applications, programming, and soft skills. Hopefully there’s something useful here for you!

Networking

  • Bill Doerrfeld discusses the use of WebAssembly to extend the Envoy proxy here after a discussion with Idit Levine, CEO and founder of Solo.io. Envoy is increasingly becoming an important component in many cloud-native networking solutions, so the ability to extend or customize Envoy’s behavior is quite important in my opinion.

Servers/Hardware

Security

Cloud Computing/Cloud Management

Adding Multiple Items Using Kustomize JSON 6902 Patches

Recently, I needed to deploy a Kubernetes cluster via Cluster API (CAPI) into a pre-existing AWS VPC. As I outlined in this post from September 2019, this entails modifying the CAPI manifest to include the VPC ID and any associated subnet IDs, as well as referencing existing security groups where needed. I knew that I could use the kustomize tool to make these changes in a declarative way, as I’d explored using kustomize with Cluster API manifests some time ago. This time, though, I needed to add a list of items, not just modify an existing value. In this post, I’ll show you how I used a JSON 6902 patch with kustomize to add a list of items to a CAPI manifest.

By the way, if you’re not familiar with kustomize, you may find my introduction to kustomize post to be helpful. Also, for those readers who are unfamiliar with JSON 6902 patches, the associated RFC is useful, as is this site.

In this particular case, the addition of the VPC ID and the subnet IDs were easily handled with a strategic merge patch that referenced the AWSCluster object. More challenging, though, was the reference to the existing security Continue reading

Using WireGuard on macOS via the CLI

I’ve written a few different posts on WireGuard, the “simple yet fast and modern VPN” (as described by the WireGuard web site) that aims to supplant tools like IPSec and OpenVPN. My first post on WireGuard showed how to configure WireGuard on Linux, both on the client side as well as on the server side. After that, I followed it up with posts on using the GUI WireGuard app to configure WireGuard on macOS and—most recently—making WireGuard from Homebrew work on an M1-based Mac. In this post, I’m going to take a look at using WireGuard on macOS again, but this time via the CLI.

Some of this information is also found in this WireGuard quick start. Here I’ll focus only on using macOS as a WireGuard client, not as a server; refer to the WireGuard docs (or to my earlier post) for information on setting up a WireGuard server. I’ll also assume that you’ve installed WireGuard via Homebrew.

Generating Keys

The first step is to generate the public/private keys you’ll need. If the /usr/local/etc/wireguard (or the /opt/homebrew/etc/wireguard for users on an M1-based Mac) directory doesn’t exist, you’ll need to first create that directory. (It didn’t exist Continue reading

Installing Older Versions of Kumactl on an M1 Mac

The Kuma community recently released version 1.2.0 of the open source Kuma service mesh, and along with it a corresponding version of kumactl, the command-line utility for interacting with Kuma. To make it easy for macOS users to get kumactl, the Kuma community maintains a Homebrew formula for the CLI utility. That includes providing M1-native (ARM64) macOS binaries for kumactl. Unfortunately, installing an earlier version of kumactl on an M1-based Mac using Homebrew is somewhat less than ideal. Here’s one way—probably not the only way—to work around some of the challenges.

Note that this post really only applies to users of M1-based Macs; users of Intel-based Macs can extract the kumactl binary from the release archive available linked from the Kuma install docs. (The same goes for users of Linux distributions running on Intel-based hardware.) On the Kuma website, simply select the desired version of Kuma from the drop-down in the upper left, check the page for the direct download link, and off you go. This doesn’t work for M1-based Macs because at the time this post was written, the Kuma community was not providing ARM64 binaries. This leaves Homebrew as the only way (aside Continue reading

Making WireGuard from Homebrew Work on an M1 Mac

After writing the post on using WireGuard on macOS (using the official WireGuard GUI app from the Mac App Store), I found the GUI app’s behavior to be less than ideal. For example, tunnels marked as on-demand would later show up as no longer configured as an on-demand tunnel. When I decided to set up WireGuard on my M1-based MacBook Pro (see my review of the M1 MacBook Pro), I didn’t want to use the GUI app. Fortunately, Homebrew has formulas for WireGuard. Unfortunately, the WireGuard tools as installed by Homebrew on an M1-based Mac won’t work. Here’s how to fix that.

The key issues with WireGuard as installed by Homebrew on an M1-based Mac are:

  • On an M1-based Mac, Homebrew installs (by default) to the /opt/homebrew prefix. By comparison, Homebrew uses /usr/local on Intel-based Macs. Some of the WireGuard-related scripts are hard-coded to use /usr/local as the Homebrew prefix. Because the prefix has changed, though, these scripts now don’t work on an M1-based Mac.
  • WireGuard has a dependency on Bash. Unfortunately, the version of Bash supplied by macOS isn’t supported by WireGuard (it’s too old). Without a very specific PATH configuration, even installing the Homebrew version of Bash—which is Continue reading

Kubernetes Port Names and Terminating HTTPS Traffic on AWS

I recently came across something that wasn’t immediately intuitive with regard to terminating HTTPS traffic on an AWS Elastic Load Balancer (ELB) when using Kubernetes on AWS. At least, it wasn’t intuitive to me, and I’m guessing that it may not be intuitive to some other readers as well. Kudos to my teammates Hart Hoover and Brent Yarger for identifying the resolution, which I’m going to call out in this post.

This AWS Premium Support post outlines the basic scenario:

  • You’re running Kubernetes on AWS. The post references EKS, but as far as I know the issue is not limited to EKS, and should apply to self-managed Kubernetes clusters on AWS (assuming these clusters are configured with the AWS cloud provider).
  • You’ve published a Service of type LoadBalancer (which, in turn, creates a classic ELB). For self-managed clusters, this requires the AWS cloud provider to be installed and configured.
  • You want to terminate HTTPS traffic on the ELB. The post references the use of an ACM certificate, but I suspect it’s not limited to ACM certificates.

Consider the following YAML, taken directly from the previously-referenced AWS Premium Support article:

apiVersion: v1
kind: Service
metadata:
  name:  Continue reading

Technology Short Take 141

Welcome to Technology Short Take #141! This is the first Technology Short Take compiled, written, and published entirely on my M1-based MacBook Pro (see my review here). The collection of links shared below covers a fairly wide range of topics, from old Sun hardware to working with serverless frameworks in the public cloud. I hope that you find something useful here. Enjoy!

Networking

Review: Logitech Ergo K860 Ergonomic Keyboard

As part of an ongoing effort to refine my work environment, several months ago I switched to a Logitech Ergo K860 ergonomic keyboard. While I’m not a “keyboard snob,” I am somewhat particular about the feel of my keyboard, so I wasn’t sure how I would like the K860. In this post, I’ll provide my feedback, and provide some information on how well the keyboard works with both Linux and macOS.

Setup

Setting up the K860 is remarkably easy. The first system I tried to pair it with was an older Mac Pro workstation, and apparently the Bluetooth hardware on that particular workstation wasn’t new enough to support the K860 (Logitech indicates that Bluetooth 5.0 is needed; more on that in a moment). Instead, I popped in the USB-A wireless receiver, and was up and running with the K860 less than a minute later. This was using macOS, but the Mac Pro also dual-booted Linux, so I rebooted into Linux and found that the K860 with the Logitech-supplied USB receiver continued to work without any issues.

Linux, macOS, and Dual Boot Support

The key takeaway regarding Linux is this: if you’re interested in getting the K860 for use with Continue reading

Review: 2020 M1-Based MacBook Pro

I hadn’t done a personal hardware refresh in a while; my laptop was a 2017-era MacBook Pro (with the much-disliked butterfly keyboard) and my tablet was a 2014-era iPad Air 2. Both were serviceable but starting to show their age, especially with regard to battery life. So, a little under a month ago, I placed an order for some new Apple equipment. Included in that order was a new 2020 13" MacBook Pro with the Apple-designed M1 CPU. In this post, I’d like to provide a brief review of the 2020 M1-based MacBook Pro based on the past month of usage.

The “TL;DR” of my review is this: the new M1-based MacBook Pro offers impressive performance and even more impressive battery life. While the raw performance may not “blow away” its 2020 Intel-based counterpart—at least, it didn’t in my real-world usage—the M1-based MacBook Pro offered consistently responsive performance with a battery life that easily blew past any other laptop I’ve ever used, bar none.

Read on for more details.

Hardware

The build quality is really good, with a significant improvement in keyboard quality relative to the earlier butterfly keyboard models (such as my 2017-era MacBook Pro). However, the overall design Continue reading

The Next Step

The Greek philosopher Heraclitus is typically attributed as the creator of the well-known phrase “Change is the only constant.” Since I left VMware in 2018 to join Heptio, change has been my companion. First, there was the change of focus, moving to a focus on Kubernetes and related technologies. Then there was the acquisition of Heptio by VMware, and all the change that comes with an acquisition. Just when things were starting to settle down, along came the acquisition of Pivotal by VMware and several more rounds of changes as a result. Today, I mark the start of another change, as I begin a new role and take the next step in my career journey.

Last week, I announced via Twitter that I was leaving VMware to explore a new opportunity. Today, I start at Kong, Inc., as a Principal Field Engineer. Kong, if you aren’t already familiar, is a company focused on service connectivity for modern architectures, with products like their eponymous API gateway and the Envoy-powered Kuma service mesh. I’m really looking forward to getting much more familiar with Envoy, the Kong API gateway, Kuma, and related projects and technologies. I still get to be Continue reading

Technology Short Take 140

Welcome to Technology Short Take #140! It’s hard to believe it’s already the start of May 2021—my how time flies! In this Technology Short Take, I’ve gathered some links for you covering topics like Azure and AWS networking, moving from macOS to Linux (and back again), and more. Let’s jump right into the content!

Networking

Servers/Hardware

Security

Making Firefox on Linux use Private Browsing by Default

While there are a couple different methods to make Firefox use private browsing by default (see this page for a couple methods), these methods essentially force private browsing and disable the ability to use “regular” (non-private) browsing. In this post, I’ll describe what I consider to be a better way of achieving this, at least on Linux.

It’s possible this method will also work on Windows, but I haven’t tested it. If anyone gets a chance to test it and let me know, I’ll update this post and credit you accordingly. Just hit me on Twitter and let me know what you’ve found in your testing. I’ve also only tested this on Fedora, but it should be the same or very similar for any distribution that uses GNOME.

GNOME uses the idea of “desktop files” (typically found in /usr/share/applications or ~/.local/share/applications) to enable the launching of applications via the Activities screen or other mechanisms. (For more information on desktop files, see here.) These desktop files specify where the executable is found, what command-line parameters to use, what icon to use, what name the application should go by, etc. Desktop files also allow application developers or users Continue reading

Technology Short Take 139

Welcome to Technology Short Take #139! This Technology Short Take is a bit heavy on cloud, OS, and programming topics, but there should be enough other interesting links to be useful to plenty of folks. (At least, I hope that’s the case!) Now, let’s get on to the content!

Networking

  • Tony Mackay has a tutorial showing how to use Traefik to rate-limit requests to a WordPress instance.
  • Ali Al Idrees has a post on using NSX ALB (formerly Avi Networks) with Kubernetes clusters in a vSphere with Tanzu environment.
  • This post provides some examples of shared control planes (and thus shared failure domains) within networking.
  • In this post, Jakub Sitnicki digs way deep into the Linux kernel to uncover the answer to the question, “Why are there no entries in the conntrack table for SYN packets dropped by the firewall?” Get ready to get nerdy!
  • This article on eBPF and Isovalent (the company behind the Cilium CNI plugin for Kubernetes) has some statements with which I agree, and some that don’t make sense to me. For example, I agree with the statement that the “impact eBPF will have on networking, security and observability will be widespread”. However, Continue reading
1 2 3 … 15