Scott Lowe

Author Archives: Scott Lowe

Technology Short Take 137

Welcome to Technology Short Take #137! I’ve got a wide range of topics for you this time around—eBPF, Falco, Snort, Kyverno, etcd, VMware Code Stream, and more. Hopefully one of these links will prove useful to you. Enjoy!

Networking

Servers/Hardware

  • I recently mentioned on Twitter that I was considering building out a new Linux PC to replace my aging Mac Pro (it’s a 2012 model, so going on 9 years old). Joe Utter shared with me his new lab build information, and now I’m sharing it with all of you. Sharing is caring, you know.

Security

Cloud Computing/Cloud Management

Technology Short Take 136

Welcome to Technology Short Take #136, the first Short Take of 2021! The content this time around seems to be a bit more security-focused, but I’ve still managed to include a few links in other areas. Here’s hoping you find something useful!

Networking

Servers/Hardware

  • Thinking of buying an M1-powered Mac? You may find this list helpful.

Security

Using Velero to Protect Cluster API

Cluster API (also known as CAPI) is, as you may already know, an effort within the upstream Kubernetes community to apply Kubernetes-style APIs to cluster lifecycle management—in short, to use Kubernetes to manage the lifecycle of Kubernetes clusters. If you’re unfamiliar with CAPI, I’d encourage you to check out my introduction to Cluster API before proceeding. In this post, I’m going to show you how to use Velero (formerly Heptio Ark) to backup and restore Cluster API objects so as to protect your organization against an unrecoverable issue on your Cluster API management cluster.

To be honest, this process is so straightforward it almost doesn’t need to be explained. In general, the process for backing up the CAPI management cluster looks like this:

  1. Pause CAPI reconciliation on the management cluster.
  2. Back up the CAPI resources.
  3. Resume CAPI reconciliation.

In the event of catastrophic failure, the recovery process looks like this:

  1. Restore from backup onto another management cluster.
  2. Resume CAPI reconciliation.

Let’s look at these steps in a bit more detail.

Pausing and Resuming Reconciliation

The process for pausing and resuming reconciliation of CAPI resources is outlined in this separate blog post. To summarize that post here for convenience, the Cluster Continue reading

Details on the New Desk Layout

Over the holiday break I made some time to work on my desk layout, something I’d been wanting to do for quite a while. I’d been wanting to “up my game,” so to speak, with regard to producing more content, including some video content. Inspired by—and heavily borrowing from—this YouTube video, I decided I wanted to create a similar arrangement for my desk. In this post, I’ll share more details on my setup.

I’ll start with the parts list, which contains links to everything I’m using in this new arrangement.

Parts List

When I shared a picture of the desk layout on Twitter, a number of folks expressed interest in the various components that I used. To make it easier for others who may be interested in replicating their own variation of this setup, here are Amazon links for all the parts I used to build this setup (these are not affiliate links):

  1. WALI Extra Tall Single LCD Monitor Fully Adjustable Desk Mount (qty 2)
  2. WALI Single Fully Adjustable Arm (qty 2)
  3. Pergear TH5 Camera Swivel Mini Tripod Ball Head (qty 2)
  4. Pergear TH3 Pro DSLR Camera Tripod Ball Head
  5. PULUZ Metal Handheld Adjustable 3/8” Tripod/Monopod Extension
  6. Continue reading

Technology Short Take 135

Welcome to Technology Short Take #135! This will likely be the last Technology Short Take of 2020, so it’s a tad longer than usual. Sorry about that! You know me—I just want to make sure everyone has plenty of technical content to read during the holidays. And speaking of holidays…whatever holidays you do (or don’t) celebrate, I hope that the rest of the year is a good one for you. Now, on to the content!

Networking

  • Arthur Chiao cracks open kube-proxy, a key part of Kubernetes networking, to expose the internals, and along the way exposes readers to a few different technologies. This is a good read if you’re trying to better understand some aspects of Kubernetes networking.
  • Gian Paolo takes a look at using tools like curl and jq when working with networking-related APIs.
  • It’s not unusual to see “networking professionals need to learn developer tools,” but how often do you see “developers need to learn these networking tools”? Martin Heinz discusses that very topic in this post.

Servers/Hardware

Continue reading

Bootstrapping a Cluster API Management Cluster

Cluster API is, if you’re not already familiar, an effort to bring declarative Kubernetes-style APIs to Kubernetes cluster lifecycle management. (I encourage you to check out my introduction to Cluster API post if you’re new to Cluster API.) Given that it is using Kubernetes-style APIs to manage Kubernetes clusters, there must be a management cluster with the Cluster API components installed. But how does one establish that management cluster? This is a question I’ve seen pop up several times in the Kubernetes Slack community. In this post, I’ll walk you through one way of bootstrapping a Cluster API management cluster.

The process I’ll describe in this post is also described in the upstream Cluster API documentation (see the “Bootstrap & Pivot” section of this page).

At a high level, the process looks like this:

  1. Create a temporary bootstrap cluster.
  2. Make the bootstrap cluster into a temporary management cluster.
  3. Use the temporary management cluster to establish a workload cluster (through Cluster API).
  4. Convert the workload cluster into a permanent management cluster.
  5. Remove the temporary bootstrap cluster.

The following sections describe each of these steps in a bit more detail.

Create a Temporary Bootstrap Cluster

The first step is Continue reading

Some Site Updates

For the last three years, the site has been largely unchanged with regard to the structure and overall function even while I continue to work to provide quality technical content. However, time was beginning to take its toll, and some “under the hood” work was needed. Over the Thanksgiving holiday, I spent some time updating the site, and there are a few changes I wanted to mention.

  1. The site has been updated to use a much more recent version of Hugo. This change is largely invisible to readers, but a couple of the site changes are related to this upgrade. Specifically…
  2. Although the main RSS feed for the site (found here) remains a full content feed, I ran into issues getting Hugo to use my custom RSS templates for generating the category and tag feeds (for example, the RSS feed for the “Tutorial” category, or the RSS feed for the “Kubernetes” tag). You’ll now find that the category and tag feeds are summary feeds only as opposed to full content feeds. I do intend to restore them to full content feeds as soon as possible.
  3. Finally, I’ve updated the “metadata line” when viewing a single article Continue reading

Assigning Node Labels During Kubernetes Cluster Bootstrapping

Given that Kubernetes is a primary focus of my day-to-day work, I spend a fair amount of time in the Kubernetes Slack community, trying to answer questions from users and generally be helpful. Recently, someone asked about assigning node labels while bootstrapping a cluster with kubeadm. I answered the question, but afterward started thinking that it might be a good idea to also share that same information via a blog post—my thinking being that others who also had the same question aren’t likely to be able to find my answer on Slack, but would be more likely to find a published blog post. So, in this post, I’ll show how to assign node labels while bootstrapping a Kubernetes cluster.

The “TL;DR” is that you can use the kubeletExtraArgs field in a kubeadm configuration file to pass the node-labels command to the Kubelet, which would allow you to assign node labels when kubeadm bootstraps the node. Read on for more details.

Testing with Kind

kind is a great tool for testing this sort of configuration, since kind uses kubeadm to bootstrap its nodes. If you aren’t familiar with kind, I encourage you to visit the kind website; in Continue reading

Pausing Cluster API Reconciliation

Cluster API is a topic I’ve discussed here in a number of posts. If you’re not already familiar with Cluster API (also known as CAPI), I’d encourage you to check out my introductory post on Cluster API first; you can also visit the official Cluster API site for more details. In this short post, I’m going to show you how to pause the reconciliation of Cluster API cluster objects, a task that may be necessary for a variety of reasons (including backing up the Cluster API objects in your management cluster).

Since CAPI leverages Kubernetes-style APIs to manage Kubernetes cluster lifecycle, the idea of reconciliation is very important—it’s a core Kubernetes concept that isn’t at all specific to CAPI. This article on level triggering and reconciliation in Kubernetes is a great article that helps explain reconciliation, as well as a lot of other key concepts about how Kubernetes works.

When reconciliation is active, the controllers involved in CAPI are constantly evaluating desired state and actual state, and then reconciling differences between the two. There may be times when you need to pause this reconciliation loop. Fortunately, CAPI makes this pretty easy: there is a paused field that allows users Continue reading

Technology Short Take 134

Welcome to Technology Short Take #134! I’m publishing a bit early this time due to the Thanksgiving holiday in the US. So, for all my US readers, here’s some content to peruse while enjoying some turkey (or whatever you’re having this year). For my international readers, here’s some content to peruse while enjoying dramatically lower volumes of e-mail because the US is on holiday. See, something for everyone!

Networking

Security

  • I’m glad to see this. Open source has become so critical to so many aspects of our computing infrastructure.
  • OpenCSPM looks like it could be quite a useful tool. I haven’t yet had time to dig in and get familiar with the details, but what I have seen so far looks good.
  • Uh oh…more hardware exploits.
  • The macOS OCSP fiasco generated quite a bit of attention and analysis (see here and here).

Cloud Computing/Cloud Management

Review: CPLAY2air Wireless CarPlay Adapter

In late September, I was given a CPLAY2air wireless CarPlay adapter as a gift. Neither of my vehicles support wireless CarPlay, and so I was looking forward to using the CPLAY2air device to enable the use of CarPlay without having to have my phone plugged into a cable. Here’s my feedback on the CPLAY2air device after about six weeks of use.

In general, the device works reasonably well. Setup with the factory radio in my GMC Sierra truck was pretty straightforward, and only took a few minutes. Since then, the device connects to my phone every time I start the vehicle, and all CarPlay functions—music, maps, Siri, sending/reading messages, etc.—all work as expected. I can leave my phone in my pocket and still gain all the benefits of CarPlay, which is incredibly convenient.

The CPLAY2air is not without a few caveats, however. The wireless connection between the CPLAY2air and my iPhone does introduce some noticeable latency. When switching music tracks, for example, it will take between one and three seconds to stop playing the old track and start playing the new track. (By comparison, the same operation using traditional wired CarPlay is nearly instantaneous.) Similarly, when talking on Continue reading

Resizing Windows to a Specific Size on macOS

I recently had a need (OK, maybe more a desire than a need) to set my browser window(s) on macOS to a specific size, like 1920x1080. I initially started looking at one of the many macOS window managers, but after reading lots of reviews and descriptions and still being unclear if any of these products did what I wanted, I decided to step back to using AppleScript to accomplish what I was seeking. In this post, I’ll share the solution (and the articles that helped me arrive at the solution).

My first stop was this blog post by Ethan Banks. I tried replicating the AppleScript he used, but couldn’t get it to work. I’m still running macOS 10.14 “Mojave,” so perhaps his code was specific to macOS 10.15 “Catalina.” I moved on, never realizing there was another section to his post that had the information I needed (and would eventually find). Let that be a lesson to be sure to read the entire post next time.

Moving on, I arrived at this post. OK, this used a different mechanism than Ethan’s post. I tried it, and it sort of worked, but it didn’t create the window geometry Continue reading

Technology Short Take 133

Welcome to Technology Short Take #133! This time around, I have a collection of links featuring the new Raspberry Pi 400, some macOS security-related articles, information on AWS Nitro Enclaves and gVisor, and a few other topics. Enjoy!

Networking

Servers/Hardware

  • The Raspberry Pi 400 is a neat offering. See this post for more details.

Security

Technology Short Take 132

Welcome to Technology Short Take #132! My list of links and articles from around the web seems to be a bit heavy on security-related topics this time. Still, there’s a decent collection of networking, cloud computing, and virtualization articles as well as a smattering of other topics for you to peruse. I hope you find something useful!

Networking

  • I think a fair number of folks may not be aware that the Nginx ingress controller for Kubernetes—both the community version and the Nginx-maintained open source version—do suffer from timeouts and errors resulting from changes in the back-end application’s list of endpoints (think pods being added or removed). This performance testing post lays out all the details. In particular, see the section titled “Timeout and Error Results for the Dynamic Deployment.”
  • Ivan Pepelnjak attempts to answer the question, “How much do I need to know about Linux networking?”
  • Speaking of Linux networking…Marek Majkowski of Cloudflare digs deep into conntrack, used for stateful firewalling functionality.

Servers/Hardware

  • Normally I talk about server hardware and such here, but with so much moving to public cloud providers, let’s expand that focus a little bit: in this post, Jeramiah Dooley provides his perspective Continue reading

Considerations for using IaC with Cluster API

In other posts on this site, I’ve talked about both infrastructure-as-code (see my posts on Terraform or my posts on Pulumi) and somewhat separately I’ve talked about Cluster API (see my posts on Cluster API). And while I’ve discussed the idea of using existing AWS infrastructure with Cluster API, in this post I wanted to try to think about how these two technologies play together, and provide some considerations for using them together.

I’ll focus here on AWS as the cloud provider/platform, but many of these considerations would also apply—in concept, at least—to other providers/platforms.

In no particular order, here are some considerations for using infrastructure-as-code and Cluster API (CAPI)—specifically, the Cluster API Provider for AWS (CAPA)—together:

  • If you’re going to need the CAPA workload clusters to have access to other AWS resources, like applications running on EC2 instances or managed services like RDS, you’ll need to use the additionalSecurityGroups functionality, as I described in this blog post.
  • The AWS cloud provider requires certain tags to be assigned to resources (see this post for more details), and CAPI automatically provisions new workload clusters with the AWS cloud provider when running on AWS. Thus, you’ll want to make Continue reading

Technology Short Take 131

Welcome to Technology Short Take #131! I’m back with another collection of articles on various data center technologies. This time around the content is a tad heavy on the security side, but I’ve still managed to pull in articles on networking, cloud computing, applications, and some programming-related content. Here’s hoping you find something useful here!

Networking

  • This recent Ars Technica article points out that a feature in Chromium—the open source project leveraged by Chrome and Edge, among others—is having a significant impact on root DNS traffic. More technical details can be found in an associated APNIC blog post.
  • Here’s a few details around Open Service Mesh.
  • Quentin Machu outlines a series of problems his company experienced using Weave Net as the CNI for their Kubernetes clusters, as well as describes the migration process to a new CNI. His blog post is well worth a read, IMO.

Security

Updating AWS Credentials in Cluster API

I’ve written a bit here and there about Cluster API (aka CAPI), mostly focusing on the Cluster API Provider for AWS (CAPA). If you’re not yet familiar with CAPI, have a look at my CAPI introduction or check the Introduction section of the CAPI site. Because CAPI interacts directly with infrastructure providers, it typically has to have some way of authenticating to those infrastructure providers. The AWS provider for Cluster API is no exception. In this post, I’ll show how to update the AWS credentials used by CAPA.

Why might you need to update the credentials being used by CAPA? Security professionals recommend that users rotate credentials on a regular basis, and when those credentials get rotated you’ll need to update what CAPA is using. There are other reasons, too; perhaps you started with one set of credentials but now want to move to a different set of credentials. Fortunately, the process for updating the CAPA credentials isn’t too terribly tedious.

CAPA stores the credentials it uses as a Secret in the “capa-system” namespace. You can use kubectl -n capa-system get secrets and you’ll see the “capa-manager-bootstrap-credentials” Secret. The credentials themselves are stored as a key named credentials; you Continue reading

Behavior Changes in clusterawsadm 0.5.5

Late last week I needed to test some Kubernetes functionality, so I thought I’d spin up a test cluster really quick using Cluster API (CAPI). As often happens with fast-moving projects like Kubernetes and CAPI, my existing CAPI environment had gotten a little out of date. So I updated my environment, and along the way picked up an important change in the default behavior of the clusterawsadm tool used by the Cluster API Provider for AWS (CAPA). In this post, I’ll share more information on this change in default behavior and the impacts of that change.

The clusterawsadm tool is part of CAPA and is used to help manage AWS-specific aspects, particularly around credentials and IAM (Identity and Access Management). As outlined in this doc, users use clusterawsadm to create a CloudFormation stack that prepares an AWS account for use with CAPA. This stack contains roles and policies that enable CAPA to function as expected.

Here’s the change in default behavior:

  • In clusterawsadm 0.5.4 and earlier, using clusterawsadm to create or update the CloudFormation stack would also create a bootstrap IAM user and group by default.
  • In clusterawsadm 0.5.5 and later, creating or updating the Continue reading

Technology Short Take 130

Welcome to Technology Short Take #130! I’ve had this blog post sitting in my Drafts folder waiting to be published for almost a month, and I kept forgetting to actually make it live. Sorry! So, here it is—better late than never, right?

Networking

Security

Cloud Computing/Cloud Management

Creating an AWS ELB using Pulumi and Go

In case you hadn’t noticed, I’ve been on a bit of a kick with Pulumi and Go recently. There are two reasons for this. First, I have a number of “learning projects” (things that I decide I’d like to try or test) that would benefit greatly from the use of infrastructure as code. Second, I’ve been working on getting more familiar with Go. The idea of combining both those reasons by using Pulumi with Go seemed natural. Unfortunately, examples of using Pulumi with Go seem to be more limited than examples of using Pulumi with other languages, so in this post I’d like to share how to create an AWS ELB using Pulumi and Go.

Here’s the example code:

elb, err := elb.NewLoadBalancer(ctx, "elb", &elb.LoadBalancerArgs{
	NamePrefix:             pulumi.String(baseName),
	CrossZoneLoadBalancing: pulumi.Bool(true),
	AvailabilityZones:      pulumi.StringArray(azNames),
	Instances:              pulumi.StringArray(cpNodeIds),
	HealthCheck: &elb.LoadBalancerHealthCheckArgs{
		HealthyThreshold:   pulumi.Int(3),
		Interval:           pulumi.Int(30),
		Target:             pulumi.String("SSL:6443"),
		UnhealthyThreshold: pulumi.Int(3),
		Timeout:            pulumi.Int(30),
	},
	Listeners: &elb.LoadBalancerListenerArray{
		&elb.LoadBalancerListenerArgs{
			InstancePort:     pulumi.Int(6443),
			InstanceProtocol: pulumi.String("TCP"),
			LbPort:           pulumi.Int(6443),
			LbProtocol:       pulumi.String("TCP"),
		},
	},
	Tags: pulumi.StringMap{
		"Name": pulumi.String(fmt.Sprintf("cp-elb-%s", baseName)),
		k8sTag: pulumi.String("shared"),
	},
})

You can probably infer from the code above that this Continue reading

1 … 4 5 6 7 8 … 19