Scott Lowe

Author Archives: Scott Lowe

Revisiting X.509 Certificates in Kubeconfig Files

In 2018, I wrote an article on examining X.509 certificates embedded in Kubeconfig files. In that article, I showed one way of extracting client certificate data from a Kubeconfig file and looking at the properties of the client certificate data. While there’s nothing technically wrong with that article, since then I’ve found another tool that makes the process a tad easier. In this post, I’ll revisit the topic of examining embedded X.509v3 certificates in Kubeconfig files.

The tool that I’ve found is yq, which is an incredibly useful tool when it comes to parsing YAML (much in the same way that jq is an incredibly useful tool when it comes to parsing JSON). I should probably write some sort of introductory post on yq.

In any case, you can use yq to replace the grep plus awk combo outlined in my earlier article on examining certificate data in Kubeconfig files. Instead, to pull out only the client certificate data, just use this yq command (you did know that Kubeconfig files are YAML, right?):

yq '.users[0].user.client-certificate-data' < ~./kube/config

(Of course, this command assumes your Kubeconfig file is named config in the ~/.kube Continue reading

Posts from the Past, August 2022

I thought I might start highlighting some older posts here on the site through a semi-regular “Posts from the Past” series. I’ll start with posts published in the month of August through the years. Here’s hoping you find something that is useful (or perhaps entertaining, at least)!

August 2021

Last year, I had a couple of posts that I think are still relevant today. First, I talked about using Pulumi with Go to create a VPC Peering relationship on AWS. Second, I showed readers how to have Wireguard interfaces start automatically (using launchd) on macOS.

August 2020

I didn’t write too much in August 2020; my wife and I took a big road trip around the US to visit family and such. However, I did publish a post on some behavior changes in version 0.5.5 of the Cluster API tool clusterawsadm.

August 2019

This was a busy month for the blog! In addition to two Technology Short Takes, I also published posts on converting Kubernetes to an HA control plane, reconstructing the kubeadm join command (in the event you didn’t write down the output of kubeadm init), and one introducing Cluster API.

August 2018

In Continue reading

Site Category Changes

This weekend I made a couple of small changes to the categories on the site, in an effort to make navigation a bit more intuitive. In the past, readers had expressed some confusion over the “Education” and “Explanation” categories, and—to be frank—their confusion was warranted. I also wasn’t clear on the distinction between those categories, so this post explains the changes I’ve made.

The following category changes are now in effect on the site:

  • First, the “Education” category has been completely removed. I try to make almost everything on this site educational in nature, so why have an “Education” category? This really only affects you if you’d subscribed to that category’s RSS feed. (Did you know that every category and every tag has its own RSS feed?)
  • A lot of the content from the “Education” category has been moved into the “Explanation” category. This is the category that will contain posts where I provide some level of explanation around a concept, technology, product, or project.
  • The “Tutorial” category also picked up some new articles from the now-eliminated “Education” category. The “Tutorial” category contains walkthroughs or step-by-step instructions for doing something. There’s most likely going to be explanation along the Continue reading

Using Default AWS Resources with Pulumi

Per the AWS documentation (although I’m sure there are exceptions), when you start using AWS you are given some automatically-created resources: a default VPC that contains public subnets in each availability zone in the region along with an Internet gateway and settings to enable DNS resolution. Most of the infrastructure-as-code tutorials that I’ve seen start with creating a VPC and subnets and gateway, but what if you wanted to use these default resources instead? I wasn’t really able to find a good walkthrough on how to do this, so this post provides some sample Go code you can use with Pulumi to identify these default AWS resources and use them.

I’ll approach this from the perspective of wanting to launch an EC2 instance in the default infrastructure that AWS provides for you in a region. To launch an EC2 instance using Pulumi (and most other infrastructure-as-code tools), there are several pieces of information you need:

  1. An AMI ID
  2. The instance type
  3. The name of an SSH keypair that’s been uploaded to/created in AWS
  4. A subnet ID
  5. A security group ID

The first three are probably things you’ll want to parameterize (i.e., make it possible for you to pass Continue reading

Technology Short Take 157

Welcome to Technology Short Take 157! I hope that this collection of links I’ve gathered is useful to someone out there. In particular, the “Career/Soft Skills” section is a bit bigger than usual this time around, as is the “Security” section.

Networking

  • Interested in understanding how NAT Traversal works? David Anderson’s post on how NAT Traversal works should help.
  • This happened a couple of months ago, but I don’t think I’ve linked to it in a Technology Short Take: the Envoy Proxy open source project announced Envoy Gateway, a “new member of the Envoy Proxy family aimed at significantly decreasing the barrier to entry when using Envoy for API Gateway (sometimes known as ’north-south’) use cases”.
  • This is a slightly older article from Ivan Pepelnjak on using netsim-tools to build Vagrant boxes, but let’s be real—his stuff is kind of timeless anyway, right?

Security

Network Programmability and Automation, Second Edition

In late 2015, I was lucky enough to be part of a small crew of authors who launched a new book project targeting “next-generation network engineering skills.” That book, Network Programmability and Automation, was published by O’Reilly and has garnered praise and accolades for tackling head-on the topics that network engineers should consider mastering as the field of network engineering continues to grow and evolve. I was excited about that announcement, and I’m even more excited to announce that the early release of the second edition of Network Programmability and Automation is now available!

2nd edition book cover

The original team of authors—Jason Edelman, Matt Oswalt, and myself—are joined this time around by Christian Adell. Christian works with Jason at Network to Code, and it has been a tremendous pleasure to get to know Christian (a little bit, at least!) as part of this project so far. I am impressed with his knowledge and experience, and I think it really adds to the book. Jason and Matt, of course, need no introductions; they are both industry leaders and are well-known in the network automation space.

Check out Jason and Christian’s announcement blog post here.

I am, once again, humbled and honored Continue reading

Technology Short Take 156

Welcome to Technology Short Take #156! It’s been about a month since the last Technology Short Take, and in that time I’ve been gathering links that I wanted to share with my readers. (I still have quite the backlog of links to read!) Hopefully something I share here will prove useful to someone. Enjoy the links below, and enjoy your weekend!

Networking

  • I’d never heard of Pipy before seeing it in this article, but it look like it could be quite useful for a number of use cases.
  • William Morgan, one of the creators of Linkerd, has a lengthy treatise on eBPF, sidecars, and the future of the service mesh. As a (relative) layperson—meaning I’m not an eBPF expert—I don’t know if I should believe the eBPF cheerleaders (some of whom I know personally and are familiar with their technical expertise) or folks like William who have clearly “been there, done that” with service mesh. I certainly think there’s a place for eBPF in service meshes, but I’m not yet on board with sidecar-less service meshes (or per-node proxy models).

Security

  • BPFDoor, as it is known, is a passive backdoor that allows threat actors to remotely connect Continue reading

Making Flatpak Firefox use Private Browsing by Default

In April 2021 I wrote a post on making Firefox use Private Browsing by default, in which I showed how to modify the GNOME desktop file so that Firefox would open private windows by default without restricting access to normal browsing windows and functionality. I’ve used that technique on all my Fedora-based systems since that time, until just recently. What happened recently, you ask? I switched to the Flatpak version of Firefox. Fortunately, with some minor tweaks, this technique works with the Flatpak version of Firefox as well. In this post, I’ll share with you the changes needed to make the Flatpak version of Firefox also use private browsing by default.

When working with the non-Flatpak version of Firefox, the GNOME desktop file installed with the Firefox package is found at /usr/share/applications. In my earlier article, I suggested editing that file to add the --private-window parameter to the Exec line. Unfortunately, that change gets overwritten every time the Firefox package is updated. It’s better, actually, to use a locally customized desktop file placed in ~/.local/share/applications instead, which will take precedence over the shared desktop file.

With the Flatpak version of Firefox, there is still a shared Continue reading

Git Difftool and Meld as a Flatpak

I’ve recently started migrating many of the applications on my Fedora 36 laptop to their Flatpak versions. For the most part, this has been pretty straightforward, although there isn’t really any method for migrating configuration and data. Today I ran into a problem with Meld, a graphical diff utility, and using it with the git difftool command. Below I’ll share how I worked around this problem.

Normally, the integration between Git and Meld—which is what enables you to run git difftool and have the results show up in Meld—would look something like this (this is from ~/.gitconfig):

[merge]
    tool = meld
[diff]
    tool = meld
[difftool]
    prompt = no
[difftool "meld"]
    cmd = /usr/bin/meld "$LOCAL" "$REMOTE"
[mergetool "meld"]
    cmd = /usr/bin/meld "$LOCAL" "$REMOTE"

However, when Meld is installed as a Flatpak, /usr/bin/meld doesn’t exist. In order to continue using Meld with the git difftool command, you must change the Git configuration to look like this instead:

[merge]
    tool = meld
[diff]
    tool = meld
[difftool]
    prompt = no
[difftool "meld" Continue reading

Technology Short Take 155

Welcome to Technology Short Take #155, just in time for the 2022 Memorial Day holiday weekend! (Here in the US, at least.) I mean, don’t you want to spend this weekend catching up on some technology-related articles instead of cooking on the grill and gathering with friends and family? I certainly hope not! Still, for those who need a little technology fix over the weekend, hopefully I’ve included something useful in the list of articles below. Enjoy!

Networking

  • Isovalent—the company behind the Cilium project—has been talking a lot about how the use of eBPF will transform things, including the architecture of a service mesh. Along those lines, one of their latest articles discusses how to achieve identity-based mutual authentication leveraging eBPF. If I’m understanding the article correctly (and feel free to correct me if I am mistaken) it looks as if Cilium Service Mesh will leverage/does leverage a combination of certificate-based mTLS for identity at the workload level and node-based transport encryption (via WireGuard) for data confidentiality. Even though I know that the underlying mechanisms are different, subjectively this feels a lot like using tunnels to connect workloads on different compute nodes (i.e., network virtualization). Is the Continue reading

Fine-Tuning Control Plane Access with Cluster API

When Cluster API creates a workload cluster, it also creates a load balancing solution to handle traffic to the workload cluster’s control plane. This is necessary so that the control plane endpoint is decoupled from the underlying control plane nodes (which facilitates scaling the control plane, among other things). On AWS, this mean creating an ELB and a set of security groups. For flexibility, Cluster API provides a limited ability to customize this control plane load balancer. In this post, I’ll show you how to use this functionality to fine-tune access to a workload cluster’s control plane when using Cluster API with AWS.

If you’re not familiar with Cluster API (hereafter just referred to as “CAPI”), then my introduction to CAPI article may be useful. Keep in mind that article was written in 2019, while the project was still in its early stages. The high-level concepts are correct, but some of the details may have shifted slightly over the last three years as the project progressed from v1alpha1 APIs to the now-current v1beta1 APIs.

The key here is the controlPlaneLoadBalancer object, which is part of the AWSCluster object (see details here in the code or here via pkg.go.dev Continue reading

Technology Short Take 154

Welcome to Technology Short Take #154! My link of links and articles from around the Internet is a bit light on networking and virtualization this time around, but heftier in the security, cloud, and OS/application sections. I hope that I’ve managed to include something that you’ll find useful. Enjoy the content!

Networking

  • Lucas Pardue and Christopher Wood share a primer on proxies. (Hat tip to Matt Oswalt for putting this on my Twitter timeline.)
  • Ivan Pepelnjak talks about the 1.2.1 release of netsim-tools in this blog post.

Servers/Hardware

  • It seems like silicon photonics is something the industry has been talking about for years, but something that never seems to come to fruition. The Next Platform recently chatted with Andy Bechtolsheim about it; you can read the results of that discussion here.

Security

Technology Short Take 153

Welcome to Technology Short Take #153! My personal and professional life has kept me busy over the last couple of months, so things have been quiet here on the blog. I’ve still been collecting links to share with you, though, and here’s the latest collection. I hope you’re able to find something useful here!

Networking

  • This article contains some good information on IPv6 for those who are just starting to get more familiar with it, although toward the end it turns into a bit of an advertisement.
  • Want to understand kube-proxy, a key part of Kubernetes networking, a bit better? Start here. Arthur Chiao’s post on cracking kube-proxy is also an excellent resource—in fact, there’s so much information packed in there you may need to read it more than once.
  • Xavier Avrillier walks readers through using Antrea (a Kubernetes CNI built on top of Open vSwitch—a topic I’ve touched on a time or two) to provide on-premise load balancing in Kubernetes.

Servers/Hardware

  • Cabling is hardware, right? What happens to submarine cables when there are massive events, like a volcanic eruption? Ulrich Speidel shares some of the findings after the volcanic eruption in Tonga.

Security

Technology Short Take 152

Welcome to Technology Short Take #152! Normally I’d publish a Technology Short Take in the morning on a Friday, but I really wanted to get this one out so I’m making it live late in the day on a Monday. Here’s hoping I’ve included some content below that you find useful!

Networking

  • I was (and am) familiar with RFC 1918 and the concept of non-routable address spaces. However, I was not familiar with the term “bogons” to refer to such prefixes that should not be publicly routed. Thanks to this article, that oversight is now corrected. Oh, and the article shares a handy Python script to help implement bogon filtering in NSX-T.
  • Koyeb describes, at a high level, the global networking stack for their serverless platform. Components involved include the open source Kuma service mesh (in turn leveraging Envoy), anycast BGP, and mutual TLS (mTLS).
  • Ivan Pepelnjak does a great job of describing all the things you really shouldn’t do (or don’t really need to do) when trying to deal with migrating container hosts in a data center fabric. In truth, the answer is exactly as Ivan says at the top of the article: when it comes to Continue reading

Using cert-manager with Kuma for mTLS

When configuring mutual TLS (mTLS) on the open source Kuma service mesh, users have a couple of different options. They can use a “builtin” certificate authority (CA), in which Kuma itself will generate a CA certificate and key for use in creating service-specific mTLS certificates. Users also have the option of using a “provided” CA, in which they must supply a CA certificate and key for Kuma to use when creating service-specific mTLS certificates. Both of these options are described on this page in the Kuma documentation. In this post, I’d like to explore the use of cert-manager as a “provided” CA for mTLS on Kuma.

Currently, Kuma lacks direct integration with cert-manager, so the process is a bit more manual than I’d prefer. If direct cert-manager integration is something you’d find useful, please consider opening an issue to that effect on the Kuma GitHub repository.

Assuming you have cert-manager installed already, the process for using cert-manager as the CA for a “provided” CA mTLS backend looks like this:

  1. Define the root CA in cert-manager.
  2. Prepare the secrets for Kuma.
  3. Configure the Kuma mesh object for mTLS.

I know these steps are really too high level to be useful Continue reading

Follow Up: Bootstrapping Servers into Ansible

Seven years ago, I wrote a quick post on bootstrapping servers into Ansible. The basic gist of the post was that you can use variables on the Ansible command-line to specify hosts that aren’t part of your inventory or log in via a different user (useful if the host doesn’t yet have a dedicated Ansible user account because you want to use Ansible to create that account). Recently, though, I encountered a situation where this approach doesn’t work, and in this post I’ll describe the workaround.

In one of the Slack communities I frequent, someone asked about using the approach described in the original blog post. However, they were having issues connecting. Specifically, this error was cropping up in the Ansible output (names have been changed to protect the innocent):

fatal: [new-server.int.domain.test]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: [email protected]: Permission denied (publickey,password).", "unreachable": true}

Now, this is odd, because the Ansible command-line being executed included the parameters I mentioned in the original blog post:

ansible-playbook bootstrap.yml -i inventory/hosts -K --extra-vars "hosts=new-server.int.domain.test user=john"

For some reason, though, it was ignoring that parameter and Continue reading

Technology Short Take 151

Welcome to Technology Short Take #151, the first Technology Short Take of 2022. I hope everyone had a great holiday season and that 2022 is off to a wonderful start! I have a few more links than normal this time around, although I didn’t find articles in a couple categories. Don’t worry—I’ll keep my eyes peeled and my RSS reader ready to pull in new articles in those categories for next time. And now for the content!

Networking

Servers/Hardware

Getting Certificate Details from HashiCorp Vault

It seems there are lots of tutorials on setting up a PKI (public key infrastructure) using HashiCorp Vault. What I’ve found missing from most of these tutorials, however, is how to get details on certificates issued by a Vault-driven PKI after the initial creation. For example, someone other than you issued a certificate, but now you need to get the details for said certificate. How is that done? In this post, I’ll show you a couple ways to get details on certificates issued and stored in HashiCorp Vault.

For the commands and API calls I’ve shared below, I’m using “pki” as the name/path you (or someone else) assigned to a PKI secrets engine within Vault. If you’re using a different name/path, then be sure to substitute the correct name/path as appropriate.

To use the Vault CLI to see the list of certificates issued by Vault, you can use this command:

vault list pki/certs

This will return a list of the serial numbers of the certificates issued by this PKI. Looking at just serial numbers isn’t terribly helpful, though. To get more details, you first need to read the certificate details (note singular “cert” here versus plural “certs” in the previous Continue reading

Using Test-Driven Development for Kustomize Overlays

I am by no means a developer (not by a long shot!), but I have been learning lots of development-related things over the last several years and trying to incorporate those into my workflows. One of these is the idea of test-driven development (see Wikipedia for a definition and some additional information), in which one writes tests to validate functionality before writing the code to implement said functionality (pardon the paraphrasing). In this post, I’ll discuss how to use conftest to (loosely) implement test-driven development for Kustomize overlays.

If you’re unfamiliar with Kustomize, then this introductory article I wrote will probably be useful.

For the discussion around using the principles of test-driven development for Kustomize overlays, I’ll pull in a recent post I did on creating reusable YAML for installing Kuma. In that post, I pointed out four changes that needed to be made to the output of kumactl install control-plane to make it reusable:

  1. Remove the caBundle value for all webhooks.
  2. Annotate all webhooks so that cert-manager will inject the correct caBundle value.
  3. Add a volume and volume mount to the “kuma-control-plane” Deployment.
  4. Change one of the environment variables for the “kuma-control-plane” Deployment to reference the volume added Continue reading

Technology Short Take 150

Welcome to Technology Short Take #150! This is the last Technology Short Take of 2021, so hopefully I’ll close the year out “with a bang” with this collection of links and articles on various technology areas. Bring on the content!

Networking

  • Ivan Pepelnjak has a post on running network automation tools in a container. In fact, he’s already built some container images, and the post has information on running tools from his prebuilt container image. Well worth reading!
  • Tom Hollingsworth likens networking disaggregation to “cutting the cord” and switching away from cable.

Servers/Hardware

Security

  • Nicholas Weaver (no, not that Nick Weaver) discusses the Log4Shell vulnerability.
  • The Log4J vulnerability and associated exploits has been on many folks' minds, so it’s only natural that many security companies have been looking into how to mitigate this attack vector. Aqua Security has a write-up on some of their analysis here.
  • This is an older post, but it doesn’t look like I’ve linked to it before, so I thought I’d include Continue reading
1 2 3 4 5 6 … 20