Scott Lowe

Author Archives: Scott Lowe

Quick Post: Parsing AWS Instance Data with JQ

I recently had a need to get a specific subset of information about some AWS instances. Naturally, I turned to the CLI and some CLI tools to help. In this post, I’ll share the command I used to parse the AWS instance data down using the ever-so-handy jq tool.

What I needed, specifically, was the public IP address and the private IP address for each instance. That information is readily accessible using the aws ec2 describe-instances command, but that command provides a ton more information than I needed. So, I decided to try to use jq to parse the JSON output from the AWS CLI. If you’re not familiar with jq, I recommend you take a look at this brief introductory post I wrote back in 2015.

After some trial and error, here’s the final command I used:

aws ec2 describe-instances | jq '.Reservations[] | .Instances[] | \
{Id: .InstanceId, PublicAddress: .PublicIpAddress, \
PrivateAddress: .PrivateIpAddress}'

I’ll refer you to the jq manual for details on breaking down how this filter works. I’ll also point out that there’s nothing terribly groundbreaking or revolutionary about this command; I wanted to share it here just in case it may save someone Continue reading

Posts from the Past, May 2018

This month—May 2018—marks thirteen years that I’ve been generating content here on this site. It’s been a phenomenal 13 years, and I’ve enjoyed the opportunity to share information with readers around the world. To celebrate, I thought I’d do a quick “Posts from the Past” and highlight some content from previous years. Enjoy!

May 2017

A year ago, I touched on the topic of using a Makefile with Markdown documents to help streamline the process of generating various output formats.

I also explored the use of custom SSH configurations with SSH bastion hosts and uncovered a very basic (but important) error I’d previously overlooked.

May 2016

Two years ago in May I was using Terraform to build an etcd v2 cluster on OpenStack.

May 2015

Three years ago, I was doing a lot of work in my home lab, automating the setup of physical hosts. That led to a post on a fully automated Ubuntu install, which was also related to this post on using an Apt proxy (via apt-cacher-ng).

May 2014

Four years ago, I shared some useful Markdown tools for OS X. Of those tools, I still use pandoc pretty extensively.

May 2013

Five years ago, Continue reading

DockerCon SF 18 and Spousetivities

DockerCon SF 18 is set to kick off in San Francisco at the Moscone Center from June 12 to June 15. This marks the return of DockerCon to San Francisco after being held in other venues for the last couple of years. Also returning to San Francisco is Spousetivities, which has organized activities for spouses, significant others/domestic partners, friends, and family members traveling with conference attendees!

Registration is open right now, so hurry on over and sign up for one or more activities. What’s that—you’re wondering what’s been planned? Here’s a quick overview:

  • The activities kick off on Tuesday, June 12, with a tour of Monterey and Carmel. I must admit I’m a bit jealous; I’d love to have the opportunity to visit Cannery Row and see the canneries that inspired John Steinbeck. Join this tour and you’ll have that opportunity! This is a full-day activity, leaving the InterContinental at 8pm and returning around 7pm that evening. Lunch is included, of course.
  • On Wednesday, June 13, Spousetivities has arranged for private transportation to the Sonoma Valley for some wine tastings! You’ll visit a couple different wineries and get to enjoy lunch in Sonoma. This event will wrap up back Continue reading

Manually Installing Firefox 60 on Fedora 27

Mozilla recently released version 60 of Firefox, which contains a number of pretty important enhancements (as outlined here). However, the Fedora repositories don’t (yet) contain Firefox 60 (at least not for Fedora 27), so you can’t just do a dnf update to get the latest release. With that in mind, here are some instructions for manually installing Firefox 60 on Fedora 27.

These instructions assume you have a dnf-installed version of Firefox (typically Firefox 59) already installed on your Fedora system. These steps should allow you to upgrade your Fedora system to Firefox 60:

  1. Download the Firefox 60 archive (typically named firefox-60.0.tar.bz2 or similar) onto your Fedora system. You can do this with your already-installed version of Firefox, but be sure to close/quit Firefox before proceeding with the rest of the instructions.
  2. Make a copy of /usr/share/applications/firefox.desktop; you’ll use this later.
  3. Remove the version of Firefox installed from the Fedora repositories with dnf remove firefox. This will remove the firefox.desktop file you copied in the previous step (which is why you copied it somewhere else).

  4. Use bunzip2 to decompress the downloaded Firefox 60 archive. This will leave you with a plain . Continue reading

One Week Until Spousetivities in Vancouver

Only one week remains until Spousetivities kicks off in Vancouver at the OpenStack Summit! If you are traveling to the Summit with a spouse, significant other, family member, or friend, I’d encourage you to take a look at the great activities Crystal has arranged during the Summit.

Here’s a quick sneak peek at what’s planned:

  • On Monday, May 21, Spousetivities attendees will enjoy a tour of the highlights of Vancouver (including things like Stanley Park, Gastown, Chinatown, and Granville Island Public Market), followed by fun at the Capilano Suspension Bridge Park. (If for no other reason, you’ll want to attend this to see Crystal face her fear of heights and suspension bridges!)
  • On Tuesday, May 22, Spousetivities is off to Whistler Village. Along the way, see wonderful sights like Howe Sound, Britannia Beach, and Shannon Falls. Then you’ll get to ride the Sea to Sky Gondola up to Squamish for an adventure-filled time.
  • On Wednesday, May 23, the activities wrap up with a wine tour. This will include tastings at three beautiful wineries and a lovely picnic lunch at one of the venues.

All of these tours includes private transportation, and the pricing for each of the events is Continue reading

Technology Short Take 99

Welcome to Technology Short Take 99! What follows below is a collection of various links and articles about (mostly) data center-related technologies. Hopefully something I’ve included will be useful. Here goes!

Networking

  • David Gee makes the connection between coffee and network automation. No, really. It’s worth reading.
  • Matt Oswalt, one of the co-authors of our recently-released network automation book from O’Reilly, recently tackled the topic of running Kubernetes with Tungsten Fabric (formerly known as OpenContrail). A network engineer using AWS and CloudFormation? Yep, get used to it folks—it’s where the industry is headed.
  • Vince Power provides a high-level overview of some of the key principles underlying Kubernetes networking.

Servers/Hardware

Sorry, I don’t have anything for you. Feel free to send me links you’d like me to consider for inclusion in the next Tech Short Take!

Security

Installing GitKraken on Fedora 27

GitKraken is a full-featured graphical Git client with support for multiple platforms. Given that I’m trying to live a multi-platform life, it made sense for me to give this a try and see whether it is worth making part of my (evolving and updated) multi-platform toolbelt. Along the way, though, I found that GitKraken doesn’t provide an RPM package for Fedora, and that the installation isn’t as straightforward as one might hope. I’m documenting the procedure here in the hope of helping others.

First, download the latest release of GitKraken. You can do this via the terminal with this command:

curl -LO https://release.gitkraken.com/linux/gitkraken-amd64.tar.gz

Extract the contents of the GitKraken download into its own directory under /opt using this command (you can use a different directory if you like, but I prefer to install third-party applications like this under /opt):

sudo tar -C /opt -xvf gitkraken-amd64.tar.gz

This will extract everything into /opt/gitkraken.

Next, you’ll create a symbolic link to an existing library to fix an error with GitKraken when running on Fedora (this is documented here):

sudo ln -s /usr/lib64/libcurl.so.4 /usr/lib64/libcurl-gnutls.so.4

Once this is done, you could just run Continue reading

An Updated Look at My Multi-Platform Toolbelt

In early 2017 I posted about my (evolving) multi-platform toolbelt, describing some of the applications, standards, and services that I use across my Linux and macOS systems. In this post, I’d like to provide an updated review of that toolbelt.

  • Visual Studio Code: I switched from Sublime Text to Visual Studio Code during my latest migration to Fedora 27 on a Lenovo ThinkPad X1 Carbon. Since I’m also planning on expanding my coding skills with Golang, I felt that Visual Studio Code would be a better choice than Sublime Text. I’m still generating the majority of my content in Markdown (MultiMarkdown is the flavor that I generally use), and I’ve found Visual Studio Code to be pretty decent as a Markdown editor.

  • IMAP/SMTP: I’ve standardized on using IMAP/SMTP for all my e-mail accounts, which gives me quite a bit of flexibility in clients and OSes. It’s very likely I’ve pretty much standardized on Thunderbird (which supports OS X, Linux, and Windows).

  • Unison: This cross-platform file synchronization tool helps keep my files in sync across my macOS and Linux systems.

  • Dropbox: Dropbox gives me access to non-confidential files from any of my devices or platforms (macOS, iOS, and Linux).

  • jrnl: Continue reading

Technology Short Take 98

Welcome to Technology Short Take #98! Now that I’m starting to get settled into my new role at Heptio, I’ve managed to find some time to pull together another collection of links and articles pertaining to various data center technologies. Feedback is always welcome!

Networking

  • VMware has released a PowerCLI preview/fling for NSX-T; Kyle Ruddy has a write-up here. Looks like this preview provides some high-level cmdlets for NSX-T that weren’t available before.
  • Cilium, the open source project working to bring eBPF-powered networking and security to Kubernetes environments, has hit the 1.0 release. I will freely admit that I am a fan of what the Cilium folks are doing.
  • What’s that? Don’t know what eBPF is? Or XDP? Not to worry, the nice folks over at Netronome have a post that explains it all.
  • People do all kinds of interesting things with Raspberry Pis; here’s an article by Scott Helme on using a Pi to secure DNS traffic.

Servers/Hardware

  • This is more of a follow-up to one of my own articles than a pointer to someone else’s article. After continued use (including on business trips) of my Lenovo ThinkPad X1 Carbon running Fedora 27, I continue to be impressed Continue reading

List of Kubernetes Folks on Twitter

Earlier this morning, I asked on Twitter about good individuals to follow on Twitter for Kubernetes information. I received quite a few good responses (thank you!), and I though it might be useful to share the list of the folks that were recommended across all those responses.

The list I’ve compiled is clearly incomplete! If you think someone should be added to this list, feel free to hit me up on Twitter and let me know. Alternately, feel free to submit a pull request (PR) that adds them to this list. I’m not going to “vet” the list, so I’ll add any and all recommendations (unless they are clearly not related to Kubernetes, such as a news anchorman someone recommended to me—not sure about that one!).

Without further ado, here is the list I compiled from the responses to my tweet, in no particular order (I’ve included full name and employer, where that information is available):

  • Kelsey Hightower (Google) - @kelseyhightower
  • Jessie Frazelle (Microsoft) - @jessfraz
  • Alex Ellis (VMware) - @alexellisuk
  • Michael Hausenblas (Red Hat) - @mhausenblas
  • Ahmet Alp Balkan (Google) - @ahmetb
  • AdNaN Abdulhussein (Bitnami) - @prydonius
  • Tim Hockin (Google) - @thockin
  • Joe Beda (Heptio) - @jbeda
  • Continue reading

Review: Lenovo ThinkPad X1 Carbon

As part of the transition into my new role at Heptio (see here for more information), I had to select a new corporate laptop. Given that my last attempt at running Linux full-time was thwarted due primarily to work-specific collaboration issues that would no longer apply (see here), and given that other members of my team (the Field Engineering team) are also running Linux full-time, I thought I’d give it another go. Accordingly, I’ve started working on a Lenovo ThinkPad X1 Carbon (5th generation). Here are my thoughts on this laptop.

This is now my second non-Apple laptop in the last year. My previous non-Apple laptop, a Dell Latitude E7370, was a pretty decent laptop (see my review). As good as the E7370 was, though, the X1 Carbon is better.

The X1 Carbon features a dual-core i7 7500U CPU, which (subjectively, anyway) outperforms the mobile CPU in the E7370. This makes the X1 Carbon feel quite snappy and responsive. CPU performance was an issue for me with the Dell—it didn’t take much to tax that mobile CPU. I haven’t seen that issue so far with the X1 Carbon. Coupled with 16GB of RAM, the X1 Carbon is no Continue reading

The Future is Containerized

Last week I announced my departure from VMware, and my intention to step away from VMware’s products and platforms to focus on a new technology area moving forward. Today marks the “official” start of a journey that’s been building for a couple years, a journey that will take me into a future that’s containerized. That journey starts in Seattle, Washington.

Why Seattle, Washington? Because that’s where Heptio is based, and because today I am joining Heptio as a senior member of the field engineering team to help drive the adoption of Kubernetes across the industry. Only a couple of folks guessed that I was headed to Heptio. If you were one of those folks, you guessed correctly!

Two questions are probably rolling around in your head right now:

  1. Why Kubernetes?
  2. Why Heptio?

Good questions!

It’s clear to me that containers will have a significant impact on how we as IT professionals will develop, deploy, upgrade, and manage applications. It’s also clear to me that when it comes to orchestrating containers, Kubernetes is the clear leader. So, if I accept that containers are going to be a significant part of IT moving forward, then it logically follows that Kubernetes is Continue reading

Technology Short Take 97

Welcome to Technology Short Take 97! This Tech Short Take marks the end of an era (sort of); it’s the last Tech Short Take published while I’m a VMware employee (today is my last day; see here for more details). But enough about me—let’s talk some tech! This Short Take may be a bit longer than some, so buckle up.

Networking

Time to Evolve

I first started getting into VMware around 2003, possibly earlier (I can’t recall exactly when it was). I remember thinking that VMware’s impact on the industry was going to be significant, and I wanted to be part of this industry change. I was right—virtualization like what VMware offers has fundamentally changed the industry. However, just as technology evolves, technology careers must evolve as well. Specifically, my technology career must change and grow. It’s time to evolve.

This need to evolve has been building for a couple years. You’ve probably observed that the amount of VMware-centric content produced here on the site has slowly been replaced by topics like Linux, Docker, Vagrant, Terraform, AWS, Azure, and others. These topics represent where I think my next period of growth and change resides, and after a couple years of slow growth in these areas it’s now time to “put the pedal to the metal” and accelerate things.

As of this coming Friday, March 30, 2018, I will be leaving VMware after a little over 5 years with the company. My time with VMware (as an employee) has been an amazing adventure. I’m thankful to Brad Hedlund for his Continue reading

Interop ITX, Dell Technologies World, and Spousetivities

Spousetivities will be present at two additional events this year—in fact, these events are only about 6 weeks away! Both Dell Technologies World and Interop ITX are in Las Vegas the last week of April (both starting April 30), and Spousetivities is running events for both conferences.

<aside>In case you’re wondering why I blog about Spousetivities, it’s not only because my wife runs it (seriously). It’s primarily because I’m committed to supporting families, marriages, and relationships in the IT industry. IT companies ask a lot of their employees—often asking employees to give up evenings and/or weekends, or setting unfair expectations on employee responsiveness via email/Slack/IM during off-hours—so a program that enables spouses and/or significant others to join IT employees during a conference helps provide a little bit of balance, in my view.</aside>

Here’s a look at what’s planned during these two IT conferences:

  • On Monday, April 30, there’s a full-day tour of Death Valley planned. This event is leaving the Mirage at 8:00 am and includes photo opportunities at Dante’s View and Zabriskie Point, a scenic drive through the Artist’s Pallet, and a stop at Bad Water Basin—the lowest point in the Western Hemisphere!

  • On Tuesday, May 1, Spousetivities Continue reading

Technology Short Take 96

Welcome to Technology Short Take 96! Ahead, lying in wait, is a unique collection of links, articles, and thoughts about various data center technologies. Browse if you dare…OK, so I’m being a bit melodramatic. It’s still some good stuff here!

Networking

  • Via Matt Oswalt and Michael Bushong, I came across this article on Juniper’s use of P4. Interesting stuff…P4 definitely has the potential to dramatically reshape networking in new ways, in my humble opinion.
  • Maxime Lagresle of XING outlines how they went about troubleshooting an unexplained connection timeout on Kubernetes/Docker.
  • Ajay Chenampara outlines how POAP (Power On Auto Provisioning), a feature of Cisco NX-OS, works to streamline provisioning new network switches.
  • Don Schenck has a high-level overview of Istio and service meshes.
  • Daniel Álvarez has a good article describing some OVN profiling and optimizing he recently performed. I believe the patches he mentioned in the post have already been accepted into the OVN codebase.

Servers/Hardware

Nothing this time around; sorry! If you have some articles you feel are worthy of inclusion in the next Tech Short Take, send them my way!

Security

Recent Changes in my “Learning Tools” Repository

A couple years ago, I created a “learning-tools” repository on GitHub with the goal of creating environments/tools that would help others learn new technologies. At first, the contents of the repository were almost exclusively leveraging Vagrant, but over time I’ve extended the environments to also leverage Ansible and to use tools such as Terraform. Over the past month or so, I’ve made a few additional (albeit relatively minor) updates that I also wanted to share.

As I said, the updates are relatively minor:

  • I’ve added environments for running generic versions of Fedora Atomic Host (26 and 27), Ubuntu 16.04, and Debian 9.x. These environments are probably of limited value by themselves, but in the future I may use them as the basis for more complex environments based on these operating systems. Of course, others may leverage them as the basis for projects of their own.
  • I’ve added Libvirt support for a number of the Vagrant-based environments, based on my experience with the Vagrant Libvirt provider. This support is limited to areas where I was able to find Libvirt-formatted Vagrant boxes, so you’ll find Libvirt support for the environments using CentOS Atomic Host, Fedora Atomic Host, and Debian. Continue reading

Looking Ahead: My 2018 Projects

For the last six years or so, I’ve been publishing a list of projects/goals for the upcoming year (followed by a year-end review of how I did with those projects/goals). For example, here are my goals for 2017, and here’s my year-end review of my progress in 2017. In this post, I’m going to share with you my list of projects/goals for 2018.

As I’ve done in previous years, I’ll list the projects/goals, along with an optional stretch goal (where it makes sense).

  1. Become extremely fluent with Kubernetes. I’m focusing all my technical skills on Kubernetes this year, with the goal of becoming extremely fluent with the project in all its aspects. There are some aspects—like networking, for example—where some specialization/additional focus will be needed (focusing on particular network architectures/plugins). That means “leaving behind” other technologies, like OpenStack, in order to more fully focus on Kubernetes. (Stretch goal: Pass the Certified Kubernetes Administrator [CKA] exam.)

  2. Learn to code/develop in Go. Given that Kubernetes is written in Go and that Go seems to be the language of choice for many new projects, tools, and utilities, I’m going to learn to code/develop in Go in 2018. Because I learned Continue reading

Technology Short Take 95

Welcome to Technology Short Take 95! This Short Take was a bit more challenging than normal to compile, given that I spent the week leading up to its publication visiting customers in Europe. (My travel schedule in Europe is also why it didn’t get published until Saturday instead of the typical Friday.) Nevertheless, I have persevered in order to deliver you this list of links and articles. I hope it proves useful!

Networking

  • Larry Smith Jr. has a nice write-up on Cisco XR stemming from a presentation at NFD 17.
  • VMware recently released a reference design guide for NSX-T; see here for more details.
  • The engineering team at Lyft recently discussed a new overlay-free networking approach they’ve been working on for Kubernetes: IPVLAN-based CNI stack for running within VPCs on AWS. This is pretty cool, but does introduce some potential design considerations for deploying Kubernetes on AWS. (For those that may be unfamiliar: CNI, or Container Network Interface, is the means whereby network mechanisms “plug into” Kubernetes. IPVLAN is a low-latency means of providing IP connectivity to containers. VPCs, or Virtual Private Clouds, are Amazon’s software-defined networking mechanism for workloads running on AWS.)
  • Viktor van den Berg writes Continue reading

Some Tools to Help Learn Kubernetes

Kubernetes is emerging as the clear leader in the container orchestration space. This makes it an important technology to know and understand. However, like other distributed systems, learning something like Kubernetes can be challenging due to the effort involved in getting Kubernetes up and running. It’s not about learning to set up Kubernetes (although that comes in time); at first, it’s about understanding what Kubernetes does and how to use Kubernetes. In this post, I’ll share some tools to help learn what Kubernetes does and how to use Kubernetes.

Note that this post is not intended to be a comprehensive list of learning resources for Kubernetes. Also, this post is not focused on providing resources to help you learn to deploy Kubernetes. Instead, I’m focusing here on tools and services that let you get Kubernetes up and running quickly and easily so that you can focus on using Kubernetes (deploying applications and workloads onto Kubernetes). I’m sure there are many more tools/options than what I have listed here; these are just some that I have used and feel might be useful for others.

I’ll briefly cover the following tools and services:

  • Minikube
  • Kops
  • Kube-aws
  • Azure Container Service (ACS/AKS)

You’ll note Continue reading

1 … 13 14 15 16 17 … 20