In this post I’d like to share a couple of things I recently learned about the interaction between cloud-init and OpenStack Orchestration (aka “Heat”). This may be stuff that you already know, but in the interest of helping others who may not know I’m posting it here.
One issue that I’d been repeatedly running into was an apparent “failure” on the part of Heat to properly apply cloud-init configurations to deployed Ubuntu instances. So, using a Heat template with an OS::Nova::Server resource defined like this would result in an instance that apparently wasn’t reachable via SSH (I’d get back Permission denied (publickey)):
resources:
instance0:
type: OS::Nova::Server
properties:
name: cloud-init-test-01
image: { get_param: image_id }
flavor: m1.xsmall
networks:
- port: { get_resource: instance0_port0 }
key_name: labDeploying an instance manually from the same image worked perfectly. So what was the deal?
The first thing I learned was that, in some circumstances (more on this in a moment) defaults to injecting SSH keys (like the key named lab specified in the template) to a user account named “ec2-user”. Ah! I’d been using the default “ubuntu” account specified in Continue reading
Most of you have probably picked up on the news of VMware’s new container-optimized Linux distribution, code-named “Photon”. (More details here.) In this post, I’m going to provide a very quick walkthrough on running Photon on VMware Fusion via Vagrant. This walkthrough will leverage a Vagrant box for Photon that has already been created.
To make things easier, I’ve added a photon directory to my GitHub “learning-tools” repository. Feel free to pull those files down to make it easier to follow along.
I assume that you’ve already installed Vagrant, VMware Fusion, and the Vagrant plugin for VMware. If you haven’t, you’ll want to complete those tasks—and verify that everything is working as expected—before proceeding.
Install an additional Vagrant plugin that enables Vagrant to better detect and interact with Photon using this command:
vagrant plugin install vagrant-guests-photon
If you don’t install this plugin, you’ll likely get a non-fatal error about Vagrant being unable to perform the networking configuration. (Review the GitHub repository for this plugin if you want/need more details. Also, note that a PR against Vagrant to eliminate the need for this plugin was opened and merged; this fix should show up in a future release of Vagrant.)
In this post, I’ll build on a couple of previous posts and show you how to build your own Docker Swarm cluster that leverages etcd for cluster node discovery. This post builds on the information presented on how to run an etcd 2.0 cluster on Ubuntu as well as the information found in this post on running a Consul-backed Docker Swarm cluster.
To help you follow along, I’ve created a Vagrant environment that you can use to turn up the configuration described in this blog post. These files are found in the “docker-swarm-etcd” directory of my GitHub learning-tools repository. Feel free to use the files in this directory/repository to help with the learning process.
There are 3 major components to this configuration:
In this post, I’m going to show you how to set up a cluster of three nodes running etcd 2.0 (specifically, etcd 2.0.9). While I’ve discussed etcd before, that was in the context of using etcd with CoreOS Linux. In this case, I’ll use Ubuntu 14.04 as the base OS, along with the latest released version of etcd.
To help you follow along, I’ve created a set of files that will allow you to use Vagrant to turn up an etcd 2.0 cluster on Ubuntu 14.04 (on your laptop, if so desired). You can find all these files in the “etcd-2.0” directory of my learning-tools GitHub repository.
You don’t need anything special when setting up etcd; a straightforward Ubuntu Server 14.04 x64 installation will work just fine. If you’re using the files in my learning-tools repository, you’ll see that Vagrant simply turns up a VM based on a plain-jane Ubuntu 14.04 box. If you’re building this from scratch (why?!), simply create a VM and install Ubuntu 14.04 into it. As long as it has Internet connectivity, that’s all that’s needed.
Installing etcd Continue reading
I’m a big fan of plain text-based formats and tools, for a variety of reasons. As a result, I’ve moved almost all of my writing into plain text formats, primarily Markdown (MultiMarkdown, to be specific), and I recently spent a couple of months experimenting to see if a plain text-based productivity system would work for me as well. Here’s what I found.
<aside>I won’t go into a great amount of detail on why I wanted to see if a plain text-based productivity system would work, as I’ve discussed the merits of using plain text-based formats before (platform portability, application portability, longevity, etc.).</aside>
I first looked at Gina Trapani’s todo.txt system, which is plain text-based and has a thriving community built around its format. The todo.txt format is very straightforward and very simple—almost too simple. After trying it for a short while I found that it just wasn’t flexible enough to meet my needs.
A few weeks later, I stumbled on the TaskPaper format (named after the Mac app of the same name). Also plain text-based, the TaskPaper format also had a thriving community of applications and tools built around the format, and while also Continue reading
“You’re a rockstar!” Chances are, you’ve either a) been told this as a compliment for some work you’d done; b) heard this told to someone else for some work they’d done; or c) told someone this for some work they’d done. If you said this to someone else—I just told someone this quite recently—chances are also very likely that you had nothing but positive intentions behind this statement and your goal was to compliment them on what you saw as outstanding work. But is “rockstar” the wrong term to use? And if so, what is the right term?
Recently, Tyler Britten (a very talented professional and a former colleague when I worked as an EMC vSpecialist) posted an article titled “Time to Retire the Rockstar,” in which he draws a connection between the use of terms like “rockstar,” “superstar,” “genius,” or “guru” and the myth of the lone genius. I see his point, and don’t necessarily disagree with it. Something can be said that calling someone a rockstar (or any of the other terms listed) isn’t automatically encouraging them to “eschew teams and communities and to work alone”, but that isn’t the point of this post. Here I’d rather Continue reading
OpenStack Congress, a project aimed at providing “policy as a service” for OpenStack clouds, is a project I’ve had the privilege of being involved in from very early days. I first mentioned Congress almost a year ago, and since then the developers have been hard at work on the project. Recently, one of the lead developers posted a summary of some pretty impressive performance improvements that have been made with Congress.
I won’t repeat all the sordid details here; for all the details, I encourage you to go read the full post over at ruleyourcloud.com. Just to give you a quick highlight of some of the performance gains they’ve been able to realize, consider these numbers:
Given the nature of Congress—that it must, by its very definition, import data from multiple cloud services and perform queries across that data to determine policy violations—the performance improvements seen in query performance and data import speeds are quite significant.
For the detailed explanation of how the developers were able to see such incredible performance improvements, see the full post. If you’re interested in Continue reading
Welcome to Technology Short Take #49 (also known as Distraction-as-a-Service)! I have here for your reading pleasure an eclectic collection of links and articles from around the web, focusing on data center-related technologies. Here’s hoping you find something useful. Bring on the content!
Upon hearing the news that Red Hat had released the Atomic Host variant of Red Hat Enterprise Linux, I decided that it would be a good idea for me to take a look at the CentOS flavor of the Atomic Host variant. In case you’re unfamiliar, the Atomic Host variant is the result of Project Atomic, which aimed to provide a container-optimized flavor of RHEL/CentOS/Fedora. This container-optimized flavor would leverage rpm-ostree for atomic system updates (hence the name) and come with Docker preinstalled. What I found, frankly, disappointed me.
Before I continue, I will make two very important disclaimers:
Note that there has been no official announcement of the release of final builds of an Atomic Host variant for CentOS 7. So, it’s entirely possible that all the issues I mention here are known issues that will be addressed. That being said, I did find CentOS 7 Atomic Host builds dated March 5, 2015; this is the same date as the Red Hat announcement. It’s reasonable, therefore, to believe that these builds are very close to final builds.
It’s entirely possible these issues are the result of errors on my part. I’ve spent most of my time with Continue reading
In this post, I’m going to show you how to set up and run your own Docker Swarm cluster. Docker Swarm is a relatively new orchestration tool from Docker (the company) that allows you to create a cluster of hosts running Docker (the open source project) and schedule containers across the cluster. However, just scheduling and running containers across a cluster isn’t enough, so I’ll show you how to add service registration and service discovery to this environment using Consul.
In the event you’re interested in following along, I’ve created a set of files that will allow you to use Vagrant to run this Docker Swarm cluster (on your laptop, if so desired). You can find all these files in the “docker-swarm” folder of my GitHub learning-tools repository.
The Docker Swarm cluster I’m going to show you how to build has 3 major components:
Over the last few days, I’ve been experimenting with Docker, Registrator, and Consul in an effort to explore some of the challenges involved in building a robust containerized infrastructure. While I haven’t finished fully exploring the idea (and documenting what I’ve learned), I did discover one interesting—and unexpected—interaction.
Here’s a quick overview of my testing environment:
Registrator, if you’re not already familiar with it, is a service registry tool that listens to the Docker Continue reading
I’ve written about CoreOS a fair amount (see here, here, and here), but one of the things that is both good and bad about CoreOS is the automatic update mechanism. It’s good because you know your systems will stay up to date, but it’s bad if you haven’t taken the time to properly address how automatic updates will affect your environment (for example, you’ve manually started some Docker containers instead of using systemd unit files—when the CoreOS system reboots after an update, your Docker containers will no longer be running). Re-architecting your environment to fully account for this change in architecture and behavior is a larger discussion than can be addressed in a single blog post, but in this post I want to at least tackle one small part of the discussion: separating your persistent data. In this post, I’ll show you how to mount an NFS share on a CoreOS instance deployed on OpenStack (or any cloud that leverages cloud-init).
Now, you could probably go into your CoreOS instance and manually make these changes, but that’s still thinking the old way. In addition to thinking about keeping persistent data separate, we (data center/cloud architects) also need Continue reading
Over the past couple of years, Markdown has become an important part of my computing landscape. I’ve transitioned almost all of my text-based content creation, including this blog, over to Markdown. I’d also been looking for ways that I might be able to extend my use of Markdown into creating presentations as well, but hadn’t—until recently—found a tool that fit into my workflow. Then I started using Deckset.
The idea behind Deckset is not unique; there are other projects out there that do the same sort of thing. (Remarkjs is one example that I’ve also used; more on that in a moment.) You create your presentation in Markdown, using headings, bulleted lists, numbered lists, etc. Markdown is just plain text, so you can use any plain text editing tool you like for this part. Deckset itself is OS X-specific, but the content remains platform- and application-independent (use any text editing tool on any platform you like).
Because Markdown isn’t natively suited to creating presentations, Deckset—along with all the other solutions I tried—have to add some “extensions” to Markdown. For example, in Deckset’s case:
---) to denote the start of a new slide.As you may have picked up from some of my recent posts, I’m focused on building content and tools that will help others learn new projects, products, and technologies that I think will be very relevant in the future. One such project is Open vSwitch (OVS), which I’ve written about quite a bit (you can see all OVS-related posts here). To help others work with and learn about Open vSwitch, I’ve published a new Vagrant base box.
(In the event you’re not familiar with Vagrant, take a look at this quick introduction.)
The new Vagrant box I’ve created is running Ubuntu 14.04.1 and has Open vSwitch 2.3.1—the latest available release—pre-installed. To install this Vagrant box for use in your Vagrant environments, simply run this command:
vagrant box add slowe/ubuntu-1404-x64-ovs
Vagrant will download and install the box. (Note that this box is formatted for the “vmware_desktop” provider, which means you’ll need VMware Fusion or VMware Workstation as well as the Vagrant plugin for VMware.) Once the box is installed on your system, then you can begin using it in a Vagrantfile by just referencing the box name. As with the other Vagrant boxes Continue reading
A colleague recently pointed me to the recent Mesosphere announcement of Mesos-DNS, a DNS-based service discovery mechanism for Apache Mesos clusters. A comment made in the announcement got me thinking, and I wanted to briefly share my thoughts.
The comment that got me thinking was this:
Mesos-DNS is simple and stateless. Unlike Consul and SkyDNS, it does not require consensus mechanisms, persistent storage, or a replicated log.
If you’ve been following along here on my site, you know that I’ve written about Consul before (see here), and I have more Consul content planned. I’m reasonably familiar with Consul’s architecture and requirements. Likewise, although I haven’t specifically written about SkyDNS, it’s based on etcd, which I have talked about (see here). The Mesos-DNS article seems to imply that Mesos-DNS is somehow less complex than either of these two solutions because it doesn’t require consensus mechanisms, persistent storage, or a replicated log.
However, in my mind that’s a misleading statement. Yes, Consul does require a consensus mechanism (it uses Raft, like etcd). SkyDNS (as I understand it, at least) simply leverages etcd, so technically SkyDNS itself doesn’t require a consensus mechanism. And the assertion that SkyDNS itself Continue reading
In this post, I’ll provide an example of using YAML to create a multi-container Docker environment in Vagrant. I made a brief mention of this technique in my earlier post talking about how to use Docker with Vagrant, but wanted to provide an example. For me, I know that examples are often quite helpful when I’m learning something new. Since one of my primary goals here is to help enable others to learn these technologies, I figured an example would be helpful. So, to that end, here’s an example that I hope will help others.
As is becoming my custom, you can find resources to help you replicate this environment on your own laptop/desktop/whatever via my “learning-tools” GitHub repository.
Before I get into the details, I want to just very quickly recap some information from my earlier post on using Docker with Vagrant:
Vagrantfile that Continue readingAs part of my ongoing effort to create tools to assist others in learning some of the new technologies out there, I spent a bit of time today working through the use of Docker with Vagrant. Neither of these technologies should be new to my readers; I’ve already provided quick introductory posts to both (see here and here). However, using these two together may provide a real benefit for users who are new to either technology, so I’d like to take a bit and show you how to use Docker with Vagrant.
Vagrant first started shipping with a Docker provider as part of the core product in version 1.6 (recall that Vagrant uses the concept of providers to support multiple backend virtualization solutions). Therefore, if you’ve installed any recent version of Vagrant, you already have the Docker provider as part of your Vagrant installation.
However, while you may have the Docker provider as part of Vagrant, you still need Docker itself (just like if you have the VMware provider for Vagrant, you still need the appropriate VMware product—VMware Fusion on the Mac or VMware Workstation on Windows/Linux) in order to provide the functionality Vagrant will consume. Continue reading
As part of my ongoing effort to create tools to assist others in learning some of the new technologies out there, I spent a bit of time today working through the use of Docker with Vagrant. Neither of these technologies should be new to my readers; I’ve already provided quick introductory posts to both (see here and here). However, using these two together may provide a real benefit for users who are new to either technology, so I’d like to take a bit and show you how to use Docker with Vagrant.
Vagrant first started shipping with a Docker provider as part of the core product in version 1.6 (recall that Vagrant uses the concept of providers to support multiple backend virtualization solutions). Therefore, if you’ve installed any recent version of Vagrant, you already have the Docker provider as part of your Vagrant installation.
However, while you may have the Docker provider as part of Vagrant, you still need Docker itself (just like if you have the VMware provider for Vagrant, you still need the appropriate VMware product—VMware Fusion on the Mac or VMware Workstation on Windows/Linux) in order to provide the functionality Vagrant will consume. Continue reading
Welcome to Technology Short Take #48, another installation in my irregularly-published series that collects links, articles, and thoughts from around the web. This time around, the content is a bit heavier on cloud management and applications/operating systems, but still lots of good content all the way around (I hope, anyway).
For reasons that (hopefully) will become clear in the relatively near future, I decided I needed to take a look at Consul, a distributed service discovery tool and key value store. I know Consul’s description sounds like a mouthful of buzzwords, but it’s pretty accurate. This post provides a quick introduction to Consul, in which I’ll break down what Consul does and how it works (at a high level). I’ll then build on this introduction in later posts.
There’s a lot to Consul, so let’s start by breaking down the description of Consul, which I provided as “a distributed service discovery tool and key value store”. What does this mean, exactly?