At Interop ITX 2017 in Las Vegas, I had the privilege to lead a half-day workshop on options for deploying containers to cloud providers. As part of that workshop, I gave four live demos of using different deployment options. Those demos—along with the slides I used for my presentation along the way—are now available to anyone who might like to try them on their own.
The slides and all the resources for the demos are available in this GitHub repository. The four demos are:
Docker Swarm on EC2: This demo leverages Terraform and Ansible to stand up and configure a Docker Swarm cluster on AWS.
Amazon EC2 Container Service (ECS): This demo uses AWS CloudFormation to create an EC2 Container Service cluster with 3 instances and an Amazon RDS instance for backend database storage.
Kubernetes on AWS using kops: Using the kops CLI tool, this demo turns up a Kubernetes cluster on AWS to show how to deploy containerized applications on Kubernetes.
Google Container Engine: The final demo shows using Google Container Engine—which is Kubernetes—to deploy an application.
In the coming weeks, I plan to recreate the demos, record them, and publish them via YouTube, so that Continue reading
This is a “liveblog” (not quite live, but you get the idea) of the Open vSwitch Open Source Day happening at the OpenStack Summit in Boston. Summaries of each of the presentations are included below.
The first session was led by Cloudbase Solutions, a company out of Italy that has been heavily involved in porting OVS to Windows with Hyper-V. The first part of the session focused on bringing attendees up to speed on the current state of OVS and OVN on Hyper-V. Feature parity and user interface parity between OVS/OVN on Hyper-V is really close to OVS/OVN on Linux, which should make it easier for Linux sysadmins to use OVS/OVN on Hyper-V as well.
The second part of the session showed using OVN under Kubernetes to provide networking between Windows containers on Windows hosts and Linux containers on Linux hosts, including networking across multiple cloud providers.
The lightning talks were all under 5 minutes, so a brief summary of these are provided below:
This is a “liveblog” (not quite live, but you get the idea) of the Open vSwitch Open Source Day happening at the OpenStack Summit in Boston. Summaries of each of the presentations are included below.
The first session was led by Cloudbase Solutions, a company out of Italy that has been heavily involved in porting OVS to Windows with Hyper-V. The first part of the session focused on bringing attendees up to speed on the current state of OVS and OVN on Hyper-V. Feature parity and user interface parity between OVS/OVN on Hyper-V is really close to OVS/OVN on Linux, which should make it easier for Linux sysadmins to use OVS/OVN on Hyper-V as well.
The second part of the session showed using OVN under Kubernetes to provide networking between Windows containers on Windows hosts and Linux containers on Linux hosts, including networking across multiple cloud providers.
The lightning talks were all under 5 minutes, so a brief summary of these are provided below:
This is a liveblog for an OpenStack Summit session on containerized OpenStack and a comparison of the tools used for containerized OpenStack. The speaker is Jaivish Kothari, from NEC Technologies. Two other speakers were listed on the title slide, but were apparently unable to make it to the Summit to present.
Kothari provides a brief overview of the session, then jumps into a discussion of deployment tools. As illustrated by one of his slides, there’s a huge collection of tools that are used to deploy OpenStack; some are “pure” deployment tools, others are configuration management tools. In this presentation, Kothari says he will focus specifically on OpenStack deployment tools, like Juju (Canonical), Fuel (Mirantis), Crowbar (Dell), and PackStack/TripleO (Red Hat), but I’m not sure how this relates to containerized OpenStack (per the session title).
According to Kothari, some of the challenges in “traditional” (non-containerized) deployment tools are best understood by looking at the challenges in deploying OpenStack:
This whole first section of the presentation was setting up the argument that containerizing your OpenStack control plane will help address these challenges. Continue reading
This is a liveblog of the day 2 keynote of the OpenStack Summit in Boston, MA. (I wasn’t able to liveblog yesterday’s keynote due to a schedule conflict.) It looks as if today’s keynote will have an impressive collection of speakers from a variety of companies, and—judging from the number of laptops on the stage—should feature a number of demos (hopefully all live).
The keynote starts with the typical high-energy video that’s intended to “pump up” the audience, and Mark Collier (COO, OpenStack Foundation) takes the stage promptly at 9am. Collier re-iterates a few statistics from yesterday’s keynote (attendees from 63 countries, for example). Collier shares that he believes that all major challenges humanity is trying to solve counts on computing. “All science is computer science,” according to Collier, which is both great but also represents a huge responsibility. He leads this discussion by pointing out what he believes to be the fundamental role of open source in machine learning and artificial intelligence (ML/AI). Collier also mentions a collection of “composable” open source projects that are leading the way toward a “cloud-native” future. All of these projects are designed in a way to be combined together in a “mix-and-match” Continue reading
This is a liveblog of an OpenStack Summit session providing an update on the Kuryr project. The speakers are Antoni Segura Puimedon and Irena Berezovsky. Kuryr, if you recall, was a project aimed at making OpenStack Neutron functionality available to Docker containers; it has since expanded to also offer Cinder and Manila storage to Docker containers, and has added support for both Docker Swarm and Kubernetes as well.
According to Puimedon, the latest release of Kuryr has a diverse base of contributors, with over 45 active contributors.
So, what will be in the Pike release? For the Kubernetes-specific support:
What’s planned for Pike, but may not actually make it? (Again, this is for Kubernetes support.)
On the Docker side, the following new features and enhancements will arrive in Pike:
On the Fuxi side, Kuryr is adding support for Manila shares.
At this point, Berezovsky takes over to discuss the release Continue reading
This is a liveblog of the OpenStack Summit session titled “AT&T’s Container Strategy and OpenStack’s Role in it”. The speakers are Kandan Kathirvel and Amit Tank, both from AT&T. I really wanted to sit in on Martin Casado’s presentation next door (happening at the same time), but as much as I love watching/hearing Martin speak, I felt this like presentation might expose me to some new information.
Kathirvel kicks off the session with some quick introductions, then sets the stage for the session. Naturally, Kathirvel starts out by describing AT&T’s cloud deployment. (I say “naturally” because it seems that every presentation starts out with describing how great and how awesome the presenter’s company’s OpenStack cloud is.)
Following the discussion of AT&T’s cloud, Kathirvel launches into a discussion of container trends and demands. He indicates that he believes container usage (or demand?) for enterprise IT applications is huge (and will continue to be large), but doesn’t believe that will hold true for virtual network functions (VNFs) in telco clouds.
As for how containers and OpenStack may be coming together, Kathirvel describes three different use cases:
The first use case has OpenStack managing the infrastructure, with Kubernetes (or another container Continue reading
This is a liveblog of the day 2 keynote of the OpenStack Summit in Boston, MA. (I wasn’t able to liveblog yesterday’s keynote due to a schedule conflict.) It looks as if today’s keynote will have an impressive collection of speakers from a variety of companies, and—judging from the number of laptops on the stage—should feature a number of demos (hopefully all live).
The keynote starts with the typical high-energy video that’s intended to “pump up” the audience, and Mark Collier (COO, OpenStack Foundation) takes the stage promptly at 9am. Collier re-iterates a few statistics from yesterday’s keynote (attendees from 63 countries, for example). Collier shares that he believes that all major challenges humanity is trying to solve counts on computing. “All science is computer science,” according to Collier, which is both great but also represents a huge responsibility. He leads this discussion by pointing out what he believes to be the fundamental role of open source in machine learning and artificial intelligence (ML/AI). Collier also mentions a collection of “composable” open source projects that are leading the way toward a “cloud-native” future. All of these projects are designed in a way to be combined together in a “mix-and-match” Continue reading
It’s no secret that I’m a big fan of using Markdown (specifically, MultiMarkdown) for the vast majority of all the text-based content that I create. Over the last few years, I’ve created used various tools and created scripts to help “reduce the friction” involved with outputting Markdown source files into a variety of destination formats (HTML, RTF, or DOCX, for example). Recently, thanks to Cody Bunch, I was pointed toward the use of a Makefile to assist in this area. After a short period of experimentation, I’m finding that I really like this workflow, and I wanted to share some details here with my readers.
First, if you’re not familiar with make and its use of a Makefile, check out this introduction. There’s a ton of power and flexibility here, of which I’ve only scratched the surface so far. The basic gist behind a Makefile is that it provides a set of instructions to the make command. Each set of instructions is tied to a target, which has one or more dependencies. In the “traditional” use cases for make, this is to allow programmers to define how a set of files should be compiled as well Continue reading
Welcome to Technology Short Take #82! This issue is a bit behind schedule; I’ve been pretty heads-down on some projects. That work will come to fruition in a couple weeks, so I should be able to come up for some air soon. In the meantime, here’s a few links and articles for your reading pleasure.
ovs-dpctl command to “program” the Open vSwitch (OVS) kernel module. It’s a bit geeky, but does provide some insight into Continue readingIf you’ve been following the Full Stack Journey podcast, you know that the podcast has been silent for a few months. Some of that was due to some adverse situations in life (it happens to all of us from time to time), but some of it was due to the coordination of a major transition in the podcast. And that’s the big news I’m here to share—read on for the full details!
If you’ve been in the IT industry for any reasonable length of time, especially in the networking space, you’ve probably heard of the Packet Pushers Podcast. It’s a hugely popular podcast created by Greg Ferro and Ethan Banks. In recent years, Packet Pushers has expanded from the “main show” to include other shows, including the Datanauts podcast (led by Chris Wahl and Ethan Banks). They’ve also been looking to expand their stable of podcasts to include additional relevant content.
This brings me to the big news: the Full Stack Journey podcast is joining the Packet Pushers network of podcasts! That’s right—the Full Stack Journey will be part of Packet Pushers’ growing network of podcasts. In talking with Greg and Ethan and the rest of the Packet Pushers team, Continue reading
A couple years ago, I wrote an article about how I was choosing CoreOS over Project Atomic based on some initial testing with CentOS Atomic Host builds. As it turns out—and as I pointed out in the “Update” section of that article—the Atomic Host builds I was using were pre-release builds, and therefore it wasn’t really appropriate to form an assessment based on pre-release builds. Now that both CentOS Atomic Host and CoreOS Container Linux have both grown and matured, I thought I’d revisit the topic and see how—if at all—things have changed.
In my original post, there were 4 major issues I identified (not necessarily in the same order as the original post):
So how do these areas look now, 2 years later?
Container-specific cloud-init extensions: Upon a closer examination of this issue, I realized that the cloud-init extensions were actually specific to CoreOS projects, like etcd and fleet. Thus, it wouldn’t make sense for these sorts of cloud-init extensions to exist on Atomic Hosts. What would make sense would be extensions that help configure Atomic Host-specific functionality, though (to be honest) Continue reading
This is a liveblog of the day 2 keynote (general session) of DockerCon 2017 in Austin, TX. For a look at what was announced or discussed in the day 1 keynote yesterday, see this liveblog. You can also see all DockerCon 2017-related posts by browsing the posts tagged with “DockerCon2017” (see the links at the bottom of this page). Before the keynote starts, there’s some nice live music playing; a welcome change (in my opinion) from yesterday’s video game.
At 9:03am, Ben Golub takes the stage to kick off the day 2 general session. He starts off by reviewing some proposed Docker logos, with a hint toward an announcement at the end of the session (presumably around changing Docker’s logo).
Golub then transitions into the meat of the general session presentation, which (understandably) is focused on Docker in the enterprise. He reviews the usual slide with notable logos from Docker customers. He also discusses some results from a company called ETR, which (apparently) shows Docker is “off the charts” in terms of adoption and market penetration within the enterprise. Golub also debunks the bi-modal IT structure model, saying that Docker’s customers only want one thing: speed (as in moving faster, Continue reading
This is a liveblog of the day 1 keynote (general session) of DockerCon 2017 in Austin, TX.
At 9:05am, Ben Golub, CEO of Docker, Inc., takes the stage to kick off the general session and the conference. Golub starts the presentation by reviewing Docker’s four-year history and all the things that have changed over the last three years since the very first DockerCon—from the size of Gordon (Docker’s tortoise mascot) to the amount of growth in Docker usage (via statistics in the number of Docker hosts, the number of Docker-ized apps, the number of image pulls from Docker Hub, and so forth).
Golub continues by mentioning some of the various use cases for Docker. One use case mentioned is Intuit’s use of Docker, and Golub points out that the person responsible for running Intuit’s systems is confident enough in their systems that they’re attending DockerCon on Tax Day (when as many as 25 million tax returns are expected to be processed).
Shifting gears a bit, Golub talks a bit more about the changes over the last 3 years in regards to Docker (the open source project) itself. Stakeholders have changed, and the nature of the project (now projects) has Continue reading
This is a liveblog of the DockerCon 2017 Black Belt session led by Thomas Graf on Cilium, a new startup that focuses on using eBPF and XDP for network and application security.
Graf starts by talking about how BPF (specifically, extended BPF or eBPF) can be used to rethink how the Linux kernel handles network traffic. Graf points out that there is another session by Brendan Gregg on using BPF to do analysis performance and profiling.
Why is it necessary to rethink how networking and security is handled? A lot of it has not evolved as application deployments have evolved from low complexity/low deployment frequency to high complexity/high deployment frequency. Further, the age of unique protocol ports (like SMTP on port 25 or SSH on port 22) is coming to a close, as now many different applications or services simply run over HTTP. This leads to “overloading” the HTTP port and a loss of visibility into which applications are talking over that port. Opening TCP port 80 in a situation like this means potentially exposing more privileges than desired (the example to use other HTTP verbs, like PUT or POST instead of just GET).
Graf quickly moves into a Continue reading
This is a liveblog for the DockerCon 2017 session titled “Creating Effective Images.” The speaker is Abby Fuller, a Senior Technical Evangelist with Amazon Web Services. Abby is a former operations engineer who was an early consumer of Amazon’s Elastic Container Service (ECS), and some of her learnings came about the “hard way.” This session is from the “Using Docker” track.
Fuller starts with reviewing the agenda, and shares that she’s intent on providing some practical tips that attendees can put to work immediately.
The first topic that Fuller tackles is the topic of container layers. A Docker container is made up of the read-only layers from the image itself, and a read/write layer at “the top” of the layers. Why do we care? Fewer layers means a smaller image, and smaller images means faster builds and faster deploys. (You may also see a reduced attack surface.)
The differences in making smaller images is important, Fuller explains, because the frequency of deployments is increasing (more deployments happening more quickly), and more containers are being deployed (sometimes at the behest of a CI/CD pipeline). This can result in significant amounts of disk space being consumed unnecessarily.
Some high-level Continue reading
In December 2016, I kicked off a migration to Linux (from OS X) as my primary laptop OS. In the nearly 4 months since the initial progress report, I’ve published a series of articles providing updates on things like which Linux distribution I selected, how I’m handling running VMs on my Linux laptop, and integration with corporate collaboration systems (here, here, and here). I thought that these “along the way” posts would be sufficient to keep readers informed, but I’ve had a couple of requests in the last week about how the migration is going. This post will help answer that question by summarizing what’s happened so far.
Let me start by saying that I am actively using a Linux-powered laptop as my primary laptop right now, and I have been doing so since early February. All the posts I’ve published so far have been updates of how things are going “in production,” so to speak. The following sections describe my current, active environment.
In my initial progress report, I’d tentatively chosen to use Ubuntu 16.04 LTS (“Xenial Xerus”). However, a short while later I switched to Fedora 25, and have settled Continue reading
Welcome to Technology Short Take #81! I have another collection of links, articles, and thoughts about key data center technologies, and hopefully I’ve managed to include something here that will prove useful or thought-provoking. Enjoy!
In discussing support for corporate communication and collaboration systems as part of my Linux migration, I’ve so far covered e-mail in part 1 and calendaring in part 2. In this post, I’m going to discuss the last few remaining aspects of corporate collaboration: instant messaging/chat, meetings and teleconferences, and document sharing.
The topic of teleconferences and meetings is closely related to calendaring—it’s often necessary to access your calendar or others’ calendars when coordinating meetings or teleconferences—so I encourage you to read part 2 to get a better feel for the challenges around calendaring/scheduling. All the same challenges from that post apply here. GNOME Calendar, although it offers basic Exchange Web Services (EWS) support, does not support meeting invitations, looking up attendees, free/busy information, etc. This makes it completely unusable for setting up meetings. Evolution provides the backend support that GNOME Calendar uses but may be better suited as a frontend; I haven’t tested this functionality so I don’t know. This EWS provider for Lightning does support free/busy information, inviting attendees, etc., so it may be a good option (I’m still testing it).
The second aspect of teleconferences/meetings is the actual conduct of the meeting itself. Hosting Continue reading
It seems as if finding the right Amazon Machine Image (AMI) ID for the workload you’d like to deploy can sometimes be a bit of a challenge. Each combination of region and AMI produces a unique ID, so you have to look up the AMI for the particular region where you’re going to deploy the workload. This in and of itself wouldn’t be so bad, but then you have to wade through multiple versions of the same AMI in each region. Fortunately, if you’re using CoreOS Container Linux on AWS, there’s an easy way to find the right AMI ID. Here’s how it works.
CoreOS publishes a JSON feed of the latest AMI for each of their channels (stable, beta, and alpha). You can find links to these JSON feeds on this page. This is powerful for 2 reasons:
Because it’s available via HTTP, you can use curl to retrieve it anytime you need it.
Because it’s in JSON, you can use jq (see my post on jq for more information) to easily parse it to find the information you need. (Not super comfortable with JSON? Check out my introductory post.)
Putting these two reasons together, you end up Continue reading