Technology Short Take #75
Welcome to Technology Short Take #75, the final Technology Short Take for 2016. Fortunately, it’s not the final Technology Short Take ever, as I’ll be back in 2017 with more content. Until then, here’s some data center-related articles and links for your enjoyment.
Networking
- Ajay Chenampara has some observations about running Ansible at scale against network devices.
- Andrey Khomyakov shares some information on automating the setup of whitebox switches running Cumulus Linux in part 2 of this series on learning network automation.
- Russell Bryant has shared the results of some testing comparing ML2+OVS and OVN as backends for OpenStack networking. As Russell indicates in his post, some additional analysis is needed to truly understand what’s happening, but early looks at the results of his tests show performance improvements in OVN versus ML2+OVS when it comes to total time required to boot a VM.
- Ivan Pepelnjak shares a Python script that creates Ansible inventory from Vagrant’s SSH configuration. Handy.
Servers/Hardware
Nothing this time around!
Security
- James Green shares some information on Docker Bench, a tool designed to help secure Docker environments.
- Via Mike Foley, the community now has a new resource to help educate on VM escapes.
- Gordon Turner Continue reading
Top Docker content of 2016
2016 has been an amazing year for Docker and the container industry. We had 3 major releases of Docker engine this year , and tremendous increase in usage. The community has been following along and contributing amazing Docker resources to help you learn and get hands-on experience. Here’s some of the top read and viewed content for the year:
Releases
Of course releases are always really popular, particularly when they fit requests we had from the community. In particular, we had:
- Docker for Mac & Docker for Windows Beta and GA release blog posts, and the video
- Docker 1.12 Built-in Orchestration release, and the DockerCon keynote where we announced it
- And the release of the Docker for AWS and Azure beta
Windows Containers
When Microsoft made Windows 2016 generally available, people rushed to
- Our release blog to read the news
- Tutorials to find out how to use Windows containers powered by Docker
- The commercial relationship blog post to understand how it all fits together
About Docker
We also provide a lot of information about how to use Docker. In particular, these posts and articles that we shared on social media were the most read:
Looking Back: 2016 Project Report Card
As I’ve done for the last few years, in early 2016 I published a list of my planned personal projects for the year. In this post, I’d like to look back on that list of projects and grade myself on my progress (or lack of progress, if that is the case). Even though 2016 isn’t over yet, it’s close enough to the end of the year that things won’t change that much before 2017 is upon us.
For reference, here’s the list of planned 2016 projects:
- Complete a new book (again)
- Make more open source contributions
- Expand my knowledge and use of Python
- Expand my knowledge, use, and focus on public cloud services
- Complete a “wildcard project”
Let’s look at each of these planned projects and see how I fared.
-
Complete a new book (again): Well—in the spirit of total honesty and transparency—this was a major failure. Not only did I fail to complete the network automation book I’ve been working on with Matt Oswalt and Jason Edelman, but the other book project I had planned also did not go anywhere. Granted, the circumstances preventing the second book project were outside my control, but the fact remains I still did Continue reading
Our Most Popular Blog Posts of 2016
2016, what a year. Ansible upgrades galore, Tower 3 was released, a tipping point for DevOps, and much more.
All these themes were reflected in our blog this year. From doing more with automation, working across platforms (think Windows automation), orchestrating containers at scale, to exploring all the great new features in Tower 3, we covered a lot.
Just in case you missed them, here are our 10 most viewed blog posts of the year (plus a sneaky few honorable mentions).
1. 6 Ways Ansible Makes Docker-Compose Better
Containers are an integral part of DevOps workflows. With containers you can be sure that if you build an application once, you can run it in the same way across every environment along the application lifecycle. That’s great, until one developer announces the need for a second, third, or fourth container. More of them, all doing different things, and all connecting together – somehow. But how? Docker has a tool that can help – docker-compose. But it’s limited to environments with a Docker-centric view of the world and doesn’t solve non-Docker orchestration problems. That’s where Ansible comes in. Read more
2. Testing Ansible Roles with Docker
Ansible plus Docker was a big deal in 2016, Continue reading
Opening Web Internet Location Files on Ubuntu
As part of my effort to make myself and my workflows more “cross-platform friendly,” I’ve been revisiting certain aspects of how I do things. One of the things I’m reviewing is how I capture—and later review—posts or articles on the web. On OS X, I would run an AppleScript that generated a .webloc file (aka an Internet location file). This is an XML file that OS X understands. However, Linux doesn’t natively understand these files, so today I came up with a solution to reading .webloc files with Ubuntu and Firefox.
The solution to the file involves the use of xmllint, a tool that you can install on Ubuntu as part of the “libxml2-utils” package. Using xmllint, you can easily extract a single XML element from an XML file—and .webloc files are just XML files. For the sake of illustration, here’s the contents of a .webloc file generated on OS X:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>URL</key>
<string>http://blog.fntlnz.wtf/post/systemd-nspawn/</string>
</dict>
</plist>
Using xmllint, you can extract the URL value, and then pass Continue reading
Understanding Docker Networking Drivers and their use cases
Applications requirements and networking environments are diverse and sometimes opposing forces. In between applications and the network sits Docker networking, affectionately called the Container Network Model or CNM. It’s CNM that brokers connectivity for your Docker containers and also what abstracts away the diversity and complexity so common in networking. The result is portability and it comes from CNM’s powerful network drivers. These are pluggable interfaces for the Docker Engine, Swarm, and UCP that provide special capabilities like multi-host networking, network layer encryption, and service discovery.
Naturally, the next question is which network driver should I use? Each driver offers tradeoffs and has different advantages depending on the use case. There are built-in network drivers that come included with Docker Engine and there are also plug-in network drivers offered by networking vendors and the community. The most commonly used built-in network drivers are bridge, overlay and macvlan. Together they cover a very broad list of networking use cases and environments. For a more in depth comparison and discussion of even more network drivers, check out the Docker Network Reference Architecture.
Bridge Network Driver
The bridge networking driver is the first driver on our list. It’s simple to understand, Continue reading
Docker & Prometheus Joint Holiday Meetup Recap
Last Wednesday we had our 52nd meetup at Docker HQ, but this time we joined forces with the Prometheus user group to host a mega-meetup! There was a great turnout and members were excited to see the talks on using Docker with Prometheus, OpenTracing and the new Docker playground; play-with-docker.
First up was Stephen Day, a Senior Software Engineer at Docker, who presented a talk entitled, ‘The History of Metrics According to Me’. Stephen believes that metrics and monitoring should be built into every piece of software we create, from the ground up. By solving the hard parts of application metrics in Docker, he thinks it becomes more likely that metrics are a part of your services from the start. See the video of his intriguing talk and slides below.
‘The History of Metrics According to Me’ @stevvooe talking #metrics and #monitoring at the #Docker SF #meetup! @prometheusIO @CloudNativeFdn pic.twitter.com/6hk0yAtats
— Docker (@docker) December 15, 2016
Next up was Ben Sigelman, an expert in distributed tracing, whose talk ‘OpenTracing Isn’t Just Tracing: Measure Twice, Instrument Once’ was both informative and humorous. He Continue reading
The Linux Migration: Initial Progress Report
About 4 years ago, I discussed some changes in the Apple ecosystem that might lead me to move away from OS X. To be honest, I’ve made only token efforts since that time to actually migrate away, even though the forces that I described in that post are still in full effect. In fact, some might say that the “iOS-ification” of OS X (now rebranded as “macOS”) is even stronger now. As a result, I’ve stepped up my work on a Linux migration, and I’m happy to report that I’ve made some progress.
Here’s a quick update on where things stand so far.
Linux Distribution
I’ve looked at a fair number of Linux distributions. I tried Elementary OS, which some have raved about but which I found too simplistic. I also went back and looked again at Ubuntu derivatives like Linux Mint. Given that Ubuntu is itself derived from Debian, I also took a look at Debian “Jessie”. Finally, I tested Fedora 25. For a number of reasons—which I’ll describe in more detail in a moment—I’ve settled on Ubuntu 16.04.
So, why Ubuntu 16.04 “Xenial Xerus”? Keep in mind that the reasons I list below are my Continue reading
Announcing Federal Security and Compliance Controls for Docker Datacenter
Security and compliance are top of mind for IT organizations. In a technology-first era rife with cyber threats, it is important for enterprises to have the ability to deploy applications on a platform that adheres to stringent security baselines. This is especially applicable to U.S. Federal Government entities, whose wide-ranging missions, from public safety and national security, to enforcing financial regulations, are critical to keeping policy in order.
Hiding the User List on the Ubuntu Login Screen
In this post, I’m going to share how to hide the user list on the login screen for Ubuntu 16.04. The information here isn’t necessarily new or ground-breaking; however, in searching for the solution myself I found a lot of conflicting information as to how this may or may not be accomplished. I’m publishing this post in the hopes of providing a bit more clarity around this topic.
I’ve verified that this procedure works on the desktop distribution of Ubuntu 16.04. Note also that this is probably not the only way of making this work; it’s likely there are other ways of accomplishing the same thing.
To make configuration changes to the login screen, you’ll want to add configuration files to /etc/lightdm/lightdm.conf.d. I used a single file to hide the user list and disable guest logins; presumably, you could use separate files for each configuration directive.
To disable the user list and disallow guest logins, add this content to a file in the etc/lightdm/lightdm.conf.d directory (I used the filename 00-hide-user-list.conf):
[SeatDefaults]
greeter-hide-users=true
greeter-show-manual-login=true
allow-guest=false
Once this file is in place, you’ll need to either restart your Ubuntu system, or restart the LightDM Continue reading
An Early Look at Ansible Container v0.3.0
The Ansible Container project is targeting mid-January for its next release, and so we thought now would be a good time to check in and look at the features actively under development and anticipated to ship.
With only a glance at the roadmap page, the casual visitor may think it seems a bit smallish, having only three items. However, the items represent features that are important to the project, and require a level of effort that’s anything but small, as we’ll see.
Building container images
The first item up is an image build cache. Building container images is of course a core function of the tool, and having a caching mechanism can improve the speed at which images are built.
If you’re not familiar with container images and how they’re built, think of an image as a tall building with dozens of floors, where each floor is layered on top of the previous floor, starting with the building’s foundation, and adding one floor or layer at a time until you reach the top. In the same way, a container image is a file system built in layers.
The build process starts with a base image, say Fedora 25, Continue reading
More details about containerd, Docker’s core container runtime component
Today we announced that Docker is extracting a key component of its container platform, a part of the engine plumbing–containerd a core container runtime–and commits to donating it to an open foundation. containerd is designed to be less coupled, and easier to integrate with other tools sets. And it is being written and designed to address the requirements of the major cloud providers and container orchestration systems.
Because we know a lot of Docker fans want to know how the internals work, we thought we would share the current state of containerd and what we plan for version 1.0. Before that, it’s a good idea to look at what Docker has become over the last three and a half years.
The Docker platform isn’t a container runtime. It is in fact a set of integrated tools that allow you to build ship and run distributed applications. That means Docker handles networking, infrastructure, build, orchestration, authorization, security, and a variety of other services that cover the complete distributed application lifecycle.
The core container runtime, which is containerd, is a small but vital part of the platform. We started breaking out containerd from the rest of the engine in Docker 1.11, Continue reading
containerd – a core container runtime project for the industry
Today Docker is spinning out its core container runtime functionality into a standalone component, incorporating it into a separate project called containerd, and will be donating it to a neutral foundation early next year. This is the latest chapter in a multi-year effort to break up the Docker platform into a more modular architecture of loosely coupled components.
Over the past 3 years, as Docker adoption skyrocketed, it grew into a complete platform to build, ship and run distributed applications, covering many functional areas from infrastructure to orchestration, the core container runtime being just a piece of it. For millions of developers and IT pros, a complete platform is exactly what they need. But many platform builders and operators are looking for “boring infrastructure”: a basic component that provides the robust primitives for running containers on their system, bundled in a stable interface, and nothing else. A component that they can customize, extend and swap out as needed, without unnecessary abstraction getting in their way. containerd is built to provide exactly that.
What Docker does best is provide developers and operators with great tools which make them more productive. Those tools come from integrating many different components into a Continue reading
Installing Ansible 2.2 on Ubuntu 16.04
A few weeks ago I wrote a post about installing Ansible 2.2 on Fedora 25; today, I’d like to tackle what’s involved in installing Ansible 2.2 on Ubuntu 16.04. This post, like its Fedora counterpart, stems from my ongoing evaluation of Linux distributions and desktop environments. While the information here is very similar to the information in the Fedora post, I’m putting it in its own post in the hopes of making the information easier for readers to find.
It’s not really a secret that I like to run Ansible in a Python virtualenv, but I don’t believe that it will make any difference to the procedure described in this post. The errors that result when trying to install Ansible 2.2 without the necessary prerequisite packages should be the same either way (in a virtualenv or not). However, I’m happy to be corrected if someone knows otherwise.
To create a Python virtualenv, you’ll first need virtualenv installed. I prefer to install virtualenv globally for all users using this command:
sudo -H pip install virtualenv
Alternately, you could install it via a package, with apt install virtualenv. As far as I can tell, either approach Continue reading
‘Tis the Season to Decorate Your Playbook
The holidays are upon us. And while we can't automate your gift wrapping, we can make your Playbook more festive.
Introducing Ansible holiday-themed cowsay!
Copy and paste this code:
---
curl "https://gist.githubusercontent.com/jlaska/14bc829af01526add07efcaa83582aaf/raw/5f4918be06ffa69ea848354c5563c6a2d7b59807/happy_holidays.sh" | bash
Decorations are popping up everywhere so why not in your Playbook?
Share your designs with #AnsibleCowsay @ansible
Tweet #AnsibleCowsayAscii sources: chris.com and ascii-code.com
Convert ASP.NET Web Servers to Docker with Image2Docker
A major update to Image2Docker was released last week, which adds ASP.NET support to the tool. Now you can take a virtualized web server in Hyper-V and extract a Docker image for each website in the VM – including ASP.NET WebForms, MVC and WebApi apps.
Image2Docker is a PowerShell module which extracts applications from a Windows Virtual Machine image into a Dockerfile. You can use it as a first pass to take workloads from existing servers and move them to Docker containers on Windows.
The tool was first released in September 2016, and we’ve had some great work on it from PowerShell gurus like Docker Captain Trevor Sullivan and Microsoft MVP Ryan Yates. The latest version has enhanced functionality for inspecting IIS – you can now extract ASP.NET websites straight into Dockerfiles.
In Brief
If you have a Virtual Machine disk image (VHD, VHDX or WIM), you can extract all the IIS websites from it by installing Image2Docker and running ConvertTo-Dockerfile like this:
Install-Module Image2Docker Import-Module Image2Docker ConvertTo-Dockerfile -ImagePath C:\win-2016-iis.vhd -Artifact IIS -OutputPath c:\i2d2\iis
That will produce a Dockerfile which you can build into a Windows container image, using docker build.
How It Works
The Image2Docker Continue reading
DockerCon 2017: Registration And CFP Now Open!
DockerCon 2017 tickets are now available! Take advantage of our lowest pricing today – tickets are limited and Early Bird will sell out fast! We have extended DockerCon to a three-day conference with repeat sessions, hands-on labs and summits taking place on Thursday.
Register for DockerCon
The DockerCon 2017 Call for Proposals is open! Before you submit your cool hack or session proposals, take a look at our tips for getting selected below. We have narrowed the scope of sessions we’re looking for this year down to Cool Hacks and Use Cases. The deadline for submissions is January 14, 2017 at 11:59 PST.
Submit a talk
Proposal Dos:
Submitting a Cool Hack:
Be novel
Show us your cool hacks and wow us with the interesting ways you can push the boundaries of the Docker stack. Check out past audience favorites like Serverless Docker, In-the-air update of a drone with Docker and Resin.io and building a UI for container management with Minecraft for inspiration.
Be clear
You do not have to have your hack ready by the submission deadline, rather, plan to clearly explain your hack, what makes it cool and the technologies you will use.
All Continue reading
Docker for Azure Public Beta
Last week Docker for AWS went public beta, and today Docker for Azure reached the same milestone and is ready for public testing. Docker for Azure is a great way for ops to setup and maintain secure and scalable Docker deployments on Azure.
With Docker for Azure, IT ops teams can:
- Deploy a standard Docker platform to ensure teams can seamlessly move apps from developer laptops to Dockerized staging and production environments, without risk of incompatibilities or lock-in.
- Integrate deeply with underlying infrastructure to ensure Docker takes advantage of the host environment’s native capabilities and exposes a familiar interface to administrators.
- Deploy the platform to all the places where you want to run Dockerized apps, simply and efficiently
- Make sure the latest and greatest Docker versions are available for the hardware, OSs, and infrastructure you love, and provide solid upgrade paths from one Docker version to the next.
To try the latest Docker for Azure beta based on the latest Docker Engine betas, click the button below or get more details on the beta site:
Installation takes a few minutes, and will give you a fully functioning swarm, ready to deploy and scale Dockerized apps.
We first unveiled the Docker for Continue reading
Tips for Troubleshooting Apps in Production with Docker Datacenter
If you have been using Docker for some time, after the initial phases of building Dockerfiles and running a container here and there, the real work begins in building, deploying and operating multi-container applications in a production environment. Are you operationally ready to take your application to production? Docker Datacenter provides an integrated management framework for your Dockerized environment and applications and when coupled with clear strategies in approaching and resolving anomalies, IT ops teams can be assured in successfully operationalizing Docker.
Let’s use a sports metaphor to approach troubleshooting:
- Pre-Game will cover the planning phase for your applications
- Game Time will cover troubleshooting tools available in Docker Datacenter
- Post-Game will discuss complementary tools to aid in ongoing insights
Pre-Game
Whether or not you are sports fan, you can appreciate the importance of the planning out any task. This is no different than what you would do for your applications. Health checks are a great way to provide a deeper level of insight into how your application is performing. Since Docker 1.12 there is a new HEALTHCHECK directive. We can use this directive to signal to the Docker Engine whether or not the application is healthy.
There are a two ways to implement the HEALTHCHECK Continue reading
Learn Docker with More Hands-On Labs
Docker Labs is a rich resource for technical folks from any background to learn Docker. Since the last update on the Docker Blog, three new labs have been published covering Ruby, SQL Server and running a Registry on Windows. The self-paced, hands-on labs are a popular way for people to learn how to use Docker for specific scenarios, and it’s a resource which is growing with the help of the community.
New Labs
- Ruby FAQ. You can Dockerize Ruby and Ruby on Rails apps, but there are considerations around versioning, dependency management and the server runtimes. The Ruby FAQ walks through some of the challenges in moving Ruby apps to Docker and proposes solutions. This lab is just beginning, we would love to have your contributions.
- SQL Server Lab. Microsoft maintain a SQL Server Express image on Docker Hub that runs in a Windows container. That image lets you attach an existing database to the container, but this lab walks you through a full development and deployment process, building a Docker image that packages your own database schema into an image.
- Registry Windows Lab. Docker Registry is an open-source registry server for storing Docker images, which you can run Continue reading