CentOS Atomic Host Customization Using cloud-init

Back in early March of this year, I wrote a post on customizing the Docker Engine on CentOS Atomic Host. In that post, I showed how you could use systemd constructs like drop-in units to customize the behavior of the Docker Engine when running on CentOS Atomic Host. In this post, I’m going to build on that information to show how this can be done using cloud-init on a public cloud provider (AWS, in this case).

Although I haven’t really blogged about it, I’d already taken the information in that first post and written some Ansible playbooks to do the same thing (see here for more information). Thus, one could use Ansible to do this when running CentOS Atomic Host on a public cloud provider. However, much like the original post, I wanted to find a very “cloud-native” way of doing this, and cloud-init seemed like a pretty good candidate.

All in all, it was pretty straightforward—with one significant exception. As I was testing this, I ran into an issue where the Docker daemon wouldn’t start after cloud-init had finished. Convinced I’d done something wrong, I kept going over the files, testing and re-testing (I’ve been working on this, off Continue reading

CentOS Atomic Host Customization Using cloud-init

Back in early March of this year, I wrote a post on customizing the Docker Engine on CentOS Atomic Host. In that post, I showed how you could use systemd constructs like drop-in units to customize the behavior of the Docker Engine when running on CentOS Atomic Host. In this post, I’m going to build on that information to show how this can be done using cloud-init on a public cloud provider (AWS, in this case).

Although I haven’t really blogged about it, I’d already taken the information in that first post and written some Ansible playbooks to do the same thing (see here for more information). Thus, one could use Ansible to do this when running CentOS Atomic Host on a public cloud provider. However, much like the original post, I wanted to find a very “cloud-native” way of doing this, and cloud-init seemed like a pretty good candidate.

All in all, it was pretty straightforward—with one significant exception. As I was testing this, I ran into an issue where the Docker daemon wouldn’t start after cloud-init had finished. Convinced I’d done something wrong, I kept going over the files, testing and re-testing (I’ve been working on this, off Continue reading

Announcing the Docker Student Developer Kit & Campus Ambassador Program!

For quite some time now we have been receiving daily requests from students all over the world, asking for our help learning Docker, using Docker and teaching their peers how to use Docker. We love their enthusiasm, so we decided it was time to reach out to the student community and give them the helping hand they need!

Understanding how to use Docker is now a must have skill for students. Here are 5 reasons why:

1. Understanding how to use Docker is one of the most important skills to learn if you want to advance in a career in tech, according to Business Insider.
2. You can just start coding instead of spending time setting up your environment.
3. You can collaborate easily with your peers and enable seamless group work: Docker eliminates any ‘works on my machine’ issues.
4. Docker allows you to easily build applications with a modern microservices architecture.
5. Using Docker will greatly enhance the security of your applications.

Getting Started with Docker

Are you a student who is excited about the prospect of using Docker but still don’t know exactly what Docker is or where to start learning? Now that your finals are over and you have all Continue reading

Docker Security at PyCon: Threat Modeling & State Machines

The Docker Security Team was out in force at PyCon 2017 in Portland, OR, giving two talks focussed on helping the Python Community to achieve better security. First up was David Lawrence and Ying Li with their “Introduction to Threat Modelling talk”.

Threat Modelling is a structured process that aids an engineer in uncovering security vulnerabilities in an application design or implemented software. The great majority of software grows organically, gaining new features as some critical mass of users requests them. These features are often implemented without full consideration of how they may impact every facet of the system they are augmenting.

Threat modelling aims to increase awareness of how a system operates, and in doing so, identify potential vulnerabilities. The process is broken up into three steps: data collection, analysis, and remediation. An effective way to run the process is to have a security engineer sit with the engineers responsible for design or implementation and guide a structured discussion through the three steps.

For the purpose of this article, we’re going to consider how we would  threat model a house, as the process can be applied to both real world scenarios in addition to software.

Data Collection

Five categories of Continue reading

Getting Started: Tower Projects and Inventories

Welcome to another post in our Getting Started series. In our previous post, we discussed how you can equip your Ansible Tower instance with users and credentials.

In this post, we will discuss how to set up projects and inventories in your Ansible Tower instance.

What Is A Tower Project?

Tower projects are a logical collection of Ansible Playbooks that are set up with each other based on what they might be doing or which hosts they might interact with.

Playbooks can be managed within Tower projects by either adding them manually to the project base path on your Tower server, (/var/lib/awx/projects) or by importing them from a source control management system (SCM) that is supported by Tower. Examples of SCMs supported by Tower are Git, Subversion and Mercurial. Managing your projects with an SCM is recommended to ensure that only users with assigned access to the repository can change the Playbook before execution, and for the extra layer of accountability and change control it provides. If your Playbooks are managed by an SCM, update options can be selected to “update on launch”, “delete on update” and “clean”.

If you select “update on launch", Tower will sync each Continue reading

Technology Short Take #83

Welcome to Technology Short Take #83! This is a slightly shorter TST than usual, which might be a nice break from the typical information overload. In any case, enjoy!

Networking

• I enjoyed Dave McCrory’s series on the future of the network (see part 1, part 2, part 3, and part 4—part 5 hadn’t gone live yet when I published this). In my humble opinion, he’s spot on in his viewpoint that network equipment is increasingly becoming more like servers, so why not embed services and functions in the network equipment? However, this isn’t enough; you also need a strong control plane to help manage and coordinate these services. Perhaps Istio will help provide that control plane, though I suspect something more will be needed.
• Michael Kashin has a handy little tool that functions like ssh-copy-id on servers, but for network devices (leveraging Netmiko). Check out the GitHub repository.
• Anthony Shaw has a good comparison of Ansible, StackStorm, and Salt (with a particular view at applicability in a networking context). This one is definitely worth a read, in my opinion.
• Miguel Gómez of Telefónica Engineering discusses maximizing performance in VXLAN overlay networks.
• Nicolas Michel has a good Continue reading

Bastion Hosts and Custom SSH Configurations

The idea of an SSH bastion host is something I discussed here about 18 months ago. For the most part, it’s a pretty simple concept (yes, things can get quite complex in some situations, but I think these are largely corner cases). For the last few months, though, I’ve been trying to use an SSH bastion host and failing, and I could not figure out why it wouldn’t work. The answer, it turns out, lies in custom SSH configurations.

In my introduction on using SSH bastion hosts (linked above)—or in just about any tutorial out there on using SSH bastion hosts—brief mention is made of adding configuration information to SSH to use the bastion host. Borrowing from my original post, if you had an instance named “private1” that you wanted to access via a bastion named “bastion”, the SSH configuration information might look like this:

Host private1
  IdentityFile ~/.ssh/rsa_private_key
  ProxyCommand ssh user@bastion -W %h:%p

Host bastion
  IdentityFile ~/.ssh/rsa_private_key

Normally, that information would go into ~/.ssh/config, which is the default SSH configuration file.

In my case, I only allow public key authentication to “trusted” systems (I vaguely recall an article I read a while ago about a Continue reading

Technology Short Take #83

Welcome to Technology Short Take #83! This is a slightly shorter TST than usual, which might be a nice break from the typical information overload. In any case, enjoy!

Networking

• I enjoyed Dave McCrory’s series on the future of the network (see part 1, part 2, part 3, and part 4—part 5 hadn’t gone live yet when I published this). In my humble opinion, he’s spot on in his viewpoint that network equipment is increasingly becoming more like servers, so why not embed services and functions in the network equipment? However, this isn’t enough; you also need a strong control plane to help manage and coordinate these services. Perhaps Istio will help provide that control plane, though I suspect something more will be needed.
• Michael Kashin has a handy little tool that functions like ssh-copy-id on servers, but for network devices (leveraging Netmiko). Check out the GitHub repository.
• Anthony Shaw has a good comparison of Ansible, StackStorm, and Salt (with a particular view at applicability in a networking context). This one is definitely worth a read, in my opinion.
• Miguel Gómez of Telefónica Engineering discusses maximizing performance in VXLAN overlay networks.
• Nicolas Michel has a good Continue reading

Bastion Hosts and Custom SSH Configurations

The idea of an SSH bastion host is something I discussed here about 18 months ago. For the most part, it’s a pretty simple concept (yes, things can get quite complex in some situations, but I think these are largely corner cases). For the last few months, though, I’ve been trying to use an SSH bastion host and failing, and I could not figure out why it wouldn’t work. The answer, it turns out, lies in custom SSH configurations.

In my introduction on using SSH bastion hosts (linked above)—or in just about any tutorial out there on using SSH bastion hosts—brief mention is made of adding configuration information to SSH to use the bastion host. Borrowing from my original post, if you had an instance named “private1” that you wanted to access via a bastion named “bastion”, the SSH configuration information might look like this:

Host private1
  IdentityFile ~/.ssh/rsa_private_key
  ProxyCommand ssh user@bastion -W %h:%p

Host bastion
  IdentityFile ~/.ssh/rsa_private_key

Normally, that information would go into ~/.ssh/config, which is the default SSH configuration file.

In my case, I only allow public key authentication to “trusted” systems (I vaguely recall an article I read a while ago about a Continue reading

Bastion Hosts and Custom SSH Configurations

The idea of an SSH bastion host is something I discussed here about 18 months ago. For the most part, it’s a pretty simple concept (yes, things can get quite complex in some situations, but I think these are largely corner cases). For the last few months, though, I’ve been trying to use an SSH bastion host and failing, and I could not figure out why it wouldn’t work. The answer, it turns out, lies in custom SSH configurations.

In my introduction on using SSH bastion hosts (linked above)—or in just about any tutorial out there on using SSH bastion hosts—brief mention is made of adding configuration information to SSH to use the bastion host. Borrowing from my original post, if you had an instance named “private1” that you wanted to access via a bastion named “bastion”, the SSH configuration information might look like this:

Host private1
  IdentityFile ~/.ssh/rsa_private_key
  ProxyCommand ssh user@bastion -W %h:%p

Host bastion
  IdentityFile ~/.ssh/rsa_private_key

Normally, that information would go into ~/.ssh/config, which is the default SSH configuration file.

In my case, I only allow public key authentication to “trusted” systems (I vaguely recall an article I read a while ago about a Continue reading

Technology Short Take #83

Welcome to Technology Short Take #83! This is a slightly shorter TST than usual, which might be a nice break from the typical information overload. In any case, enjoy!

Networking

• I enjoyed Dave McCrory’s series on the future of the network (see part 1, part 2, part 3, and part 4—part 5 hadn’t gone live yet when I published this). In my humble opinion, he’s spot on in his viewpoint that network equipment is increasingly becoming more like servers, so why not embed services and functions in the network equipment? However, this isn’t enough; you also need a strong control plane to help manage and coordinate these services. Perhaps Istio will help provide that control plane, though I suspect something more will be needed.
• Michael Kashin has a handy little tool that functions like ssh-copy-id on servers, but for network devices (leveraging Netmiko). Check out the GitHub repository.
• Anthony Shaw has a good comparison of Ansible, StackStorm, and Salt (with a particular view at applicability in a networking context). This one is definitely worth a read, in my opinion.
• Miguel Gómez of Telefónica Engineering discusses maximizing performance in VXLAN overlay networks.
• Nicolas Michel has a good Continue reading

Get involved with the Moby Project by attending upcoming Moby Summits!

Last month at DockerCon, we introduced the Moby Project: an open-source project sponsored by Docker to advance the software containerization movement. The idea behind the project is to help the ecosystem take containers mainstream by providing a library of components, a framework for assembling them into custom container-based systems and a place for all container enthusiasts to experiment and exchange ideas. Going forward, Docker will be assembled using Moby, see Moby and Docker or the diagram below for more details.

Moby Summit at DockerCon 2017

Knowing that that a good number of maintainers, contributors and advanced Docker users would be attending DockerCon, we decided to organize the first Moby Summit in collaboration with the Cloud Native Computing Foundation (CNCF). The summit was a small collaborative event for container hackers who are actively maintaining, contributing or generally involved or interested in the design and development of components of the Moby project library in particular: LinuxKit, containerd, Infrakit, SwarmKit, libnetwork and Notary.

Here’s what we covered during the first part of the summit:

• 0:05 – Opening words by Patrick Chanezon
• 9:05 – Moby Project Q&A with Solomon Hykes and Justin Cormack
• 60:14 – Quick update on containerd by Michael Continue reading

Container Deployment Demos from Interop ITX

At Interop ITX 2017 in Las Vegas, I had the privilege to lead a half-day workshop on options for deploying containers to cloud providers. As part of that workshop, I gave four live demos of using different deployment options. Those demos—along with the slides I used for my presentation along the way—are now available to anyone who might like to try them on their own.

The slides and all the resources for the demos are available in this GitHub repository. The four demos are:

1. Docker Swarm on EC2: This demo leverages Terraform and Ansible to stand up and configure a Docker Swarm cluster on AWS.

2. Amazon EC2 Container Service (ECS): This demo uses AWS CloudFormation to create an EC2 Container Service cluster with 3 instances and an Amazon RDS instance for backend database storage.

3. Kubernetes on AWS using kops: Using the kops CLI tool, this demo turns up a Kubernetes cluster on AWS to show how to deploy containerized applications on Kubernetes.

4. Google Container Engine: The final demo shows using Google Container Engine—which is Kubernetes—to deploy an application.

In the coming weeks, I plan to recreate the demos, record them, and publish them via YouTube, so that Continue reading

Container Deployment Demos from Interop ITX

At Interop ITX 2017 in Las Vegas, I had the privilege to lead a half-day workshop on options for deploying containers to cloud providers. As part of that workshop, I gave four live demos of using different deployment options. Those demos—along with the slides I used for my presentation along the way—are now available to anyone who might like to try them on their own.

The slides and all the resources for the demos are available in this GitHub repository. The four demos are:

1. Docker Swarm on EC2: This demo leverages Terraform and Ansible to stand up and configure a Docker Swarm cluster on AWS.

2. Amazon EC2 Container Service (ECS): This demo uses AWS CloudFormation to create an EC2 Container Service cluster with 3 instances and an Amazon RDS instance for backend database storage.

3. Kubernetes on AWS using kops: Using the kops CLI tool, this demo turns up a Kubernetes cluster on AWS to show how to deploy containerized applications on Kubernetes.

4. Google Container Engine: The final demo shows using Google Container Engine—which is Kubernetes—to deploy an application.

In the coming weeks, I plan to recreate the demos, record them, and publish them via YouTube, so that Continue reading

Container Deployment Demos from Interop ITX

At Interop ITX 2017 in Las Vegas, I had the privilege to lead a half-day workshop on options for deploying containers to cloud providers. As part of that workshop, I gave four live demos of using different deployment options. Those demos—along with the slides I used for my presentation along the way—are now available to anyone who might like to try them on their own.

The slides and all the resources for the demos are available in this GitHub repository. The four demos are:

1. Docker Swarm on EC2: This demo leverages Terraform and Ansible to stand up and configure a Docker Swarm cluster on AWS.

2. Amazon EC2 Container Service (ECS): This demo uses AWS CloudFormation to create an EC2 Container Service cluster with 3 instances and an Amazon RDS instance for backend database storage.

3. Kubernetes on AWS using kops: Using the kops CLI tool, this demo turns up a Kubernetes cluster on AWS to show how to deploy containerized applications on Kubernetes.

4. Google Container Engine: The final demo shows using Google Container Engine—which is Kubernetes—to deploy an application.

In the coming weeks, I plan to recreate the demos, record them, and publish them via YouTube, so that Continue reading

Spring Boot Development with Docker

The AtSea Shop is an example storefront application that can be deployed on different operating systems and can be customized to both your enterprise development and operational environments. In my last post, I discussed the architecture of the app. In this post, I will cover how to setup your development environment to debug the Java REST backend that runs in a container.

Building the REST Application

I used the Spring Boot framework to rapidly develop the REST backend that manages products, customers and orders tables used in the AtSea Shop. The application takes advantage of Spring Boot’s built-in application server, support for REST interfaces and ability to define multiple data sources. Because it was written in Java, it is agnostic to the base operating system and runs in either Windows or Linux containers. This allows developers to build against a heterogenous architecture.

Project setup

The AtSea project uses multi-stage builds, a new Docker feature, which allows me to use multiple images to build a single Docker image that includes all the components needed for the application. The multi-stage build uses a Maven container to build the the application jar file. The jar file is then copied to a Java Development Kit image. This Continue reading

Docker Federal Summit Recap and videos

On May 2nd, Docker returned to the Newseum to host the second annual Docker Federal Summit.  This one day event is designed to bring government agency developers, IT ops, program leaders and the ecosystem together to share and learn about the trends driving change in IT from containers, cloud and devops.  We expanded the agenda this year two tracks, with presentations from Docker, ecosystem partners, agency and community leaders to drive discussions, technology deep dives and hands on tutorials.

View the general session replay here:

General session table of content and slides

• 13:05 Iain Gray, SVP Customer Success discusses how Docker delivers a unique secure supply chain for all applications and infrastructure
• 33:35 Nathan McCauley, Director Security Engineering discusses the principles of least privilege design on which Docker is built
• 55:30 Modernize Traditional Apps to gain portability, security and efficiency without changing source code
• 59:13 Banjot Chanana, Senior Director Products delivers an overview and demo of Docker Enterprise Edition

In addition, the following breakout sessions dove deeper into pragmatic advice, security, development, cloud and compliance.

• Lessons Learned from Deploying Containers in Production featuring a panel discussion with Booz Allen Hamilton, JIDO, GSA and USCIS
• Scaling and Securing Applications on Your Continue reading

Ansible + Windows Webinar Q&A

The Ansible Ask an Expert webinar series continues to be one of the most popular series we’ve ever hosted. During these Q&A style webinars, our Ansible experts take questions from the audience about specific topics.

In April, we covered Ask an Expert: Windows. We’ve compiled the questions and answers below for your reference.

Interested in more? Our next Ask an Expert: Windows webinar is scheduled for August 10th at 2PM EDT. Register here.

Q: Any update on support for Windows machine as the control machine? This would make a lot of sense for Windows-only administrators who don't use Linux all the time.

A: There are several technical limitations that prevent the Ansible controller from running as a native Win32 application. However, Ansible does work under the new Windows Subsystem for Linux on Windows 10. While we don't officially support it for production workloads (nor does Microsoft), it does work quite well for developing and testing Ansible content.

Q: Is it possible to manage MySQL under Windows with Ansible?

A: Yes, the MySQL modules can manage Windows-hosted MySQL the same way as Linux-hosted MySQL. The modules themselves still need to actually run on a Linux/Mac host, but they're usually run from Continue reading

Developing a Spring Boot app on Docker: The AtSea Demo App

This is the first of a series of blog posts that demonstrates using Docker to develop a typical web application and deploying it in production. For DockerCon 2017, we wanted to build a new demo application that would demonstrate the flexibility of using Docker in development as well as showcase the features of Docker in a production environment. The result was the AtSea Shop, a storefront application that can be deployed on different operating systems and can be customized to both your enterprise development and operational environment.

A Hybrid Architecture

The team decided on a few ground rules. First, we wanted to use modern components commonly used in enterprise applications. We decided to build a Java application using the Spring Boot framework. The web client is a javascript application written using React as a framework.  Second, the application should be able to use any relational database and that it could be deployed on a Linux or Windows environment or cluster. Finally, the team wanted to show the process from development to deployment including building the application, implementing security, and deploying the application.

The application combines a typical Java n-tier architecture that uses Spring Boot’s web MVC framework for the REST API Continue reading

Docker at Microsoft Build 2017

Build is Microsoft’s premier developer event, run annually. This year Docker, Inc. and containers were everywhere, starting with a dedicated container pre-day, then with constant traffic to the Docker booth, and many shared container success stories.

Container Fest Pre-Day

Build is usually a three-day event, but this year saw the very first pre-day – run jointly by Docker and Microsoft. “Container Fest” was a whole-day event focused on containers and Docker, running on Windows and Linux, on-premises and in Azure.

There were 12 sessions throughout the day, presented by engineers and architects from Microsoft and Docker, Inc. They covered everything from the internals of Docker on Windows Server, through modernizing .NET Framework apps with Docker, to the options for running Docker containers on Azure.

A popular first step for modernizing traditional Windows applications is to use Image2Docker, which we demonstrated at the event. Image2Docker can extract existing applications from Windows machines into Dockerfiles, so you can automate the conversion of your app landscape to Docker. You can see Image2Docker in action from our session at DockerCon:

Over 300 people were at the Container Fest pre-day, and when the sessions had finished, they stayed on to run through the Hands-On Labs Continue reading