How Docker for Mac helps me sleep better at night
My name is Matt Aimonetti, I’m the co-founder and CTO of Splice. At Splice, we built a cloud platform for music producers, this platform is made of elements engineers often take for granted. We invented version control for music, a distributed collaboration flow and a subscription based marketplace for samples, loops presets and MIDI. All that without changing the creation tools musicians already know and like.
I’m a developer and an entrepreneur, the last thing I want to worry about are ops concerns.
Building serverless apps with Docker
Every now and then, there are waves of technology that threaten to make the previous generation of technology obsolete. There has been a lot of talk about a technique called “serverless” for writing apps. The idea is to deploy your application as a series of functions, which are called on-demand when they need to be run. You don’t need to worry about managing servers, and these functions scale as much as you need, because they are called on-demand and run on a cluster.
But serverless doesn’t mean there is no Docker – in fact, Docker is serverless. You can use Docker to containerize these functions, then run them on-demand on a Swarm. Serverless is a technique for building distributed apps and Docker is the perfect platform for building them on.
From servers to serverless
So how might we write applications like this? Let’s take our example a voting application consisting of 5 services:
This consists of:
- Two web frontends
- A worker for processing votes in the background
- A message queue for processing votes
- A database
The background processing of votes is a very easy target for conversion to a serverless architecture. In the voting app, we can run a Continue reading
Introducing the Docker Store Private Beta
We are very excited to announce the private beta of Docker Store, a marketplace for trusted and validated dockerized software – free, open source and commercial.
Our goals with Docker Store are designed around bringing Docker users and ecosystem partners together.
- Provide a scalable self-service system for ISVs to publish and distribute trusted and enterprise-ready content
- Provide a publishing process that validates software quality, including; security scanning, component inventory, the open-source license usage and use of best practices in image construction.
- Provide enterprise users with compliant, commercially supported software from trusted and verified publishers, that is packaged as Docker images.
- We’ve added powerful search and browsing capabilities, including categorization.
The use and creation of dockerized content has grown exponentially in the last couple of years. This demand on content and the expanded use of Docker within the enterprise naturally led to the need for more content, entitlement, visibility into security profiles and compliance.
Docker Store builds on the Official Images and the popularity of the Docker Hub for community content by providing an official marketplace that provides workflows for those who wish to create and distribute content and those that wish to download content to build their Continue reading
Docker Datacenter in AWS and Azure in a few clicks
Introducing Docker Datacenter AWS Quickstart and Azure Marketplace Templates production-ready, high availability deployments in just a few clicks.
The Docker Datacenter AWS Quickstart uses a CloudFormation templates and pre-built templates on Azure Marketplace to make it easier than ever to deploy an enterprise CaaS Docker environment on public cloud infrastructures.
The Docker Datacenter Container as a Service (CaaS) platform for agile application development provides container and cluster orchestration and management that is simple, secure and scalable for enterprises of any size. With our new cloud templates pre-built for Docker Datacenter, developers and IT operations can frictionlessly move dockerized applications to an Amazon EC2 or Microsoft Azure environment without any code changes. Now businesses can quickly realize greater efficiency of computing and operations resources and Docker supported container management and orchestration in just a few steps.
What is Docker Datacenter?
Docker Datacenter includes Docker Universal Control Plane, Docker Trusted Registry (DTR), CS Docker Engine with commercial support & subscription to align to your application SLAs:
- Docker Universal Control Plane (UCP), an enterprise-grade cluster management solution that helps you manage your whole cluster from a single pane of glass
- Docker Trusted Registry (DTR), an image storage solution that helps securely store and manage the Docker Continue reading
Docker for the Enterprise with Docker Datacenter
Businesses today are digital and fundamentally powered by applications – software that drives revenue, engages with customers and runs their operations. The process of making that software has, until recently, been long and cumbersome. The addition of Docker and containerization, a new software supply chain is enabled to bring agility, portability and control together into the enterprise. From security and compliance to shipping more software faster to migrating workloads around sites for the best cost to performance ratio, Docker Datacenter is helping businesses transform their software supply chain.
At DockerCon in Seattle, business of all sizes are sharing their stories Dockering for transformation in development, CI and production environments for all kinds of apps. In this post, I wanted to highlight those companies who are using Docker Datacenter.
HealthDirect Australia: Docker in Production, Look No Hands!
Healthdirect Australia was a long time open source Docker user who in the last year transitioned to a Docker Datacenter environment to gain the benefit of having out of the box integrations of a full Docker supported stack. Scott Coulton, the lead architect (and also Docker Captain and DockerCon speaker!) is the driving force behind deploying and running the applications Continue reading
DockerCon 2016 Vendor Meetings
While at DockerCon 2016 in Seattle today, I took some time on the expo floor to talk to a number of different vendors, mostly focused on networking solutions. Here are some notes from these discussions. I may follow up with additional posts on some of these technologies; it will largely depend on time and the ease by which the technologies/products may be consumed.
Plumgrid
My first stop was the Plumgrid booth. I’d heard of Plumgrid, but wanted to take this time to better understand their architecture. As it turns out, their architecture is quite interesting. Plumgrid is one of the primary commercial sponsors behind the IO Visor project, a Linux Foundation project, which leverages the extended Berkeley Packet Filter (eBPF) subsystem in the Linux kernel. Using eBPF, Plumgrid has created in-kernel virtual network functions (VNFs) that do things like bridging, routing, network address translation (NAT), and firewalling. Combined with a scale-out central control plane and leveraging the Linux kernel’s built-in support for VXLAN, this enables Plumgrid to create overlay networks and apply very granular security policies to attached workloads (which could be VMs or containers).
Project Calico
Next, I stopped by the Calico booth. Unlike many of the networking Continue reading
DockerCon 2016 Day 2 Keynote
This is a liveblog for the day 2 keynote of DockerCon 2016, which wraps up today in Seattle, WA. While today’s pre-keynote warm-up doesn’t include laser-equipped kittens, the music is much more upbeat and energetic (as opposed to yesterday’s more somber, dramatic music). If the number of laptops on the podium is any indicator (yesterday it was a cue to the number of demos planned), then today’s keynote will include a few demos as well.
Ben Golub kicks off the day 2 keynote—with the requisite coffee shot that is a sacrifice to the “demo gods”—and offers up some thanks to the supporters of last night’s party at the Space Needle. Golub quick reviews the key announcements and demos from the day 1 keynote (see my liveblog here). Today, though, will be focused on democratizing Docker in the enterprise. In referring to Docker’s adoption in the enterprise, Golub shares some numbers that vary widely, and admits that it’s really difficult to know what the real adoption rate is. He points to multiple “critical transformations” occurring within the enterprise: application modernization, cloud adoption, and DevOps (process/procedure/culture changes).
This leads Golub into a discussion of anti-patterns, or fallacies. The first fallacy he Continue reading
Docker 1.12: Now with Built-in Orchestration!
Three years ago, Docker made an esoteric Linux kernel technology called containerization simple and accessible to everyone. Today, we are doing the same for container orchestration.
Container orchestration is what is needed to transition from deploying containers individually on a single host, to deploying complex multi-container apps on many machines. It requires a distributed platform, independent from infrastructure, that stays online through the entire lifetime of your application, surviving hardware failure and software updates. Orchestration is at the same stage today as containerization was 3 years ago. There are two options: either you need an army of technology experts to cobble together a complex ad hoc system, or you have to rely on a company with a lot of experts to take care of everything for you as long as you buy all hardware, services, support, software from them. There is a word for that, it’s called lock-in.
Docker users have been sharing with us that neither option is acceptable. Instead, you need a platform that makes orchestration usable by everyone, without locking you in. Container orchestration would be easier to implement, more portable, secure, resilient, and faster if it was built into the platform.
Starting with Docker 1.12, Continue reading
Announcing the Docker for Mac and Windows Public Beta
Back in March, we launched a private beta for a new ambitious project called Docker for Mac and Docker for Windows. Our major goal was to bring a native Docker experience to Mac and Windows, making it easier for developers to work with Docker in their own environments. And thousands agreed. Over thirty thousand applied in the first 24 hours. And by last week, we let in over seventy thousand.
And now all you need to get started developing is Docker and a text editor. No more installing dependencies and runtimes just to debug applications.
Introducing the Docker for AWS and Azure Beta
Today, we’re excited to announce Docker for AWS and Docker for Azure: the best ways to install, configure and maintain Docker deployments on AWS and Azure.
Our goals for Docker for AWS and Azure are the same as for Docker for Mac and Windows:
- Deploy a standard Docker platform to ensure teams can seamlessly move apps from developer laptops to Docker staging and production environments, without risk of incompatibilities or lock-in.
- Integrate deeply with underlying infrastructure to make sure Docker takes advantage of the host environment’s native capabilities and exposes a familiar interface to administrators.
- Deploy the Docker platform to all the places where you want to run containerized apps, simply and efficiently and at no extra cost.
- Make sure the latest and greatest Docker versions are available for the hardware, OSs, and infrastructure you love, and provide solid upgrade paths from one Docker version to the next.
Introducing Experimental Distributed Application Bundles
The built-in orchestration features announced today with Docker 1.12 will revolutionize how IT teams build, ship and run containerized apps. With Docker 1.12, developers and ops now share a set of simple and powerful APIs, tools, and formats for building agile delivery pipelines that ship software from development through CI to production in the cloud with Docker for AWS and Azure.
To facilitate that revolution, we’re introducing Distributed Application Bundles—an experimental open file format for bundling up all the artifacts required to ship and deploy multi-container apps: a DAB contains a description of all the services required to run the application and details images to use, ports to expose, and the networks used to link services.
5 Reasons We Started the Ansible Container Project
Here at Ansible, we recently announced a new project called Ansible Container. Its purpose is to allow users to build, deploy, and orchestrate containers at scale, all from Ansible playbooks.
It’s still a young project, barely a month old at this point -- but we’re excited by it, and we think it has a great deal of potential. Here are five reasons why.
1. Because our community has been using Ansible to manage containers for quite a while now.
Ansible has been successful, in large part, by following where our community leads, and our community has been using Ansible to help manage containers for nearly as long as Ansible has been around. Our community wrote the original Docker module in October 2013, and that module and other container modules have been among the most frequently used Ansible modules ever since. There are hundreds of community-maintained Ansible container images in Dockerhub, and there are excellent blog posts in which Ansible community members describe their own best practices for building and deploying containers. The next logical step was to start a project to bring together some of these best practices into tools that anyone could use.
2. Because the new Docker Continue reading
DockerCon 2016 Day 1 Keynote
This is a liveblog for the day 1 keynote of DockerCon 2016, taking place over the next couple of days in Seattle, WA. Before the keynote starts in earnest, Gordon the Turtle entertains attendees with some “special” Docker containers that affect the display on the main stage: showing butterflies, playing sounds, launching a Docker-customized version of Pac-Man, or initiating a full-out battle of laser-shooting kittens.
The keynote starts with Ben Golub taking the stage to kick things off. Golub begins his portion with a quick “look back” at milestones from previous Docker events and the history of Docker (the open source project). Golub calls out a few particular sessions—protein folding, data analysis in sports, and extending a video game—and then unveils that these sessions are being presented by kids under the age of 13.
This leads Golub into a review of the efforts of Docker (the company) to democratize containers:
- Increasing usability
- Enhancing portability
- Extending community
Golub gives a “shout out” to the technologies underpinning modern Linux containers (namespaces, cgroups, etc., and their predecessors) and calls out the 2,900+ contributors to the open source Docker project. He then spends the next several minutes talking about various metrics—pull requests, containers Continue reading
DockerCon 2016 Day 2 Keynote
This is a liveblog for the day 2 keynote of DockerCon 2016, which wraps up today in Seattle, WA. While today’s pre-keynote warm-up doesn’t include laser-equipped kittens, the music is much more upbeat and energetic (as opposed to yesterday’s more somber, dramatic music). If the number of laptops on the podium is any indicator (yesterday it was a cue to the number of demos planned), then today’s keynote will include a few demos as well.
Ben Golub kicks off the day 2 keynote—with the requisite coffee shot that is a sacrifice to the “demo gods”—and offers up some thanks to the supporters of last night’s party at the Space Needle. Golub quick reviews the key announcements and demos from the day 1 keynote (see my liveblog here). Today, though, will be focused on democratizing Docker in the enterprise. In referring to Docker’s adoption in the enterprise, Golub shares some numbers that vary widely, and admits that it’s really difficult to know what the real adoption rate is. He points to multiple “critical transformations” occurring within the enterprise: application modernization, cloud adoption, and DevOps (process/procedure/culture changes).
This leads Golub into a discussion of anti-patterns, or fallacies. The first fallacy he Continue reading
DockerCon Cool Hack Challenge: Tyrion Cannister Neural Style GUI
As students at Holberton School, a software engineering school based in San Francisco, Siphan and I are exposed to lots of exciting technology! The main goal of the school is to produce full-stack engineers in two years. Although we are only four months into the program, we are already learning how to use the Docker platform.
A few weeks ago, we held our very first hackathon at school – and of course it was focused on Docker! The school’s founders (one of whom was an original member of the Docker marketing #boomteam) thought now was the perfect time for us to participate in a hackathon, so they teamed up with Docker to make it happen.
In the spirit of DockerCon, our guidelines were basically the same as those of DockerCon Cool Hack Challenge: make the most awesome things we could think of using Docker, and do it in ten hours. Cue the intense music.
Webinar: Containerization for the Virtualization Admin
If you’re a virtualization admin there’s a good chance that the word “container” has floated by in the last few months. This is because enterprises of all sizes are now looking to leverage the power of containerization. But what is containerization and how can you leverage containers while also making use of your investment in virtualization?
Announcing Our Next Docker Hackathon!
In just a couple of days, over four thousand people will be joining us in Seattle for DockerCon 2016 to learn from top practitioners, take part in hands-on labs, engage with Docker ecosystem innovators and meet others in the Docker Community.
We realize that attendees were bummed when we didn’t announce an in-person hackathon but we have been working hard to remedy this by organizing an online hackathon for participants to hack on the newest features and products coming out of DockerCon!
Help Docker’s Initiatives to Promote Tech Diversity Including Bump Up at DockerCon
Docker is actively working to improve opportunities for women and underrepresented minorities throughout the global ecosystem and promote diversity in the larger tech community.
We’re proud to have contributed close to $100,000 in sponsorships, scholarships and complementary tickets to DockerCon. In addition to these funds, the Docker Team has launched and supports several ongoing initiatives with like-minded partners. Continue reading
Configuring Linux Policy Routing using Ansible
In this post, I’m going to talk about using Ansible to configure policy routing on Linux. If you’re not familiar with Linux policy routing, have a look at this post, and also review this post for one potential use case (I’m sure there are a number of other quite valuable use cases).
As you may recall from the policy routing introductory post, there are three steps involved in configuring policy routing:
- You must define the new routing table in
/etc/iproute2/rt_tables - You must add routes to the new routing tables
- You must define rules for when the new routing table is consulted
All three of these tasks can be handled via Ansible.
To address step #1, you can use Ansible’s “lineinfile” module to add a reference to the new routing table in /etc/iproute2/rt_tables. For example, consider this Ansible task:
- lineinfile: dest=/etc/iproute2/rt_tables line="200 eth1"
This snippet of Ansible code would add the line “200 eth1” to the end of the etc/iproute2/rt_tables file (if the line does not already exist). This takes care of task #1.
For tasks #2 and #3, you can use a Jinja2 template. Because the creation of the policy routing rule and the routing table entries can Continue reading
Open Source at Docker, Part 3: The Tooling and Automation
The Docker open source project is among the most successful in recent history by every possible metric: number of contributors, GitHub stars, commit frequency, … Managing an open source project at that scale and preserving a healthy community doesn’t come without challenges.
This post is the last of a 3-part series on how we deal with those challenges on the Docker Engine project. Part 1 was all about the people behind the project, and part 2 focused on the processes. In Part 3, we will cover tooling and automation.
There are many areas for automation in a project such as Docker. We wanted to present and share some of our tooling with you: the CI, the utility bots, and the project dashboards.