Technology Short Take #68
Welcome to Technology Short Take #68, my erratically-published collection of links, articles, and posts from around the web—all focused on today’s major data center technologies. I’ve been trying to stick to a schedule that has these posts published on a Friday, but given the pending holiday weekend I wanted to get this out a bit early. As always, I hope that something I’ve included here proves useful to you.
Networking
- Brent Salisbury retweeted a ton of MACVLAN- and IPVLAN-related posts right after I published the last Technology Short Take, so they had to get bumped to this one. First, we have this post on HiCube, which covers MACVLAN vs. IPVLAN. Next, we have a pair of articles by Sreenivas Makam; the first covers MACVLAN and IPVLAN basics, while the second tackles the Docker MACVLAN and IPVLAN network plugins. Both articles are useful, IMHO, especially considering that MACVLAN is no longer experimental as of the Docker 1.12 release.
- I recently came across this mention of in-band network telemetry (INT), which looks really powerful. I think this will be something worth watching.
- Oh, and while we’re discussing P4: Diego Dompe has an article talking about OpenSwitch and P4 that Continue reading
OVS Integration with Debian Network Scripts
I had a reader contact me recently with some questions regarding the use of Open vSwitch (OVS) on Debian “Jessie” 8.5 and using the OVS integration with the Debian network scripts. For those of you that might be unfamiliar with this functionality, it’s the ability to configure OVS via instructions and directives found in the /etc/network/interfaces file. As I was helping this reader, I came across a couple potential “gotchas” that I wanted to point out here.
First, I’ll point you to the documentation for the Debian network scripts integration, which is found in this file in the “Debian network scripts integration” section. This documentation provides the complete breakdown of the various commands that can be used in /etc/network/interfaces to configure OVS.
Based on that documentation, you could create an OVS bridge and add a physical port to that bridge by including the following stanzas in /etc/network/interfaces:
allow-ovs ovsbr0
iface ovsbr0 inet manual
ovs_type OVSBridge
ovs_ports eth1
allow-ovsbr0 eth1
iface eth1 inet manual
ovs_bridge ovsbr0
ovs_type OVSPort
Now for the gotchas…
The Debian “Jessie” repos include version 2.3.0 of OVS; the latest release in the 2.3.x train is 2.3.3. As it turns Continue reading
DockerCon 2016: Part 1 of Top 10 Videos
DockerCon 2016 was packed lots with great conference sessions! Attendees enjoyed the variety of topics in the agenda including advanced technical deep dives in the Black Belt track and practical applications of Docker in the Use Case track.
Through the mobile app (powered by Docker of course!), DockerCon 2016 attendees voted on their favorite sessions at the conference. Here are 5 of the top 10 highest rated sessions at DockerCon 2016: Continue reading
DockerCon videos: Moby’s cool hack session
Those of you who have attended previous DockerCons or watched the videos already know that with the help of the community, the Docker team tries it’s best to end each conference with a bang!
From Docker running on small IoT hardware devices (Raspberry Pi) to the biggest iron (IBM mainframes) to live migrating a Quake container around the world or running unikernels in Docker, DockerCon closing keynotes, also known as Moby’s Cool Hacks, aim to give everyone a sneak peak into what’s next for the Docker community. Check out the videos from the previous DockerCons to see the cool hacks and demos that stood out of the Docker team and Community.
DockerCon General Session Day 1 and Day 2 videos
In case you missed DockerCon and the livestream last week, you can now watch the video recording of the general sessions packed with a lot of exciting announcements and live demos of Docker 1.12 and Docker Datacenter!
The rest of the video will be posted on the Docker Blog and Docker Youtube Channel over the next few days. You can already find most of the slides on the Docker Slideshare page.
DockerCon 2016: Just Wow!
The Docker Community has continued to impress us with so much awesomeness packed in just a few days at DockerCon 2016!
Big thank you to all of the 141 confirmed speakers (not including Open Forum track sessions!), 100+ sponsors and over 4,000 attendees for helping us make such a big splash in Seattle.
We’ll post the recorded videos and slides soon starting with the general sessions but here is a brief recap of all of the news DockerCon 2016!
Continue reading
Register Now for the Docker 1.12 Hackathon
With the recent announcement of Docker 1.12, we are happy to announce that today marks the first day of the Docker 1.12 Hackathon! Now that the hackathon is open for registration, participants are encouraged to hack using the new features included in Docker 1.12 including: Swarm Mode, the Service Deployment API, and the built-in routing mesh.
For the Docker 1.12 Hackathon, we are using a platform called DevPost, which allows participants to review the rules, submit their hacks, view other hacks, participate in and start discussions, and easily find other participants with similar interests or complementary skills to join forces with! Submissions are due on Monday, July 25th followed by a week-long judging period.
The judging panel for the Docker 1.12 Hackathon include three Docker Captains and two Docker employees:
- Phil Estes – Docker Captain and Senior Technical Staff Member at IBM Cloud Open Technologies
- Arun Gupta – Docker Captain and VP of Developer Relations at Couchbase
- Laura Frank – Docker Captain and Senior Engineer at Codeship
- Mano Marks – Director of Developer Relations at Docker
- Mike Coleman – Technical Evangelist at docker
Each of our five judges will assign a rating of 1-5 Continue reading
Free Ansible ebooks
Need some summer reading for your trip to the beach? We are pleased to offer three free ebook previews from our friends at Packt Publishing featuring their most popular Ansible books.
Mastering Ansible by Jesse Keating
Design, develop, and solve real world automation and orchestration needs by unlocking the automation capabilities of Ansible
Excerpt includes:
Chapter 1 - System Architecture and Design of Ansible: A detailed in and out view of Ansible's task performance
Chapter 3 - Unlocking the Power of Jinja2 Templates: Usage of the Jinja2 templating engine within Ansible
Download Mastering Ansible by Jesse Keating
OpenStack Administration with Ansible by Walter Bentley
Design, build, and automate 10 real-world OpenStack administrative tasks with Ansible
Excerpt includes:
Chapter 1 - An Introduction to OpenStack: A level setter on OpenStack components, concepts, and verbiage
Chapter 8 - Deploying OpenStack Features: Adding Docker to OpenStack with Ansible
Download OpenStack Administration with Ansible by Walter Bentley
Extending Ansible by Rishabh Das
Discover how to efficiently deploy and customize Ansible in the way your platform demands
Excerpt includes:
Chapter 1 - Getting Started with Ansible: Introduction to Ansible as a tool
Chapter 4 - Exploring API: The Python API for Ansible
Download Extending Ansible by Rishabh Das
How Docker for Mac helps me sleep better at night
My name is Matt Aimonetti, I’m the co-founder and CTO of Splice. At Splice, we built a cloud platform for music producers, this platform is made of elements engineers often take for granted. We invented version control for music, a distributed collaboration flow and a subscription based marketplace for samples, loops presets and MIDI. All that without changing the creation tools musicians already know and like.
I’m a developer and an entrepreneur, the last thing I want to worry about are ops concerns.
Building serverless apps with Docker
Every now and then, there are waves of technology that threaten to make the previous generation of technology obsolete. There has been a lot of talk about a technique called “serverless” for writing apps. The idea is to deploy your application as a series of functions, which are called on-demand when they need to be run. You don’t need to worry about managing servers, and these functions scale as much as you need, because they are called on-demand and run on a cluster.
But serverless doesn’t mean there is no Docker – in fact, Docker is serverless. You can use Docker to containerize these functions, then run them on-demand on a Swarm. Serverless is a technique for building distributed apps and Docker is the perfect platform for building them on.
From servers to serverless
So how might we write applications like this? Let’s take our example a voting application consisting of 5 services:
This consists of:
- Two web frontends
- A worker for processing votes in the background
- A message queue for processing votes
- A database
The background processing of votes is a very easy target for conversion to a serverless architecture. In the voting app, we can run a Continue reading
Introducing the Docker Store Private Beta
We are very excited to announce the private beta of Docker Store, a marketplace for trusted and validated dockerized software – free, open source and commercial.
Our goals with Docker Store are designed around bringing Docker users and ecosystem partners together.
- Provide a scalable self-service system for ISVs to publish and distribute trusted and enterprise-ready content
- Provide a publishing process that validates software quality, including; security scanning, component inventory, the open-source license usage and use of best practices in image construction.
- Provide enterprise users with compliant, commercially supported software from trusted and verified publishers, that is packaged as Docker images.
- We’ve added powerful search and browsing capabilities, including categorization.
The use and creation of dockerized content has grown exponentially in the last couple of years. This demand on content and the expanded use of Docker within the enterprise naturally led to the need for more content, entitlement, visibility into security profiles and compliance.
Docker Store builds on the Official Images and the popularity of the Docker Hub for community content by providing an official marketplace that provides workflows for those who wish to create and distribute content and those that wish to download content to build their Continue reading
Docker Datacenter in AWS and Azure in a few clicks
Introducing Docker Datacenter AWS Quickstart and Azure Marketplace Templates production-ready, high availability deployments in just a few clicks.
The Docker Datacenter AWS Quickstart uses a CloudFormation templates and pre-built templates on Azure Marketplace to make it easier than ever to deploy an enterprise CaaS Docker environment on public cloud infrastructures.
The Docker Datacenter Container as a Service (CaaS) platform for agile application development provides container and cluster orchestration and management that is simple, secure and scalable for enterprises of any size. With our new cloud templates pre-built for Docker Datacenter, developers and IT operations can frictionlessly move dockerized applications to an Amazon EC2 or Microsoft Azure environment without any code changes. Now businesses can quickly realize greater efficiency of computing and operations resources and Docker supported container management and orchestration in just a few steps.
What is Docker Datacenter?
Docker Datacenter includes Docker Universal Control Plane, Docker Trusted Registry (DTR), CS Docker Engine with commercial support & subscription to align to your application SLAs:
- Docker Universal Control Plane (UCP), an enterprise-grade cluster management solution that helps you manage your whole cluster from a single pane of glass
- Docker Trusted Registry (DTR), an image storage solution that helps securely store and manage the Docker Continue reading
Docker for the Enterprise with Docker Datacenter
Businesses today are digital and fundamentally powered by applications – software that drives revenue, engages with customers and runs their operations. The process of making that software has, until recently, been long and cumbersome. The addition of Docker and containerization, a new software supply chain is enabled to bring agility, portability and control together into the enterprise. From security and compliance to shipping more software faster to migrating workloads around sites for the best cost to performance ratio, Docker Datacenter is helping businesses transform their software supply chain.
At DockerCon in Seattle, business of all sizes are sharing their stories Dockering for transformation in development, CI and production environments for all kinds of apps. In this post, I wanted to highlight those companies who are using Docker Datacenter.
HealthDirect Australia: Docker in Production, Look No Hands!
Healthdirect Australia was a long time open source Docker user who in the last year transitioned to a Docker Datacenter environment to gain the benefit of having out of the box integrations of a full Docker supported stack. Scott Coulton, the lead architect (and also Docker Captain and DockerCon speaker!) is the driving force behind deploying and running the applications Continue reading
DockerCon 2016 Vendor Meetings
While at DockerCon 2016 in Seattle today, I took some time on the expo floor to talk to a number of different vendors, mostly focused on networking solutions. Here are some notes from these discussions. I may follow up with additional posts on some of these technologies; it will largely depend on time and the ease by which the technologies/products may be consumed.
Plumgrid
My first stop was the Plumgrid booth. I’d heard of Plumgrid, but wanted to take this time to better understand their architecture. As it turns out, their architecture is quite interesting. Plumgrid is one of the primary commercial sponsors behind the IO Visor project, a Linux Foundation project, which leverages the extended Berkeley Packet Filter (eBPF) subsystem in the Linux kernel. Using eBPF, Plumgrid has created in-kernel virtual network functions (VNFs) that do things like bridging, routing, network address translation (NAT), and firewalling. Combined with a scale-out central control plane and leveraging the Linux kernel’s built-in support for VXLAN, this enables Plumgrid to create overlay networks and apply very granular security policies to attached workloads (which could be VMs or containers).
Project Calico
Next, I stopped by the Calico booth. Unlike many of the networking Continue reading
DockerCon 2016 Day 2 Keynote
This is a liveblog for the day 2 keynote of DockerCon 2016, which wraps up today in Seattle, WA. While today’s pre-keynote warm-up doesn’t include laser-equipped kittens, the music is much more upbeat and energetic (as opposed to yesterday’s more somber, dramatic music). If the number of laptops on the podium is any indicator (yesterday it was a cue to the number of demos planned), then today’s keynote will include a few demos as well.
Ben Golub kicks off the day 2 keynote—with the requisite coffee shot that is a sacrifice to the “demo gods”—and offers up some thanks to the supporters of last night’s party at the Space Needle. Golub quick reviews the key announcements and demos from the day 1 keynote (see my liveblog here). Today, though, will be focused on democratizing Docker in the enterprise. In referring to Docker’s adoption in the enterprise, Golub shares some numbers that vary widely, and admits that it’s really difficult to know what the real adoption rate is. He points to multiple “critical transformations” occurring within the enterprise: application modernization, cloud adoption, and DevOps (process/procedure/culture changes).
This leads Golub into a discussion of anti-patterns, or fallacies. The first fallacy he Continue reading
Docker 1.12: Now with Built-in Orchestration!
Three years ago, Docker made an esoteric Linux kernel technology called containerization simple and accessible to everyone. Today, we are doing the same for container orchestration.
Container orchestration is what is needed to transition from deploying containers individually on a single host, to deploying complex multi-container apps on many machines. It requires a distributed platform, independent from infrastructure, that stays online through the entire lifetime of your application, surviving hardware failure and software updates. Orchestration is at the same stage today as containerization was 3 years ago. There are two options: either you need an army of technology experts to cobble together a complex ad hoc system, or you have to rely on a company with a lot of experts to take care of everything for you as long as you buy all hardware, services, support, software from them. There is a word for that, it’s called lock-in.
Docker users have been sharing with us that neither option is acceptable. Instead, you need a platform that makes orchestration usable by everyone, without locking you in. Container orchestration would be easier to implement, more portable, secure, resilient, and faster if it was built into the platform.
Starting with Docker 1.12, Continue reading
Announcing the Docker for Mac and Windows Public Beta
Back in March, we launched a private beta for a new ambitious project called Docker for Mac and Docker for Windows. Our major goal was to bring a native Docker experience to Mac and Windows, making it easier for developers to work with Docker in their own environments. And thousands agreed. Over thirty thousand applied in the first 24 hours. And by last week, we let in over seventy thousand.
And now all you need to get started developing is Docker and a text editor. No more installing dependencies and runtimes just to debug applications.
Introducing the Docker for AWS and Azure Beta
Today, we’re excited to announce Docker for AWS and Docker for Azure: the best ways to install, configure and maintain Docker deployments on AWS and Azure.
Our goals for Docker for AWS and Azure are the same as for Docker for Mac and Windows:
- Deploy a standard Docker platform to ensure teams can seamlessly move apps from developer laptops to Docker staging and production environments, without risk of incompatibilities or lock-in.
- Integrate deeply with underlying infrastructure to make sure Docker takes advantage of the host environment’s native capabilities and exposes a familiar interface to administrators.
- Deploy the Docker platform to all the places where you want to run containerized apps, simply and efficiently and at no extra cost.
- Make sure the latest and greatest Docker versions are available for the hardware, OSs, and infrastructure you love, and provide solid upgrade paths from one Docker version to the next.
Introducing Experimental Distributed Application Bundles
The built-in orchestration features announced today with Docker 1.12 will revolutionize how IT teams build, ship and run containerized apps. With Docker 1.12, developers and ops now share a set of simple and powerful APIs, tools, and formats for building agile delivery pipelines that ship software from development through CI to production in the cloud with Docker for AWS and Azure.
To facilitate that revolution, we’re introducing Distributed Application Bundles—an experimental open file format for bundling up all the artifacts required to ship and deploy multi-container apps: a DAB contains a description of all the services required to run the application and details images to use, ports to expose, and the networks used to link services.
5 Reasons We Started the Ansible Container Project
Here at Ansible, we recently announced a new project called Ansible Container. Its purpose is to allow users to build, deploy, and orchestrate containers at scale, all from Ansible playbooks.
It’s still a young project, barely a month old at this point -- but we’re excited by it, and we think it has a great deal of potential. Here are five reasons why.
1. Because our community has been using Ansible to manage containers for quite a while now.
Ansible has been successful, in large part, by following where our community leads, and our community has been using Ansible to help manage containers for nearly as long as Ansible has been around. Our community wrote the original Docker module in October 2013, and that module and other container modules have been among the most frequently used Ansible modules ever since. There are hundreds of community-maintained Ansible container images in Dockerhub, and there are excellent blog posts in which Ansible community members describe their own best practices for building and deploying containers. The next logical step was to start a project to bring together some of these best practices into tools that anyone could use.
2. Because the new Docker Continue reading