Swarm Orchestration in Docker Enterprise Edition

At DockerCon Europe, we announced that the next release of Docker Enterprise Edition (Docker EE) would include Kubernetes integration. We’re really excited about bringing Kubernetes to our customer base and continuing to increase our involvement within the community. But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time.

Here are just a few reasons that Swarm is integral to the Docker EE solution:

1. Support our existing customers and ecosystem integrations
2. Provide customers the flexibility in choice of orchestrators
3. Provide secure and highly available clustering architecture

Existing Customers and Ecosystem Integrations

Docker now has hundreds of Docker EE customers who have standardized on Swarm orchestration. In fact, at our Customer Summit during DockerCon, all of the customers stated that they intend to continue using Swarm even with the Kubernetes announcement. Having both orchestration options available is definitely a plus for some of these customers that have organizations within the company using both Swarm and Continue reading

Docker Enterprise Edition Certified as Kubernetes Conformant

Yesterday, the Cloud Native Computing Foundation® (CNCF®) announced that Docker Enterprise Edition (Docker EE) passed the Certified Kubernetes™ conformance program. Based on the upcoming release of Docker EE that was demonstrated onstage at DockerCon Europe, this certification guarantees that all Kubernetes APIs function as specified to deliver a consistent and portable Kubernetes experience within Docker EE.

In addition to a fully-conformant, unmodified  Kubernetes experience, users of Docker EE will also have access to the advanced capabilities of the Docker platform including simple and powerful clustering, a secure end-to-end supply chain, and validation to run on all major Linux distributions, Windows, IBM mainframe, and several leading public clouds. By integrating Kubernetes into Docker EE, we simplify and advance the management of Kubernetes for enterprise IT.  

Conformance Testing

Docker EE was certified against Kubernetes v1.8 by passing a test suite overseen by SIG Architecture. The test gives end users the confidence that Docker EE delivers a high level of common functionality to the main Kubernetes distribution and these results are validated by CNCF.

As a Certified Kubernetes platform, we are also promising to release new versions of Docker EE with Kubernetes to ensure that customers can take advantage of the rapid pace Continue reading

Integrating Docker EE Into Société Générale’s Existing Enterprise IT Systems

Société Générale is a 153-year old French multinational bank that believes technology and innovation are key to enriching the customer experience and advancing economic development. A few years ago, the bank started a project to define their next generation application platform that would help them get 80% of their applications running in the cloud by 2020. Société Générale chose Docker Enterprise Edition (Docker EE) to be the foundation of their application platform and began working with it 15 months ago. This year at DockerCon Europe, Stephan Dechoux, DevOps architect, and Thomas Boussardon, Middleware Specialist, shared their journey over this time integrating Docker Enterprise Edition [Docker EE] into Société Générale IT systems.

You can watch their breakout session here:

A New Platform For Today and Tomorrow

Société Générale has a diverse application portfolio that includes many different types of applications, including legacy monolithic apps, SOA, distributed apps and REST APIs. The bank is also a global organization with teams and data centers around the world. A primary goal  was to deliver a new application platform to improve time-to-market and lower costs, while accelerating innovation. Initially Société Générale considered off-the-shelf PaaS solutions, but realized that these were better suited for greenfield applications Continue reading

Making AWS re:Invent More Family-Friendly

AWS re:Invent is just around the corner, and Spousetivities will be there to help bring a new level of family friendliness to the event. If you’re thinking of bringing a spouse, partner, or significant other with you to Las Vegas, I’d encourage you to strongly consider getting him or her involved in Spousetivities.

Want a sneak peek at what’s planned? Have a look:

• Monday’s activity is a full-day trip to Death Valley, including a stop at Bad Water Basin (significant because it is 280 feet below sea level, making it the lowest place in the Western Hemisphere!). Lunch is included, of course.
• On Tuesday, Spousetivities participants will get to visit a number of locations on the Las Vegas Strip, including Siegfried and Roy’s Secret Garden, the Wildlife Habitat at the Flamingo, and the Shark Reef at Mandalay Bay. Transportion is provided for longer connections, but there will be some walking involved—wear comfortable shoes!
• Wednesday includes a visit to Red Rock Canyon and Hoover Dam. There will some opportunities for short sightseeing walks in Red Rock Canyon (plus the 13-mile scenic drive), and the Hoover Dam tour includes access to the generator room (a very cool sight).
• Wrapping up the Continue reading

THE ENTERPRISE IT CHECKLIST FOR DOCKER OPERATIONS

At Docker, we believe the best insights come from the developers and IT pros using the Docker platform every day. Since the launch of Docker Enterprise Edition, we learned three things from our customers.

1. First, a top goal in enterprise IT is to deliver value to customers (internal business units or external clients)…and to do so fast.
2. Second, most enterprises believe that Docker is at the center of their IT platform.
3. Finally, most enterprises’ biggest challenge is moving their containerized applications to production in time to prove value. My DockerCon talk focused on addressing the third item, which seems to be a critical one for many of our customers.

In our recent customer engagements, we’ve seen a pattern of common challenges when designing and deploying Docker in an enterprise environment. Particularly, customers are struggling to find best practices to speed up their move to production. To address some of these common challenges, we put together a production readiness checklist (https://github.com/nicolaka/checklist) for Docker Enterprise Edition. This list was discussed thoroughly during my DockerCon EU 2017 session. Here’s a video of that talk:

I go through 10 key topics (shown below) that a typical enterprise should  go through when deploying Continue reading

Technology Short Take 90

Welcome to Technology Short Take 90! This post is a bit shorter than most, as I’ve been on the road quite a bit recently. Nevertheless, there’s hopefully something here you’ll find useful.

Networking

• VMware recently released an updated to PowerCLI (version 6.5.3), and with it comes a set of cmdlets that provide low-level access to the NSX-T REST API. Romain Decker provides a walkthrough. Have fun automating!
• Looking for some “MPLS 101”-type material? Jon Langemak has you covered.
• I really enjoyed David Gee’s series on the network automation engineer persona (part 1, part 2, part 3, and part 4). David really unpacks the changes that are happening in the networking space and describes the key skills the “next-generation network engineer” will have. Good stuff!
• This post by Patrick Ogenstad on Ansible action plugins for Cisco IOS is just outside my comfort zone with regards to Python knowledge and skill, but it’s only by pushing the boundaries that we learn, right?
• Romain Decker discusses the ten things you should know before starting with NSX-T.
• Mark Betz digs into some Kubernetes networking concepts in detail.

Security

• Tony Reeves shares a procedure for setting up encryption Continue reading

How to Tag Docker Images with Git Commit Information

I’ve recently been working on a very simple Flask application that can be used as a demo application in containerized environments (here’s the GitHub repo). It’s nothing special, but it’s been useful for me as a learning exercise—both from a Docker image creation perspective as well as getting some additional Python knowledge. Along the way, I wanted to be able to track versions of the Docker image (and the Dockerfile used to create those images), and link those versions back to specific Git commits in the source repository. In this article, I’ll share a way I’ve found to tag Docker images with Git commit information.

Before I proceed any further, I’ll provide the disclaimer that this information isn’t unique; I’m building on the work of others. Other articles sharing similar information include this one; no doubt there are countless more I haven’t yet seen. I’m presenting this information here simply to show one way (not the only way) of including Git commit information with a Docker image.

Getting the necessary information from Git is actually far easier than one might think. This variation of the git log command will print only the full hash of the last commit Continue reading

Deep Dive into Container Images in Kolla

This is a liveblog of my last session at the Sydney OpenStack Summit. The session title is “OpenStack images that fit your imagination: deep dive into container images in Kolla.” The presenters are Vikram Hosakote and Rich Wellum, from Cisco and Lenovo, respectively.

Hosakote starts with an overview of Kolla. Kolla is a project to deploy OpenStack services into Docker containers. There are two ways to use Kolla: using Ansible (referred to as Kolla-Ansible) or using Kubernetes (referred to as Kolla-Kubernetes). Hosakote mentions that Kolla-Kubernetes also uses Helm and Helm charts; this makes me question the relationship between Kolla-Kubernetes and OpenStack-Helm.

Why Kolla? Some of the benefits of Kolla, as outlined by Hosakote, include:

• Fast(er) deployment (Hosakote has deployed in as few as 9 minutes)
• Easy maintenance, reconfiguration, patching, and upgrades
• Containerized services are found in container registry
• One tool to do multiple things

Hosakote briefly mentions his preference for Kolla over other tools, including Juju, DevStack, PackStack, Fuel, OpenStack-Ansible, TripleO, OpenStack-Puppet, and OpenStack-Chef.

Other benefits of using containers for OpenStack:

• Reproduce golden state easily
• No more “Works in DevStack” but doesn’t work in production
• Production-ready images (this seems specific to Kolla, not just containers for OpenStack control plane)
• Continue reading

Carrier-Grade SDN-Based OpenStack Networking Solution

This session was titled “Carrier-Grade SDN Based OpenStack Networking Solution,” led by Daniel Park and Sangho Shin. Both Park and Shin are from SK Telecom (SKT), and (based on the description) this session is a follow-up to a session from the Boston summit where SK Telecom talked about an SDN-based networking solution they’d developed and released for use in their own 5G-based network.

Shin starts the session with some presenter introductions, and sets the stage for the presentation. Shin first provides some background on SKT, and discusses the steps that SKT has been taking to prepare their network for 5G infrastructure/services. This involves more extensive use of virtual network functions (VNFs) and software-defined infrastructure based on open software and open hardware. Shin reinforces that the SKT project (which is apparently called COSMOS?) exclusively leverages open source software.

Diving into a bit more detail, Shin talks about SONA Fabric (which is used to control the leaf/spine fabric used as the network underlay), SONA (which handles virtual network management), and TACO (which is an SKT-specific version of OpenStack). The network monitoring solution is called TINA, and this feeds into an integrated monitoring system known as 3DV.

TACO (stands for SKT All Continue reading

Can OpenStack Beat AWS in Price

This is a liveblog of the session titled “Can OpenStack Beat AWS in Price: The Trilogy”. The presenters are Rico Lin, Bruno Lago, and Jean-Daniel Bonnetot. The “trilogy” refers to the third iteration of this presentation; each time the comparison has been done in a different geographical region (first in Europe, then in North America, and finally here in Asia-Pacific).

Lago starts the presentation with an explanation of the session, and each of the presenters introduce themselves, their companies, and their backgrounds. In this particular case, the presenters are representing Catalyst (runs Catalyst Cloud in New Zealand), OVH, and EasyStack—all three are OpenStack-powered public cloud offerings.

Lago explains that they’ll cover three common OpenStack scenarios:

• Private cloud
• Managed private cloud
• OpenStack-powered public cloud

Lin takes point to talk a bit about price differences in different geographical regions. Focusing on AWS, Lin points out that AWS services are about 8% higher in Europe than in North America. Moving to APAC, AWS services are about 29% higher than in North America. With this 29% price increase, I can see where OpenStack might be much more competitive in APAC than in North America (and this, in turn, may explain why OpenStack seems much Continue reading

Lessons Learnt from Running a Container-Native Cloud

This is a liveblog of the session titled “Lessons Learnt from Running a Container-Native Cloud,” led by Xu Wang. Wang is the CTO and co-founder of Hyper.sh, a company that has been working on leveraging hypervisor isolation for containers. This session claims to discuss some lessons learned from running a cloud leveraging this sort of technology.

Wang starts with a brief overview of Hyper.sh. The information for this session comes from running a Hypernetes (Hyper.sh plus Kubernetes)-based cloud for a year.

So, what is a “container-native” cloud? Wang provides some criteria:

• A container is a first-class citizen in the cloud. This means container-level APIs and the ability to launch containers without a VM.
• The cloud offers container-centric resources (floating IPs, security groups, etc.).
• The cloud offers container-based services (load balancing, scheduled jobs, functions, etc.).
• Billing is handled on a per-container level (not on a VM level).

To be honest, I don’t see how any cloud other than Hyper.sh’s own offering could meet these criteria; none of the major public cloud providers (Microsoft Azure, AWS, GCP) currently satisfy Wang’s requirements. A “standard” OpenStack installation doesn’t meet these requirements. This makes the session more like a Continue reading

Make Your Application Serverless

This is a liveblog from the last day of the OpenStack Summit in Sydney, Australia. The title of the session is “Make Your Application Serverless,” and discusses Qinling, a project for serverless (Functions-as-a-Service, or FaaS) architectures/applications on OpenStack. The presenters for the session are Lingxian Kong and Feilong Wang from Catalyst Cloud.

Kong provides a brief background on himself and his co-presenter (Wang), and explains that Catalyst Cloud is an OpenStack-based public cloud based in New Zealand. Both presenters are active technical contributors to OpenStack projects.

Kong quickly transitions into the core content of the presentation, which focuses on serverless computing and Qinling, a project for implementing serverless architectures on OpenStack. Kong points out that serverless computing doesn’t mean there are no servers, only that the servers (typically VMs) are hidden from view. Functions-as-a-Service, or FaaS, is a better term that Kong prefers. He next provides an example of how a FaaS architecture may benefit applications, and contrasts solutions like AutoScaling Groups (or the equivalent in OpenStack) with FaaS.

Some key characteristics of serverless, as summarized by Kong:

• No need to think about servers
• Run your code, not the whole application
• Highly available and horizontally scalable
• Stateless/ephemeral
• Lightweight/single-purpose functions
• Event-driven Continue reading

How to Deploy 800 Servers in 8 Hours

This is a liveblog of the session titled “How to deploy 800 nodes in 8 hours automatically”, presented by Tao Chen with T2Cloud (Tencent).

Chen takes a few minutes, as is customary, to provide an overview of his employer, T2cloud, before getting into the core of the session’s content. Chen explains that the drive to deploy such a large number of servers was driven in large part by a surge in travel due to the Spring Festival travel rush, an annual event that creates high traffic load for about 40 days.

The “800 servers” count included 3 controller nodes, 117 storage nodes, and 601 compute nodes, along with some additional bare metal nodes supporting Big Data workloads. All these nodes needed to be deployed in 8 hours or less in order to allow enough time for T2cloud’s customer, China Railway Corporation, to test and deploy applications to handle the Spring Festival travel rush.

To help with the deployment, T2cloud developed a “DevOps” platform consisting of six subsystems: CMDB, OS installation, OpenStack deployment, task management, automation testing, and health check/monitoring. Chen doesn’t go into great deal about any of these subsystems, but the slide he shows does give away some information:

• Continue reading

How to Deploy 800 Servers in 8 Hours

This is a liveblog of the session titled “How to deploy 800 nodes in 8 hours automatically”, presented by Tao Chen with T2Cloud (Tencent).

Chen takes a few minutes, as is customary, to provide an overview of his employer, T2cloud, before getting into the core of the session’s content. Chen explains that the drive to deploy such a large number of servers was driven in large part by a surge in travel due to the Spring Festival travel rush, an annual event that creates high traffic load for about 40 days.

The “800 servers” count included 3 controller nodes, 117 storage nodes, and 601 compute nodes, along with some additional bare metal nodes supporting Big Data workloads. All these nodes needed to be deployed in 8 hours or less in order to allow enough time for T2cloud’s customer, China Railway Corporation, to test and deploy applications to handle the Spring Festival travel rush.

To help with the deployment, T2cloud developed a “DevOps” platform consisting of six subsystems: CMDB, OS installation, OpenStack deployment, task management, automation testing, and health check/monitoring. Chen doesn’t go into great deal about any of these subsystems, but the slide he shows does give away some information:

• Continue reading

IPv6 Primer for Deployments

This is a liveblog of the OpenStack Summit Sydney session titled “IPv6 Primer for Deployments”, led by Trent Lloyd from Canonical. IPv6 is a topic with which I know I need to get more familiar, so attending this session seemed like a reasonable approach.

Lloyd starts with some history. IPv6 was released in 1980, and uses 32-bit address (with a total address space of around 4 billion). IPv4, as most people know, is still used for the majority of Internet traffic. IPv6 was released in 1998, and uses 128-bit addresses (for a theoretical total address space of 3.4 x 10 to the 38th power). IPv5 was an experimental protocol, which is why the IETF used IPv6 as the version number for the next production version of the IP protocol.

Lloyd shows a graph showing the depletion of IPv4 address space, to help attendees better understand the situation with IPv4 address allocation. The next graph Lloyd shows illustrates IPv6 adoption, which—according to Google—is now running around 20% or so. (Lloyd shared that he naively estimated IPv4 would be deprecated in 2010.) In Australia it’s still pretty difficult to get IPv6 support, according to Lloyd.

Next, Lloyd reviews decimal and Continue reading

Battle Scars from OpenStack Deployments

This is the first liveblog from day 2 of the OpenStack Summit in Sydney, Australia. The title of the session is “Battle Scars from OpenStack Deployments.” The speakers are Anupriya Ramraj, Rick Mathot, and Farhad Sayeed (two vendors and an end-user, respectively, if my information is correct). I’m hoping for some useful, practical, real-world information out of this session.

Ramraj starts the session, introducing the speakers and setting some context for the presentation. Ramraj and Mathot are with DXC, a managed services provider. Ramraj starts with a quick review of some of the tough battles in OpenStack deployments:

• Months to deploy OpenStack at scale
• Chaos during incidents due to lack of OpenStack skills and knowledge
• Customers spend lengthy periods with support for troubleshooting basic issues
• Applications do not get onboarded to OpenStack
• Marooned on earlier version of OpenStack
• OpenStack skills are hard to recruit and retain

Ramraj recommends using an OpenStack distribution versus “pure” upstream OpenStack, and recommends using new-ish hardware as opposed to older hardware. Given the last bullet, this complicates rolling out OpenStack and resolving OpenStack issues. A lack of DevOps skills and a lack of understanding around OpenStack APIs can impede the process of porting applications Continue reading

Kubernetes on OpenStack: The Technical Details

This is a liveblog of the OpenStack Summit session titled “Kubernetes on OpenStack: The Technical Details”. The speaker is Angus Lees from Bitnami. This is listed as an Advanced session, so I’m hoping we’ll get into some real depth in the session.

Lees starts out with a quick review of Bitnami, and briefly clarifies that this is not a talk about OpenStack on Kubernetes (i.e., using Kubernetes to host the OpenStack control plane); instead, this is about Kubernetes on OpenStack (OpenStack as IaaS, Kubernetes to do container orchestration on said IaaS).

Lees jumps quickly into the content, providing a “compare-and-contrast” of Kubernetes versus OpenStack. One of the key points is that Kubernetes is more application-focused, whereas OpenStack is more machine-focused. Kubernetes’ multi-tenancy story is shaky/immature, and the nature of containers means there is a larger attack surface (VMs provide a smaller attack surface than containers). Lees also points out that Kubernetes is implemented mostly in Golang (versus Python for OpenStack), although I’m not really sure why this matters (unless you are planning to contribute to one of these projects).

Lees next provides an overview of the Kubernetes architecture (Kubernetes master node containing API server talking to controller manager Continue reading

Issues with OpenStack That Are Not OpenStack Issues

This is a liveblog of OpenStack Summit session on Monday afternoon titled “Issues with OpenStack that are not OpenStack Issues”. The speaker for the session is Sven Michels. The premise of the session, as I understand it, is to discuss issues that arise during OpenStack deployments that aren’t actually issues with OpenStack (but instead may be issues with process or culture).

Michels starts with a brief overview of his background, then proceeds to position today’s talk as a follow-up (of sorts) to a presentation he did in Boston. At the Boston Summit, Michels discussed choosing an OpenStack distribution for your particular needs; in this talk, Michels will talk about some of the challenges around “DIY” (Do It Yourself) OpenStack—that is, OpenStack that is not based on some commercial distribution/bundle.

Michels discusses that there are typically two approaches to DIY OpenStack:

• The “Donald” approach leverages whatever around, including older hardware.
• The “Scrooge” approach is one in which money is available, which typically means newer hardware.

Each of these approaches has its own challenges. With older hardware, it’s possible you’ll run into older firmware that may not be supported by Linux, or hardware that no longer works as expected. With new hardware, Continue reading

To K8s or Not to K8s Your OpenStack Control Plane

This is a liveblog of the Monday afternoon OpenStack Summit session titled “To K8s or Not to K8s Your OpenStack Control Plane”. The speaker is Robert Starmer of Kumulus Technologies. This session is listed as a Beginner-level session, so I’m hoping it’s not too basic for me (and that readers will still get some value from the liveblog).

Starmer begins with a quick review of his background and expertise, and then proceeds to provide—as a baseline—an overview of containers and Kubernetes for container orchestration. Starmer covers terminology and concepts like Pods, Deployments (and Replica Sets), Services, StatefulSets, and Persistent Volumes. Starmer points out that StatefulSets and Persistent Volumes are particularly applicable to the discussion about using Kubernetes to handle the OpenStack control plane. Following the discussion of Kubernetes components, Starmer points out that the Kubernetes architecture is designed to be resilient, talking about the use of etcd as a distributed state storage system, multiple API servers, separate controller managers, etc.

Next, Starmer spends a few minutes talking about Kubernetes networking and some of the components involved, followed by a high-level discussion around persistent volumes and storage requirements, particularly for StatefulSets.

Having covered Kubernetes, Starmer now starts talking about the requirements Continue reading

OpenStack Summit Sydney Day 1 Keynote

This is a liveblog of the day 1 keynote here at the OpenStack Summit in Sydney, Australia. I think this is my third or fourth trip to Sydney, and this is the first time I’ve run into inclement weather; it’s cloudy, rainy, and wet here, and forecasted to remain that way for most of the Summit.

At 9:02am, the keynotes (there are actually a set of separate keynote presentations this morning) kicks off with a video with Technical Committee memebers, community members, and others talking about the OpenStack community, the OpenStack projects, and the Summit itself. At 9:05am, the founders of the Australian OpenStack User Group—Tristan Goode and Tom Fifield—take the stage to kick off the general session. Goode and Fifield take a few minutes to talk about the history of the Australian OpenStack User Group and the evolution of the OpenStack community in Australia. Goode also takes a few moments to talk about his company, Aptira.

After a few minutes, Goode and Fifield turn the stage over to Mark Collier and Lauren Sell from the OpenStack Foundation. Collier and Sell set the stage for the upcoming presentations, do some housekeeping announcements, and talk about sponsors and support partners. Sell Continue reading