Archive

Category Archives for "Docker Blog"

Proactive Ops for Container Orchestration Environments: Monitoring and Logging Strategies with Docker Enterprise

Over the last decade, the popularity of microservices and highly-scalable systems has increased, leading to an overall increase in the complexity of applications that are now distributed heavily across the network with many moving pieces and potential failure modes.

This architectural evolution has changed the monitoring requirements and led to a need for scalable and insightful tooling and practices to enable us to better identify, debug and resolve issues in our systems before they impact the business and our end users (internal and/or external).

I recently gave a talk at DockerCon SF 18 discussing functionality in Docker Enterprise that enables operators to more easily monitor their container platform environment, along with some key metrics and best practices to triage and remediate issues before they cause downtime.

You can watch the full talk here:

 

Monitoring Methodologies

One of the most well-known early monitoring techniques was the USE method from Brendan Gregg at Netflix. USE specified that for every resource we should be monitoring utilization (time spent servicing work), saturation (the degree to which a resource had work it couldn’t service) and errors (number of error events). This model worked well for more hardware / node centric metrics but network-based Continue reading

Docker SF Chapter Meetup: BuildKit and Container Orchestration

Last week, we had a great turnout for the Docker San Francisco chapter event at HQ! The meetup was focused on two of the most popular talks from DockerCon on BuildKit and container orchestration with Kubernetes and Docker Swarm. The purpose was to give people who couldn’t attend these talks a chance to hear to hear them live,  provide the opportunity to ask the speakers questions and network with other members of the container ecosystem.

‘Introduction to BuildKit: the Next Generation Builder Component’

Watch Tibor Vass’ Presentation

First, we had Tibor Vass, a software engineer on the Engine Team at Docker and maintainer of the Docker project since 2014, take the podium with his talk ‘Introduction to BuildKit: the Next Generation Builder Component’. In this talk, Tibor went into detail about the latest developments around the builder component of the Docker Platform including BuildKit, a toolkit that builds on containerd to provide next-generation capabilities for building software with the help of containers. Watch his talk to learn about the architecture and design choices of BuildKit’s frontend model, how to use BuildKit directly, and how to leverage its features in Docker. There were a number of great questions from particularly engaged community members!

 

How BCG Gamma is Transforming Analytics with Docker

Changing the culture and service offerings of a big consulting firm isn’t easy, but BCG has been on that path for the past five years. BCG has evolved from traditional consulting services into a digital transformation powerhouse with six divisions that deliver strategic and technical services to clients.

One of those divisions, BCG Gamma, is a global team of world-class data scientists who build data analytics, machine learning, and artificial intelligence solutions for the firm’s clients. But building and shipping analytics, ML and AI applications to clients is challenging. Andrea Gallego, CTO of the division, is charged with creating an infrastructure that can support delivering high-quality ML and AI models at scale.

The Challenge: Shipping ML and AI Software to Clients at Scale

The big question on her mind was how BCG Gamma could not only build models, but deliver them to clients at the edge with real-time orchestration, monitoring and updates. GDPR and other regulations also meant she had to do this while ensuring integrity, consistency and lineage across data models.

Andrea and her team launched the BCG GammaX initiative, a core team of 30 engineers specializing in analytics software engineering, data engineering, UX design, distributed systems, and Continue reading

Kubernetes is Now Available In Docker Desktop Stable Channel

Back in January we made Kubernetes available in our Edge release channels for Docker Desktop on macOS and on Windows. Today we’re excited to announce that Kubernetes orchestration has graduated to the Stable release channels for Docker Desktop!

Docker Desktop is the fastest and simplest way to get a Kubernetes cluster running on your desktop machine, while still giving you the freedom to choose Docker Swarm if you prefer. Docker Developer Advocate Elton Stoneman recently created a short video demonstrating Docker Desktop on both Windows and Mac. In the video, Elton demonstrates:

  • Enabling Kubernetes and alternating between Kubernetes and Swarm
  • Integrating Docker Desktop and containers in to your environment and workflow
  • Deploying .NET, NodeJS, and Java apps with Docker Desktop, including deploying to Kubernetes with a Compose file

Click image below to watch video:

Docker Desktop is simple to install on macOS and Windows 10 Pro or Enterprise, and is available at the links below. If you’re already using Docker Desktop and you’re in the Stable channel (which is default) then you should see an auto-update notification soon.

 

What You Can Do with Kubernetes on your desktop?

Docker Desktop is the most popular way to configure a Docker dev Continue reading

How to Get Your Apps off Windows 2003 and Into the Cloud with Docker Enterprise

Docker Windows Containers with Docker Enterprise

 

A huge number of companies are still running apps on Windows Server 2003 and 2008 in the data center. They want to move to a modern, secure, supported platform which gives them the flexibility to run in the data center today – and in any cloud tomorrow. Docker gives them that flexibility, and you can move your apps to Docker without changing any code.

That was the focus of our recent webinar, where we showed several apps currently running on Windows Server 2003, and packaged them to run as Docker Windows containers. We showed all the steps to migrate the apps with no code changes, and then we ran them in Docker Enterprise on a Windows Server 2016 VM running in Azure.

You can watch the full video of the webinar here – it comes in at just over 60 minutes:

In the webinar, you see the portability that Docker Enterprise gives you. The applications we move are a mixture of older web technologies – static HTML, classic ASP and ASP.NET WebForms. The apps from the demo and the Dockerfiles are on GitHub here. They could be 15-year old apps and you Continue reading

Introducing Docker Application Guides

In April of 2017 we announced the Modernize Traditional Applications (MTA) program at Docker. The goal of MTA is to take the vast back catalogs of existing applications that are running in enterprise organizations today, and bring them to a modern container platform, without requiring extensive rewrites or refactoring. I’m excited to share part of our learning from the MTA program and announce the release of Docker Application Guides.

 

Oracle WebLogic MedRec Sample Application on Docker Enterprise Edition

Docker Application Guides demonstrate how to deploy popular enterprise applications – Oracle WebLogic and IBM MQ with WebSphere Liberty – on Docker Enterprise and Docker Desktop. Application Guides include example architectures and guidance for selecting Certified Docker container images from Docker Store and deploying a prototype application, orchestrated by Docker Swarm or Kubernetes.

It is important to note that Docker Application Guides are one piece of our prescriptive Docker customer journey to production. In addition to the knowledge transfer and process transformation that come with our full approach, Application Guides provide a reference for deploying common enterprise applications on the Docker Enterprise platform.

The first Docker Application Guides are designed to help you plan and deploy an Oracle WebLogic application Continue reading

Extending Support Cycle for Docker Community Edition

We’re excited to share the release of Docker 18.06 Community Edition (CE) and also share some changes that will be implemented in the next release. Based on feedback we’ve been hearing from the community, we are implementing some changes to deliver higher quality Community Edition (CE) releases, while also providing faster access to new features as they get added:

  • Docker CE Stable is changing to twice-a-year release cadence
  • Docker CE Edge is deprecated in favor of a nightly build channel

As a result of these changes, Docker 18.06 CE will be the last release with a 4-month maintenance lifecycle. The planned Docker 18.09 CE release will be supported for 7 months with Docker 19.03 CE being the next release in line. Further, the monthly Edge releases of Docker CE (Engine) are now replaced with nightly builds. Note that Docker Desktop (Docker for Mac, Docker for Windows) edge channels will still provide monthly updates.

Docker CE Stable Moving to Longer Release and Maintenance Cycles

We will be delivering a more thoroughly tested Docker CE release twice a year instead of quarterly. We will also be separating the packaging for different operating systems from the Continue reading

Making Application Collaboration Easier with Docker Application Packages

In the first blog post about the experimental Docker Application Packages, Gareth showed how our new open-source docker-app can be used to augment Compose files by adding metadata and separate settings.

Now that you know how to create an Application Package, how do you share it? Using a Docker registry solution like Docker Hub or Docker Enterprise of course! Let’s look at an example that works with the latest release of docker-app. Here’s a simple single-file format Application Package with the filename hello.dockerapp:

# This section contains your application metadata.

version: 0.1.0
name: hello
description: "A simple HTTP echo server"
maintainers:
- name: Chris Crone
  email: [email protected]
targets:
  swarm: true
  kubernetes: true

---

# This section contains the Compose file that describes your application services.

version: '3.6'
services:
  hello:
    image: hashicorp/http-echo:${version}
    command: ["-text", "${text}"]
    ports:
      - ${port}:5678

 ---

# This section contains the default values for your application settings.

port: 5678
text: hello development
version: latest

We can save this Application Package as a Docker image using the save command:

$ docker-app save
Saved application as image: hello.dockerapp:0. Continue reading

Cool Hacks Spotlight: DART

Docker container platforms  are being used to support mission-critical efforts all over the world. The Planetary Defense Coordination Office out of NASA is using Docker’s platform to support a critical mission that could potentially affect everyone on the planet! The office is responsible for tracking near-earth asteroids, characterizing them and determining how to deflect them if one were to find its way to earth. 

DART, led by the Johns Hopkins Applied Physics Laboratory by way of NASA, is the Double Asteroid Redirection Test. The team has chosen a potentially hazardous asteroid to hit in order to measure the impact and determine how effective this type of mission would be.

Developing the software for this mission is no easy feat, because space is hard! The team has one shot to make this mission work, there’s no rebooting in space. Space physics constraints lead to very low bandwidth, and low density memory due to the turbulent effects of radiation. So what did the software team want to solve for using Docker? Hardware scarcity. The development systems used in this project are very expensive ($300K), so not every developer is going to get their own system. This led to a time constraint, Continue reading

Jabil Enables a Global Software Supply Chain with Docker Enterprise Edition

Jabil, one of world’s most technologically advanced manufacturing solutions provider with over 100 sites in 29 countries is embarking on a digital journey to modernize their technology infrastructure so the company is better able to deliver the right solutions at the right time to their global customer base.

 

Starting the Digital Journey By Modernizing .NET Apps 

As Jabil embarked on their digital journey with a cloud-first approach in mind, they investigated how to best migrate their applications to the cloud. Jabil partnered with Docker and Microsoft to leverage Docker Enterprise Edition with Windows Server 2016 and Microsoft Azure for this initiative through Docker’s Modernize Traditional Application (MTA) Program – starting with  a .NET 4.5 monitoring application to containerize.

Since completing the initial POC, Jabil has continued containerizing more applications and has started scaling their Docker Enterprise usage globally. Sujay Pillai, a Senior DevOps Engineer at Jabil, participated at DockerCon SF 2018 in June and shared with the attendees insights on how Jabil is scaling Docker Enterprise Edition.

Improving Application Security while Reducing Costs at the Edge

One growing use case for Jabil is monitoring of the manufacturing floors. Jabil uses lightweight edge devices to run the monitoring Continue reading

Cool Hacks Spotlight: Gloo Function Gateway

To close DockerCon Cool Hacks keynote, Idit Levine from Solo.io presented Gloo, a high-performance, plugin-extendable, platform-agnostic function Gateway built on top of Envoy.

Idit showed a demo that involved modernizing a traditional application; the classic Spring Pet Clinic sample app, by containerizing it and deploying it to Docker Enterprise Edition. She added functionality to the app by adding a microservice written in Go through a Gloo route. Then added more functionality by adding a Gloo route to an AWS Lambda function, creating a true hybrid cloud application combining legacy, microservices and serverless components.

She then provided a demo of Squash, that works with Gloo to live debug two microservices forming an application running in Kubernetes on Docker Enterprise Edition, one in Java from IntelliJ, one in Go from Visual Studio Code.

She finished her presentation by announcing and open sourcing Qloo, a GraphQL Server built on top of Gloo and the Envoy Proxy. This allows you to add GraphQL support without any coding to your existing application, and combining functions together in a workflow described as a graph.

See all these excellent demos in the video below, and view the presentation on SlideShare.


 #Docker Spotlights: @gloo a Continue reading

DockerCon SF 2018 Cool Hacks Spotlight: Kubeflow

The closing keynote at DockerCon is traditionally the Cool Hacks keynote. This year, we featured three great hacks showing off innovative uses of Docker. In this post, I’d like to highlight one in particular, the Kubeflow demo with David Aronchick, and Michelle Casbon.

Machine Learning (ML) is becoming increasing popular, and important to enterprises. Kubeflow is an ML toolkit for Kubernetes, developed by Google. It’s a dedicated, portable and scalable approach to machine learning, using tools you’re already using to deploy other applications. It’s great because data scientists can use it to test out model creation on their laptops. And data engineers can take the models and use the power of Docker Enterprisein the cloud to further train and use the models in production.

In their demo, David and Michelle showed building an app using Kubeflow first with Docker Desktop and then on Docker Enterprise in the cloud. And they even took advantage of Google Cloud Tensorflow Processing Units native to the platform.

Check out their presentation and demo, and also check out Kubeflow to learn more.

 


Cool Hacks Spotlight: @kubeflow, a ML toolkit for #Kubernetes
Click To Tweet


Stay connected with Docker by taking advantage of the Continue reading

DockerCon Updates on Containerd, BuildKit, and a Reflection about the Enduring Value of Docker Engine

Two weeks ago was our eighth DockerCon in just four years. Our community of contributors, developers, IT users, enterprises and ecosystem partners has grown exponentially into the millions,  anchored on our founder Solomon Hykes’ simple premise of democratizing the use of the software container. Today as was from the beginning, Docker creates simple tooling and a universal packaging approach that bundles up all application dependencies inside the container.  Docker Engine enables applications to run anywhere consistently on any infrastructure, solving “dependency hell” for developers and operations teams, and eliminating the “it works on my laptop!” problem.

In the past 2 years, Docker Engine’s codebase has been refactored into several reusable components, the most important being containerd, the core container runtime, and BuildKit, the part of Docker Engine used to build images. In the contribute and collaborate track at DockerCon, Michael Crosby and Tonis Tiigli gave an update on these two projects (video, slides)

Docker platform internals from Docker, Inc.

containerd, the core container runtime in Docker Engine has been leveraged by millions of users and is run in production by tens of thousands of organizations. Eighteen months ago, Docker spun out containerd from Docker Engine; donated Continue reading

DockerCon 2018: Top 5 Rated Sessions

We are happy to share the videos from the top 5 rated sessions from last week’s DockerCon 2018 in San Francisco. The DockerCon on-demand sessions videos are ready for you to explore.

Note: The Ecosystem sessions will not be available.

Demystifying Container Connectivity with Kubernetes in Docker

by Nicola Kabar & Karthik Prabhakar

Message-Based Microservices Architectures Driven with Docker

by Michele Leroux Bustamante

Docker Storage: Designing a Platform for Persistent Data

by Dan Finneran

Istio: Managing, Securing, and Observing Microservices

by Lin Sun & Zack Butcher 

Tips and Tricks of the Docker Captains

by Adrian Mouat

Watch the past videos from DockerCon SF:

Pre-registration open for:

 


Missed @DockerCon SF? Here are the top 5 rated sessions.
Click To Tweet


The post DockerCon 2018: Top 5 Rated Sessions appeared first on Docker Blog.

Disruption from Within: Driving Innovation at Franklin American with Docker EE

When you think of mortgage companies, you think of paperwork. You probably don’t think agile and responsive to customers. But Franklin American Mortgage wanted to disrupt that model. With their investments in innovation, microservices and Docker Enterprise Edition, they’ve been able to quickly create a platform to make technology core to their success.

Don Bauer, the DevOps lead at Franklin American is part of an innovation team the mortgage company had established last year to challenge the status quo. Franklin American was doing amazing things and transforming their business — all with Docker Enterprise Edition as the foundation.

Don presented at DockerCon 2018 along with Franklin American’s VP of Innovation, Sharon Frazier. They’ve been able to quickly build a DevOps culture around four pillars: Visibility, Simplification, Standardization and Experimentation. Experimentation is key. It lets them fail fast, and fearlessly.

In an interview at DockerCon, Don explained how they use Docker Enterprise Edition to drive innovation.

“Docker has allowed us to fail fearlessly. We can test new things easily and quickly and if they work, awesome. But if they don’t, we didn’t spend weeks or months on it.” – Don Bauer, DevOps Lead 

For the company, innovation is about Continue reading

Making Compose Easier to Use with Application Packages

Docker Compose is wildly popular with developers for describing an application. In fact, there are more than 300,000 Docker Compose files on GitHub. With a set of services described in a docker-compose.yml file, it’s easy to launch a complex multi-service application (or a simple, single-service app) on Docker by running a single command. This ease of use makes Docker Compose perfect for development teams striving for a quick way of getting started with projects.

Over time Compose has evolved, adding lots of features which help when deploying those same applications to production environments, for example specifying a number of replicas, memory resource constraints or a custom syslog server. But those attributes can become specific to your own environment. There are a number of different strategies for trying to address this situation, but the most common is relying on copy and paste. It’s fairly common to maintain multiple Compose files for the same application running in different environments for example. This leads to two problems:

  1. We share Docker images all the time, but don’t have a good way of sharing the multi-service applications that use them
  2. It’s hard to collaborate between developers and operators around a Compose file. This waters Continue reading

It’s a wrap! The DockerCon Cool Hacks closing keynote.

Yesterday we continued a long tradition at DockerCon, the Cool Hacks closing keynote. In our Cool Hacks keynote, we like to emphasize applications that push the limits and applications that represent major future trends in container workloads. We also like to feature applications that demonstrate how Docker fueled innovation can be used every day.

This DockerCon, the three applications we chose embodied all of these characteristics.

Our first hack, by Christopher Heistand of the Johns Hopkins University Applied Physics Laboratory is helping save the world. The Double Asteroid Redirect Mission Test (DART) is testing kinetic impact against an asteroid to measure whether one can be redirected. They use Docker to emulate the specialized and expensive hardware, saving them money and development time.

David Aronchick (@aronchick ‏) and Michelle Casbon (@texasmichelle)  demonstrated our second hack with Kubeflow. Machine learning in production workloads, at scale.

And finally, Idit Levine (@Idit_Levine) showed us Gloo. Gloo gives you the portability and choice of a serverless framework, from cloud services like AWS Lambda to running one of the several containerized self-hosted serverless frameworks. All running in Docker EE.

Check out our Cool Hacks closing keynote.

And finally, we wrapped up inviting Continue reading

DockerCon Guest Speaker: Liberty Mutual

In yesterday’s DockerCon keynote, Eric Drobisewski, Senior Architect at Liberty Mutual Insurance, shared how Docker Enterprise Edition has been a foundational technology for their digital transformation.

If you missed it, the replay of the keynote is available below:

Turning Disruption into Opportunities

Liberty Mutual – the 3rd largest property and casualty insurance provider in the United States –  recognizes that the new digital economy is bringing a faster cycle of technology evolution. Disruptive technologies like autonomous vehicles and smart homes are changing the way customers interact and transact. Liberty Mutual sees these as opportunities to bring new services to market and ways to reinvent traditional insurance models, but they needed to become more flexible and agile while managing their technical debt.

Rapid Expansion of Docker EE

As a 106-year old company, Liberty Mutual recognized that they were not going to become agile overnight. Liberty Mutual has instead built a “multi-lane highway” that enables both traditional apps and new microservices apps to modernize at different speeds according to their needs, all based on Docker Enterprise Edition.

“(Docker Enterprise Edition) began to open multiple paths for our teams to modernize traditional applications and move them to the cloud in a Continue reading

Open Source Serverless Frameworks on Docker EE

Since the advent of AWS Lambda in 2014, the Function as a Service (FaaS) programming paradigm has gained a lot of traction in the cloud community. At first, only large cloud providers such as AWS Lambda, Google Cloud Functions or Azure Functions provided such services with a pay-per-invocation model, but since then interest has increased for developers and entreprises to build their own solutions on an open source model.

The maturation of container platforms such as Docker EE has made this process even easier, resulting in a number of competing frameworks in this space. We have identified at least 9 different frameworks*. In this study, we start with the following six: OpenFaaS, nuclio, Gestalt, Riff, Fn and OpenWhisk. You can find an introduction (including slides and videos) to some of these frameworks in this blog post from the last DockerCon Europe.

These frameworks vary a lot in feature set, but can be generalized as having several key elements shown in the following diagram, from the Serverless Architecture from CNCF Serverless Working Group whitepaper:

serverless Docker

  • Event sources – trigger or stream events into one or more function instances
  • Function instances – a single function/microservice, that can be Continue reading

Day 1 Keynote Highlights from DockerCon San Francisco 2018

Hello from San Francisco! Tuesday we kicked off the first day of DockerCon with general session jam packed with inspiration, demos and customer guest speakers.

Steve Singh, our CEO and Chairman opened the session with Docker’s promise to ensure freedom of choice, agility in development and operations and pervasive security in a container platform that can help unlock the potential for innovation in every company. Docker will deliver an integrated toolset with a delightful user experience that needs innovators like you.

Docker EE promise

Day one also featured three demos of new technologies capabilities for both Docker Desktop and Docker Enterprise Edition. These features are not yet generally available but released and those interested in the beta can sign up here to be notified.

1 2 3 40