0

Faster builds in Docker Compose 1.25.1 thanks to BuildKit Support

One of the most requested features for the docker-compose tool is definitely support for building using Buildkit which is an alternative builder with great capabilities, like caching, concurrency and ability to use custom BuildKit front-ends just to mention a few… Ahhh with a nice blue output! And the good news is that Docker Compose 1.25.1 – that was just released early January – includes BuildKit support!

BuildKit support for Docker Compose is actually achieved by redirecting the docker-compose build to the Docker CLI with a limited feature set.

Enabling Buildkit build

To enable this, we have to align some stars.

First, it requires that the Docker CLI binary present in your PATH:

$ which
docker/usr/local/bin/docker

Second, docker-compose has to be run with the environment variable COMPOSE_DOCKER_CLI_BUILD set to 1 like in:

$ COMPOSE_DOCKER_CLI_BUILD=1 docker-compose build

This instruction tells docker-compose to use the Docker CLI when executing a build. You should see the same build output, but starting with the experimental warning.

As docker-compose passes its environment variables to the Docker CLI, we can also tell the CLI to use BuildKit instead of the default builder. To accomplish that, we can execute this:

$ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build

A Continue reading

0

Docker Desktop release 2.2 is here!

We are excited to announce that we released a new Docker Desktop version today! Thanks to the user feedback on the new features initially released in the Edge channel, we are now ready to publish them into Stable. 

Before getting to each feature into detail, let’s see what’s new in Docker Desktop 2.2:

• WSL 2 as a technical preview, allowing access to the full system resources, improved boot time, access to Linux workspaces and improved file system performance
• A new file sharing implementation for Windows, improving the developer inner loop user experience
• A New Integrated Desktop Dashboard, to see at once glance your local running containers and Compose applications, and easily manage them.

WSL 2 – New architecture 

Back in July we released on Edge the technical preview of Docker Desktop for WSL 2, where we included an experimental integration of Docker running on an existing user Linux distribution. We learnt from our experience and re-architected our solution (covered in Simon’s blog) . 

This new architecture for WSL 2 allows users to: 

• Use Kubernetes on the WSL 2 backend
• Work with just WSL 2/turn off the traditional HyperV VM while working with WSL 2
• Continue Continue reading

0

Capturing Logs in Docker Desktop

Docker Desktop runs a Virtual Machine to host Docker containers. Each component within the VM (including the Docker engine itself) runs as a separate isolated container. This extra layer of isolation introduces an interesting new problem: how do we capture all the logs so we can include them in Docker Desktop diagnostic reports? If we do nothing then the logs will be written separately into each individual container which obviously isn’t very useful!

The Docker Desktop VM boots from an ISO which is built using LinuxKit from a list of Docker images together with a list of capabilities and bind mounts. For a minimal example of a LinuxKit VM definition, see https://github.com/linuxkit/linuxkit/blob/master/examples/minimal.yml — more examples and documentation are available in the LinuxKit repository. The LinuxKit VM in Docker Desktop boots in two phases: in the first phase, the init process executes a series of one-shot “on-boot” actions sequentially using runc to isolate them in containers. These actions typically format disks, enable swap, configure sysctl settings and network interfaces. The second phase contains “services” which are started concurrently and run forever as containerd tasks.

The following diagram shows a simplified high-level view of the boot process:

By default Continue reading

0

5 Software Development Predictions for 2020

• 

To kick off the new year, we sat down with Docker CEO Scott Johnston and asked him what the future holds for software development. Here are his 2020 predictions and trends to keep an eye on.

Existing Code and Apps Become New Again

Developers will find new ways to reuse existing code instead of reinventing the wheel to start from scratch. Additionally, we’ll see companies extend the value to existing apps by adding more functionality via microservices.

The Changing Definition of a Modern Application

Today’s applications are more complex than those of yesterday. In 2020, modern apps will power tomorrow’s innovation and this requires a diverse set of tools, languages and frameworks for developers. Developers need even more flexibility to address this new wave of modern apps and evolve with the rest of the industry.

Containers Pave the Way to New Application Trends

Now that containers are typically considered a common deployment mechanism, the conversation will evolve from the packaging of individual containers to the packaging of the entire application (which are becoming increasingly diverse and distributed). Organizations will increasingly look for guidance and solutions that help them unify how they build and manage Continue reading

0

2019 Docker Community Awards

0

Year in Review: The Most Loved Docker Articles, Blogs and Tweets of 2019

As this decade comes to a close, we are rounding up some of your favorite content from 2019. Catch up on anything you missed and get ready for a lot more to come in 2020!

Docker Captain Content

Brian Christner did an analysis of VMware, Docker, and Kubernetes Google Trends and the results just might surprise you. . . or maybe not.

John Lees Miller updated his 2016 Lessons from Building a Node App in Docker. Run through the updated tutorial to learn how to Dockerize your node.js apps by setting up the socket.io chat example with Docker, from scratch to production-ready. 

Ajeet Singh Raina wrote nearly 30 blogs in 2019, and the most popular was 5 Minutes to Kubernetes Dashboard running on Docker Desktop for Windows 2.0.0.3. Find yourself five minutes before the end of the year to try this out yourself.

Łukasz Lach and Thomas Shaw spread holiday cheer with some seasonal docker run commands:

$ docker run -it lukaszlach/merry-christmas 

docker run --rm -t tomwillfixit/hohoho

Bret Fisher hosts a weekly Docker and DevOps YouTube live show – a fun and educational way to spend an hour on Thursdays. Continue reading

0

Containers Today Recap: The Future of the Developer Journey

There was amazing attendance at Containers Today in Stockholm a couple of weeks ago. For those who were unable to make it, here is a quick overview of what I talked about at the event in my session around the future of the developer journey. 

Before we talk about what we think will change the journey, we need to think about why it changes. The fundamental goal of any change to the way of working for developers should be to reduce the number of boring, mundane and repetitive tasks that developers have to do or to allow them to reach new customers/solve new problems. Developers create amazing value for companies and provide solutions to customers’ real world problems. But if they are having to spend half of their time working out how to get things into the hands of their customers, then you are getting half the value.

Developer Evolution

The role of developers has changed a lot over the last ~40 years. Developers no longer deploy to mainframes or in house hardware, they don’t do waterfall deployments and not many of them write in machine code. Developers have to now think about web languages and ML, work in Continue reading

0

Deep Dive Into the New Docker Desktop filesharing Implementation Using FUSE

The latest Edge release of Docker Desktop for Windows 2.1.7.0 has a completely new filesharing system using FUSE instead of Samba. The initial blog post we released presents the performance improvements of this new implementation and explains how to give feedback. Please try it out and let us know what you think. Now, we are going to go into details to give you more insight about the new architecture.

New Architecture

Instead of Samba running over a Hyper-V virtual network, the new system uses a Filesystem in Userspace (FUSE) server running over gRPC over Hypervisor sockets.

The following diagram shows the path taken by a single request from a container, for example to read a PHP file:

In step (1) the web-server in the container calls “read” which is a Linux system call handled by the kernel’s Virtual File System (VFS) layer. The VFS is modular and supports many different filesystem implementations. In our case we use Filesystem in Userspace (FUSE) which sends the request to a helper process running inside the VM labelled “FUSE client.” This process runs within the same namespace as the Docker engine. The FUSE client can handle some requests locally, but Continue reading

0

New Filesharing Implementation in Docker Desktop Windows Improves Developer Inner Loop UX

• 

A common developer workflow when using frameworks like Symfony or React is to edit the source code using a Windows IDE while running the app itself in a Docker container. The source is shared between the host and the container with a command like the following:

$ docker run -v C:\Users\me:/code -p 8080:8080 my-symfony-app

This allows the developer to edit the source code, save the changes and immediately see the results in their browser. This is where file sharing performance becomes critical.

The latest Edge release of Docker Desktop for Windows 2.1.7.0 has a completely new filesharing implementation using Filesystem in Userspace (FUSE) instead of Samba which:

• uses caching to (for example) reduce page load time in Symfony by up to 60%;
• supports Linux inotify events, triggering automatic recompilation / reload when the source code is changed;
• is independent of how you authenticate to Windows: smartcard, Azure AD are all fine;
• always works irrespective of whether your VPN is connected or disconnected;
• reduces the amount of code running as Administrator.

Your feedback needed!

This improvement is available today in the Edge 2.1.7.0 release and will roll-out to the stable Continue reading

0

Managing the TICK Stack with Docker App

Docker Application eases the packaging and the distribution of a Docker Compose application. The TICK stack – Telegraf, InfluxDB, Chronograf, and Kapacitor – is a good candidate to illustrate how this actually works. In this blog, I’ll show you how to deploy the TICK stack as a Docker App.

About the TICK Stack

This application stack is mainly used to handle time-series data. That makes it a great choice for IoT projects, where devices send data (temperature, weather indicators, water level, etc.) on a regular basis.

Its name comes from its components:

– Telegraf

– InfluxDB

– Chronograf

– Kapacitor

The schema below illustrates the overall architecture, and outlines the role of each component.

Data are sent to Telegraph and stored in an InfluxDB database. Chronograf can query the database through a web interface. Kapacitor can process, monitor, and raise alerts based on the data.

Defining the Application in a Compose File

The tick.yml file below defines the four components of the stack and the way they communicate with each other:

version: '3.7'
services:
  telegraf:
    image: telegraf
    configs:
    - source: telegraf-conf
      target: /etc/telegraf/telegraf.conf
    ports:
    - 8186:8186
  influxdb:
    image: influxdb
  chronograf:
 Continue reading

0

AWS IoT Greengrass 1.10 Now Supports Docker Containers

On November 25, 2019, AWS announced the release of AWS IoT Greengrass 1.10 allowing developers to package applications into Docker container images and deploy these to edge devices. Deploying and running Docker containers on AWS IoT Greengrass devices enables application portability across development environments, edge locations, and the cloud. Docker images can easily be stored in Docker Hub, private container registries, or with Amazon Elastic Container Registry (Amazon ECR).

Docker is committed to working with cloud service provider partners such as AWS who offer Docker-compatible on-demand container infrastructure services for both individual containers as well as multi-container apps. To make it even easier for developers to benefit from the speed of these services but without giving up app portability and infrastructure choice, Docker Hub will seamlessly integrate developers’ “build” and “share” workflows with the cloud “run” services of their choosing.

“Docker and AWS are collaborating on our shared vision of how workloads can be more easily deployed to edge devices. Docker’s industry-leading container technology including Docker Desktop and Docker Hub are integral to advancing developer workflows for modern apps and IoT solutions. Our customers can now deploy and run Docker containers seamlessly on AWS IoT Greengrass devices, enabling development Continue reading

0

Black Friday Deals on Docker + Kubernetes Courses

In honor of Black Friday, America’s favorite shopping holiday, we’ve rounded up the best deals on Docker + Kubernetes learning materials from Docker Captains. Docker Captain is a distinction that Docker awards to select members of the community that are both experts in their field and are committed to sharing their Docker knowledge with others. 

Books:

Manning publications is also offering half off when you spend $50 this week.

Self-Paced Online Courses:

0

Docker’s Next Chapter: Advancing Developer Workflows for Modern Apps

Today we start the next chapter in the Docker story, one that’s focused on developers. That we have the opportunity to write this next chapter is thanks to you, our community, for without you we wouldn’t be here. And while our focus on developers builds on recent history, it’s a focus also grounded in Docker’s beginning.

In The Beginning

When Solomon Hykes, Docker’s founder, unveiled the Docker project in 2013, he succinctly stated the problem Docker aimed to solve as, “for a developer, shipping code to the server is hard.” To address, Docker abstracted out OS kernels’ complex container primitives, provided a developer-friendly, CLI-based workflow and defined an immutable, portable image format. The result transformed how developers work, making it much easier to build, ship and run their apps on any server. So while container primitives had existed for decades, Docker democratized them and made them as easy to use as

docker run hello-world

The rest is history. Over the last six years, Docker containerization catalyzed the growth of microservices-based applications, enabled development teams to ship apps many times faster and accelerated the migration of apps from the data center to the cloud. Far from a Docker-only effort, a Continue reading

0

Celebrating Veterans Day: Docker Employee Profiles

On Veterans Day, and every day, we give thanks to our veterans. We are fortunate to have Brent Salisbury, Siobhan Casey, and Johnny Gonzalez, as Docker colleagues who were in the United States Marine Corps Reserve, the United States Army Reserve, and the United States Marine Corps. Thank you all for your service, hard work, and dedication. As a thank you for their service, we’re profiling them on our blog.

Brent Salisbury, Software Alliance Engineer

What is your job? 

Software Alliance Engineer.

How long have you worked at Docker?

4.5 years.

Is your current role one that you always intended on your career path? 

Data Networking has been my passion since college. Working at Docker has afforded me the opportunity to help usher in a new software paradigm in what can be achieved in host networking and security versus the traditional proprietary hardware models of the past.

What is your advice for someone entering the field?

It may sound cliche, but find your passion. Everyone in technology is Continue reading

0

A Roadmap for Building Modern Applications

No matter what industry you’re in, your application modernization strategy matters. Overlooking or downplaying its importance is a quick way for customers to sour and competitors to gain an edge. It’s why 91% of executives believe their revenues will take a hit without successful digital transformation.

The good news is modern applications offer a clear path forward. Creating a roadmap for your modern application strategy is a critical step toward a more agile and continuous model of software development and delivery – one that’s centered on delivering perpetually expanding value and new experiences to customers. 

This is the first of a series of blogs where we will look at industry viewpoints, different approaches, underlying platforms and real-world stories that are foundational to successful modern application development in order to provide a roadmap for application modernization.

What’s in Your Environment? 

The technology inventory at companies today is as diverse, distributed and complex as ever. It includes a variety of technology stacks, application frameworks, services and languages. During a modernization process, new Open Source technologies are often integrated with legacy solutions. Existing applications need to be maintained and enhanced, modern applications need to be Continue reading

0

Depend on Docker for Kubeflow

Run Kubeflow natively on Docker Desktop for Mac or Windows

This is a guest post by Alex Iankoulski, Docker Captain and full stack software and infrastructure architect at Shell New Energies. The views expressed here are his own and are neither opposed or endorsed by Shell or Docker. 

In this blog, I will show you how to use Docker Desktop for Mac or Windows to run Kubeflow. To make this easier, I used my Depend on Docker project, which you can find on Github.

Rationale

Even though we are experiencing a tectonic shift of development workflows in the cloud era towards hosted and remote environments, a substantial amount of work and experimentation still happens on developer’s local machines. The ability to scale down allows us to mimic a cloud deployment locally and enables us to play, learn quickly, and make changes in a safe, isolated environment. A good example of this rationale is provided by Kubeflow and MiniKF.

Overview

Since Kubeflow was first released by Google in 2018, adoption has increased significantly, particularly in the data science world for orchestration of machine learning pipelines. There are various ways to deploy Kubeflow both on desktops and servers as described in Continue reading

0

For Liberty Mutual, the Openness and Flexibility of the Cloud Means Better Business Outcomes

We had the chance recently to sit down with the Liberty Mutual Insurance team at their Portsmouth, New Hampshire offices and talk about how they deliver better business outcomes with the cloud and containerization.

At this point, Liberty Mutual has moved about 30 percent of their applications to the cloud. One of big improvements the team has seen with the cloud and Docker is the speed at which developers can develop and deploy their applications. That means better business outcomes for Liberty Mutual and its customers.

Here’s what they told us. You can also catch the highlights in this two-minute video:

On how tech is central to Liberty Mutual’s business

Mark Cressey, SVP and GM, IT Hosting Services: Tech and the digitization it’s allowed has really enabled Liberty Mutual to get deeply ingrained in our customers’ lives and support them through their major life journeys. We’re able to be more predictive of what our customer’s needs and get in front of them as a proactive step. How can we help? How can we assist you? Is this the right coverage? And even to the point where using real time information, we can warn them about approaching windstorms or warn our Continue reading

0

Docker’s Recommended Sessions for KubeCon 2019

The Docker team is gearing up for another great KubeCon this year in San Diego, November 17-21. As a Platinum sponsor of this year’s event, we are excited to bring Docker employees, community members and Docker captains together to demonstrate and celebrate  the combined impact of Docker and Kubernetes.

Stop by Booth P37 to learn how to leverage the Docker platform to securely build, share and run modern applications for any Kubernetes environment. We will demonstrate Docker Desktop Enterprise and how it accelerates container application development while supporting developer choice. Experts will be on hand to answer questions about Docker Kubernetes Services (DKS), a secure and production-ready Kubernetes environment. Or come to learn more about Docker’s contributions to Kubernetes while picking up some great Docker swag.

Learn More from Docker Experts

KubeCon will also provide a great opportunity to learn from industry experts and hear from people who run production applications on Kubernetes. Here’s a helpful guide from the Docker team of our recommended talks:

Monday, Nov 18

• Kubernetes 101 Workshop – Docker Captain Nigel Poulton

Tuesday, Nov 19

• Securing the Software Supply Chain with in-toto – Justin Cappos & Marina Moore, NYU  
• Introduction to Windows Containers in Kubernetes Continue reading

0

Learn About Modern Apps on Azure with Docker at Microsoft Ignite

The Docker team will be on the show floor at Microsoft Ignite the week of November 4. We’ll be talking about the state of modern application development, how to accelerate innovation efforts, and the role containerization, Docker, and Microsoft Azure play in powering these initiatives.

Come by booth #2414 at Microsoft Ignite to check out the latest developments in the Docker platform. Learn why over 1.8 million developers build modern applications on Docker, and over 800 enterprises rely on Docker Enterprise for production workloads. 

At Microsoft Ignite, we will be talking about:

How to Develop and Deliver Modern Applications for Azure Kubernetes Service (AKS)

Docker Enterprise 3.0 shipped back in April 2019, making it the first and only desktop-to-cloud container platform in the market that lets you build and share any application and securely run them anywhere – from hybrid cloud to the edge. At Microsoft Ignite, we’ll have demos that shows how Docker Enterprise 3.0 simplifies Kubernetes for Azure Kubernetes Service (AKS) and enables companies to more easily build modern applications with Docker Desktop Enterprise and Docker Application. 

Learn how to accelerate your journey to the cloud with Docker’s Dev Team Starter Bundle for Continue reading

0

Don’t Be Scared of Kubernetes

5 Reasons You Might Be Afraid to Get Started with Kubernetes

Kubernetes has the broadest capabilities of any container orchestrator available today, which adds up to a lot of power and complexity. That can be overwhelming for a lot of people jumping in for the first time – enough to scare people off from getting started. There are a few reasons it can seem intimidating:

• It’s complicated, isn’t it? As we noted in a previous post, jumping into the cockpit of a state-of-the-art jet puts a lot of power under you, but how to actually fly the thing is not obvious. If you’ve never done more than play a flight simulator game, it can be downright scary.
• Is it production-ready? Everyone is talking about Kubernetes, but it’s only emerged as a major technology in the past few years. Many companies take a wait-and-see approach on new technologies. Building out a Kubernetes deployment on your own means solving challenging problems without enterprise support. 
• Do I have the people and skills to support it? IT teams are just beginning to learn Kubernetes. If it’s complicated, it means you’ll need people with the right experience to support it. According to industry Continue reading