Depend on Docker for Kubeflow

Run Kubeflow natively on Docker Desktop for Mac or Windows

This is a guest post by Alex Iankoulski, Docker Captain and full stack software and infrastructure architect at Shell New Energies. The views expressed here are his own and are neither opposed or endorsed by Shell or Docker. 

In this blog, I will show you how to use Docker Desktop for Mac or Windows to run Kubeflow. To make this easier, I used my Depend on Docker project, which you can find on Github.

Rationale

Even though we are experiencing a tectonic shift of development workflows in the cloud era towards hosted and remote environments, a substantial amount of work and experimentation still happens on developer’s local machines. The ability to scale down allows us to mimic a cloud deployment locally and enables us to play, learn quickly, and make changes in a safe, isolated environment. A good example of this rationale is provided by Kubeflow and MiniKF.

Overview

Since Kubeflow was first released by Google in 2018, adoption has increased significantly, particularly in the data science world for orchestration of machine learning pipelines. There are various ways to deploy Kubeflow both on desktops and servers as described in Continue reading

The “Depend on Docker” Philosophy at Baker Hughes, a GE Company

BHGE is the world’s leading full stream Oil & Gas company on a mission to find better ways to deliver energy to the world. BHGE Digital develops enterprise grade cloud-first SaaS solutions to improve efficiency and reduce non-productive time for the Oil & Gas industry.

In our group, we have developed an analytics-driven product portfolio to enable company-wide digital transformation for our customers. Challenges ranging from predicting the failures of mission-critical industrial assets such as gas turbines to optimizing the conditions of an Electric Submersible Pump (ESP) to increase production, which require building and maintaining sophisticated analytics at scale.

The past few years have taught us this: where there is a whale, there is a way!

We were happy to share our story at DockerCon recently, and wanted to share it here on the Docker blog as well. You can watch the session here:

We face two major challenges in delivering advanced analytics:

1. Data silos
   We must handle a multitude of data sources that range from disconnected historical datasets to high speed sensor streams. Industrial data volumes and velocities dwarf even the largest ERP implementations as shown below.