The Linux Migration Series

In early 2017 I kicked off an effort to start using Linux as my primary desktop OS, and I blogged about the journey. That particular effort ended in late October 2017. I restarted the migration in April 2018 (when I left VMware to join Heptio), and since that time I’ve been using Linux (Fedora, specifically) full-time. However, I thought it might be helpful to collect the articles I wrote about the experience together for easy reference. Without further ado, here they are.

Initial Progress Report

Final Linux Distro Selection

Virtualization Provider

Other Users’ Stories: Part 1, Part 2, Part 3, Part 4

Creating Presentations

Corporate Collaboration: Part 1, Part 2, Part 3

April 2017 Progress Report

July 2017 Progress Report

Wrap-Up

These are only the articles directly related to the migration efforts, but many more articles were spawned as a result of the project. Browse through all the Fedora-tagged articles to see some related articles.

If you have any questions about migrating to Linux or about any of these articles (or related articles), you’re welcome to contact me on Twitter. I look forward to hearing from you!

Docker Certified Containers From IBM

As cloud computing continues to transform every business and industry, developers at global enterprises and emerging startups alike are increasingly leveraging container technologies to accelerate how they build modern web, mobile and IoT applications.  

IBM has achieved certification of its flagship Db2 database, Websphere-Liberty middleware server and Security Access Manager products now available on Docker Hub. These Certified Containers enable developers to accelerate building cloud-native applications for the Docker Enterprise platform.  Developers can deploy these solutions from IBM to any on-premises infrastructure or public cloud.  They are designed to assist in the modernization of traditional applications moving from on-premises monoliths into hybrid cloud microservices.

These solutions are validated by both Docker and IBM and are integrated into a seamless support pipeline that provides customers the world-class support they have become accustomed to when working with Docker and IBM.

Check out the latest certified technology available from IBM Continue reading

Technology Short Take 108

Welcome to Technology Short Take #108! This will be the last Technology Short Take of 2018, so here’s hoping I can provide something useful for you. Enjoy!

Networking

• Maish Saidel-Keesing has a 5-part series on replacing the AWS ELB. This is an older post (from August) that I’ve had in my backlog for a while, and I’m just now getting around to reading the series. There’s some really good information in here. I won’t link to all five, but rather just point you at the introductory post (and Maish has done a great job—better than a lot of bloggers—making the entire series easily accessible).
• Quentin Machu delves deep into an obscure DNS resolution issue that was introducing seemingly-erratic delays with DNS lookups. Quentin’s post is very detailed, and has lots of good information.
• Anthony Burke has some information on preparing a node for Kubernetes and the NSX NCP.
• Benjamin Dale suggests that you don’t know JunOS.

Servers/Hardware

• On the hardware side of things, we have James Hamilton of AWS discussing two hardware-related items coming out of AWS re:Invent 2018. First, Hamilton discusses the Arm-based AWS Graviton processor powering the newly-announced A1 family of EC2 instances. Next, Hamilton tackles the need Continue reading

Speak at DockerCon San Francisco 2019 – Call for Papers is Open

Whether you missed DockerCon EU in Barcelona, or you already miss the fun, connections and learning you experienced at DockerCon – you won’t have to wait long for the next one. DockerCon returns to San Francisco on April 29 and extends through May 2, 2019 and the Call for Papers is now open. We are accepting talk submissions through January 18th at 11:59 PST.  

Attending DockerCon is an awesome experience, but so is speaking at DockerCon – it’s a great way to get to know the community, share ideas and collaborate. Don’t be nervous about proposing your idea – no topic is too small or too big. And for some speakers, DockerCon is their first time speaking publicly. Don’t be intimidated, DockerCon attendees are all looking to level up their skills, connect with fellow container fans and go home inspired to implement new containerization initiatives. Here are some suggested topics from the conference committee:

• “How To” type sessions for developers or IT teams
• Case Studies
• Technical deep dives into container and distributed systems related components
• Cool New Apps built with Docker containers
• The craziest thing you have containerized
• Wild Card – anything and everything!
• The Continue reading

Desigual Transforms the In-Store Customer Experience with Docker Enterprise

At DockerCon Barcelona, we awarded Desigual with the first ever Rising Star Docker Customer Innovation Award. The Desigual team earned the award by building a brand new in-store shopping assistant application in just 5 months thanks to Docker Enterprise. The digital shopping assistant is already deployed at over 100 stores, and is being rolled out to all of Desigual’s 500-plus clothing stores worldwide in the coming months.

In this 2 minute video, Desigual gives the highlights of their story:

https://youtu.be/BKbD-gW6yI0

The Desigual team analyzed existing sales data and found that of lost in-store sales, 60 percent were because a particular size was out of stock, and 40 percent were because a product wasn’t available in the catalog.

They wanted to create a customer-first shopping experience that would stand out among retail clothing brands and help store associates recommend alternatives to customers. To do that, they needed to tie multiple elements together: Store point-of-sale (POS), the online catalog, mobile capability, and personal attention through the shopper profile.

Mathias Kriegel, IT Ops Lead and Cloud Architect, and Joan Anton Sances, Software Architect, discussed the project and why they selected Docker Enterprise in their presentation at DockerCon Barcelona 2018.

KubeCon NA 2018 Wrap Up: Docker and the Kubernetes Community

Right on the heels of DockerCon Europe, the Docker team was excited to be a part of KubeCon in Seattle last week for great conversations and collaboration with the Kubernetes community. In addition to our commitment to delivering a simple, integrated experience with Kubernetes in our Docker Desktop and Docker Enterprise products, we’re also excited by our work with the community at the very foundation of Kubernetes with projects like containerd and Notary/TUF and to talk container standards with the members of the Open Container Initiative (OCI). KubeCon is an opportunity for project maintainers to explain the status and roadmap of projects, but also to meet face to face and collaborate with contributors to determine what is next for cloud native applications.

Giving Back to the Kubernetes Community

The Docker and Kubernetes communities have been working together closely since Kubernetes was announced at DockerCon 2014. In line with our commitment to continue to make containerization technology like Kubernetes easier to use: a few weeks ago we open sourced Docker Compose on Kubernetes, a project that provides a simple way to define cloud native applications with a higher-level abstraction, the Docker Compose file. Docker Compose is a tool Continue reading

Running Fedora on my Mac Pro

I’ve been working on migrating off macOS for a couple of years (10+ years on a single OS isn’t undone quickly or easily). I won’t go into all the gory details here; see this post for some background and then see this update from last October that summarized my previous efforts to migrate to Linux (Fedora, specifically) as my primary desktop operating system. (What I haven’t blogged about is the success I had switching to Fedora full-time when I joined Heptio.) I took another big step forward in my efforts this past week, when I rebuilt my 2011-era Mac Pro workstation to run Fedora.

When I mentioned this on Twitter, a few people asked the question every parent dreads hearing: “Why?” (If you’ve been a parent for more than a couple years you’ll understand this.) The motivation for using Linux is something I’ve already discussed. As for the hardware, it’s simple: the hardware for the Mac Pro is very good (see the base specs here), so why not re-use the hardware for use with Linux? I mean, if I’ve already decided on running Linux (which I have), then why spend money on new hardware Continue reading

Ansible Tips and Tricks: Dealing with Unreliable Connections and Services

Red Hat Ansible Automation is widely known to automate and configure Linux and Windows hosts, as well as network automation for routers, switches, firewalls and load balancers. Plus, there are a variety of modules that deal with the cloud and the API around it such as Microsoft Azure, Amazon Web Services (AWS) and Google Compute Engine.  And there are other modules that interact with Software as a Service (SaaS) tools like Slack or ServiceNow. Although the downtime for these APIs is very minimal, it does happen, and it is even more likely that the connection between your Ansible control host (where you are running Ansible from) and the cloud-centric API could be down.

In this blog post, I will cover some tips and tricks for dealing with unreliable connections to cloud-centric APIs and how I build Ansible Playbooks in a reliable manner. As a technical marketing engineer, I consider my customers the Red Hat field teams, and often Solutions Architects are running playbooks from unreliable hotel wireless, coffee shops and sometimes even airplanes! I have to make sure playbooks have some more robustness built in for these odd situations. It is hair-pulling frustrating to get through a 20 task Continue reading

Introducing the New Docker Hub

Today, we’re excited to announce that Docker Store and Docker Cloud are now part of Docker Hub, providing a single experience for finding, storing and sharing container images. This means that:

• Docker Certified and Verified Publisher Images are now available for discovery and download on Docker Hub
• Docker Hub has a new user experience

Millions of individual users and more than a hundred thousand organizations use Docker Hub, Store and Cloud for their container content needs. We’ve designed this Docker Hub update to bring together the features that users of each product know and love the most, while addressing known Docker Hub requests around ease of use, repository and team management.

Here’s what’s new:

Repositories

• View recently pushed tags and automated builds on your repository page
• Pagination added to repository tags
• Improved repository filtering when logged in on the Docker Hub home page

• As an organization Owner, see team permissions across all of your repositories at a glance.
• Add existing Docker Hub users to a team via their email (if you don’t remember their of Docker ID)

New Automated Builds

• Speed up builds using Build Caching
• Add environment variables and run tests in your builds
• Continue reading

KubeCon 2018 Day 2 Keynote

This is a liveblog of the day 2 (Wednesday) keynotes at KubeCon/CloudNativeCon 2018 in Seattle, WA. For additional KubeCon 2018 coverage, check out other articles tagged KubeCon2018.

Kicking off the day 2 keynotes, Liz Rice takes the stage at 9:02am (same time as yesterday, making me wonder if my clock is off by 2 minutes). Rice immediately brings out Janet Kuo, Software Engineer at Google and co-chair with Rice of the KubeCon/CloudNativeCon event program. Kuo will be delivering a Kubernetes project update.

Kuo starts off by reiterating the announcement of the Kubernetes 1.13 release, and looking back on her very first commit to Kubernetes in 2015 (just prior to the 1.0 release and the formation of the CNCF). Kuo talks about how Kubernetes, as a software cycle, has matured through the cycle of first focusing on innovation, then expanding to include scale, and finally expanding again to include stability (critical for enterprise adopters).

Reviewing usage details, Kuo states that she believes Kubernetes has moved—in the context of the technology adoption curve—from early adopters to early majority, the first phase in the mainstream market (and, for those who think in these terms, has crossed the chasm). However, this also Continue reading

Liveblog: Hardening Kubernetes Setups

This is a liveblog of the KubeCon NA 2018 session titled “Hardening Kubernetes Setup: War Stories from the Trenches of Production.” The speaker is Puja Abbassi (@puja108 on Twitter) from Giant Swarm. It’s a pretty popular session, held in one of the larger ballrooms up on level 6 of the convention center, and nearly every seat was full.

Abbassi starts by talking about Giant Swarm’s environment, in which they run more than 100 clusters across different clouds and different regions. These clusters are running for different companies, different industries, and they serve different use cases for various constituents of users. Abbassi says that Giant Swarm opts to give users more freedom in how they use (and potentially misuse) the clusters.

Obviously, this can lead to problems, and that’s where the postmortems come into play. Abbassi explains the idea behind postmortems by quoting a definition from the Google SRE book, and then provides some context about the process that Giant Swarm follows when conducting postmortems. That leads into a discussion of various postmortems conducted at Giant Swarm.

The first one mentioned by Abbassi concerns a memory leak first fixed in 1.11.4 and 1.12.0. Prior to Continue reading

Liveblog: Linkerd 2.0, Now with Extra Prometheus

This is a liveblog of the KubeCon NA 2018 session titled “Linkerd 2.0, Now with Extra Prometheus.” The speakers are Frederic Branczyk from Red Hat and Andrew Seigner with Buoyant.

Seigner kicks off the session with a quick introduction before handing off to Branczyk. Prometheus, for folks who didn’t know, originated at SoundCloud with a couple of ex-Googlers. Prometheus is one of the graduated CNCF projects and—judging by a show of hands in response to a speaker question—lots of folks here at KubeCon know about Prometheus and are using Prometheus in production.

Branczyk provides an overview of Prometheus, explaining that it pulls metrics from a target on a set of regular intervals (like every 15 seconds, for example). Prometheus stores those metrics in a time-series database, so every time it pulls metrics it stores them in a time series. As a monitoring solution, it also has to provide alerting, to notify cluster operators/administrators that some metric is outside of some predefined threshold.

With regards to Kubernetes, Prometheus has built-in support to perform service discovery in Kubernetes by querying the Kubernetes API. This enables it to discover Pods backing a Service and scrape (pull) the metrics from those discovered Continue reading

KubeCon 2018 Day 1 Keynote

This is a liveblog from the day 1 (Tuesday, December 11) keynote of KubeCon/CloudNativeCon 2018 in Seattle, WA. This will be my first (and last!) KubeCon as a Heptio employee, and looking forward to the event.

The keynote kicks off at 9:02am with Liz Rice, Technology Evangelist at Aqua Security. Rice welcomes attendees (back) to Seattle, and she shares that this year’s event in Seattle is 8x the size of the same event in Seattle just two years ago. Rice also shares some statistics from other CNCF events around the world, stressing the growth of these events both in size and in the number of events happening worldwide.

Rice next shares some entertaining statistics about web site visits to kubernetes.io versus some other popular brands. (TL;DR: Kubernetes gets more web site visits than the Seahawks and Manchester United, but not as many as Starbucks.)

Moving on, Rice talks for a few minutes about the strategy or purpose behind the collection of projects that fall under the CNCF umbrella (to provide some of the important building blocks in the full stack of technologies to support cloud-native environments). At this point, Rice turns it over to Michelle Noorali, Continue reading

How to find the correct MTU and MRU of your link

Overview

In the previous post, I talked about Network IP Fragmentation, what it is and why it’s needed (You are advised to read it before continuing). I also covered the so called PMTUD Black hole effect.

Fixing a PMTUD Black hole is a multistep process, and it starts with finding the correct MTU/MRU of your link.

Now as I’ve discussed, every path can have its own unique MTU/MRU value, but we are usually interested in the max value that is dictated by your ISP.

When you send a packet, it always routes through your ISP. Because of different protocols in place and their overheads (mostly layer 2 ones), it is common for your ISP to force MTU/MRU of less than 1500 bytes on your link.

If a packet exceeds these values, your ISP is required to send the appropriate ICMP messages either back to you (for the MTU), or to the server sending the data (for the MRU). These messages give the corresponding hosts a chance to adapt themselves to the link.

If your ISP decides to not send the required ICMP messages (or they get lost in transaction for some reason), all sorts of issues could arise. And Continue reading

Docker App and CNAB

Docker App is a new tool we spoke briefly about back at DockerCon US 2018. We’ve been working on `docker-app` to make container applications simpler to share and easier to manage across different teams and between different environments, and we open sourced it so you can already download Docker App from GitHub at https://github.com/docker/app.

In talking to others about problems they’ve experienced sharing and collaborating on the broad area we call “applications” we came to a realisation: it’s a more general problem that others have been working on too. That’s why we’re happy to collaborate with Microsoft on the new Cloud Native Application Bundle (CNAB) specification.

Today’s cloud native applications typically use different technologies, each with their own toolchain. Maybe you’re using ARM templates and Helm charts, or CloudFormation and Compose, or Terraform and Ansible. There is no single solution in the market for defining and packaging these multi-service, multi-format distributed applications.

CNAB is an open source, cloud-agnostic specification for packaging and running distributed applications that aims to solve some of these problems. CNAB unifies the management of multi-service, distributed applications across different toolchains into a single all-in-one packaging format.

The draft specification is available at cnab.io and Continue reading

Announcing Cloud Native Application Bundle (CNAB)

As more organizations pursue cloud-native applications and infrastructures for creating modern software environments, it has become clear that there is no single solution in the market for defining and packaging these multi-service, multi-format distributed applications. Real-world applications can now span on-premises infrastructure and cloud-based services, requiring multiple tools like Terraform for the infrastructure, Helm charts and Docker Compose files for the applications, and CloudFormation or ARM templates for the cloud-services. Each of these need to be managed separately.

To address this problem, Microsoft in collaboration with Docker are announcing Cloud Native Application Bundle (CNAB) – an open source, cloud-agnostic specification for packaging and running distributed applications. CNAB unifies the management of multi-service, distributed applications across different toolchains into a single all-in-one packaging format.The CNAB specification lets you define resources that can be deployed to any combination of runtime environments and tooling including Docker Engine, Kubernetes, Helm, automation tools and cloud services.

Docker is the first to implement CNAB for containerized applications and will be expanding it across the Docker platform to support new application development, deployment and lifecycle management. Initially CNAB support will be released as part of our docker-app experimental tool for building, packaging and managing Continue reading

Simplifying Kubernetes with Docker Compose and Friends

Today we’re happy to announce we’re open sourcing our support for using Docker Compose on Kubernetes. We’ve had this capability in Docker Enterprise for a little while but as of today you will be able to use this on any Kubernetes cluster you choose.

Why do I need Compose if I already have Kubernetes?

The Kubernetes API is really quite large. There are more than 50 first-class objects in the latest release, from Pods and Deployments to ValidatingWebhookConfiguration and ResourceQuota. This can lead to a verbosity in configuration, which then needs to be managed by you, the developer. Let’s look at a concrete example of that.

The Sock Shop is the canonical example of a microservices application. It consists of multiple services using different technologies and backends, all packaged up as Docker images. It also provides example configurations using different tools, including both Compose and raw Kubernetes configuration. Let’s have a look at the relative sizes of those configurations:

$ git clone https://github.com/microservices-demo/microservices-demo.git
$ cd deployment/kubernetes/manifests
$ (Get-ChildItem -Recurse -File | Get-Content | Measure-Object -line).Lines
908
$ cd ../../docker-compose
$ (Get-Content docker-compose.yml | Measure-Object -line).Lines
174

Describing the exact same multi-service application using just Continue reading

Introducing Docker Desktop Enterprise

Nearly 1.4 million developers use Docker Desktop every single day because it is the simplest and easiest way for container-based development. Docker Desktop provides the Docker Engine with Swarm and Kubernetes orchestrators right on the desktop, all from a single install. While this is great for an individual user, in enterprise environments administrators often want to automate the Docker Desktop installation and ensure everyone on the development team has the same configuration following enterprise requirements and creating applications based on architectural standards.

Docker Desktop Enterprise is a new desktop offering that is the easiest, fastest and most secure way to create and deliver production-ready containerized applications. Developers can work with frameworks and languages of their choice, while IT can securely configure, deploy and manage development environments that align to corporate standards and practices. This enables organizations to rapidly deliver containerized applications from development to production.

Enterprise Manageability That Helps Accelerate Time-to-Production

Docker Desktop Enterprise provides a secure way to configure, deploy and manage developer environments while enforcing safe development standards that align to corporate policies and practices. IT teams and application architects can present developers with application templates designed specifically for their team, to bootstrap and standardize Continue reading

Announcing the Docker Customer Innovation Awards

We are excited to announce the first annual Docker Customer Innovation Award winners at DockerCon Barcelona today! We launched the awards this year to recognize customers who stand out in their adoption of Docker Enterprise platform to drive transformation within IT and their business.

38 companies were nominated, all of whom have spoken publicly about their containerization initiatives recently, or plan to soon. From looking at so many excellent nominees, we realized there were really two different stories — so we created two award categories. In each category, we have a winner and three finalists.

Business Transformation

Customers in this category have developed company-wide initiatives aimed at transforming IT and their business in a significant way, with Docker Enterprise as a key part if it. They typically started their journey two or more years ago and have containerized multiple applications across the organization.

WINNER:

• Societe Generale transformed how the bank develops its software by building a container platform for migrating thousands of its applications to the cloud.

FINALISTS:

• Bosch built a global platform that enables developers to build and deliver new software solutions and updates at digital speed.

• Liberty Mutual consolidated infrastructure and VMs significantly, paving Continue reading

Technology Short Take 107

Welcome to Technology Short Take #107! In response to my request for feedback in the last Technology Short Take, a few readers responded in favor of a more regular publication schedule even if that means the articles are shorter in length. Thus, this Tech Short Take may be a bit shorter than usual, but hopefully you’ll still find something useful.

Networking

• Matt Oswalt shares some details on how NRE Labs implements “curriculum-as-code.”

Servers/Hardware

• Christian Kellner provides a brief reminder that not all USB-C ports are Thunderbolt ports, and updates everyone on the status of bolt (Linux utility for working with Thunderbolt ports and peripherals).

Security

• Troy Hunt has a good article on security measures other than just passwords, explaining some of the differences between multi-factor authentication and multi-step authentication (for example). Highly recommended reading.

Cloud Computing/Cloud Management

• Another post from Matt Oswalt, also related to the NRE Labs post I mentioned above, discusses troubleshooting NGINX Ingress rewrites in Kubernetes.
• Arush Salil reviews using ingress on Kubernetes with cert-manager.
• Michael Hausenblas has an informative article with information on Kubernetes RBAC defaults.
• This post is a couple months old (a lifetime in the cloud-native world): the AWS Service Operator enables Continue reading