0

New Year, New Adventure

I’ll skip the build-up and jump straight to the whole point of this post: a once-in-a-lifetime opportunity has come up and I’m embarking on a new adventure starting in early 2020. No, I’m not changing jobs…but I am changing time zones.

Sometime in the next month or two (dates are still being finalized), I’ll be temporarily relocating to Tokyo, Japan, to help build out VMware’s Cloud Native Field Engineering team to provide consulting and professional services around cloud-native technologies and modern application platforms for customers in Japan. Basically, my charter is to replicate the former Heptio Field Engineering team (now the Cloud Native Field Engineering Practice within VMware) in Japan.

Accomplishing this feat will involve a variety of responsibilities: a pretty fair amount of training/enablement, engaging customers on the pre-sales side, helping lead projects on the post-sales (delivery) side, mentoring team members, performing some project management, probably some people management, and the infamous “other duties as required.” All in about six months (the inital duration of my assignment), and all while learning Japanese! No big deal, right?

I’m both simultaneously excited and scared. I’m excited by the idea of living in Tokyo, but let’s be honest—the language barrier is Continue reading

0

Technology Short Take 122

Welcome to Technology Short Take #122! Luckily I did manage to get another Tech Short Take squeezed in for 2019, just so all my readers could have some reading materials for the holidays. I’m kidding! No, I mean I really am kidding—don’t read stuff over the holidays. Spend time with your family instead. The investment in your family will pay off in later years, trust me.

Networking

• Teammate Alex Brand has a look at VMware’s new OVS-based CNI plugin, Antrea.

Servers/Hardware

• Adam Leventhal takes a closer look at the financial side of AWS Outposts, which was announced as generally available at this year’s re:Invent.

Security

• Bruce Schneier writes about how some Chinese hackers are bypassing RSA software token authentication (the title is a bit more broad, implying other forms of two-factor authentication are affected, but the article focuses on attacks against the use of RSA software tokens).
• Robin Moffatt discusses detecting and analysing SSH attacks with ksqlDB.

Cloud Computing/Cloud Management

• Ahmet Alp Balkan talks about how the Knative Serving component addresses some key networking challenges within Kubernetes.
• Teri Radichel has a post with some structural tips to better understand and utilize AWS CloudFormation.
• Geert Baeke walks readers through deploying Continue reading

0

Year in Review: The Most Loved Docker Articles, Blogs and Tweets of 2019

As this decade comes to a close, we are rounding up some of your favorite content from 2019. Catch up on anything you missed and get ready for a lot more to come in 2020!

Docker Captain Content

Brian Christner did an analysis of VMware, Docker, and Kubernetes Google Trends and the results just might surprise you. . . or maybe not.

John Lees Miller updated his 2016 Lessons from Building a Node App in Docker. Run through the updated tutorial to learn how to Dockerize your node.js apps by setting up the socket.io chat example with Docker, from scratch to production-ready. 

Ajeet Singh Raina wrote nearly 30 blogs in 2019, and the most popular was 5 Minutes to Kubernetes Dashboard running on Docker Desktop for Windows 2.0.0.3. Find yourself five minutes before the end of the year to try this out yourself.

Łukasz Lach and Thomas Shaw spread holiday cheer with some seasonal docker run commands:

$ docker run -it lukaszlach/merry-christmas 

docker run --rm -t tomwillfixit/hohoho

Bret Fisher hosts a weekly Docker and DevOps YouTube live show – a fun and educational way to spend an hour on Thursdays. Continue reading

0

Containers Today Recap: The Future of the Developer Journey

There was amazing attendance at Containers Today in Stockholm a couple of weeks ago. For those who were unable to make it, here is a quick overview of what I talked about at the event in my session around the future of the developer journey. 

Before we talk about what we think will change the journey, we need to think about why it changes. The fundamental goal of any change to the way of working for developers should be to reduce the number of boring, mundane and repetitive tasks that developers have to do or to allow them to reach new customers/solve new problems. Developers create amazing value for companies and provide solutions to customers’ real world problems. But if they are having to spend half of their time working out how to get things into the hands of their customers, then you are getting half the value.

Developer Evolution

The role of developers has changed a lot over the last ~40 years. Developers no longer deploy to mainframes or in house hardware, they don’t do waterfall deployments and not many of them write in machine code. Developers have to now think about web languages and ML, work in Continue reading

0

Rebooting Network Devices with Ansible

With the Red Hat Ansible Automation Platform release in November, we released over 50 network resource modules to help make automating network devices easier and more turn-key for network engineers.  In addition to the new resource modules, Andrius also discussed fact gathering enhancements in his blog post, which means with every new resource module, users gain increased fact coverage for network devices.  For this blog post I want to cover another cool enhancement that may have gone unnoticed. This is the ability for network devices to make use of the wait_for_connection module.  If you are a network engineer that has operational Ansible Playbooks that need to reboot devices or take them offline, this module will help you make more programmatic playbooks to handle disconnects.  By leveraging wait_for_connection network automation playbooks can look and behave more like playbooks for Linux or Windows hosts.

 

Table of Contents

Comparing wait_for and wait_for_connection

Dealing with prompts

Using reset_connection in combination

Where to go next?

 

Comparing wait_for and wait_for_connection 

There are two great modules that can wait for a condition to be met, wait_for and the wait_for_connection.  I highly recommend against using the pause module if you Continue reading

0

Ansible Content Collections: The Sensu Go Use Case

In Getting Started With Ansible Content Collections, which presented the general idea behind what is becoming a new standard in the distribution of Ansible content, we learned about the what, the why and the how of Ansible Collections (and hopefully it got you excited about Ansible Collections!). In this post, we'll take things a bit further, continuing the journey into the world of Ansible Collections accompanied by the certified Sensu Go Ansible Collection that our team at XLAB Steampunk developed and supports for Sensu.

 

What to expect?

This article will guide you through the process of creating a fully functioning automated deployment of the Sensu Go monitoring agent and backend with the help of roles and modules included in the Sensu Go Ansible Collection. 

If you are not familiar with Sensu Go, this quick introduction to Sensu Go will help you get up to speed.

Before we begin, let's first talk about the collection we're taking along for the ride.

 

The Sensu Go Ansible Collection: the basics

What exactly do we need for a complete and fully functioning deployment of Sensu Go? First, the Sensu Go monitoring backend. Then, to allow the backend to Continue reading

0

Deep Dive Into the New Docker Desktop filesharing Implementation Using FUSE

The latest Edge release of Docker Desktop for Windows 2.1.7.0 has a completely new filesharing system using FUSE instead of Samba. The initial blog post we released presents the performance improvements of this new implementation and explains how to give feedback. Please try it out and let us know what you think. Now, we are going to go into details to give you more insight about the new architecture.

New Architecture

Instead of Samba running over a Hyper-V virtual network, the new system uses a Filesystem in Userspace (FUSE) server running over gRPC over Hypervisor sockets.

The following diagram shows the path taken by a single request from a container, for example to read a PHP file:

In step (1) the web-server in the container calls “read” which is a Linux system call handled by the kernel’s Virtual File System (VFS) layer. The VFS is modular and supports many different filesystem implementations. In our case we use Filesystem in Userspace (FUSE) which sends the request to a helper process running inside the VM labelled “FUSE client.” This process runs within the same namespace as the Docker engine. The FUSE client can handle some requests locally, but Continue reading

0

New Filesharing Implementation in Docker Desktop Windows Improves Developer Inner Loop UX

• 

A common developer workflow when using frameworks like Symfony or React is to edit the source code using a Windows IDE while running the app itself in a Docker container. The source is shared between the host and the container with a command like the following:

$ docker run -v C:\Users\me:/code -p 8080:8080 my-symfony-app

This allows the developer to edit the source code, save the changes and immediately see the results in their browser. This is where file sharing performance becomes critical.

The latest Edge release of Docker Desktop for Windows 2.1.7.0 has a completely new filesharing implementation using Filesystem in Userspace (FUSE) instead of Samba which:

• uses caching to (for example) reduce page load time in Symfony by up to 60%;
• supports Linux inotify events, triggering automatic recompilation / reload when the source code is changed;
• is independent of how you authenticate to Windows: smartcard, Azure AD are all fine;
• always works irrespective of whether your VPN is connected or disconnected;
• reduces the amount of code running as Administrator.

Your feedback needed!

This improvement is available today in the Edge 2.1.7.0 release and will roll-out to the stable Continue reading

0

Technology Short Take 121

Welcome to Technology Short Take #121! This may possibly be the last Tech Short Take of 2019 (not sure if I’ll be able to squeeze in another one), so here’s hoping that you find something useful, helpful, or informative in the links that I’ve collected. Enjoy some light reading over your festive holiday season!

Networking

• Brian Linkletter has a post on how to run the Antidote network emulator on KVM for better performance. Antidote, as I understand it, is the network emulator that runs the labs on the NRELabs web site (which is another cool thing in and of itself). In perusing some of Brian’s content, I see lots of stuff at the intersection of networking and virtualization. That’s a good place to be right now, in my opinion.
• Anders Olsson explains how to map NSX-T UUIDs to the actual logical objects.
• Julien Demierre has three-part series on interfaces and uplinks for ESXi/Edge in NSX-T deployments (part 1, part 2, and part 3).
• This is an old(er) post that I’ve probably mentioned here before, but I think it still has relevance. Brent Salisbury’s post on Golang for Network Ops not only points out why networking pros Continue reading

0

Managing the TICK Stack with Docker App

Docker Application eases the packaging and the distribution of a Docker Compose application. The TICK stack – Telegraf, InfluxDB, Chronograf, and Kapacitor – is a good candidate to illustrate how this actually works. In this blog, I’ll show you how to deploy the TICK stack as a Docker App.

About the TICK Stack

This application stack is mainly used to handle time-series data. That makes it a great choice for IoT projects, where devices send data (temperature, weather indicators, water level, etc.) on a regular basis.

Its name comes from its components:

– Telegraf

– InfluxDB

– Chronograf

– Kapacitor

The schema below illustrates the overall architecture, and outlines the role of each component.

Data are sent to Telegraph and stored in an InfluxDB database. Chronograf can query the database through a web interface. Kapacitor can process, monitor, and raise alerts based on the data.

Defining the Application in a Compose File

The tick.yml file below defines the four components of the stack and the way they communicate with each other:

version: '3.7'
services:
  telegraf:
    image: telegraf
    configs:
    - source: telegraf-conf
      target: /etc/telegraf/telegraf.conf
    ports:
    - 8186:8186
  influxdb:
    image: influxdb
  chronograf:
 Continue reading

0

AWS IoT Greengrass 1.10 Now Supports Docker Containers

On November 25, 2019, AWS announced the release of AWS IoT Greengrass 1.10 allowing developers to package applications into Docker container images and deploy these to edge devices. Deploying and running Docker containers on AWS IoT Greengrass devices enables application portability across development environments, edge locations, and the cloud. Docker images can easily be stored in Docker Hub, private container registries, or with Amazon Elastic Container Registry (Amazon ECR).

Docker is committed to working with cloud service provider partners such as AWS who offer Docker-compatible on-demand container infrastructure services for both individual containers as well as multi-container apps. To make it even easier for developers to benefit from the speed of these services but without giving up app portability and infrastructure choice, Docker Hub will seamlessly integrate developers’ “build” and “share” workflows with the cloud “run” services of their choosing.

“Docker and AWS are collaborating on our shared vision of how workloads can be more easily deployed to edge devices. Docker’s industry-leading container technology including Docker Desktop and Docker Hub are integral to advancing developer workflows for modern apps and IoT solutions. Our customers can now deploy and run Docker containers seamlessly on AWS IoT Greengrass devices, enabling development Continue reading

0

Automation Analytics: Part 2 – Looking at Data Collection

We recently released Red Hat Ansible Automation Platform which now includes multiple Software-as-a-Service (SaaS) offerings, one of which is Automation Analytics.  This offering provides a visual dashboard, health notifications and organization statistics for your Ansible Automation. Automation Analytics works across multiple Ansible Tower clusters allowing holistic analytics across your entire automation infrastructure.

In a previous blog I wrote about Getting Started with Automation Analytics, but now want to expand on what data is collected and how to gain access to that data.  I highly recommend reading the previous blog if you are new to Red Hat Ansible Automation Platform, Ansible Tower concepts and our SaaS offerings. This is important to many customers because they all have their own security concerns with what data leaves their premises as well as  obligations to their own customers and stakeholders to make sure data sent will not be compromised in any way.

 

Retrieving the data:

Analyzing the data

config.json

counts.json

cred_type_counts.json

events_table.csv

instance_info.json

inventory_counts.json

job_counts.json

job_instance_counts.json

manifest.json

org_counts.json

projects_by_scm_type.json

query_info.json

unified_jobs_table.csv

unified_job_template_table.csv

Where to go next?

Retrieving the data:

Login to the Ansible Tower host with Continue reading

0

Black Friday Deals on Docker + Kubernetes Courses

In honor of Black Friday, America’s favorite shopping holiday, we’ve rounded up the best deals on Docker + Kubernetes learning materials from Docker Captains. Docker Captain is a distinction that Docker awards to select members of the community that are both experts in their field and are committed to sharing their Docker knowledge with others. 

Books:

Manning publications is also offering half off when you spend $50 this week.

Self-Paced Online Courses:

0

What’s New In Red Hat Ansible Tower 3.6

As part of the release of Red Hat Ansible Automation Platform, we’re happy to announce the release of Red Hat Ansible Tower 3.6. Ansible Tower is the scalable execution framework of the Ansible Automation Platform, providing an API around automation that you can use to scale automation across your enterprise and integrate automation into your tools and processes.

 

Workflow Pause and Approvals

Not all automation processes can proceed entirely without human input. In Ansible Tower 3.6, we’ve added pause and approval to Ansible Tower workflows to help enable more flexible automation. At any step in a workflow you can pause and wait for an approval from an administrator, or any other you delegate approval permissions to. Need to verify that your deployment was fully successful before updating the external DNS entries? Need to ensure that your developers won’t spin up 300 extra cloud servers when provisioning new dev environments? Now you can do that, integrated directly in Ansible Tower workflows.

 

Customizable Notifications

Notifications were introduced in Ansible Tower 3.0, allowing the status of any job to be reported out via email, Slack, IRC, and more. In Ansible Tower 3.6, we’ve made the content Continue reading

0

Technology Short Take 120

Welcome to Technology Short Take #120! Wow…hard to believe it’s been almost two months since the last Tech Short Take. Sorry about that! Hopefully something I share here in this Tech Short Take is useful or helpful to readers. On to the content!

Networking

• In the event you are seeking more information on NAT hole punching, here you go. You’re welcome. (I found this link in the serverless mullet architectures post, see the “Operating Systems/Applications” section below.)
• Mohamad Alhussein shares how to filter the transit subnets from being redistributed from NSX-T to upstream physical routers (by transit subnets Alhussein is referring to the NSX-managed subnets used to connect logical routers together).
• Ahmet Alp Balkan shows how to use mitmproxy to inspect kubectl traffic. I’m now inspired to go do this myself and see what knowledge I can gain.
• Corey Quinn proclaims “BGP is a hot mess”; Ivan Pepelnjak lays out some facts.

Servers/Hardware

I don’t have anything to share this time around, but I’ll stay alert for content to include future Tech Short Takes.

Security

• This basic introduction of firewalld as found in CentOS 8 may prove useful to some readers. I’ve been messing around with firewalld ever since Continue reading

0

KubeCon 2019 Day 3 and Event Summary

Keynotes

Bryan Liles kicked off the day 3 morning keynotes with a discussion of “finding Kubernetes’ Rails moment”—basically focusing on how Kubernetes enables folks to work on/solve higher-level problems. Key phrase from Bryan’s discussion (which, as usual, incorporated the humor I love to see from Bryan): “Kubernetes isn’t the destination. Kubernetes is the vehicle that takes us to the destination.” Ian Coldwater delivered a talk on looking at Kubernetes from the attacker’s point of view, and using that perspective to secure and harden Kubernetes. Two folks from Walmart also discussed their use case, which involves running Kubernetes clusters in retail locations to support a point-of-sale (POS) application at the check-out register. Finally, there was a discussion of chaos engineering from folks at Gremlin and Target.

No Breakout Sessions

Due to booth duty and my flight home, I wasn’t able to attend any breakout sessions today.

Event Summary

If I’m completely honest, I didn’t get as much out of the event as I’d hoped. I’m not yet sure if that is because I didn’t get to attend as many sessions as I’d hoped/planned (due to problems with sessions being moved/rescheduled or whatever), if my choice of sessions was just poor, Continue reading

0

KubeCon 2019 Day 2 Summary

Keynotes

This morning’s keynotes were, in my opinion, better than yesterday’s morning keynotes. (I missed the closing keynotes yesterday due to customer meetings and calls.) Only a couple of keynotes really stuck out. Vicki Cheung provided some useful suggestions for tools that are helping to “close the gap” on user experience, and there was an interesting (but a bit overly long) session with a live demo on running a 5G mobile core on Kubernetes.

Running Large-Scale Stateful Workloads

Due to some power outages at the conference venue resulting from rain in San Diego, the Prometheus session I had planned to attend got moved to a different time. As a result, I sat in this session by Lyft instead. The topic was about running large-scale stateful workloads, but the content was really about a custom solution Lyft built (called Flyte) that leveraged CRDs and custom controllers to help manage stateful workloads. While it’s awesome that companies like Lyft can extend Kubernetes to address their specific needs, this session isn’t helpful to more “ordinary” companies that are trying to figure out how to run their stateful workloads on Kubernetes. I’d really like the CNCF and the conference committee to try Continue reading

0

Manage Secrets and Protect Sensitive Data

Automation is an essential part of modern IT. In this blog I focus on Ansible credential plugins integration via Hashicorp Vault, an API addressable secrets engine which will make life easier for anyone wishing to handle secrets management and automation better. In order to automate effectively, modern systems require multiple secrets: certificates, database credentials, keys for external services, operating systems, networking. Understanding who is accessing secret credentials and when is difficult and often platform-specific and to manage key rotation, secure storage and detailed audit logging across a heterogeneous toolset is almost impossible. Red Hat Ansible Tower solves many of these issues on its own, but its integration with enterprise secret management solutions means it can utilize secrets on demand without human interaction.

In terms of secrets management, I will demonstrate how some of the risks associated with an automation service account can be mitigated by replacing password authentication with ssh certificate based authentication. In the context of automation, a service account is used to provide authorised access into endpoints from a central location.  Best practices around security state that, shared accounts could pose a risk. While Red Hat Ansible Tower has the ability to obfuscate passwords, private keys, etc. Continue reading

0

KubeCon 2019 Day 1 Summary

This week I’m in San Diego for KubeCon + CloudNativeCon. Instead of liveblogging each session individually, I thought I might instead attempt a “daily summary” post that captures highlights from all the sessions each day. Here’s my recap of day 1 at KubeCon + CloudNativeCon.

Keynotes

KubeCon + CloudNativeCon doesn’t have “one” keynote; it uses a series of shorter keynotes by various speakers. This has advantages and disadvantages; one key advantage is that there is more variety, and the attendees are more likely to stay engaged. I particularly enjoyed Bryan Liles’ CNCF project updates; I like Bryan’s sense of humor, and getting updates on some of the CNCF projects is always useful. As for some of the other keynotes, those that were thinly-disguised vendor sales pitches were generally pretty poor.

Introduction to the Virtual Kubelet

I was running late for the start of this session due to booth duty, and I guess the stuff I needed most was presented in that portion I missed. Most of what I saw was about Netflix Titus, and how the Netflix team ported Titus from Mesos to Virtual Kubelet. However, that information was so specific to Netflix’s particular use of Virtual Kubelet that it Continue reading

0

Continuous Improvements in Ansible and Kubernetes Automation

Ansible is an ideal tool for managing many different types of Kubernetes resources. There are four key features that really help:

• Modules and plugins for creating, updating, removing and obtaining information about Kubernetes resources
• Templating of Kubernetes resource definitions
• Powerful inventory system
• Secrets management 

Together these combine to help enable repeatable deployment and management of applications and multiple Kubernetes clusters in a single role for every resource.

Since the last blog post on Kubernetes features for Ansible Engine 2.6, there have been a number of improvements to Ansible's Kubernetes capabilities. Let’s go over some of the improvements to the modules and libraries and other new features that have been added in the last year, and also highlight what is in the works.

 

New features

Better change management through apply mode

The k8s module now accepts an apply parameter, which approximates the behavior of kubectl apply. When apply is set to True, the k8s module will store the last applied configuration in an annotation on the object. When the object already exists, instead of just sending the new manifest to the API server, the module will now do a 3-way merge, combining the existing cluster state, the Continue reading