HelloFresh: Navigating the rough seas of environment scaling with Aho

In the past couple of years, our engineering team at HelloFresh has experienced a huge growth spurt. Within 2 years our team grew from 35 engineers to well over 150. One of the biggest challenges we faced was how to enable over 20 teams to test their code independently to other teams in a stable environment.

HelloTech Team Growth

I’m going to tell you about how we solved scaling our staging and local environments using phoenix environments.

The Problem

When we were smaller we had only 2 environments, what we called stagingand production. This setup was great for simple applications and small teams. Once we moved to a microservice architecture with 90+ services it quickly became apparent that it was not a scalable solution.

With more teams blocking staging for their testing, the environment quickly became unusable. The solution seemed simple, create an environment for each team called team staging.

Team Staging: Not the solution

We hoped that by creating a staging environment (which is a subset of all services from the main staging) for each team, it would enable them to test whatever they wanted in isolation. Or so we thought; in reality, it was much more complicated than that.

The Continue reading

Google Boots Up Tensor Processors On Its Cloud

Google laid down its path forward in the machine learning and cloud computing arenas when it first unveiled plans for its tensor processing unit (TPU), an accelerator designed by the hyperscaler to speeding up machine learning workloads that are programmed using its TensorFlow framework.

Almost a year ago, at its Google I/O event, the company rolled out the architectural details of its second-generation TPUs – also called the Cloud TPU – for both neural network training and inference, with the custom ASICs providing up to 180 teraflops of floating point performance and 64 GB of High Bandwidth Memory.

Google Boots Up Tensor Processors On Its Cloud was written by Jeffrey Burt at The Next Platform.

Why hiring the ‘best’ people produces the least creative results — Quartz

The complexity of modern problems often precludes any one person from fully understanding them. Factors contributing to rising obesity levels, for example, include transportation systems and infrastructure, media, convenience foods, changing social norms, human biology and psychological factors. Designing an aircraft carrier, to take another example, requires knowledge of nuclear engineering, naval architecture, metallurgy, hydrodynamics, […]

Context-Aware Micro-segmentation – Remote Desktop Session Host Enhancements for VMware Horizon

In a previous post my colleague, Stijn, discussed the enhancements to how NSX for vSphere 6.4 handles Remote Desktop Session Host, RDSH, systems with the Identity-based Firewall and Context-Aware Micro-segmentation.

Remote Desktop Services is an underlying technology from Microsoft that many vendors take advantage of to provide overlay management and application deployment technologies for.  In this post, we’re going to discuss how NSX for vSphere 6.4 allows customers to run RDS hosts with granular security for VMware Horizon systems.

VMware Horizon can provide multiple users the ability to connect to a single system to access their applications using the RDSH technology.  These users can be of the same type, for example all HR users, or of multiple types, HR and Engineering users.  In previous versions of NSX, it was not possible to individually secure user sessions and create Distributed Firewall (DFW) rule sets according to the user session logged into an RDSH server.  This meant less flexibility in controlling what users could access data center application servers without isolating one set of users to one RDSH server.  This model created a very rigid architecture for Horizon customers to follow.

Horizon allows customers Continue reading

IDG Contributor Network: The top 4 industrial enterprise requirements of IoT application enablement platforms (AEP)

With over 400 self-proclaimed IoT platforms in the market, it doesn’t surprise me that industrial enterprises are hindered trying to identify, test and select a high quality IoT platform. Platform vendors’ marketing materials contain the same messages, their RFX responses always affirm “full compliance” with all requested capabilities and they have partnerships with the same cloud vendors. With over 400 self-proclaimed IoT platforms in the market, the only way to truly know each platform is to use it.What makes a great IoT AEP? An Application Enablement Platform (AEP) is a technology-centric offering optimized to deliver a best-of-breed, industry-agnostic, extensible middleware core for building a set of interconnected or independent IoT solutions for customers. An AEP links IoT devices and applications, delivering data to allow industrial enterprises to implement predictive maintenance, machine learning, factory automation, asset logistics, surveillance and many other applications. With IoT platform revenue slated to grow to USD 63.4 billion by 2026, IoT application enablement is one of the most highly demanded enterprise IoT platforms.To read this article in full, please click here

The internet will miss John Perry Barlow

John Perry Barlow, who died in San Francisco last week at age 70, was an important pioneer for internet freedom. But he was much, much more than that. He was the kind of Renaissance Man that today’s internet moguls can’t even dream of emulating. And that is a huge loss for the world of technology — and the world at large.Barlow’s wide-ranging influence You may not have heard of Barlow, but you’ve probably been influenced by him in a wide variety of surprising ways. For one thing, he was a co-founder — and at his passing, vice chairman — of the Electronic Frontier Foundation, which is considered “the leading nonprofit organization defending civil liberties in the digital world.” Back in 1990, when the EFF was formed, Barlow helped popularize the term “cyberspace.” He was a director of the WELL (Whole Earth ’Lectronic Link), the seminal online community, and he was an influential early voice at Wired magazine.To read this article in full, please click here

IDG Contributor Network: Cloud strategy: hybrid and multi cloud are not the same

IDC tells us that most companies are using more than one cloud and that cloud usage isn’t just about cost savings. Three out of every four companies are using cloud to chase additional revenue in the form of new customers, risk mitigation, IoT enablement or time to market gains. Most are using multiple external cloud services.However, especially as microservices become the dominant approach to new application development because of the iteration speed improvements that it provides, it has become important to distinguish the different ways that more than one cloud can be utilized. Specifically, the differences lie in where you sit in an organization and what you are trying to optimize from that seat. Although historically we’ve used the terms interchangeably, hybrid and multi cloud are not the same.To read this article in full, please click here

IDG Contributor Network: Cloud strategy: hybrid and multi cloud are not the same

IDC tells us that most companies are using more than one cloud and that cloud usage isn’t just about cost savings. Three out of every four companies are using cloud to chase additional revenue in the form of new customers, risk mitigation, IoT enablement or time to market gains. Most are using multiple external cloud services.However, especially as microservices become the dominant approach to new application development because of the iteration speed improvements that it provides, it has become important to distinguish the different ways that more than one cloud can be utilized. Specifically, the differences lie in where you sit in an organization and what you are trying to optimize from that seat. Although historically we’ve used the terms interchangeably, hybrid and multi cloud are not the same.To read this article in full, please click here

IDG Contributor Network: Why the edge has moved to the forefront

Remember just a few years ago, when everyone was talking about cloud computing? While cloud was consuming all the air in the room, few people were paying attention to another technology trend—one with the potential to transform industrial enterprises. I’m talking about edge computing.The idea of placing computing resources at the network’s edge—at or near where production processes are occurring—is not a completely new idea. Industrial control has relied on distributed computers to control manufacturing machines and processes for decades. But as manufacturers come under increasing competitive pressure, the need to optimize their efficiency, productivity and quality has become a matter of survival. This imperative requirement is driving companies across the industrial spectrum to look at how pushing intelligence out to the edge can help them gain a competitive advantage.To read this article in full, please click here

IDG Contributor Network: Why the edge has moved to the forefront

Remember just a few years ago, when everyone was talking about cloud computing? While cloud was consuming all the air in the room, few people were paying attention to another technology trend—one with the potential to transform industrial enterprises. I’m talking about edge computing.The idea of placing computing resources at the network’s edge—at or near where production processes are occurring—is not a completely new idea. Industrial control has relied on distributed computers to control manufacturing machines and processes for decades. But as manufacturers come under increasing competitive pressure, the need to optimize their efficiency, productivity and quality has become a matter of survival. This imperative requirement is driving companies across the industrial spectrum to look at how pushing intelligence out to the edge can help them gain a competitive advantage.To read this article in full, please click here

Lustre Shines at HPC Peaks, But Rest of Market is Fertile Ground

The Lustre file system has been the canonical choice for the world’s largest supercomputers, but for the rest of high performance computing user base, it is moving beyond reach without the support and guidance it has had from its many backers, including most recently Intel, which dropped Lustre from its development ranks in mid-2017.

While Lustre users have seen the support story fall to pieces before, for many HPC shops, the need is greater than ever to look toward a fully supported scalable parallel file system that snaps well to easy to manage appliances. Some of these commercial HPC sites

Lustre Shines at HPC Peaks, But Rest of Market is Fertile Ground was written by Nicole Hemsoth at The Next Platform.

Some Tools to Help Learn Kubernetes

Kubernetes is emerging as the clear leader in the container orchestration space. This makes it an important technology to know and understand. However, like other distributed systems, learning something like Kubernetes can be challenging due to the effort involved in getting Kubernetes up and running. It’s not about learning to set up Kubernetes (although that comes in time); at first, it’s about understanding what Kubernetes does and how to use Kubernetes. In this post, I’ll share some tools to help learn what Kubernetes does and how to use Kubernetes.

Note that this post is not intended to be a comprehensive list of learning resources for Kubernetes. Also, this post is not focused on providing resources to help you learn to deploy Kubernetes. Instead, I’m focusing here on tools and services that let you get Kubernetes up and running quickly and easily so that you can focus on using Kubernetes (deploying applications and workloads onto Kubernetes). I’m sure there are many more tools/options than what I have listed here; these are just some that I have used and feel might be useful for others.

I’ll briefly cover the following tools and services:

  • Minikube
  • Kops
  • Kube-aws
  • Azure Container Service (ACS/AKS)

You’ll note Continue reading

The Blind Men and the Elephant

640px-elephas_maximus_eye_closeupBufferbloat is responsible for much of the poor performance seen in the Internet today and causes latency (called “lag” by gamers), triggered even by your own routine web browsing and video playing.

But bufferbloat’s causes and solutions remind me of the old parable:

It was six men of Indostan, to learning much inclined,
who went to see the elephant (Though all of them were blind),
that each by observation, might satisfy his mind.

      ……. (six stanzas elided)

And so these men of Indostan, disputed loud and long,
each in his own opinion, exceeding stiff and strong,
Though each was partly in the right, and all were in the wrong!

So, oft in theologic wars, the disputants, I ween,
tread on in utter ignorance, of what each other mean,
and prate about the elephant, not one of them has seen!

John Godfrey Saxe

Most technologists are not truly wise: we are usually like the blind men of Indostan. The TCP experts, network operators, telecom operators, router makers, Internet service operators, router vendors and users have all had a grip *only* on their piece of the elephant.

The TCP experts look at TCP and think “if only TCP were Continue reading

NASA Social OA-4 Tours and Launch

In December 2015 I had the distinct honor to be selected to attend a NASA Social event that coincided with the OA-4 launch, an ISS resupply mission. NASA runs these events occasionally to give the world some insight into what goes on at various locations pertaining to the space industry. There’s a lot of really cool work going on, and I’m happy to finally have a chance to publish my experience on this amazing trip.

Home network traffic analysis with a Raspberry Pi 3, a NetGear Switch and Ntop

I had the Raspberry Pi laying around for some time without doing any major function and so was a the NetGear switch [1]. So, I decided to do a weekend project to implement traffic analysis on my home network. I have a PPPoE connection to my ISP that connects to my home router [2]. The … Continue reading Home network traffic analysis with a Raspberry Pi 3, a NetGear Switch and Ntop