Technology Short Take 134

Welcome to Technology Short Take #134! I’m publishing a bit early this time due to the Thanksgiving holiday in the US. So, for all my US readers, here’s some content to peruse while enjoying some turkey (or whatever you’re having this year). For my international readers, here’s some content to peruse while enjoying dramatically lower volumes of e-mail because the US is on holiday. See, something for everyone!

Networking

Security

  • I’m glad to see this. Open source has become so critical to so many aspects of our computing infrastructure.
  • OpenCSPM looks like it could be quite a useful tool. I haven’t yet had time to dig in and get familiar with the details, but what I have seen so far looks good.
  • Uh oh…more hardware exploits.
  • The macOS OCSP fiasco generated quite a bit of attention and analysis (see here and here).

Cloud Computing/Cloud Management

Moving Quicksilver into production

Moving Quicksilver into production

One of the great arts of software engineering is making updates and improvements to working systems without taking them offline. For some systems this can be rather easy, spin up a new web server or load balancer, redirect traffic and you’re done. For other systems, such as the core distributed data store which keeps millions of websites online, it’s a bit more of a challenge.

Quicksilver is the data store responsible for storing and distributing the billions of KV pairs used to configure the millions of sites and Internet services which use Cloudflare. In a previous post, we discussed why it was built and what it was replacing. Building it, however, was only a small part of the challenge. We needed to deploy it to production into a network which was designed to be fault tolerant and in which downtime was unacceptable.

We needed a way to deploy our new service seamlessly, and to roll back that deploy should something go wrong. Ultimately many, many, things did go wrong, and every bit of failure tolerance put into the system proved to be worth its weight in gold because none of this was visible to customers.

The Bridge

Our goal Continue reading

Measuring Internet Resilience in Africa

For many in the African region, Internet interruptions or service degradations occur frequently, which results in a disjointed Internet experience. In order to help improve this experience, we need to track and measure various Internet characteristics through network telemetry. This data can help to identify infrastructure and traffic issues and can provide key information to help decision makers decide where infrastructure investment and policy change might need to be made.

However, as shown by a survey carried out by AFRINIC, the Regional Internet Registry (RIR) for Africa, in 2019, Internet measurement is not a common practice in the African region. This is largely due to a scarcity of deployed measurement tools, platforms and equipment, a lack of awareness in the subject, and the lack of relevant skills to carry out the measurement tasks. The shortage of measurement equipment in African countries makes it very challenging to accurately determine the problem areas that need to be addressed in order to improve Internet reliability and resilience in Africa.

Collaborative Measurements 

Assessing the resilience of the Internet is a key component of the Internet Society’s work on Measuring the Internet. To help identify the causes of Internet interruptions and service degradations, we Continue reading

Growing Beyond Networking Skills

One of my subscribers trying to figure out how to improve his career choices sent me this question:

I am Sr. Network Engineer with 12+ Years’ experience. I was quit happy with my networking skills but will all the recent changes I’m confused. I am not able to understand what are the key skills I should learn as a network engineer to keep myself demandable.

Before reading the rest of this blog post, please read Cloud and the Three IT Geographies by Massimo Re Ferre.

Growing Beyond Networking Skills

One of my subscribers trying to figure out how to improve his career choices sent me this question:

I am Sr. Network Engineer with 12+ Years’ experience. I was quit happy with my networking skills but will all the recent changes I’m confused. I am not able to understand what are the key skills I should learn as a network engineer to keep myself demandable.

Before reading the rest of this blog post, please read Cloud and the Three IT Geographies by Massimo Re Ferre.

BPF: The future of configs

BPF has some wow-presentations, showing how it enables new performance measuring and tracing. Brendan Gregg has a whole bunch, for example. But I don’t think’s it’s very well explained just why BPF is such a big deal.

Most of the demos are essentially cool and useful looking tools, with an “oh by the way BPF made this happen”. Similar to how it’s common to see announcements about some software, where the very title of the announcement ends with “written in Go”. It gives a vibe of “so what?”.

If you’re interested in system tooling and configuration, and aren’t already aware of BPF, then this is for you.

I’m not an expert on BPF, but this will hopefully help someone else bootstrap faster.

bpftrace

bpftrace is really cool. Clearly it’s inspired by dtrace. But one should not mistake bpftrace for BPF. bpftrace is only yet another tool that uses BPF, albeit one that allows you to create trace points in a domain specific language.

This is not the full power of BPF. It’s not at all the big picture.

BPF and configs

Let’s take packet filtering as an example. Once upon a time in Linux there was ipfwadm. I Continue reading

Facts about backup security that should scare you to death

Least privilege—the idea that each person in your organization should have the least number of privileges they need in order to accomplish a given task—is an important security concept that needs to be implemented in your backup system.The challenge here is that network, system, and backup admins all wield an incredible amount of power. If one of them makes a mistake, or worse, intentionally tries to do the company harm, limiting the amount of power they have reduces the amount of damage they can inflict.For example, you might give one network administrator the ability to monitor networks, and another one the ability to create and/or reconfigure networks. Security admins might be responsible for creating and maintaining network-administration users without getting any of those privileges themselves.To read this article in full, please click here

Facts about backup security that should scare you to death

Least privilege—the idea that each person in your organization should have the least number of privileges they need in order to accomplish a given task—is an important security concept that needs to be implemented in your backup system.The challenge here is that network, system, and backup admins all wield an incredible amount of power. If one of them makes a mistake, or worse, intentionally tries to do the company harm, limiting the amount of power they have reduces the amount of damage they can inflict.For example, you might give one network administrator the ability to monitor networks, and another one the ability to create and/or reconfigure networks. Security admins might be responsible for creating and maintaining network-administration users without getting any of those privileges themselves.To read this article in full, please click here

Pure Expands Its As-A-Service Playbook

The push by established datacenter tech vendors to get into the as-a-service game has accelerated in recent months, fueled in part by the COVID-19 pandemic and the need by organizations to more quickly embrace cloud services to help them adapt to the suddenly shifted business model that features a more widely distributed workforce, which brings a truckload of security and management issues.

Pure Expands Its As-A-Service Playbook was written by Jeffrey Burt at The Next Platform.

IETF 109

The latest IETF meeting was held in mid-November. Here I’m going to pick just one presentation from each of a small collection of the week’s working group meetings and explore that topic in a little more detail.

The History of EARN, RARE, and European Networks (part 1)

European networks from the mid-1980’s to the late 2000’s underwent a lot of change, bolstered by the rise and fall of America Online, the laying of a lot of subsea cables, and the creation of several organizations, including EARN and RARE, to bolster the spread and use of the Internet. Daniele Bovio joins Donald Sharp and Russ White on this episode of the History of Networking to give us a good overall perspective of this history.

You can find more information about the history of EARN at https://earn-history.net.

download

Sponsored Post: IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna

Who's Hiring? 

  • InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

  • IP2Location is IP address geolocation service provider since 2002. The geolocation database or API detects location, proxy and other >20 parameters. The technology has been cited in more than 700 research papers and trusted by many Fortune 500 companies. Try it today!

  • ipdata is a reliable IP Address Geolocation API that allows you to lookup the approximate location of any IP Address, detect proxies and identify a company from an IP Address. Trusted by 10,000+ developers. Try it now!

  • Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how Continue reading

New LibSSH Connection Plugin for Ansible Network Replaces Paramiko, Adds FIPS Mode Enablement

As Red Hat Ansible Automation Platform expands its footprint with a growing customer base, security continues to be an important aspect of organizations’ overall strategy. Red Hat regularly reviews and enhances the foundational codebase to follow better security practices. As part of this effort, we are introducing FIPS 140-2 readiness enablement by means of a newly developed Ansible SSH connection plugin that uses the libssh library. 

 

Ansible Network SSH Connection Basics

Since most network appliances don't support or have limited capability for the local execution of a third party software, the Ansible network modules are not copied to the remote host unlike linux hosts; instead, they run on the control node itself. Hence, Ansible network can’t use the typical Ansible SSH connection plugin that is used with linux host. Furthermore, due to this behavior, performance of the underlying SSH subsystem is critical. Not only is the new LibSSH connection plugin enabling FIPS readiness, but it was also designed to be more performant than the existing Paramiko SSH subsystem.

Screen Shot 2020-11-20 at 8.52.53 AM

The top level network_cli connection plugin, provided by the ansible.netcommon Collection (specifically ansible.netcommon.network_cli), provides an SSH based connection to the network appliance. It in turn calls the Continue reading

Leading from the Front: How the Internet Society’s Training Efforts Are Helping to Upskill Africa’s Future Digital Champions

Tech skills are important for digital transformation in Africa. To realize this transformation, the Internet Society is supporting work-ready digital skills development with local talent.

By 2030, over 230 million jobs in Sub-Saharan Africa will require digital skills. With the fourth industrial revolution taking shape, we are helping to build a community of digital champions in Africa who will fill the skills gap on the continent.

African governments are already banking big on the contributions that digital technologies will provide to this transformation. For example, the African Union’s Digital Transformation Strategy (2020-2030) wants to create a “Digital Single Market” for the continent, while national governments are increasingly embracing digitalization in several sectors of their economies.

But the continent needs the right talent for its ambitious economic agenda to succeed. Though more learners are taking interest in STEM (science, technology, engineering, and mathematics) subjects, school curricula in Africa primarily focus on theoretical learning with scarce engagement with digital subjects. This is one of the main reasons a skills gap exists for jobs that require digital skills.

To play a role in positively changing these circumstances, the Internet Society developed the Introduction to Network Operations course, which equips novice and intermediate level Continue reading