The Serverlist Newsletter: Serverless Benchmarks, Workers.dev, security implications of serverless cloud computing, and more

Check out our third edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.

Writing an API at the Edge with Workers and Cloud Firestore

We’re super stoked about bringing you Workers.dev, and we’re even more stoked at every opportunity we have to dogfood Workers. Using what we create keeps us tuned in to the developer experience, which takes a good deal of guesswork out of drawing our roadmaps.

Our goal with Workers.dev is to provide a way to deploy JavaScript code to our network of 165 data centers without requiring developers to register a domain with Cloudflare first. While we gear up for general availability, we wanted to provide users an opportunity to reserve their favorite subdomain in a fair and consistent way, so we built a system to allow visitors to reserve a subdomain where their Workers will live once Workers.dev is released. This is the story of how we wrote the system backing that submission process.

Requirements

Of course, we always want to use the best tool for the job, so designing the Workers that would back Workers.dev started with an inventory of constraints and user experience expectations:

Constraints

1. We want to limit reservations to one per email address. It’s no fun if someone writes a bot to claim every good Workers subdomain in ten seconds; they Continue reading

Spectrum for UDP: DDoS protection and firewalling for unreliable protocols

Today, we're announcing Spectrum for UDP. Spectrum for UDP works the same as Spectrum for TCP: Spectrum sits between your clients and your origin. Incoming connections are proxied through, whilst applying our DDoS protection and IP Firewall rules. This allows you to protect your services from all sorts of nasty attacks and completely hides your origin behind Cloudflare.

Last year, we launched Spectrum. Spectrum brought the power of our DDoS and firewall features to all TCP ports and services. Spectrum for TCP allows you to protect your SSH services, gaming protocols, and as of last month, even FTP servers. We’ve seen customers running all sorts of applications behind Spectrum, such as Bitfly, Nicehash, and Hypixel.

This is great if you're running TCP services, but plenty of our customers also have workloads running over UDP. As an example, many multiplayer games prefer the low cost and lighter weight of UDP and don't care about whether packets arrive or not.

UDP applications have historically been hard to protect and secure, which is why we built Spectrum for UDP. Spectrum for UDP allows you to protect standard UDP services (such as RDP over UDP), but can also protect any custom protocol Continue reading

Preventing Request Loops Using CDN-Loop

HTTP requests typically originate with a client, and end at a web server that processes the request and returns some response. Such requests may pass through multiple proxies before they arrive at the requested resource. If one of these proxies is configured badly (for instance, back to a proxy that had already processed it) then the request may be caught in a loop.

Request loops, accidental or malicious, can consume resources and degrade user's Internet performance. Such loops can even be observed at the CDN-level. Such a wide-scale attack would affect all customers of that CDN. It's been over three years since Cloudflare acknowledged the power of such non-compliant or malicious request loops. The proposed solution in that blog post was quickly found to be flawed and loop protection has since been implemented in an ad-hoc manner that is specific to each individual provider. This lack of cohesion and co-operation has led to a fragmented set of protection mechanisms.

We are finally happy to report that a recent collaboration between multiple CDN providers (including Cloudflare) has led to a new mechanism for loop protection. This now runs at the Cloudflare edge and is compliant with other CDNs, allowing us to Continue reading

Monsters in the Middleboxes: Introducing Two New Tools for Detecting HTTPS Interception

The practice of HTTPS interception continues to be commonplace on the Internet. HTTPS interception has encountered scrutiny, most notably in the 2017 study “The Security Impact of HTTPS Interception” and the United States Computer Emergency Readiness Team (US-CERT)  warning that the technique weakens security. In this blog post, we provide a brief recap of HTTPS interception and introduce two new tools:

1. MITMEngine, an open-source library for HTTPS interception detection, and
2. MALCOLM, a dashboard displaying metrics about HTTPS interception we observe on Cloudflare’s network.

In a basic HTTPS connection, a browser (client) establishes a TLS connection directly to an origin server to send requests and download content. However, many connections on the Internet are not directly from a browser to the server serving the website, but instead traverse through some type of proxy or middlebox (a “monster-in-the-middle” or MITM). There are many reasons for this behavior, both malicious and benign.

Types of HTTPS Interception, as Demonstrated by Various Monsters in the Middle

One common HTTPS interceptor is TLS-terminating forward proxies. (These are a subset of all forward proxies; non-TLS-terminating forward proxies forward TLS connections without any ability to inspect encrypted traffic). A TLS-terminating forward proxy sits Continue reading

RFC8482 – Saying goodbye to ANY

Ladies and gentlemen, I would like you to welcome the new shiny RFC8482, which effectively deprecates the DNS ANY query type. DNS ANY was a "meta-query" - think of it as a similar thing to the common A, AAAA, MX or SRV query types, but unlike these it wasn't a real query type - it was special. Unlike the standard query types, ANY didn't age well. It was hard to implement on modern DNS servers, the semantics were poorly understood by the community and it unnecessarily exposed the DNS protocol to abuse. RFC8482 allows us to clean it up - it's a good thing.

But let's rewind a bit.

Historical context

It all started in 2015, when we were looking at the code of our authoritative DNS server. The code flow was generally fine, but it was all peppered with naughty statements like this:

if qtype == "ANY" {
    // special case
}

This special code was ugly and error prone. This got us thinking: do we really need it? "ANY" is not a popular query type - no legitimate software uses it (with the notable exception of qmail).

ANY is Continue reading

Unit Testing Worker Functions

If you were not aware, Cloudflare Workers lets you run Javascript in all 165+ of our Data Centers. We’re delighted to see some of the creative applications of Workers. As the use cases grow in complexity, the need to sanity check your code also grows.  

More specifically, if your Worker includes a number of functions, it’s important to ensure each function does what it’s intended to do in addition to ensuring the output of the entire Worker returns as expected.

In this post, we’re going to demonstrate how to unit test Cloudflare Workers, and their individual functions, with Cloudworker, created by the Dollar Shave Club engineering team.

Dollar Shave Club is a Cloudflare customer, and they created Cloudworker, a mock for the Workers runtime, for testing purposes. We’re really grateful to them for this. They were kind enough to post on our blog about it.

This post will demonstrate how to abstract away Cloudworker, and test Workers with the same syntax you write them in.

Example Script

Before we get into configuring Cloudworker, let’s introduce the simple script we are going to test against in our example. As you can see this script contains two functions, both of Continue reading

Reflecting on my first year as Head of Cloudflare Asia

One year into my role as Head of Asia for Cloudflare, I wanted to reflect on what we’ve achieved, as well as where we are going next.

When I started, I spoke about growing our brand recognition in Asia and optimizing our reach to clients by building up teams and channel partners. I also mentioned a key reason behind my joining was Cloudflare’s mission to help build a better Internet and focus on democratizing Internet tools that were once only available to large companies. I’m delighted to share that we’ve made great progress and are in a strong position to continue our rapid growth. It’s been a wonderful year, and I’m thrilled that I joined the company.

There has been a lot going on in our business, as well as in the region. Let’s start with Cloudflare Asia.

Cloudflare Asia

Our Singapore team has swelled from 40 people from 11 countries to almost 100 people from 19 nations. Our team is as diverse as our client base and keeps the office lively and innovative.

Our Customers

The number of Asian businesses choosing to work with us has more than doubled. You can check out what Continue reading

Employee resource groups aren’t the answer, but they’re a first step

Why employee resource groups are important for building a great company culture but they're not enough.

Diversity and inclusion is a process. To achieve diversity and inclusion, it’s not enough to hire diverse candidates. Once hired, we must be welcomed by a safe and belonging culture, and our diverse perspectives must be honored by our coworkers.

Too many times we are approached by well-meaning companies eager to hire diverse candidates, only to look behind the curtain and discover a company culture where we will not feel safe to be ourselves, and where our perspectives will be ignored. Why would we choose to stay in such an environment? These are the companies where diverse employees leave just as quickly as they join.

Employee Resource Groups (ERGs) are an essential part of diversity and inclusion, especially as companies grow larger. Before being heard, or trying to change someone's mind, you need to feel safe.

ERGs serve as a safe haven for those with perspectives and experiences that are "diverse" compared to the company as a whole. They are a place to share stories, particular plights, and are a source of stress relief. A place where we can safely show up fully as Continue reading

Happy Birthday to the World Wide Web!

Today, March 12th 2019, marks the 30th birthday of the World Wide Web! Cloudflare is helping to celebrate in coordination with the Web Foundation, as part of a 30 hour commemoration of the many ways in which the Web has changed our lives. As we post this blog, Sir Tim Berners Lee is kicking off his journey of the web at CERN, where he wrote the first web browser.

The Web Foundation (@webfoundation) is organizing a Twitter timeline of the web, where each hour corresponds to a year starting now with 1989 at 00:00PT/ 08:00 CET. We (@cloudflare) will be tweeting out milestones in our history and the web’s history, as well as some fun infographics. We hope you will follow the journey on Twitter and contribute your own memories and thoughts to the timeline by tweeting and using #Web30 #ForTheWeb. Celebrate with us and support the Web!

A Node to Workers Story

Node.js allows developers to build web services with JavaScript. However, you're on your own when it comes to registering a domain, setting up DNS, managing the server processes, and setting up builds.

There's no reason to manage all these layers on separate platforms. For a site on Cloudflare, these layers can be on a single platform. Serverless technology simplifies developers' lives and reframes our current definition of backend.

In this article I will breeze through a simple example of how converting a former Node server into a Worker untangled a part of my teams’ code base. The conversion to Workers for this example can be found at this PR on Github.

Background

Cloudflare Marketplace hosts a variety of apps, most of which are produced by third party developers, but some are produced by Cloudflare employees.

The Spotify app is one of those apps that was written by the Cloudflare apps team. This app requires an OAuth flow with Spotify to retrieve the user’s token and gather the playlist, artists, other Spotify profile specific information. While Cloudflare manages the OAuth authentication portion, the app owner - in this case Cloudflare Apps - manages the small integration service that uses the Continue reading

Diving into Technical SEO using Cloudflare Workers

This is a guest post by Igor Krestov and Dan Taylor. Igor is a lead software developer at SALT.agency, and Dan a lead technical SEO consultant, and has also been credited with coining the term “edge SEO”. SALT.agency is a technical SEO agency with offices in London, Leeds, and Boston, offering bespoke consultancy to brands around the world. You can reach them both via Twitter.

With this post we illustrate the potential applications of Cloudflare Workers in relation to search engine optimization, which is more commonly referred to as ‘SEO’ using our research and testing over the past year making Sloth.

This post is aimed at readers who are both proficient in writing performant JavaScript, as well as complete newcomers, and less technical stakeholders, who haven’t really written many lines of code before.

Endless practical applications to overcome obstacles

Working with various clients and projects over the years we’ve continuously encountered the same problems and obstacles in getting their websites to a point of “technical SEO excellence”. A lot of these problems come from platform restriction at an enterprise level, legacy tech stacks, incorrect builds, and years of patching together various services and infrastructures.

As a team of Continue reading

Stopping Drupal’s SA-CORE-2019-003 Vulnerability

On the 20th February 2019, Drupal announced that they had discovered a severe vulnerability and that they would be releasing a patch for it the next day. Drupal is a Content Management System used by many of our customers, which made it important that our WAF protect against the vulnerability as quickly as possible.

As soon as Drupal released their patch, we analysed it to establish what kind of payloads could be used against it and created rules to mitigate these. By analysing the patch we were able to put together WAF rules to protect cloudflare customers running Drupal.

We identified the type of vulnerability we were dealing within 15 minutes. From here, we were able to deploy rules to block the exploit well before any real attacks were seen.

The exploit

As Drupal's release announcement explains, a site is affected if:

• It has the Drupal 8 RESTful API enabled                                      
• Or it uses one of the 8 modules found to be affected

From looking at the patch we very quickly realised the exploit would be based on deserialization. The option ['allowed_classes' Continue reading

Building fast interpreters in Rust

In the previous post we described the Firewall Rules architecture and how the different components are integrated together. We also mentioned that we created a configurable Rust library for writing and executing Wireshark®-like filters in different parts of our stack written in Go, Lua, C, C++ and JavaScript Workers.

With a mixed set of requirements of performance, memory safety, low memory use, and the capability to be part of other products that we’re working on like Spectrum, Rust stood out as the strongest option.

We have now open-sourced this library under our Github account: https://github.com/cloudflare/wirefilter. This post will dive into its design, explain why we didn’t use a parser generator and how our execution engine balances security, runtime performance and compilation cost for the generated filters.

Parsing Wireshark syntax

When building a custom Domain Specific Language (DSL), the first thing we need to be able to do is parse it. This should result in an intermediate representation (usually called an Abstract Syntax Tree) that can be inspected, traversed, analysed and, potentially, serialised.

There are different ways to perform such conversion, such as:

1. Manual char-by-char parsing using state machines, regular expression and/or native string APIs.
2. Parser combinators, which use Continue reading

How we made Firewall Rules

Recently we launched Firewall Rules, a new feature that allows you to construct expressions that perform complex matching against HTTP requests and then choose how that traffic is handled. As a Firewall feature you can, of course, block traffic. The expressions we support within Firewall Rules along with powerful control over the order in which they are applied allows complex new behaviour.

In this blog post I tell the story of Cloudflare’s Page Rules mechanism and how Firewall Rules came to be. Along the way I’ll look at the technical choices that led to us building the new matching engine in Rust.

The evolution of the Cloudflare Firewall

Cloudflare offers two types of firewall for web applications, a managed firewall in the form of a WAF where we write and maintain the rules for you, and a configurable firewall where you write and maintain rules. In this article, we will focus on the configurable firewall.

One of the earliest Cloudflare firewall features was the IP Access Rule. It dates backs to the earliest versions of the Cloudflare Firewall and simply allows you to block traffic from specific IP addresses:

if request IP equals 203.0.113.1 then block  Continue reading

Deploying Workers with GitHub Actions + Serverless

If you weren’t aware, Cloudflare Workers, our serverless programming platform, allows you to deploy code onto our 165 data centers around the world.

Want to automatically deploy Workers directly from a GitHub repository? Now you can with our official GitHub Action. This Action is an extension of our existing integration with the Serverless Framework. It runs in a containerized GitHub environment and automatically deploys your Worker to Cloudflare. We chose to utilize the Serverless Framework within our GitHub Action to raise awareness of their awesome work and to enable even more serverless applications to be built with Cloudflare Workers. This Action can be used to deploy individual Worker scripts as well; the Serverless Framework is being used in the background as the deployment mechanism.

Before going into the details, we’ll quickly go over what GitHub Actions are.

GitHub Actions

GitHub Actions allow you to trigger commands in reaction to GitHub events. These commands run in containers and can receive environment variables. Actions could trigger build, test, or deployment commands across a variety of providers. They can also be linked and run sequentially (i.e. ‘if the build passes, deploy the app’). Similar to many CI/CD tools, these commands run Continue reading

New Firewall Tab and Analytics

At Cloudflare, one of our top priorities is to make our products and services intuitive so that we can enable customers to accelerate and protect their Internet properties. We're excited to launch two improvements designed to make our Firewall easier to use and more accessible, and helping our customers better manage and visualize their threat-related data.

New Firewall Tabs for ease of access

We have re-organised our features into meaningful pages: Events, Firewall Rules, Managed Rules, Tools, and Settings. Our customers will see an Overview tab, which contains our new Firewall Analytics, detailed below.

All the features you know and love are still available, and can be found in one of the four new tabs. Here is a breakdown of their new locations.

If the new sub navigation has not appeared, you may need to re-login to the dashboard or clear your browser’s cookies.

New Firewall Analytics for analysing events and Continue reading

Digital Evidence Across Borders and Engagement with Non-U.S. Authorities

Since we first started reporting in 2013, our transparency report has focused on requests from U.S. law enforcement. Previous versions of the report noted that, as a U.S. company, we ask non-U.S. law enforcement agencies to obtain formal U.S. legal process before providing customer data.

As more countries pass laws that seek to extend beyond their national borders and as we expand into new markets, the question of how to handle requests from non-U.S. law enforcement has become more complicated. It seems timely to talk about our engagement with non-U.S. law enforcement and how our practice is changing. But first, some background on the changes that we’ve seen over the last year.

Law enforcement access to data across borders

The explosion of cloud services -- and the fact that data may be stored outside the countries of residence of those who generated it -- has been a challenge for governments conducting law enforcement investigations. A number of U.S. laws, like the Stored Communications Act or the Electronic Communications Privacy Act restrict companies from providing particular types of data, such as the content of communications, to any person or entity, including foreign law enforcement Continue reading

Out of the Clouds and into the weeds: Cloudflare’s approach to abuse in new products

In a blogpost yesterday, we addressed the principles we rely upon when faced with numerous and various requests to address the content of websites that use our services. We believe the building blocks that we provide for other people to share and access content online should be provided in a content-neutral way. We also believe that our users should understand the policies we have in place to address complaints and law enforcement requests, the type of requests we receive, and the way we respond to those requests. In this post, we do the dirty work of addressing how those principles are put into action, specifically with regard to Cloudflare’s expanding set of features and products.

Abuse reports and new products

Currently, we receive abuse reports and law enforcement requests on fewer than one percent of the more than thirteen million domains that use Cloudflare’s network. Although the reports we receive run the gamut -- from phishing, malware or other technical abuses of our network to complaints about content -- the overwhelming majority are allegations of copyright violations copyright or violations of other intellectual property rights. Most of the complaints that we receive do not identify concerns with particular Cloudflare services Continue reading

The Serverlist Newsletter 2nd Edition: Available Now

Check out our second edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.