RFC8482 – Saying goodbye to ANY

Ladies and gentlemen, I would like you to welcome the new shiny RFC8482, which effectively deprecates the DNS ANY query type. DNS ANY was a "meta-query" - think of it as a similar thing to the common A, AAAA, MX or SRV query types, but unlike these it wasn't a real query type - it was special. Unlike the standard query types, ANY didn't age well. It was hard to implement on modern DNS servers, the semantics were poorly understood by the community and it unnecessarily exposed the DNS protocol to abuse. RFC8482 allows us to clean it up - it's a good thing.

But let's rewind a bit.

Historical context

It all started in 2015, when we were looking at the code of our authoritative DNS server. The code flow was generally fine, but it was all peppered with naughty statements like this:

if qtype == "ANY" {
    // special case
}

This special code was ugly and error prone. This got us thinking: do we really need it? "ANY" is not a popular query type - no legitimate software uses it (with the notable exception of qmail).

ANY is Continue reading

Unit Testing Worker Functions

If you were not aware, Cloudflare Workers lets you run Javascript in all 165+ of our Data Centers. We’re delighted to see some of the creative applications of Workers. As the use cases grow in complexity, the need to sanity check your code also grows.  

More specifically, if your Worker includes a number of functions, it’s important to ensure each function does what it’s intended to do in addition to ensuring the output of the entire Worker returns as expected.

In this post, we’re going to demonstrate how to unit test Cloudflare Workers, and their individual functions, with Cloudworker, created by the Dollar Shave Club engineering team.

Dollar Shave Club is a Cloudflare customer, and they created Cloudworker, a mock for the Workers runtime, for testing purposes. We’re really grateful to them for this. They were kind enough to post on our blog about it.

This post will demonstrate how to abstract away Cloudworker, and test Workers with the same syntax you write them in.

Example Script

Before we get into configuring Cloudworker, let’s introduce the simple script we are going to test against in our example. As you can see this script contains two functions, both of Continue reading

Reflecting on my first year as Head of Cloudflare Asia

One year into my role as Head of Asia for Cloudflare, I wanted to reflect on what we’ve achieved, as well as where we are going next.

When I started, I spoke about growing our brand recognition in Asia and optimizing our reach to clients by building up teams and channel partners. I also mentioned a key reason behind my joining was Cloudflare’s mission to help build a better Internet and focus on democratizing Internet tools that were once only available to large companies. I’m delighted to share that we’ve made great progress and are in a strong position to continue our rapid growth. It’s been a wonderful year, and I’m thrilled that I joined the company.

There has been a lot going on in our business, as well as in the region. Let’s start with Cloudflare Asia.

Cloudflare Asia

Our Singapore team has swelled from 40 people from 11 countries to almost 100 people from 19 nations. Our team is as diverse as our client base and keeps the office lively and innovative.

Our Customers

The number of Asian businesses choosing to work with us has more than doubled. You can check out what Continue reading

Employee resource groups aren’t the answer, but they’re a first step

Why employee resource groups are important for building a great company culture but they're not enough.

Diversity and inclusion is a process. To achieve diversity and inclusion, it’s not enough to hire diverse candidates. Once hired, we must be welcomed by a safe and belonging culture, and our diverse perspectives must be honored by our coworkers.

Too many times we are approached by well-meaning companies eager to hire diverse candidates, only to look behind the curtain and discover a company culture where we will not feel safe to be ourselves, and where our perspectives will be ignored. Why would we choose to stay in such an environment? These are the companies where diverse employees leave just as quickly as they join.

Employee Resource Groups (ERGs) are an essential part of diversity and inclusion, especially as companies grow larger. Before being heard, or trying to change someone's mind, you need to feel safe.

ERGs serve as a safe haven for those with perspectives and experiences that are "diverse" compared to the company as a whole. They are a place to share stories, particular plights, and are a source of stress relief. A place where we can safely show up fully as Continue reading

Happy Birthday to the World Wide Web!

Today, March 12th 2019, marks the 30th birthday of the World Wide Web! Cloudflare is helping to celebrate in coordination with the Web Foundation, as part of a 30 hour commemoration of the many ways in which the Web has changed our lives. As we post this blog, Sir Tim Berners Lee is kicking off his journey of the web at CERN, where he wrote the first web browser.

The Web Foundation (@webfoundation) is organizing a Twitter timeline of the web, where each hour corresponds to a year starting now with 1989 at 00:00PT/ 08:00 CET. We (@cloudflare) will be tweeting out milestones in our history and the web’s history, as well as some fun infographics. We hope you will follow the journey on Twitter and contribute your own memories and thoughts to the timeline by tweeting and using #Web30 #ForTheWeb. Celebrate with us and support the Web!

A Node to Workers Story

Node.js allows developers to build web services with JavaScript. However, you're on your own when it comes to registering a domain, setting up DNS, managing the server processes, and setting up builds.

There's no reason to manage all these layers on separate platforms. For a site on Cloudflare, these layers can be on a single platform. Serverless technology simplifies developers' lives and reframes our current definition of backend.

In this article I will breeze through a simple example of how converting a former Node server into a Worker untangled a part of my teams’ code base. The conversion to Workers for this example can be found at this PR on Github.

Background

Cloudflare Marketplace hosts a variety of apps, most of which are produced by third party developers, but some are produced by Cloudflare employees.

The Spotify app is one of those apps that was written by the Cloudflare apps team. This app requires an OAuth flow with Spotify to retrieve the user’s token and gather the playlist, artists, other Spotify profile specific information. While Cloudflare manages the OAuth authentication portion, the app owner - in this case Cloudflare Apps - manages the small integration service that uses the Continue reading

Diving into Technical SEO using Cloudflare Workers

This is a guest post by Igor Krestov and Dan Taylor. Igor is a lead software developer at SALT.agency, and Dan a lead technical SEO consultant, and has also been credited with coining the term “edge SEO”. SALT.agency is a technical SEO agency with offices in London, Leeds, and Boston, offering bespoke consultancy to brands around the world. You can reach them both via Twitter.

With this post we illustrate the potential applications of Cloudflare Workers in relation to search engine optimization, which is more commonly referred to as ‘SEO’ using our research and testing over the past year making Sloth.

This post is aimed at readers who are both proficient in writing performant JavaScript, as well as complete newcomers, and less technical stakeholders, who haven’t really written many lines of code before.

Endless practical applications to overcome obstacles

Working with various clients and projects over the years we’ve continuously encountered the same problems and obstacles in getting their websites to a point of “technical SEO excellence”. A lot of these problems come from platform restriction at an enterprise level, legacy tech stacks, incorrect builds, and years of patching together various services and infrastructures.

As a team of Continue reading

Stopping Drupal’s SA-CORE-2019-003 Vulnerability

On the 20th February 2019, Drupal announced that they had discovered a severe vulnerability and that they would be releasing a patch for it the next day. Drupal is a Content Management System used by many of our customers, which made it important that our WAF protect against the vulnerability as quickly as possible.

As soon as Drupal released their patch, we analysed it to establish what kind of payloads could be used against it and created rules to mitigate these. By analysing the patch we were able to put together WAF rules to protect cloudflare customers running Drupal.

We identified the type of vulnerability we were dealing within 15 minutes. From here, we were able to deploy rules to block the exploit well before any real attacks were seen.

The exploit

As Drupal's release announcement explains, a site is affected if:

• It has the Drupal 8 RESTful API enabled                                      
• Or it uses one of the 8 modules found to be affected

From looking at the patch we very quickly realised the exploit would be based on deserialization. The option ['allowed_classes' Continue reading

Building fast interpreters in Rust

In the previous post we described the Firewall Rules architecture and how the different components are integrated together. We also mentioned that we created a configurable Rust library for writing and executing Wireshark®-like filters in different parts of our stack written in Go, Lua, C, C++ and JavaScript Workers.

With a mixed set of requirements of performance, memory safety, low memory use, and the capability to be part of other products that we’re working on like Spectrum, Rust stood out as the strongest option.

We have now open-sourced this library under our Github account: https://github.com/cloudflare/wirefilter. This post will dive into its design, explain why we didn’t use a parser generator and how our execution engine balances security, runtime performance and compilation cost for the generated filters.

Parsing Wireshark syntax

When building a custom Domain Specific Language (DSL), the first thing we need to be able to do is parse it. This should result in an intermediate representation (usually called an Abstract Syntax Tree) that can be inspected, traversed, analysed and, potentially, serialised.

There are different ways to perform such conversion, such as:

1. Manual char-by-char parsing using state machines, regular expression and/or native string APIs.
2. Parser combinators, which use Continue reading

How we made Firewall Rules

Recently we launched Firewall Rules, a new feature that allows you to construct expressions that perform complex matching against HTTP requests and then choose how that traffic is handled. As a Firewall feature you can, of course, block traffic. The expressions we support within Firewall Rules along with powerful control over the order in which they are applied allows complex new behaviour.

In this blog post I tell the story of Cloudflare’s Page Rules mechanism and how Firewall Rules came to be. Along the way I’ll look at the technical choices that led to us building the new matching engine in Rust.

The evolution of the Cloudflare Firewall

Cloudflare offers two types of firewall for web applications, a managed firewall in the form of a WAF where we write and maintain the rules for you, and a configurable firewall where you write and maintain rules. In this article, we will focus on the configurable firewall.

One of the earliest Cloudflare firewall features was the IP Access Rule. It dates backs to the earliest versions of the Cloudflare Firewall and simply allows you to block traffic from specific IP addresses:

if request IP equals 203.0.113.1 then block  Continue reading

Deploying Workers with GitHub Actions + Serverless

If you weren’t aware, Cloudflare Workers, our serverless programming platform, allows you to deploy code onto our 165 data centers around the world.

Want to automatically deploy Workers directly from a GitHub repository? Now you can with our official GitHub Action. This Action is an extension of our existing integration with the Serverless Framework. It runs in a containerized GitHub environment and automatically deploys your Worker to Cloudflare. We chose to utilize the Serverless Framework within our GitHub Action to raise awareness of their awesome work and to enable even more serverless applications to be built with Cloudflare Workers. This Action can be used to deploy individual Worker scripts as well; the Serverless Framework is being used in the background as the deployment mechanism.

Before going into the details, we’ll quickly go over what GitHub Actions are.

GitHub Actions

GitHub Actions allow you to trigger commands in reaction to GitHub events. These commands run in containers and can receive environment variables. Actions could trigger build, test, or deployment commands across a variety of providers. They can also be linked and run sequentially (i.e. ‘if the build passes, deploy the app’). Similar to many CI/CD tools, these commands run Continue reading

New Firewall Tab and Analytics

At Cloudflare, one of our top priorities is to make our products and services intuitive so that we can enable customers to accelerate and protect their Internet properties. We're excited to launch two improvements designed to make our Firewall easier to use and more accessible, and helping our customers better manage and visualize their threat-related data.

New Firewall Tabs for ease of access

We have re-organised our features into meaningful pages: Events, Firewall Rules, Managed Rules, Tools, and Settings. Our customers will see an Overview tab, which contains our new Firewall Analytics, detailed below.

All the features you know and love are still available, and can be found in one of the four new tabs. Here is a breakdown of their new locations.

If the new sub navigation has not appeared, you may need to re-login to the dashboard or clear your browser’s cookies.

New Firewall Analytics for analysing events and Continue reading

Digital Evidence Across Borders and Engagement with Non-U.S. Authorities

Since we first started reporting in 2013, our transparency report has focused on requests from U.S. law enforcement. Previous versions of the report noted that, as a U.S. company, we ask non-U.S. law enforcement agencies to obtain formal U.S. legal process before providing customer data.

As more countries pass laws that seek to extend beyond their national borders and as we expand into new markets, the question of how to handle requests from non-U.S. law enforcement has become more complicated. It seems timely to talk about our engagement with non-U.S. law enforcement and how our practice is changing. But first, some background on the changes that we’ve seen over the last year.

Law enforcement access to data across borders

The explosion of cloud services -- and the fact that data may be stored outside the countries of residence of those who generated it -- has been a challenge for governments conducting law enforcement investigations. A number of U.S. laws, like the Stored Communications Act or the Electronic Communications Privacy Act restrict companies from providing particular types of data, such as the content of communications, to any person or entity, including foreign law enforcement Continue reading

Out of the Clouds and into the weeds: Cloudflare’s approach to abuse in new products

In a blogpost yesterday, we addressed the principles we rely upon when faced with numerous and various requests to address the content of websites that use our services. We believe the building blocks that we provide for other people to share and access content online should be provided in a content-neutral way. We also believe that our users should understand the policies we have in place to address complaints and law enforcement requests, the type of requests we receive, and the way we respond to those requests. In this post, we do the dirty work of addressing how those principles are put into action, specifically with regard to Cloudflare’s expanding set of features and products.

Abuse reports and new products

Currently, we receive abuse reports and law enforcement requests on fewer than one percent of the more than thirteen million domains that use Cloudflare’s network. Although the reports we receive run the gamut -- from phishing, malware or other technical abuses of our network to complaints about content -- the overwhelming majority are allegations of copyright violations copyright or violations of other intellectual property rights. Most of the complaints that we receive do not identify concerns with particular Cloudflare services Continue reading

The Serverlist Newsletter 2nd Edition: Available Now

Check out our second edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.

Unpacking the Stack and Addressing Complaints about Content

Although we are focused on protecting and optimizing the operation of the Internet, Cloudflare is sometimes the target of complaints or criticism about the content of a very small percentage of the more than thirteen million websites that use our service. Our termination of services to the Daily Stormer website a year and a half ago drew significant attention to our approach to these issues and prompted a lot of thinking on our part.  

At the time, Matthew wrote that calls for service providers to reject some online content should start with a consideration of how the Internet works and how the services at issue up and down the stack interact with that content. He tasked Cloudflare’s policy team with engaging broadly to try and find an answer. With some time having passed, we want to take stock of what we’ve learned and where we stand in addressing problematic content online.  

The aftermath of the Daily Stormer decision

The weeks immediately following the decision in August 2017 were filled with conversations. Matthew made sure the Cloudflare team accepted every single invitation to talk about these issues; we didn’t simply put out a press release or “no comment” anyone. Continue reading

Cloudflare Signs European Commission Declaration on Gender Balanced Company Culture

Last week Cloudflare attended a roundtable meeting in Brussels convened by the European Commissioner for Digital Economy and Society, Mariya Gabriel, with all signatories of the Tech Leaders’ Declaration on Gender Balanced Company Culture. Cloudflare joined this European Commission initiative late last year and, along with other companies, we are committed to taking a hands-on approach to close the digital gender divide in skills, inception of technologies, access and career opportunities.

In particular, we have all committed to implementing, promoting and spreading five specific actions to achieve equality of opportunities for women in our companies and in the digital sector at large:

1. Instil an inclusive, open, female-friendly company culture
2. Recruit and invest in diversity
3. Give women in tech their voice and visibility
4. Create the leaders of the future
5. Become an advocate for change

The project, spearheaded by the Digital Commissioner as part of a range of actions to promote gender balance in the digital industry, allows for the exchange of ideas and best practices among companies, with opportunities to chart progress and also to discuss the challenges we face. Many companies around the table shared their inspiring stories of steps taken at company level to encourage diversity, push back against Continue reading

Logpush: the Easy Way to Get Your Logs to Your Cloud Storage

Introducing Logpush

Today, we’re excited to announce a new way to get your logs: Logpush, a tool for uploading your logs to your cloud storage provider, such as Amazon S3 or Google Cloud Storage. It’s now available in Early Access for Enterprise domains.

We first explained Cloudflare’s logging functionality almost six years ago. Since then, the number of domains on our network has grown by ten times. We’ve continued giving our Enterprise customers the ability to download logs using a REST API, which has gotten a large number of functional and technical updates. We’ve also been paying attention to how our customers’ needs have evolved, especially as we protect and accelerate increasingly larger domains. This led to the development of Logpush.

The Value of Logs

Cloudflare works by being an intermediary between our customers’ websites, applications, and devices, and their end-users or potential attackers. As part of providing our service, we create a record of each request that goes through our network. These records (or request logs) have detailed information regarding the connecting client, our actions—including whether the request was served by the cache or blocked by our firewall—and the response from the origin web server. For Enterprise customers Continue reading

Cloudflare Transparency Update: Joining Cloudflare’s Flock of (Warrant) Canaries

Today, Cloudflare is releasing its transparency report for the second half of 2018. We have been publishing biannual Transparency Reports since 2013.

We believe an essential part of earning the trust of our customers is being transparent about our features and services, what we do – and do not do – with our users’ data, and generally how we conduct ourselves in our engagement with third parties such as law enforcement authorities.  We also think that an important part of being fully transparent is being rigorously consistent and anticipating future circumstances, so our users not only know how we have behaved in the past, but are able to anticipate with reasonable certainty how we will act in the future, even in difficult cases.

As part of that effort, we have set forth certain ‘warrant canaries’ – statements of things we have never done as a company. As described in greater detail below, the report published today adds three new ‘warrant canaries’, which is the first time we’ve added to that list since 2013. The transparency report is also distinguished because it adds new reporting on requests for user information from foreign law enforcement, and requests for user information that we Continue reading

Cloudflare’s RPKI Toolkit

A few months ago, we made a first then a second announcement about Cloudflare’s involvement in Resource Public Key Infrastructure (RPKI), and our desire to make BGP Internet routing more secure. Our mission is to build a safer Internet. We want to make it easier for network operators to deploy RPKI.

Today’s article is going to cover our experience and the tools we are using. As a brief reminder, RPKI is a framework that allows networks to deploy route filtering using cryptography-validated information. Picture TLS certificates for IP addresses and Autonomous System Numbers (ASNs)

What it means for you:

We validate our IP routes. This means, as a 1.1.1.1 DNS resolver user, you are less likely to be victim of cache poisoning. We signed our IP routes. This means a user browsing the websites on Cloudflare’s network are unlikely to experience route hijacks.

All our Points of Presence which have a router compatible with The Resource Public Key Infrastructure (RPKI) to Router Protocol (RTR protocol) are connected to our custom software called GoRTR and are now filtering invalid routes. The deployment amounts to around 70% of our network.

We received many questions regarding the amount of invalid Continue reading