Athenian Project Turns One: Are Election Websites Safer?

One year ago, Cloudflare launched the Athenian Project to provide free Enterprise-level service to election and voter registration websites run by state and local governments in the United States. Through this project, we have helped over 100 entities in 24 states protect their websites from denial of service attacks, SQL injection, and other malicious efforts aimed at undermining the integrity of their elections. With the end of the year approaching, and the November 6th US midterm elections behind us, we wanted to look back at the project and what we have learned as we move towards 2020.

US Midterm Election Day

The morning of November 6th was full of anticipation for the Athenian Project team with the policy, engineering and support teams ready as polls opened in the East. Early in the day, we were notified by our partner at the CDT that some elections websites were experiencing downtime. Mobilizing to help these groups, we reached out to the website administrators and, through the course of the day, on-boarded over 30 new county-level websites to the Athenian Project and helped them manage the unpredictably large amounts of legitimate traffic.

This last-minute effort would not have been possible without the help Continue reading

Encrypting DNS end-to-end

Over the past few months, we have been running a pilot with Facebook to test the feasibility of securing the connection between 1.1.1.1 and Facebook’s authoritative name servers. Traditionally, the connection between a resolver and an authoritative name server is unencrypted i.e. over UDP.

In this pilot we tested how an encrypted connection using TLS impacts the end-to-end latency between 1.1.1.1 and Facebook’s authoritative name servers. Even though the initial connection adds some latency, the overhead is amortized over many queries. The resulting DNS latency between 1.1.1.1 and Facebook’s authoritative name servers is on par with the average UDP connections.

To learn more about how the pilot went, and to see more detailed results, check out the complete breakdown over on Code, Facebook's Engineering blog.

Firewall Rules – Priority and Ordering

Firewall Rules are one of the best security features we released this year and have been an overwhelming success. Customers have been using Firewall Rules to solve interesting security related use cases; for example, advanced hotlink protection, restricting access to embargoed content (e.g. productId=1234), locking down sensitive API endpoints, and more.

One of the biggest pieces of feedback from the Cloudflare community, Twitter, and via customer support, has been around the order in which rules are actioned. By default, Firewall Rules have a default precedence, based on the actions set on the rule:

If two or more rules match a request, but have different actions, the above precedence will take effect. However, what happens if you've got a bad actor who needs to be blocked from your API, and you have other specific allow or challenge rules already created for their originating ASN or a perhaps one of your URLs? Once a Firewall Rule is matched, it will not continue processing other rule, unless you are using the Log action. Without a method of overriding the default precedence, you cannot easily achieve what's needed.

Today, we’re launching the ability for customers to change the ordering of their rules. Continue reading

Ten new data centers: Cloudflare expands global network to 165 cities

Cloudflare is excited to announce the addition of ten new data centers across the United States, Bahrain, Russia, Vietnam, Pakistan and France (Réunion). We're delighted to help improve the performance and security of over 12 million domains across these diverse countries that collectively represent about half a billion Internet users.

Our global network now spans 165 cities, with 46 new cities added just this year, and several dozen additional locations being actively worked on.

United States of America

Our expansion begins in the United States, where Cloudflare's 36th and 37th data centers in the nation serve Charlotte (North Carolina) and Columbus (Ohio) respectively. They are promising markets for interconnection, and join our existing deployments in Ashburn, Atlanta, Boston, Chicago, Dallas, Denver, Detroit, Houston, Indianapolis, Jacksonville, Kansas City, Las Vegas, Los Angeles, McAllen, Memphis, Miami, Minneapolis, Montgomery, Nashville, Newark, Norfolk, Omaha, Philadelphia, Portland, Richmond, Sacramento, Salt Lake City, San Diego, San Jose, Seattle, St. Louis, Tallahassee, and Tampa.

Bahrain

Cloudflare's Manama (Bahrain) data center, our 158th globally, further expands our Middle East coverage. A growing hub for cloud computing, including public sector adoption (with the Kingdom's "Cloud First" policy), Bahrain is attracting talent and investment in innovative companies.

Russia

Cloudflare's new Continue reading

Banking-Grade Credential Stuffing: The Futility of Partial Password Validation

Recently when logging into one of my credit card providers, I was greeted by a familiar screen. After entering in my username, the service asked me to supply 3 random characters from my password to validate ownership of my account.

It is increasingly common knowledge in the InfoSec community that this practice is the antithesis of, what we now understand to be, secure password management.

For starters; sites prompting you for Partial Password Validation cannot store your passwords securely using algorithms like BCrypt or Argon2. If the service provider is ever breached, such plain-text passwords can be used to login to other sites where the account holder uses the same password (known as a Credential Stuffing attack).

Increased difficulty using long, randomly-generated passwords from Password Managers, leads to users favouring their memory over securely generated unique passwords. Those using Password Managers must extract their password from their vault, paste it somewhere else and then calculate the correct characters to put in. With this increased complexity, it further incentivises users to (re-)use simple passwords they can remember and count off on their fingers (and likely repeatedly use on other sites).

This is not to distinct thinking that originally bought us complex Continue reading

Cloudworker  –  A local Cloudflare Worker Runner

This is a guest post by Hank Jacobs, who is the Lead Software Engineer for Platform Services & Tools at Dollar Shave Club. This post originally appeared on the DSC Engineering blog.

At Dollar Shave Club, we continuously look for ways to improve how we build and ship code. Improving the time it takes for engineers to ship code is key. Providing engineers with a development environment that closely mirrors production really helps.

Earlier this year, we began evaluating Cloudflare Workers as a replacement for our legacy edge routing and caching layers. Cloudflare Workers brings the power of Javascript to Cloudflare’s Edge. Developers can write and deploy Javacript that gets executed for every HTTP request that passes through Cloudflare. This capability excited us but a critical thing was missing — a way to run Worker code locally. We couldn’t find a suitable solution, so we started to build our own. Luckily, Workers uses the open Service Workers API so we had documentation to consult. Within a few weeks, Cloudworker was born.

Cloudworker

Cloudworker is a local Cloudflare Worker runtime. With it, you can run Cloudflare Worker scripts locally (or anywhere you can run a Docker image). Our primary goal with Continue reading

Real World Serverless: Serverless Use Cases and Best Practices

Cloudflare Workers has had a very busy 2018. Throughout the year, Workers moved from beta to general availability, continued to expand its footprint as Cloudflare grew to 155 locations, and added new features and services to help developers create increasingly advanced applications.

To cap off 2018 we decided hit the road (and then head to the airport) with our Real World Serverless event series in San Francisco, Austin, London, Singapore, Sydney, and Melbourne. It was a great time sharing serverless application development insights we’ve discovered over the past year as well as demonstrating how to build applications with new services like our key value store, Cloudflare Workers KV.

Below is a recording from our Singapore Real World Serverless event. It included three talks about Serverless technology featuring Tim Obezuk, Stanley Tan, and Remy Guercio from Cloudflare. They spoke about the fundamentals of serverless technology, twelve factors of serverless application development, and achieving no ops at scale with network-based serverless.

If you’d like to join us in person to talk about serverless, we’ll be announcing 2019 event locations starting in the new year.

About the talks

Fundamentals of Serverless Technology - Tim Obezuk (0:00-13:56)

Tim explores the anatomy of Continue reading

Managing DNS Records For The People With Cloudflare Apps

DNS records are hard.  Many people, even the technically competent, don’t understand more than the basics.

I'll speak for myself — as someone who always learned just enough about DNS to get it working, then immediately forgot everything until the next time it broke. It was a vicious cycle until I convinced myself to learn it in depth.  Meanwhile, non-technical folks wisely avoid meddling in such dangerous affairs all together.

Surely, there must be a better way (this is a blog post after all).

Every day, thousands of Cloudflare users add DNS records to their Internet properties to configure awesome tools like G Suite, Shopify, Wordpress, Ghost, and thousands of others.  A new Cloudflare Apps feature allows apps to automatically set up and manage configurable DNS records on more than 12 million registered domains on the Cloudflare network. In short, Cloudflare Apps are here to alleviate the Internet’s collective DNS woes.

Gone are the days of tribulating over whether it’s A or CNAME you should set.  Gone are the days of puzzling between A and AAAA records while wondering what the heck happened to AA and AAA records?  Unload your DNS dysphoria onto highly trained developers experienced at explicating these Continue reading

Traffic Acceleration with Cloudflare Mobile SDK

We’re excited to announce early access for Traffic Acceleration with Cloudflare Mobile SDK. Acceleration uses novel transport algorithms built into the SDK to accelerate apps beyond the performance they would see with TCP. Enabling Acceleration through the SDK reduces latency, increases throughput, and improves app user experiences.

A year ago, we launched Cloudflare Mobile SDK with a set of free features focused on measuring mobile app networking performance. Apps are dependent on network connectivity to deliver their app’s user experiences, but developers have limited visibility into how network connectivity is impacting app performance. Integrating the Mobile SDK allows developers to measure and improve the speed of their app’s network interactions.

How it works

Mobile applications interact with the Internet to do everything — to fetch the weather, your email, to step through a check out flow. Everything that makes a smartphone magical is powered by a service on the Internet. How quickly those network interactions happen is dictated by two things: how large the payloads are for the given request/response, and what the available link bandwidth is.

Payload size is mostly application specific: a shopping app is going to request product images and similar medium sized assets, while a stock Continue reading

How my team wrote 12 Cloudflare apps with fewer than 20 lines of code

This is a guest post by Ben Ross. Ben is a Berkeley PhD, serial entrepreneur, and Founder and CTO and POWr.io, where he spends his days helping small businesses grow online.

I like my code the same way I like my team of POWr Rangers… DRY.

And no, I don’t mean dull and unexciting! (If you haven’t heard this acronym before, DRY stands for Don’t Repeat Yourself, the single most important principle in software engineering. Because, as a mentor once told me, “when someone needs to re-write your code, at least they only need to do it once.”)

At POWr, being DRY is not just a way to write code, it’s a way of life. This is true whether you’re an Engineer, a Customer Support agent, or an Office Manager; if you find you’re repeating yourself, we want to find a way to automate that repetition away. Our employees’ time is our company’s most valuable resource. Not to mention, who wants to spend all day repeating themselves?

We call this process becoming a Scaled Employee. A Scaled Employee leverages their time and resources to make a multifold impact compared to an average employee in their Continue reading

More consistent LuaJIT performance

This is a guest post by Laurence Tratt, who is a programmer and Reader in Software Development in the Department of Informatics at King's College London where he leads the Software Development Team. He is also an EPSRC Fellow.

A year ago I wrote about a project that Cloudflare were funding at King's College London to help improve LuaJIT. Our twelve months is now up. How did we do?

The first thing that happened is that I was lucky to employ a LuaJIT expert, Thomas Fransham, to work on the project. His deep knowledge about LuaJIT was crucial to getting things up and running – 12 months might sound like a long time, but it soon whizzes by!

The second thing that happened was that we realised that the current state of Lua benchmarking was not good enough for anyone to reliably tell if they'd improved LuaJIT performance or not. Different Lua implementations had different benchmark suites, mostly on the small side, and not easily compared. Although it wasn't part of our original plan, we thus put a lot of effort into creating a larger benchmark suite. This sounds like a trivial job, but it isn't. Many programs make Continue reading

OAuth Auth Server through Workers

Let’s pretend I own a service and I want to grant other services access to my service on behalf of my users. The familiar OAuth 2.0 is the industry standard used by the likes of Google sign in, Facebook, etc. to communicate safely without inconveniencing users.

Implementing an OAuth Authentication server is conceptually simple but a pain in practice. We can leverage the power of Cloudflare Worker to simplify the implementation, reduce latency, and segregate our service logic from the authentication layer.

For those unfamiliar with OAuth, I highly recommend reading a more in depth article.

The steps of the OAuth 2.0 workflow are as follows:

1. The consumer service redirects the user to a callback URL that was setup by the auth server. At this callback URL, the auth server asks the user to sign in and accept the consumer permissions requests.
2. The auth server redirects the user to the consumer service with a code.
3. The consumer service asks to exchange this code for an access token. The consumer service validates their identity by including their client secret in the callback URL.
4. The auth server gives the consumer the access token.
5. The consumer service can now use Continue reading

The truth about Black Friday and Cyber Monday

At Cloudflare we handle a lot of traffic on behalf of our customers. Something we all see and hear a lot about at this time of year are Black Friday (23 November this year) and Cyber Monday (26 November) - but just how important are these days on the Internet?

To try and answer this question, we took a look at anonymised samples of HTTP requests crossing our network. First of all, let’s look at total page views from across our global network from the last few weeks and see if we can spot Black Friday and Cyber Monday:

So this is total page views by day (UTC) from November 19 (a week before Cyber Monday) until Monday December 3. Other than follow-the-sun fluctuations in a repeating daily pattern, each whole day is pretty similar in shape and size compared to the last. Black Friday and Cyber Monday aren’t visible in overall traffic patterns.

Get specific

We have a very diverse set of customers across 12 million domain names and not all of them are selling products or doing so directly online. To identify those websites Continue reading

A new look on your Cloudflare dashboard

Building a great customer experience is a top priority here at Cloudflare. Over the last week, we have been  slowly rolling out an improvement to our customer dashboard - specifically the zone overview page. We are now ready to share the new dashboard with everyone.

Many of the changes you’ll notice first were driven directly by customer feedback. We’ve heard time and time again that site owners would like better visibility of their key analytics, and for it to be easier to apply certain common settings changes. This new dashboard provides both, with a snapshot of several of your key analytics, and a new toolkit area for easy access to key functions. This page has also been redesigned from top to bottom for responsiveness across various devices and screen widths.

Even more interesting is all the work under-the-hood that went into making this release and how it sets the stage for improvements to our customer experience as we look ahead to 2019. This is one of the first pieces of production Product work shipped using our new prototyping framework. We will soon be sharing details of the design systems work that underlies the new experience on Continue reading

Ampere Targets Hyperscalers, Challenges Intel With Arm-Based Chips

The year-old startup, founded by former Intel president Renee James, is also partnering with Packet, Cloudflare, and other companies on 5G and edge computing proof of concepts.

EU Terrorist Content Online proposal – political haste and unintended consequences

In September, the European Commission presented a legislative proposal to address the removal of terrorist content online. There has been significant political pressure, particularly as the EU elections of 2019 approach, towards internet companies taking on increased responsibility in the area of terrorist propaganda online. This proposal would be a marked move from various voluntary initiatives taken up by some social media companies in recent times towards a legal responsibility framework for many.  

While appreciating the concerns around terrorism, Cloudflare is not only troubled by the late presentation of this proposal – which leaves inadequate time for a thorough review before this EU legislative term expires – but also much of the substance. Along with others such as CDT, GSMA/ETNO and Mozilla, we have significant concerns around the legal implications, practical application and possible unintended consequences of the proposal, some of which we outline below. Furthermore, we believe that little evidence has been presented as to the necessity of the proposed measures.

Concerns and shortcomings

The Commission’s proposal does not account for the complexity and range of information society services having a storage component - not all services have the same Continue reading

How well do you know website performance?

How quickly did this blog post load? Did you happen to notice?

Most website visitors do notice that sort of thing, even if they don't realize it consciously. In fact, virtually all visitors have strong opinions about how quickly a website should load, how responsive it should be, and how often it should go down (preferably, never). Most users strongly prefer webpages that take under 5 seconds to load, and they're much more likely to leave without buying products or reading content if a page takes too long to render.

Everyone should understand how to keep websites fast

Part of the Cloudflare mission is to make the Internet better by helping it perform better. One way we are delivering on our mission is by sharing easy-to-read educational content to assist anyone with a web property.

We recently launched a performance-themed Learning Center: a series of educational articles on why performance matters, the factors affecting performance, and some of the best practices for making websites faster. Check out the Performance Learning Center!

This Learning Center is for anyone who wants to really do a deep dive into the complex topic of website performance, no matter what their technical background Continue reading

Cloudflare Apps Fund 2.0

Today we are launching the second iteration of the Cloudflare App Developer Fund.

This is an investment fund started in partnership with Venrock, Pelion Venture Partners, and New Enterprise Associates. We created the fund to provide developers with resources so that they can build apps using the Cloudflare network and our serverless product, Workers, which enables developers to create applications without configuring or maintaining infrastructure. The goal is to fuel a new economy that grows into a thriving ecosystem so that developers’ serverless innovations can enjoy commercial success.  

Cloudflare Apps with Workers allows developers to package Workers, delivering new worker-powered experiences to Cloudflare customers, all within a few clicks via Cloudflare Apps.

Our CTO, John Graham-Cumming talks about it in a recently recorded video:

In short, Cloudflare is looking for apps that will advance our mission to help build a better Internet: one that is safe, fast and reliable.   We are seeking out apps that have a clear vision, delight customers by the ease of use and align with our vision. Having already seen multiple use cases around custom application security, data loss prevention, and bot mitigation, we can’t wait to see what else you come up Continue reading

Logs from the Edge

With Cloudflare Workers, our JavaScript environment at the edge, it is possible to send traffic logs to arbitrary locations. In this post we are going to discuss an example Worker implementation on how to achieve this. So if you are building or maintaining your own traffic logging/analytics environment, read on.

To build the underlying script we are going to leverage sub requests. Sub requests, which can be spawned from the initial HTTP/S request, can be used to aggregate and compose a response from several back end services, or, like in the example discussed here, to post data to a specific endpoint. Sub requests can be made asynchronously and after the initial request has been fully served to avoid adding unnecessary latency to the main request.

The Worker Code

In this example we assume an Elastic stack has been set up at elk.example.com and has been configured to receive via HTTP/S PUT requests a number of fields for each log line. The full script that we are going to look at can be found below:

addEventListener('fetch', event => {
  event.respondWith(fetchAndLog(event));
})

async function fetchAndLog(event) {
  const response = await fetch(event.request);
  event.waitUntil(logToElk(event.request, response));
  return response;
 Continue reading

Know your SCM_RIGHTS

As TLS 1.3 was ratified earlier this year, I was recollecting how we got started with it here at Cloudflare. We made the decision to be early adopters of TLS 1.3 a little over two years ago. It was a very important decision, and we took it very seriously.

It is no secret that Cloudflare uses nginx to handle user traffic. A little less known fact, is that we have several instances of nginx running. I won’t go into detail, but there is one instance whose job is to accept connections on port 443, and proxy them to another instance of nginx that actually handles the requests. It has pretty limited functionality otherwise. We fondly call it nginx-ssl.

Back then we were using OpenSSL for TLS and Crypto in nginx, but OpenSSL (and BoringSSL) had yet to announce a timeline for TLS 1.3 support, therefore we had to implement our own TLS 1.3 stack. Obviously we wanted an implementation that would not affect any customer or client that would not enable TLS 1.3. We also needed something that we could iterate on quickly, because the spec was very fluid back then, and also something Continue reading