A Tour Inside Cloudflare’s G9 Servers

Cloudflare operates at a significant scale, handling nearly 10% of the Internet HTTP requests that is at peak more than 25 trillion requests through our network every month. To ensure this is as efficient as possible, we own and operate all the equipment in our 154 locations around the world in order to process the volume of traffic that flows through our network. We spend a significant amount of time specing and designing servers that makes up our network to meet our ever changing and growing demands. On regular intervals, we will take everything we've learned about our last generation of hardware and refresh each component with the next generation…

If the above paragraph sounds familiar, it’s a reflecting glance to where we were 5 years ago using today’s numbers. We’ve done so much progress engineering and developing our tools with the latest tech through the years by pushing ourselves at getting smarter in what we do.

Here though we’re going to blog about muscle.

Since the last time we blogged about our G4 servers, we’ve iterated one generation each of the past 5 years. Our latest generation is now the G9 server. From a G4 server comprising 12 Intel Continue reading

Mapping Factorio with Leaflet

The following is a guest post by Jacob Hands, Creator of FactorioMaps.com. He is building a community site for the game Factorio centered around sharing user creations.

Factorio is a game about building and maintaining factories. Players mine resources, research new technology and automate production. Resources move along the production line through multiple means of transportation such as belts and trains. Once production starts getting up to speed, alien bugs start to attack the factory requiring strong defenses.

At FactorioMaps.com, I am building a place for the community of Factorio players to share their factories as interactive Leaflet maps. Due to the size and detail of the game, it can be difficult to share an entire factory through a few screenshots. A Leaflet map provides a Google Maps-like experience allowing viewers to pan and zoom throughout the map almost as if they are playing the game.

Hosting

Leaflet maps contain thousands of small images for X/Y/Z coordinates. Amazon S3 and Google Cloud Storage are Continue reading

Leave your VPN and cURL secure APIs with Cloudflare Access

We built Access to solve a problem here at Cloudflare: our VPN. Our team members hated the slowness and inconvenience of VPN but, that wasn’t the issue we needed to solve. The security risks posed by a VPN required a better solution.

VPNs punch holes in the network perimeter. Once inside, individuals can access everything. This can include  critically sensitive content like private keys, cryptographic salts, and log files. Cloudflare is a security company; this situation was unacceptable. We need a better method that gives every application control over precisely who is allowed to  reach it.

Access meets that need. We started by moving our browser-based applications behind Access. Team members could connect to applications faster, from anywhere, while we improved the security of the entire organization. However, we weren’t yet ready to turn off our VPN as some tasks are better done through a command line. We cannot #EndTheVPN without replacing all of its use cases. Reaching a server from the command line required us to fall back to our VPN.

Today, we’re releasing a beta command line tool to help your team, and ours. Before we started using this feature at Cloudflare, curling a server required me to Continue reading

Announcing Firewall Rules

Threat landscapes change every second. As attackers evolve, becoming more dynamic and devious, vulnerabilities materialize faster than engineers can patch their applications. Part of Cloudflare’s mission is to keep you and your applications safe. Today, Cloudflare is launching a new feature, giving customers what they have been requesting - fine-grained control over their incoming requests.

Cloudflare already offers a number of powerful firewall tools such as IP rules, CIDR rules, ASN rules, country rules, HTTP user-agent blocking, Zone Lockdown (for these URIs only allow traffic from those IPs), and our comprehensive managed rules within our WAF (Web Application Firewall). But sometimes, you need to combine the power of these to fully mitigate an attack, and to express a block rule that breaks the boundaries of the existing tools, to be able to “block traffic to this URI when the request comes from that IP and the user-agent matches one of these”.

Flexibility and Control

© Stefano Kocka : Source Wikipedia

Common themes arose when we spoke to customers about their needs and also reviewed feature requests that our customer support team had seen, and we categorised the top pieces of feedback and feature requests into three core needs:

1. More flexibility Continue reading

Custom Load Balancing With Cloudflare Workers

The following is a guest post by Jayaprabhakar Kadarkarai, Developer of Codiva.io, an Online IDE used by computer science students across the world. He works full stack to deliver low latency and scalable web applications.

Have you launched your website? Getting a lot of traffic? And you are planning to add more servers? You’ll need load balancing to maintain the scalability and reliability of your website. Cloudflare offers powerful Load Balancing, but there are situations where off-the-shelf options can’t satisfy your specific needs. For those situations, you can write your own Cloudflare Worker.

In this post, we’ll learn about load balancers and how to set them up at a low cost with Cloudflare Service Workers.

This post assumes you have a basic understanding of JavaScript, as that’s the language used to write a Cloudflare Worker.

The Basic Pattern

The basic pattern starts with adding ‘fetch’ event listener to intercept the requests. You can configure which requests to intercept on the Cloudflare dashboard or using the Cloudflare API.

Then, modify the hostname of the URL and send the request to the new host.

addEventListener('fetch', event => {
  var url = new URL(event.request.url);

  // https://example.com/path/  Continue reading

Ulaanbaatar, Mongolia

Whenever you get into a conversation about exotic travel or ponder visiting the four corners of the globe, inevitably you end up discussing Ulaanbaatar in Mongolia. Travelers want to experience the rich culture and vivid blue skies of Mongolia; a feature which gives the country its nickname of “Land of the Eternal Blue Sky”.

Ulaanbaatar (or Ulan Bator; but shortened to UB by many) is the capital of Mongolia and located nearly a mile above sea level just outside the Gobi Desert - a desert that spans a good percentage of Central Asia’s Mongolia. (The rest of the Gobi Desert extends into China). The country is nestled squarely between Russia to the north and China to the south. It’s also home to some of the richest and ancient customs and festivals around. It’s those festivals that successfully draw in the tourists who want to experience something quite unique. Luckily, even with all the tourists, Mongolia has managed to keep its local customs; both in the cities and within its nomadic tribes.

via Wikipedia

History also has drawn explorers and conquerors to and from the region; but more on that later.

Cloudflare is also drawn into Mongolia

Any avid reader of Continue reading

Cloudflare Access: Sharing our single-sign on plugin for Atlassian

Here at Cloudflare, we rely on a set of productivity tools built by Atlassian, including Jira and Confluence. We secure them with Cloudflare Access. In the past, when our team members wanted to reach those applications, they first logged in with our identity provider credentials to pass Access. They then broke out a second set of credentials, specific to Atlassian tools, to reach Jira. The flow is inconvenient on a desktop and downright painful on a mobile device.

While Access can determine who should be able to reach an application, the product alone cannot decide what the user should be able to do once they arrive at the destination. The application sets those specific permissions, typically by requiring another set of user credentials. The extra step slows down and frustrates end users. Access saves time by replacing a cumbersome VPN login. However, we wanted to also solve the SSO problem for our team.

We created a plugin, specific to Atlassian, that could take identity data from the token generated by Access and map it to a user account. Our team members log in with our identity provider to pass Access, and then Access could set their user permissions in Jira Continue reading

#BetterInternet: Join the Movement

When it comes to overall awareness of Cloudflare, it seems most folks fall into one of three camps: 1) those who don’t know much about Cloudflare at all, 2) those who are familiar with one or two of Cloudflare’s many solutions (i.e. DDoS protection, caching, DNS, etc.), and finally, 3) those who understand the full breadth and scope of Cloudflare’s global cloud network. This latter group of folks are especially excited about the broad scope of Cloudflare’s mission, which is: “to help build a better Internet.” Last week our co-founder Michelle Zatlyn explained in a blog post what this mission actually means:

“Our mission at Cloudflare is to help build a better Internet. That is a big, broad mission that means many things. It means that we push to make Internet properties faster. It means respecting individual’s privacy. It means making it harder for malicious actors to do bad things. It means helping to make the Internet more reliable. It means supporting new Internet standards and protocols, and making sure they are accessible to everyone. It means democratizing technology and making sure the widest possible group has access to it. It means increasing value for our community, Continue reading

Free to code

This week at the Cloudflare Internet Summit I have the honour of sitting down and talking with Sophie Wilson. She designed the very first ARM processor instruction set in the mid-1980s and was part of the small team that built the foundations for the mobile world we live in: if you are reading this on a mobile device, like a phone or tablet, it almost certainly has an ARM processor in it.

But, despite the amazing success of ARM, it’s not the processor that I think of when I think of Sophie Wilson. It’s the BBC Micro, the first computer I ever owned. And it’s the computer on which Wilson and others created ARM despite it having just an 8-bit 6502 processor and 32k of RAM.

Luckily, I still own that machine and recently plugged it into a TV set and turned it on to make sure it was still working 36 years on (you can read about that one time blue smoke came out of it and my repair). I wanted to experience once more the machine Sophie Wilson helped to design. One vital component of that machine was BBC BASIC, stored in a ROM chip on Continue reading

WebAssembly on Cloudflare Workers

We just announced ten major new products and initiatives over Crypto Week and Birthday Week, but our work is never finished. We're continuously upgrading our existing products with new functionality.

Today, we're extending Cloudflare Workers with support for WebAssembly. All Workers customers can now augment their applications with WASM at no additional cost.

What is WebAssembly?

WebAssembly -- often abbreviated as "WASM" -- is a technology that extends the web platform to support compiled languages like C, C++, Rust, Go, and more. These languages can be compiled to a special WASM binary format and then loaded in a browser.

WASM code is securely sandboxed, just like JavaScript. But, because it is based on compiled lower-level languages, it can be much faster for certain kinds of resource-intensive tasks where JavaScript is not a good fit. In addition to performance benefits, WASM allows you to reuse existing code written in languages other than JavaScript.

What are Workers?

For those that don't know: Cloudflare Workers lets you deploy "serverless" JavaScript code directly to our 153-and-growing datacenters. Your Worker handles your site's HTTP traffic directly at the location closest to your end user, allowing you to achieve lower latency and reduce serving costs. Continue reading

Real World Serverless: The Video

We held our second Real World Serverless event in London last week and filmed the four talks about Serverless technology to share with you here, on the Cloudflare blog. Check out the recording, featuring Henry Heinemann, Sevki Hasirci, and Stephen Pinkerton from Cloudflare and Paddy Sherry from gambling.com Group.

For details of our other upcoming Real World Serverless events in Austin, Singapore, Sydney, and Melbourne, scroll to the bottom.

Video transcript:

Moderator: Okay, welcome to Cloudflare, everybody. I'm so pleased that you're here. This is the second event of our real world serverless event series. We had our first one in San Francisco just last week, and we were so excited to fly over here, both me and Jade and some other Cloudflare employees, and tap into this community here, because this is our second largest office.

We also frankly, love coming here to London to visit and engage with the developer community here. Let me hand this over to Jade and she'll speak a little bit about Cloudflare, and then we'll get started with speaker number one.

[applause]

Jade: Hello, everyone. Welcome to Cloudflare. It's on, okay cool. How many of you know Continue reading

How to save costs on your API Gateway solution using Cloudflare Workers

The following is a guest post by Janusz Jezowicz, CEO of Speedchecker. The Speedchecker team runs a global distributed measurement network and offer speed test solutions using the Cloudflare platform.

Software companies contemplating offering a public API to 3rd party developers have many options to choose from for how to offer their API securely with high reliability and with fast performance. When it comes to cost though, commercial solutions are expensive and open-source solutions require a lot of time managing servers and the synchronization between them. This blog post describes how we successfully moved our API gateway to Cloudflare Workers and slashed our costs by a factor of 10.

Our original solution based on the Kong open-source API gateway

When we built our measurement network API  for cost reasons we opted for open-source solution Kong. Kong is a great solution which has a vibrant community of users and plug-in developers who extend and maintain the platform. Kong is a good alternative to commercial solutions from companies such as Apigee or Mulesoft whose solutions are really catering for larger businesses who can afford them. Kong is free and it works. On the other hand, if your business has complex needs Continue reading

Statement concerning events at Glowbeam Technologies

All of Cloudflare's staff were shocked at the events depicted in NCIS Season 16 Episode 1 where incorrect use of random numbers for encryption resulted in the insertion of multiple trojan horses that brought a nuclear reactor within seconds of a meltdown.

Although Cloudflare has long been a competitor of the company responsible, Glowbeam Technologies, and uses similar random number generation technology, we would like to emphasize that there are significant differences between the two companies.

Firstly, Cloudflare's Lava Lamps are not an "encryption engine" and thus they are not susceptible to tampering by the janitor.

Secondly, all Cloudflare staff undergo extensive background checks.

Thirdly, we were shocked that Glowbeam Technologies' wall of Lava Lamps was a single point of failure. In contrast, Cloudflare uses multiple sources of randomness.

Lastly, Glowbeam Technologies' CEO confirmed that the company did not use "AES" or "key block ciphers", but instead relied solely on their Lava Lamp "encryption engine". Cloudflare strongly advocates for never writing or inventing encryption algorithms and works closely with groups like the IETF to use standard, well understood encryption.

As a result of these events Cloudflare has acquired the assets of Glowbeam Technologies, please visit glowbeamtechnologies.com for more information.

John Graham-Cumming
Chief Technology Officer
Cloudflare, Inc.

Birthday Week Wrap-Up: Every day is launch day at Cloudflare

Our customers are accustomed to us launching new services, features, and functionality at a feverish pace, but recently, we’ve been especially active. This week we celebrated our 8th Birthday Week by announcing new offerings that benefit our customers and the global Internet community. Our mission is to help build a better Internet, and we’re convinced that launching new capabilities that benefit not only our customers, but also the broader Internet overall, is the best way to fulfill our mission.

Helping build a better Internet, one launch at a time

As an organization, we could choose to celebrate Cloudflare’s birthday in lots of different ways (a press release, a company party, or fun gifts for all our employees). But at Cloudflare, we have a unique birthday tradition: we roll up our sleeves and give our customers and the Internet community a new capability (i.e. a gift) every day of our birthday week.

Some of this past week’s launches have been entirely new offerings, like providing key-value storage across Cloudflare’s global cloud network with Cloudflare Workers KV.  Other birthday week launches help improve the overall Internet ecosystem: the Bandwidth Alliance reduces data transfer charges from major cloud hosts and Cloudflare Registrar Continue reading

Building With Workers KV, a Fast Distributed Key-Value Store

Your Workers now have access to a low-latency key-value data store which lives inside our network all around the world!

For those who don’t know, Cloudflare Workers is a new type of compute platform, built on top of our global network of 152+ data centers around the world. It allows you to write serverless code which runs in the fabric of the Internet itself, allowing you to engage with your users faster than other platforms can even get a packet to where your code is running. It’s built on a new architecture which eliminates cold starts and dramatically reduces the memory overhead of keeping your code running when compared to a platform like Amazon Lambda.

As powerful as this is, compute is just one component of what you need to build an application, you also need the ability to store data. We evaluated many of the available open source data stores on the market, but ultimately nothing was designed for a world with quite as many distributed nodes as our network. Instead, we have begun releasing our own vision for distributed storage, beginning today.

The Workers KV is a highly distributed, eventually-consistent, key value store. It will allow you to Continue reading

Introducing Workers KV

In 1864 British computer pioneer Charles Babbage described the first key-value store. It was meant to be part of his Analytical Engine. Sadly, the Analytical Engine, which would have been the first programmable computer, was never built. But Babbage lays out clearly the design for his key-value store in his autobiography. He imagined a read-only store implemented as punched cards. He referred to these as Tables:

I explained that the Tables to be used must, of course, be computed and punched on cards by the machine, in which case they would undoubtedly be correct. I then added that when the machine wanted a tabular number, say the logarithm of a given number, that it would ring a bell and then stop itself. On this, the attendant would look at a certain part of the machine, and find that it wanted the logarithm of a given number, say of 2303. The attendant would then go to the drawer containing the pasteboard cards representing its table of logarithms. From amongst these he would take the required logarithmic card, and place it in the machine.

Punched card illustration from Babbage’s autobiography showing an integer key (2303) and value representing the decimal part of Continue reading

Bandwidth Alliance Works to Slash Cloud Interconnect Fees

AWS and Google are notably absent from the group, though Microsoft and IBM are on board.

Introducing Cloudflare Registrar: Domain Registration You Can Love

“I love my domain registrar.” Has anyone ever said this? From before Cloudflare even launched in September 2010, our early beta customers were literally begging us: "Will you please launch a registrar too?!" Today we're doing just that, launching the first registrar we hope you’ll be able to say you love. It's built around three principles: trust, security, and always-fair pricing. And it’s available to all Cloudflare customers.

Needing Secure Domain Registration Ourselves

Cloudflare has actually run a registrar for some time. Like many of our best products, it started by solving an internal issue we had. Cloudflare has several mission-critical domains. If the registration of these domains were ever compromised, it would be, in a word, bad.

For years, we worked with our original domain registrar to ensure these domains were as locked down as possible. Unfortunately, in 2013, a hacker was able to compromise several of the systems of the registrar we used and come perilously close to taking over some of our domains.

That began a process of us looking for a better registrar. Unfortunately, even the registrars that charge hefty premiums and promise to be very secure turn out to have pretty lousy security. Continue reading

Cloudflare Registrar: what happens when you register a domain?

Every website, large or small, started with an idea, rapidly followed by registering a domain. Most registrars offer promotions for your initial domain registration and then quietly hike the price with each renewal. What they don’t tell customers is that the price they pay to a registry, for your registration, is set by the registry. In some cases, we’ve found registrars charging eight times the wholesale price for a domain renewal.

Today, we’re launching Cloudflare Registrar, the first domain registrar you can love. Cloudflare Registrar will never charge you more than what we pay to the registry for your domain. No markup and no surprise fees. For eight years Cloudflare has built products that make the internet faster and safer. It's time for us to start where your internet journey starts, your domain.

A quick introduction to domain registration

When you register a domain, you become the owner, or registrant, for that domain for a set period of time. Now that you are the registrant, you can create an authoritative record that tells the world the nameservers for your domain. The domain name system, or DNS, uses those nameservers to direct traffic to the IP address of your server.

Continue reading

Bandwidth Alliance: powered by smart routing on Cloudflare’s network

Today, we’re excited to announce the launch of the Bandwidth Alliance, a group of cloud providers that have agreed to reduce data transfer fees for mutual customers.

Three things were required to make the Bandwidth Alliance a reality:

1. An ecosystem of like-minded companies who want to provide reduced data transfer fees to their customers.
2. A large global and well-connected network (Cloudflare has 150+ points of presence around the world and multiple peered and paid links at each location). Our network is connected to thousands of partners through transit providers, Internet exchanges, peering interconnects, and private network interconnects. Having a large network footprint allows us to meet our partners where their infrastructure is and exchange traffic with them over low-cost or free connections, instead of expensive paid transit.
3. Argo, our sophisticated traffic routing engine. Argo allows us to make decisions on how to carry traffic across our network in ways that optimize for a number of factors: latency, throughput, jitter, or in the case of the Bandwidth Alliance, cost to our partners to exchange traffic. This routing engine is the technical underpinning of the Bandwidth Alliance.

Typically, as traffic moves across the Internet, packets are exchanged between multiple networks as they Continue reading