CloudFlare’s JSON-powered Documentation Generator
Everything that it's possible to do in the CloudFlare Dashboard is also possible through our RESTful API. We use the same API to power the dashboard itself.
In order to keep track of all our endpoints, we use a rich notation called JSON Hyper-Schema. These schemas are used to generate the complete HTML documentation that you can see at https://api.cloudflare.com. Today, we want to share a set of tools that we use in this process.
CC BY 2.0 image by Richard Martin
JSON Schema
JSON Schema is a powerful way to describe your JSON data format. It provides complete structural validation and can be used for things like validation of incoming requests. JSON Hyper-Schema further extends this format with links and gives you a way describe your API.
JSON Schema Example
{
"type": "object",
"properties": {
"name": { "type": "string" },
"age": { "type": "number" },
"address": {
"type": "object",
"properties": {
"street_address": { "type": "string" },
"city": { "type": "string" },
"state": { "type": "string" },
"country": { "type" : "string" }
}
}
}
}
Matching JSON
{
"name": "John Doe",
"age": 45,
"address": {
"street_address": "12433 State St NW",
"city": "Atlanta",
"state": "Georgia",
"country": Continue readingIntroducing the p0f BPF compiler
Two years ago we blogged about our love of BPF (BSD packet filter) bytecode.
CC BY 2.0 image by jim simonson
Then we published a set of utilities we are using to generate the BPF rules for our production iptables: the bpftools.
Today we are very happy to open source another component of the bpftools: our p0f BPF compiler!
Meet the p0f
p0f is a tool written by superhuman Michal Zalewski. The main purpose of p0f is to passively analyze and categorize arbitrary network traffic. You can feed p0f any packet and in return it will derive knowledge about the operating system that sent the packet.
One of the features that caught our attention was the concise yet explanatory signature format used to describe TCP SYN packets.
The p0f SYN signature is a simple string consisting of colon separated values. This string cleanly describes a SYN packet in a human-readable way. The format is pretty smart, skipping the varying TCP fields and keeping focus only on the essence of the SYN packet, extracting the interesting bits from it.
We are using this on daily basis to categorize the packets that we, at CloudFlare, see when we are a target Continue reading
Ask Some HTTP/2 Pros The Hard Questions
We're big fans of HTTP/2 and our customers make up the majority of HTTP/2 enabled domains today. HTTP/2 is a key part of the modern web, and its growth and adoption is changing how websites and applications are built.
On Thursday July 21, 2016, join web performance experts Ilya Grigorik (Web Performance Engineer at Google) and Suzanne Aldrich (Solutions Engineer at CloudFlare), as they discuss the latest in HTTP/2 and Server Push, the tools and resources you can use today to build fast and scalable web apps, and ways to speed up your content on any device.
When: Thursday July 21, 2016 from 1pm-2pm Eastern Time (1500 – 1600 UTC)
Who: CloudFlare’s own Suzanne Aldrich and Ilya Grigorik from Google
Need the basics of HTTP/2 and Server Push? Visit the CloudFlare HTTP/2 website.
CloudFlare sites protected from httpoxy
CC BY 2.0 image by Joe Seggiola
We have rolled out automatic protection for all customers for the the newly announced vulnerability called httpoxy.
This vulnerability affects applications that use “classic” CGI execution models, and could lead to API token disclosure of the services that your application may talk to.
By default httpoxy requests are modified to be harmless and then request is allowed through, however customers who want to outright block those requests can also use the Web Application Firewall rule 100050 in CloudFlare Specials to block requests that could lead to the httpoxy vulnerability.
More data, more data
"multas per gentes et multa per aequora" 1
The life of a request to CloudFlare begins and ends at the edge. But the afterlife! Like Catullus to Bithynia, the log generated by an HTTP request or a DNS query has much, much further to go.
This post comes from CloudFlare's Data Team. It reports the state of processing these sort of edge logs, including what's worked well for us and what remains a challenge in the time since our last post from April 2015.
Numbers, sense
In an edge network, where HTTP and DNS clients connect to thousands of servers distributed across the world, the key is to distribute those servers across many carefully picked points of presence—and with over 80 PoPs, no network has better representation than CloudFlare. The reverse, however, has to happen for that network's logs. After anycast has scattered requests (and queries) to thousands of nodes at the edge, it's the Data Team's job to gather the resulting logs to a small number of central points and consolidate them for easy use by our customers.
The charts above depict (with some artifacts due to counter resets) the total structured logs sent from the edge to one Continue reading
Why we use the Linux kernel’s TCP stack
A recent blog post posed the question Why do we use the Linux kernel's TCP stack?. It triggered a very interesting discussion on Hacker News.
I've also thought about this question while working at CloudFlare. My experience mostly comes from working with thousands of production machines here and I can try to answer the question from that perspective.
CC BY 2.0 image by John Vetterli
Let's start with a broader question - what is the point of running an operating system at all? If you planned on running a single application, having to use a kernel consisting of multiple million lines of code may sound like a burden.
But in fact most of us decide to run some kind of OS and we do that for two reasons. Firstly, the OS layer adds hardware independence and easy to use APIs. With these we can focus on writing the code for any machine - not only the specialized hardware we have at the moment. Secondly, the OS adds a time sharing layer. This allows us to run more than one application at a time. Whether it's a second HTTP server or just a bash session, this ability to share resources Continue reading
HTTP/2 Server Push with multiple assets per Link header
In April we announced that we had added experimental support for HTTP/2 Server Push to all CloudFlare web sites. We did this so that our customers could iterate on this new functionality.
CC BY 2.0 image by https://www.flickr.com/photos/mryipyop/
Our implementation of Server Push made use of the HTTP Link header as detailed in W3C Preload Working Draft.
We also showed how to make Server Push work from within PHP code and many people started testing and using this feature.
However, there was a serious restriction in our initial version: it was not possible to specify more than one asset per Link header for Server Push and many CMS and web development platforms would not allow multiple Link headers.
We have now addressed that problem and it is possible to request that multiple assets be pushed in a single Link header. This change is live and was used to push assets in this blog post to your browser if your browser supports HTTP/2.
When CloudFlare reads a Link header sent by an origin web server it will remove assets that it pushes from the Link header passed on to the web browser. That made it a little difficult Continue reading
The complete guide to Go net/http timeouts
When writing an HTTP server or client in Go, timeouts are amongst the easiest and most subtle things to get wrong: there’s many to choose from, and a mistake can have no consequences for a long time, until the network glitches and the process hangs.
HTTP is a complex multi-stage protocol, so there's no one-size fits all solution to timeouts. Think about a streaming endpoint versus a JSON API versus a Comet endpoint. Indeed, the defaults are often not what you want.
In this post I’ll take apart the various stages you might need to apply a timeout to, and look at the different ways to do it, on both the Server and the Client side.
SetDeadline
First, you need to know about the network primitive that Go exposes to implement timeouts: Deadlines.
Exposed by net.Conn with the Set[Read|Write]Deadline(time.Time) methods, Deadlines are an absolute time which when reached makes all I/O operations fail with a timeout error.
Deadlines are not timeouts. Once set they stay in force forever (or until the next call to SetDeadline), no matter if and how the connection is used in the meantime. So to build a timeout with SetDeadline you'll have to Continue reading
Economical With The Truth: Making DNSSEC Answers Cheap
We launched DNSSEC late last year and are already signing 56.9 billion DNS record sets per day. At this scale, we care a great deal about compute cost. One of the ways we save CPU cycles is our unique implementation of negative answers in DNSSEC.
CC BY-SA 2.0 image by Chris Short
I will briefly explain a few concepts you need to know about DNSSEC and negative answers, and then we will dive into how CloudFlare saves on compute when asked for names that don’t exist.
What You Need To Know: DNSSEC Edition
Here’s a quick summary of DNSSEC:
This is an unsigned DNS answer (unsigned == no DNSSEC):
cloudflare.com. 299 IN A 198.41.214.162
cloudflare.com. 299 IN A 198.41.215.162
This is an answer with DNSSEC:
cloudflare.com. 299 IN A 198.41.214.162
cloudflare.com. 299 IN A 198.41.215.162
cloudflare.com. 299 IN RRSIG A 13 2 300 20160311145051 20160309125051 35273 cloudflare.com. RqRna0qkih8cuki++YbFOkJi0DGeNpCMYDzlBuG88LWqx+Aaq8x3kQZX TzMTpFRs6K0na9NCUg412bOD4LH3EQ==
Answers with DNSSEC contain a signature for every record type that is returned. (In this example, only A records are returned so there is only one signature.) The signatures allow Continue reading
A post-mortem on this morning’s incident
We would like to share more details with our customers and readers on the internet outages that occurred this morning and earlier in the week, and what we are doing to prevent these from happening again.
June 17 incident
On June 17, at 08:32 UTC, our systems detected a significant packet loss between multiple destinations on one of our major transit provider backbone networks, Telia Carrier.
In the timeframe where the incident was being analysed by our engineers, the loss became intermittent and finally disappeared.
Packet loss on Telia Carrier (AS1299)
Today’s incident
Today, Jun 20, at 12:10 UTC, our systems again detected massive packet loss on one of our major transit provider backbone networks: Telia Carrier.
Packet loss on Telia Carrier (AS1299)
Typically, transit providers are very reliable and transport all of our packets from one point of the globe to the other without loss - that’s what we pay them for. In this case, our packets (and that of other Telia customers), were being dropped.
While Internet users usually take it for granted that they can reach any destination in the world from their homes and businesses, the reality is harsher than that. Our planet is big, and Continue reading
Join Us And Paul Vixie On Tuesday To Discuss BIND, Root Servers, And DNS Security
CloudFlare and Gandi have been hosting a speaker series on DNS, previously bringing in the founder of DNS Paul Mockapetris and Dan Kaminsky, who uncovered one of the most critical vulnerabilities in DNS.
Our third and final talk is coming up on June 21st at 6PM PST at the Gandi office in San Francisco (live stream link will be posted on the Meetup page) and you won’t want to miss it, because our speaker is none other than Paul Vixie.
Paul wrote most of BIND Version 8 and hired the team who wrote BIND Version 9, the most widely used DNS software on the Internet. He founded ISC, home of BIND and F-root and later operated C-root.
We'll talk about alternative DNS root servers, email security and spam (Paul founded the first ever anti-spam company), and what needs to be done about DNS and security (Paul added most of BIND’s current security systems).
So come grab a beer and let's geek out about DNS one more time.
Moscow, Russia: CloudFlare’s 83rd data center
Здравствуйте! ))) CloudFlare is excited to announce the newest addition to our network in the largest country in the world (by footprint), increasing both our data center and city count to 83. Moscow is not only the capital and largest city in Russia, it is also home to several Internet exchanges which CloudFlare now participates at: the Moscow Internet Exchange (MSK-IX), Data IX and Global IX (Eurasia Peering coming soon). This raises the number of exchanges that CloudFlare is a participant of to over 120, making us one of the top interconnected networks globally.
Здравствуйте! ))) Мы рады объявить о новом пополнении в нашей сети network в самой большой стране мира (по занимаемой площади), увеличивая как количество наших датацентров, так и количество городов на 83. Москва является не только столицей и самым крупным городом в России, но она также является домом для нескольких точек обмена интернет-трафиком. CloudFlare в настоящее время принимает участие в следующих: Moscow Internet Exchange (MSK-IX), Data IX и Global IX (Eurasia Peering на подходе). Это увеличивает количество точек обмена интернет-трафиком, в которых участвует CloudFlare до 120, тем самым продвигая нас на одну из лидирующих позиций наиболее взаимосвязаных сетей в мире.
Improving performance in Continue reading
Hello, Colorado! CloudFlare’s 82nd Data Center is Live in Denver
Hello from the Mile High City! Denver is our twelfth data center in the United States, and our 82nd data center globally, improving regional web performance.
Denver Tech Scene
Near the iconic Rocky Mountains, Denver brews more beer than any other American city, and is home to a thriving technology and entrepreneurship community. The Colorado community brought us companies such as Chipotle, HomeAdvisor, and LogRhythm - and is helping build the next great Colorado company.
Mile High, Twice as Fast
Despite having a unique place on the map of the United States, and it's significant distance from other cities (900 miles to Dallas; 1,000 miles to Chicago, 1,000 miles to Los Angeles), Denver has not always been a major point of regional interconnection. Through the efforts of the community, and greater localized peering of traffic, this is changing for the better.
Visitors to millions of websites using ISPs, big or small, such as Aerux, Blackfoot, Comcast and CenturyLink, should see a 2x increase in performance, as they are now served from our Denver data center.
CloudFlare participates at two major internet exchanges: Any2 Denver and IX-Denver.
Latency in milliseconds halves for end user (Denver) to CloudFlare. Source: Cedexis
Global Expansion
Secure and fast GitHub Pages with CloudFlare
GitHub offers a web hosting service whereby you can serve a static website from a GitHub repository. This platform, GitHub Pages, can be used with CloudFlare whilst using a custom domain name.
In this tutorial, I will show you how to use CloudFlare and GitHub together. By taking advantage of CloudFlare’s global network, you can utilise our CDN service to improve your site's performance and security.
Whilst GitHub Pages doesn't ordinarily support SSL on custom domains, CloudFlare's Universal SSL allows your users to access your site over SSL, thus opening up the performance advantages of HTTP/2.
GitHub Pages is designed to host sites that only serve static HTML. The ability to only host static content isn’t as big of a restriction as you might think.
Static site generators avoid repetitive update tasks of updating “latest posts” feeds, pagination or sitemaps; whilst generating static HTML that can be uploaded to any web hosting service without a scripting engine. Unlike ancient desktop tools like FrontPage and Dreamweaver which lacked a content model, modern static site generators have the design decisively separate from content.
Typically, CMS-based sites must query a database for content, then render the HTML to be served to the end Continue reading
Brussels, Belgium: CloudFlare’s 81st Data Center is Live!
Our week begins in Brussels, where we announce our newest data center. This is our 23rd data center in Europe alone, and our 81st data center globally - providing additional redundancy to nearby facilities in Amsterdam and Paris.
Millions of websites using CloudFlare are now faster in Brussels, and we are excited to exchange traffic at the Belgium National Internet Exchange (BNIX).
Strategic City
Brussels - which hosts several strategic European institutions such as the European Commission and the Council of the European Union - is home to over 40,000 EU employees. With the turn up of our Brussels data center, they join the rest of the country in experiencing a faster Internet. Impressively, close to 90% of Belgium's citizens are online.
Two more cities
We will be announcing two more data centers this week - each in one of the world’s most populous countries. Could you guess which ones?
The CloudFlare network today:
Optimizing TLS over TCP to reduce latency
The layered nature of the Internet (HTTP on top of some reliable transport (e.g. TCP), TCP on top of some datagram layer (e.g. IP), IP on top of some link (e.g. Ethernet)) has been very important in its development. Different link layers have come and gone over time (any readers still using 802.5?) and this flexibility also means that a connection from your web browser might traverse your home network over WiFi, then down a DSL line, across fiber and finally be delivered over Ethernet to the web server. Each layer is blissfully unaware of the implementation of the layer below it.
But there are some disadvantages to this model. In the case of TLS (the most common standard used for sending encrypted data across in the Internet and the protocol your browser uses with visiting an https:// web site) the layering of TLS on top of TCP can cause delays to the delivery of a web page.
That’s because TLS divides the data being transmitted into records of a fixed (maximum) size and then hands those records to TCP for transmission. TCP promptly divides those records up into segments which are then transmitted. Ultimately, Continue reading
Perth, Australia: 80th Data Center
CloudFlare is excited to announce the launch of our newest data center in Perth, Australia. This expands the breadth of our global network to span 80 unique cities across 41 countries, and is our fourth data center in the Oceania region, joining existing data centers in Sydney, Melbourne and Auckland.
Western Australia
Perth is in a fascinating location. Home to sunny beaches and the highest number of self-made millionaires in the world, it is actually geographically closer to Singapore than to Sydney (though closer to Sydney in a “networking” sense, as determined by BGP routing).
Visitors to millions of websites across Western Australia served locally can now experience a faster and safer Internet - and ISPs can reach us at the Western Australia Internet Exchange (WA-IX), one of 119 internet exchanges that we openly peer at.
Latency in milliseconds from end user (Perth) to CloudFlare. Source: Cedexis
Missing letters?
Helping build CloudFlare’s network provides our team members with the opportunity to not just speed up the Internet, but also improve our sense of geography. Visitors to our offices in San Francisco, London and Singapore can get a sneak peek at our fast-changing map (with live and upcoming dots). Perth Continue reading
Open Sourcing CloudFlare’s UI Framework
Late last year, the CloudFlare UI team made a huge decision: to change JavaScript frameworks from Backbone & Marionette to React & Redux.
We’ve been one of the single biggest Backbone+Marionette apps on the web for a while now, and the decision to move was not taken lightly. On our team we have a former core team member of the Marionette team (myself), and the author of several popular Backbone projects: Backgrid and Backbone Paginator.
In the spirit of the open web, we want to share more about what we’re doing. This starts by open sourcing the UI framework that we have spent the last few months building.
Introducing cf-ui
While moving to React, we’ve taken our existing Backbone UI framework and rebuilt it from scratch on top of React. This includes over 50 packages that include dozens of components, utilities, test helpers, and more.
Examples: https://cloudflare.github.io/cf-ui
GitHub: https://github.com/cloudflare/cf-ui
We’re not open sourcing cf-ui because we think our buttons are any better than anyone else’s buttons, but because it’s an opportunity to share some of the technical decisions that we’ve made while building out a massive React application. The hope is that this will be Continue reading
Supporting the transition to IPv6-only networking services for iOS
Early last month Apple announced that all apps submitted to the Apple Store June 1 forward would need to support IPv6-only networking as they transition to IPv6-only network services in iOS 9. Apple reports that “Most apps will not require any changes”, as these existing apps support IPv6 through Apple's NSURLSession and CFNetwork APIs.
Our goal with IPv6, and any other emerging networking technology, is to make it ridiculously easy for our customers to make the transition. Over 2 years ago, we published Eliminating the last reasons to not enable IPv6 in celebration of World IPv6 Day. CloudFlare has been offering full IPv6 support as well as our IPv6-to-IPv4 gateway to all of our customers since 2012.
Why is the transition happening?
IPv4 represents a technical limitation, a hard stop to the number of devices that can access the Internet. When the Internet Protocol (IP) was first introduced by Vint Cerf and Bob Kahn in the late 1970s, Internet Protocol Version 4 (IPv4) used a 32-bit (four-byte) number, allowing about 4 billion unique addresses. At the time, IPv4 seemed more than sufficient to power the World Wide Web. On January 31, 2011, the top-level pool of Internet Assigned Numbers Authority Continue reading
The Sleepy User Agent
From time to time a customer writes in and asks about certain requests that have been blocked by the CloudFlare WAF. Recently, a customer couldn’t understand why it appeared that some simple GET requests for their homepage were listed as blocked in WAF analytics.
A sample request looked liked this:
GET / HTTP/1.1
Host: www.example.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (compatible; MSIE 11.0; Windows NT 6.1; Win64; x64; Trident/5.0)'+(select*from(select(sleep(20)))a)+'
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,fr;q=0.6
As I said, a simple request for the homepage of the web site, which at first glance doesn’t look suspicious at all. Unless your take a look at the User-Agent header (its value is the string that identifies the browser being used):
Mozilla/5.0 (compatible; MSIE 11.0; Windows NT 6.1; Win64; x64; Trident/5.0)'+(select*from(select(sleep(20)))a)+
The start looks reasonable (it’s apparently Microsoft Internet Explorer 11) but the agent strings ends with '+(select*from(select(sleep(20)))a)+. The attacker is attempting a SQL injection inside the User-Agent value.
It’s common to see SQL injection in URIs and form parameters, but here the attacker has hidden the SQL query select * from (select(sleep(20))) inside the User-Agent Continue reading