WHOIS going to be at the Grace Hopper Celebration?
Ubuntu us are doing the round trip! It’s time to live - WAN you arrive at GHC, come meet us and say HELO (we love GNU faces, we’ll be very api to meet you). When you’re exhausted like IPv4, git over to the Cloudflare corner to reboot –– we’ll have chargers and Wi-Fi (it’s not a SYN to REST). R booth can be your ESC. Then Thursday morning we’re hosting a breakfast bash with Zendesk –– it will be quite the Assembly, you should definitely Go, compile a bowl of serial, drink a bit of CIDR or a cup of tee.
I’m also speaking at 1:30PM on Wednesday in OCCC W414 hashing out encryption and updates for IoT –– DES should be a fun session.
ACK! I did NAT tell you how to find us. Check for sum women in capes a few hops away from the booths with the lava LAMP stack. I'm the one with cURLs.
In D air! Excited to LANd. C you soon.
Go Hack Nights at Cloudflare
At Cloudflare we're extensively using the Go programming language to build a better Internet. Go is a free and open source programming language created by Google in 2007 and open sourced in 2009. Earlier this year, Go made news when it entered the list of top 10 programming languages on the TIOBE Index.
Recently we launched an internal monthly Go Hack Night at our San Francisco office, open to anyone who works at Cloudflare regardless of their department or position. Anyone from newbie programmers to our most experienced Go engineers are encouraged to attend, and experienced engineers are asked to throw on a mentor badge and help guide colleagues with installing and learning Go.
We had over 30 attendees at our inaugural Go Hack Night, and our survey reveals some great stats:
- 26% of attendees were completely new to programming
- 61% of attendees were experienced in other languages but new to Go
- Every attendee said they learned something!
We actively encourage an inclusive learning culture and we're super excited to make the Go programming language more accessible to our entire company.
If you're interested in working with Go and helping to build a better Internet, we're hiring!
P.S. if Continue reading
A New Cybersecurity Strategy for Europe
October is European Cybersecurity Month, an annual advocacy campaign to raise awareness of cyber risks among citizens and businesses, and to share best practices in cybersecurity. This year’s campaign was launched at an event in Estonia, a country which both holds the current Presidency seat of the European Council and is well known as being highly cyber aware and digitally savvy.
It is fitting, therefore, that it is under Estonia’s Presidency that the European Commission announced a number of initiatives last month aimed at stepping up the European Union’s cybersecurity capacity and response to cyber attacks, while laying the foundations for increased cyber awareness and better cyber hygiene overall.
This EU’s Cybersecurity Strategy is a welcome initiative, as we already know that the overall cyber threat level is rising. At Cloudflare, we deal with a new type of DDoS attack every 3 minutes, and it has been that way for the last 6 months. This year alone, we've seen a DDoS attack that peaked at 300 Mpps and another at 480 Gbps. Furthermore, as DDoS mitigation companies like Cloudflare have become adept at handling 'traditional' DDoS attacks, the attackers have also adapted and increasingly try out new techniques.
A holistic Continue reading
Code Everywhere: Why We Built Cloudflare Workers
It all comes down to the speed of light. It always does. The speed of light limits the latency possible between someone using the Internet and the application they are accessing. It doesn’t matter if they are walking down the street hailing a car using a ride-sharing app, sitting in an office accessing a SaaS application on the web, or if their wearable device is reporting health information over WiFi. The speed of light is everywhere.
When you can’t fight the speed of light you only have one possible solution: move closer to where the end users are. In simplistic terms, that’s what Cloudflare has done by building its network of 117 data centers around the world. We’ve cut the latency between users and servers by moving closer.
The Missing Link
But to date all we’ve moved closer are things like SSL handshakes, WAF processing of requests and caching of content. All those things help make Internet applications faster and safer, but there’s a huge missing component... code.
The code that makes Internet applications work is still sequestered in servers and cloud services around the world. And there are only a limited number of such locations even for large cloud Continue reading
Introducing Cloudflare Workers: Run Javascript Service Workers at the Edge
TL;DR: You'll soon be able to deploy Javascript to Cloudflare's edge, written against an API similar to Service Workers.
Try writing a Worker in the playground »
Introduction
Every technology, when sufficiently complicated, becomes programmable.
You see this everywhere, but as a lifelong gamer, my personal favorite example is probably graphics cards. In the '90s, graphics hardware generally provided a fixed set of functionality. The OpenGL standard specified that the geometry pipeline would project points from 3D space onto your viewport, then the raster pipeline would draw triangles between them, with gradient shading and perhaps a texture applied. You could only use one texture at a time. There was only one lighting algorithm, which more or less made every surface look like plastic. If you wanted to do anything else, you often had to give up the hardware entirely and drop back to software.
Of course, new algorithms and techniques were being developed all the time. So, hardware vendors would add the best ideas to their hardware as "extensions". OpenGL ended up with hundreds of vendor-specific extensions to support things like multi-texturing, bump maps, reflections, dynamic shadows, and more.
Then, in 2001, everything changed. The first GPU with a programmable Continue reading
Cloudflare Apps Platform Update: September Edition
This is the September edition of our blog series showcasing the latest platform improvements in developer analytics, user feedback, release notes, and more!
Since launch, we’ve received hundreds of feature requests from developers and users alike. Feedback has been the source of some our most popular features. This month’s post is celebration of the innovation achieved when great ideas are shared.
Let’s dive in!
? Developer Analytics
Continuing with the theme of feedback, App developers can now track their apps’ popularity and growth:
The usage charts help identify which changes have a positive impact on your app.
If you’ve created a paid app you can also track its financial performance:
? User feedback
Charts and graphs are great for tracking trends, but what do your users actually think of your app? Wonder no longer; users can now leave comments when adding and removing apps from their site. Each comment includes sentiment tags and an optional message from the user.
? Page Selectors
Cloudflare users have always been able to select which routes their apps are active, though apps this was too course Continue reading
Introducing Cloudflare Warp: Hide Behind The Edge
I work at a company whose job it is to be attacked. As I’m writing this, an automatic mitigation is fighting two ongoing DDoS attacks. Any machine that’s publicly routable on the internet today can be a vector for attack, and that’s a problem.
Today we want to turn the tables and give you a new way of exposing services to the internet without having them be directly, publicly routable. Meet Cloudflare Warp.
CC BY-SA 2.0 image by Christian Ortiz
Playing Hide and Seek with Bots and Hackers
Cloudflare internally runs about 4,000 containers that make up about 1.5K services and applications. Some of these containers need to network with other local containers, and others need to accept connections over the wire.
Every devops engineer knows that bad things happen to good machines, and so our platform operations team tries to hide servers altogether from the internet. There are several ways to do this:
- Rotate IP addresses
- Deploy proxies
- Create firewall rules
- Configure IP tables
- Limit connections by client certificate
- Cross connect with an upstream provider
- Configure a GRE tunnel
- Authentication mechanisms like OAuth or OIDC
These can be complicated or time consuming, yet none of them are Continue reading
Down the Rabbit Hole: The Making of Cloudflare Warp
In the real world, tunnels are often carved out from the mass of something bigger - a hill, the ground, but also man-made structures.
CC BY-SA 2.0 image by Matt Brown
In an abstract sense Cloudflare Warp is similar; its connection strategy punches a hole through firewalls and NAT, and provides easy and secure passage for HTTP traffic to your origin. But the technical reality is a bit more interesting than this strained metaphor invoked by the name of similar predecessor technologies like GRE tunnels.
Relics
Generic Routing Encapsulation or GRE is a well-supported standard, commonly used to join two networks together over the public Internet, and by some CDNs to shield an origin from DDoS attacks. It forms the basis of the legacy VPN protocol PPTP.
Establishing a GRE tunnel requires configuring both ends of the tunnel to accept the other end’s packets and deciding which IP ranges should be routed through the tunnel. With this in place, an IP packet destined for any address in the configured range will be encapsulated within a GRE packet. The GRE packet is delivered directly to the other end of the tunnel, which removes the encapsulation and forwards the original Continue reading
Introducing Cloudflare Stream: Fixing the Streaming Video Market
Cloudflare turns seven years old today. We launched on September 27, 2010.
It was only a few days after our launch that we got our first request to support video streaming. Yet, until today, we'd avoided it.
Why? Simply put: the video streaming market is screwed up. While there's a lot of money spent on video, there are only really about 1,000 customers that do any meaningful level of streaming.
This is in large part because it's technically far too complicated. If you want to move beyond just uploading your videos to a consumer service like YouTube, then you have to use at least three different services. You need someone to encode your video into a streamable format, you need someone else to act as the content delivery network delivering the bytes, and you need someone else still to provide the player code that runs on the client device. Further, since video encoding standards keep evolving and vary across generations of devices, it becomes challenging to ensure a consistently high quality experience for all visitors.
And if that sounds like a technical mess, the business side is even worse. Encoding companies charge based on CPU usage, which is driven by Continue reading
How Cloudflare Streams
Every day, close to 1 billion people watch video through Cloudflare. That’s 100 petabytes of video every month. Last year, video made up 73% of all internet traffic, which is why earlier today we announced Cloudflare Stream, an end to end video solution designed to bring instant video to any device and network connection. Here’s how Stream works to optimize video upload, encoding and delivery.
Uploads
Using Cloudflare Stream starts with a call to the Stream API to /upload a video.
HTTP does not by default provide a reliable upload mechanism for large files, which can make it tricky to upload large media content like high quality video. If there’s any latency in the network connection, a timeout can cancel the entire upload and require the client to start over.
We use an open source upload protocol called Tus which allows for resumable uploads where the upload previously failed. Tus does this by splitting the download into manageable chunks and tracking completed chunks on the server.
Encodes
Video files are containers that hold the video file, the audio track, and some metadata. The video file is compressed by a codec. The codec is a compression algorithm for encoding video Continue reading
Geo Key Manager: How It Works
Today we announced Geo Key Manager, a feature that gives customers unprecedented control over where their private keys are stored when uploaded to Cloudflare. This feature builds on a previous Cloudflare innovation called Keyless SSL and a novel cryptographic access control mechanism based on both identity-based encryption and broadcast encryption. In this post we’ll explain the technical details of this feature, the first of its kind in the industry, and how Cloudflare leveraged its existing network and technologies to build it.
Keys in different area codes
Cloudflare launched Keyless SSL three years ago to wide acclaim. With Keyless SSL, customers are able to take advantage of the full benefits of Cloudflare’s network while keeping their HTTPS private keys inside their own infrastructure. Keyless SSL has been popular with customers in industries with regulations around the control of access to private keys, such as the financial industry. Keyless SSL adoption has been slower outside these regulated industries, partly because it requires customers to run custom software (the key server) inside their infrastructure.
One of the motivating use cases for Keyless SSL was the expectation that customers may not trust a third party like Cloudflare with their Continue reading
Introducing the Cloudflare Geo Key Manager
Cloudflare’s customers recognize that they need to protect the confidentiality and integrity of communications with their web visitors. The widely accepted solution to this problem is to use the SSL/TLS protocol to establish an encrypted HTTPS session, over which secure requests can then be sent. Eavesdropping is protected against as only those who have access to the “private key” can legitimately identify themselves to browsers and decrypt encrypted requests.
Today, more than half of all traffic on the web uses HTTPS—but this was not always the case. In the early days of SSL, the protocol was viewed as slow as each encrypted request required two round trips between the user’s browser and web server. Companies like Cloudflare solved this problem by putting web servers close to end users and utilizing session resumption to eliminate those round trips for all but the very first request.
Expanding footprint meets geopolitical concerns
As Internet adoption grew around the world, with companies increasingly serving global and more remote audiences, providers like Cloudflare had to continue expanding their physical footprint to keep up with demand. As of the date this blog post was published, Cloudflare has data centers in over 55 countries, and we continue Continue reading
Unmetered Mitigation: DDoS Protection Without Limits
This is the week of Cloudflare's seventh birthday. It's become a tradition for us to announce a series of products each day of this week and bring major new benefits to our customers. We're beginning with one I'm especially proud of: Unmetered Mitigation.
CC BY-SA 2.0 image by Vassilis
Cloudflare runs one of the largest networks in the world. One of our key services is DDoS mitigation and we deflect a new DDoS attack aimed at our customers every three minutes. We do this with over 15 terabits per second of DDoS mitigation capacity. That's more than the publicly announced capacity of every other DDoS mitigation service we're aware of combined. And we're continuing to invest in our network to expand capacity at an accelerating rate.
Surge Pricing
Virtually every Cloudflare competitor will send you a bigger bill if you are unlucky enough to get targeted by an attack. We've seen examples of small businesses that survive massive attacks to then be crippled by the bills other DDoS mitigation vendors sent them. From the beginning of Cloudflare's history, it never felt right that you should have to pay more if you came under an attack. That feels barely a Continue reading
No Scrubs: The Architecture That Made Unmetered Mitigation Possible
When building a DDoS mitigation service it’s incredibly tempting to think that the solution is scrubbing centers or scrubbing servers. I, too, thought that was a good idea in the beginning, but experience has shown that there are serious pitfalls to this approach.
A scrubbing server is a dedicated machine that receives all network traffic destined for an IP address and attempts to filter good traffic from bad. Ideally, the scrubbing server will only forward non-DDoS packets to the Internet application being attacked. A scrubbing center is a dedicated location filled with scrubbing servers.
Three Problems With Scrubbers
The three most pressing problems with scrubbing are: bandwidth, cost, knowledge.
The bandwidth problem is easy to see. As DDoS attacks have scaled to >1Tbps having that much network capacity available is problematic. Provisioning and maintaining multiple-Tbps of bandwidth for DDoS mitigation is expensive and complicated. And it needs to be located in the right place on the Internet to receive and absorb an attack. If it’s not then attack traffic will need to be received at one location, scrubbed, and then clean traffic forwarded to the real server: that can introduce enormous delays with a limited number of locations.
Meet Gatebot – a bot that allows us to sleep
In the past, we’ve spoken about how Cloudflare is architected to sustain the largest DDoS attacks. During traffic surges we spread the traffic across a very large number of edge servers. This architecture allows us to avoid having a single choke point because the traffic gets distributed externally across multiple datacenters and internally across multiple servers. We do that by employing Anycast and ECMP.
We don't use separate scrubbing boxes or specialized hardware - every one of our edge servers can perform advanced traffic filtering if the need arises. This allows us to scale up our DDoS capacity as we grow. Each of the new servers we add to our datacenters increases our maximum theoretical DDoS “scrubbing” power. It also scales down nicely - in smaller datacenters we don't have to overinvest in expensive dedicated hardware.
During normal operations our attitude to attacks is rather pragmatic. Since the inbound traffic is distributed across hundreds of servers we can survive periodic spikes and small attacks without doing anything. Vanilla Linux is remarkably resilient against unexpected network events. This is especially true since kernel 4.4 when the performance of SYN cookies was greatly improved.
But at some point, malicious traffic volume Continue reading
The History of Email
This was adapted from a post which originally appeared on the Eager blog. Eager has now become the new Cloudflare Apps.
QWERTYUIOP
— Text of the first email ever sent, 1971
The ARPANET (a precursor to the Internet) was created “to help maintain U.S. technological superiority and guard against unforeseen technological advances by potential adversaries,” in other words, to avert the next Sputnik. Its purpose was to allow scientists to share the products of their work and to make it more likely that the work of any one team could potentially be somewhat usable by others. One thing which was not considered particularly valuable was allowing these scientists to communicate using this network. People were already perfectly capable of communicating by phone, letter, and in-person meeting. The purpose of a computer was to do massive computation, to augment our memories and empower our minds.
Surely we didn’t need a computer, this behemoth of technology and innovation, just to talk to each other.
The computers which sent (and received) the first email.
The history of computing moves from massive data processing mainframes, to time sharing where many people share one computer, to the diverse collection of personal computing devices Continue reading
A New API Binding: cloudflare-php
Back in May last year, one of my colleagues blogged about the introduction of our Python binding for the Cloudflare API and drew reference to our other bindings in Go and Node. Today we are complimenting this range by introducing a new official binding, this time in PHP.
This binding is available via Packagist as cloudflare/sdk, you can install it using Composer simply by running composer require cloudflare/sdk. We have documented various use-cases in our "Cloudflare PHP API Binding" KB article to help you get started.
Alternatively should you wish to help contribute, or just give us a star on GitHub, feel free to browse to the cloudflare-php source code.
PHP is a controversial language, and there is no doubt there are elements of bad design within the language (as is the case with many other languages). However, love it or hate it, PHP is a language of high adoption; as of September 2017 W3Techs report that PHP is used by 82.8% of all the websites whose server-side programming language is known. In creating this binding the question clearly wasn't on the merits of PHP, but whether we wanted to help drive improvements to the developer experience for Continue reading
Project Jengo Strikes Its First Targets (and Looks for More)
Jango Fett by Brickset (Flickr)
When Blackbird Tech, a notorious patent troll, sued us earlier this year for patent infringement, we discovered quickly that the folks at Blackbird were engaged in what appeared to be the broad and unsubstantiated assertion of patents -- filing about 115 lawsuits in less than 3 years, and have not yet won a single one of those cases on the merits in court. Cloudflare felt an appropriate response would be to review all of Blackbird Tech’s patents, not just the one it asserted against Cloudflare, to determine if they are invalid or should be limited in scope. We enlisted your help in this endeavor by placing a $50,000 bounty on prior art that proves the Blackbird Tech patents are invalid or overbroad, an effort we dubbed Project Jengo.
Since its inception, Project Jengo has doubled in size and provided us with a good amount of high quality prior art submissions. We have received more than 230 submissions so far, and have only just begun to scratch the surface. We have already come across a number of standouts that appear to be strong contenders for invalidating many of the Blackbird Tech patents. This means it is Continue reading
#FuerzaMexico: A way to help Mexico Earthquake victims
Photo Credit: United Nations Photo (Flickr)
On September 19, 1985 Mexico City was hit with the most damaging earthquake in its history. Yesterday -exactly 32 years later- Mexico’s capital and neighbouring areas were hit again by a large earthquake that caused significant damage. While the scale of the destruction is still being assessed, countless people passed away and the lives of many have been disrupted. Today, many heroes are on the streets focusing on recovery and relief.
We at Cloudflare want to make it easy for people to help out those affected in central Mexico. The Mexico Earthquake app will allow visitors to your site to donate to one of the charities helping those impacted.
The Mexico Earthquake App takes two clicks to install and requires no code change. The charities listed are two well respected organizations that are on the ground helping people now.
If you wanted to add your own custom list of charities for disaster relief or other causes, feel free to fork the source of this app and make your own.
#FuerzaMéxico: Una manera de apoyar a los damnificados del SismoMX
El 19 de septiembre de 1985 la Ciudad de México fue afectada por Continue reading
Cloudflare and Google Offer App Developers $100,000 in Cloud Platform Credits
When Cloudflare started, our company needed two things: an initial group of users, and the finances to fund our development. We know most developers face the same issues. The Cloudflare Apps Platform solves the first problem by allowing third parties to develop applications that can be delivered across Cloudflare's edge network to any of the six million sites powered by Cloudflare. The Cloudflare Developer Fund alleviates the second by giving developers the financial support they need to fund their company. Today, we are excited to announce another initiative that will make it possible for developers to make their app dreams a reality.
Cloudflare and Google Cloud are working together to offer developers the resources needed to quickly launch and scale Cloudflare Apps. This partnership will give any Cloudflare Apps developer the chance to access a wide range of benefits including $3k - $100k of Google Cloud Platform (GCP) for one year at no cost. Some startups will also be eligible for 24/7 technical support, and access to GCP’s technical solutions team. This supports a core belief of the Cloudflare Apps initiative: we want developers to focus on building great Apps, not worry about paying for infrastructure. Hundreds of startups have Continue reading