Cloudflare goes InterPlanetary – Introducing Cloudflare’s IPFS Gateway
Today we’re excited to introduce Cloudflare’s IPFS Gateway, an easy way to access content from the InterPlanetary File System (IPFS) that doesn’t require installing and running any special software on your computer. We hope that our gateway, hosted at cloudflare-ipfs.com, will serve as the platform for many new highly-reliable and security-enhanced web applications. The IPFS Gateway is the first product to be released as part of our Distributed Web Gateway project, which will eventually encompass all of our efforts to support new distributed web technologies.
This post will provide a brief introduction to IPFS. We’ve also written an accompanying blog post describing what we’ve built on top of our gateway, as well as documentation on how to serve your own content through our gateway with your own custom hostname.
Quick Primer on IPFS
Usually, when you access a website from your browser, your browser tracks down the origin server (or servers) that are the ultimate, centralized repository for the website’s content. It then sends a request from your computer to that origin server, wherever it is in the world, and that server sends the content back to your computer. This system has served the Internet well for decades, Continue reading
Welcome to Crypto Week
The Internet is an amazing invention. We marvel at how it connects people, connects ideas, and makes the world smaller. But the Internet isn’t perfect. It was put together piecemeal through publicly funded research, private investment, and organic growth that has left us with an imperfect tapestry. It’s also evolving. People are constantly developing creative applications and finding new uses for existing Internet technology. Issues like privacy and security that were afterthoughts in the early days of the Internet are now supremely important. People are being tracked and monetized, websites and web services are being attacked in interesting new ways, and the fundamental system of trust the Internet is built on is showing signs of age. The Internet needs an upgrade, and one of the tools that can make things better, is cryptography.
Every day this week, Cloudflare will be announcing support for a new technology that uses cryptography to make the Internet better. Everything we are announcing this week is free to use and provides a meaningful step towards supporting a new capability or structural reinforcement. So why are we doing this? Because it’s good for the users and good for the Internet. Welcome to Crypto Week!
A more Continue reading
JAMstack podcast episode: Listen to Cloudflare’s Kenton Varda speak about originless code
JAMstack Radio is a show all about the JAMstack, a new way to build fast & secure apps or websites. In the most recent episode, the host, Brian Douglas, met with Kenton Varda, tech lead for Cloudflare Workers and author of Sandstorm.io to discuss some of the infinite uses for running code at the edge.
Listen to what Kenton had to say about serverless technology in this twenty two minute podcast here:
Here's the transcript of the podcast as well:
Brian Douglas: Welcome to another installment of JAMstack Radio. In the room I've got Kenton Varda from Cloudflare.
Kenton Varda: Thanks for having me.
Brian: Thanks for coming all the way across San Francisco to chat with me in person. I'm curious who Kenton is, but I'm also curious what Cloudflare is. Can you answer both questions? Let's start with, "Who is Kenton?"
Kenton: I'm an engineer. I'm the architect of Cloudflare Workers. In a past life I worked for Google for several years. I was once known as the "protocol buffers guy," I was the one who open sourced that. And I founded a company called Sandstorm that was later acquired by Cloudflare.
Brian: I'm Continue reading
Cache API for Cloudflare Workers is now in Beta!
In October of last year we announced the launch of Cloudflare Workers. Workers allows you to run JavaScript from 150+ of Cloudflare’s data centers. This means that from the moment a request hits the Cloudflare network, you have full control over its destiny. One of the benefits of using Workers in combination with Cloudflare’s cache is that Workers allow you to have programmatic, and thus very granular control over the Cloudflare cache.
You can choose what to cache, how long to cache it for, the source it should be cached from, and you can even modify the cached result after it is retrieved from the cache.
We have seen many of our existing customers use Workers to enhance their usage of the Cloudflare cache, and we have seen many new customers join Cloudflare to take advantage of these unique benefits.
(Re-)Introducing the Cache API
You can always have more control, so today we are announcing support for the Cache API! As some of you may know, Cloudflare Workers are built against the existing Service Worker APIs. One of the reasons we originally chose to model Cloudflare Workers after Service Workers was due to the existing familiarity and audience of Service Continue reading
Deploy Workers using Terraform
Today we're excited to announce that Cloudflare Workers are now supported in the Cloudflare Terraform Provider.
Terraform is a fantastic tool for configuring your infrastructure. Traditionally if you wanted to spin up, tear down or update some of your infrastructure you would have to click around on a website or make some API calls, which is prone to human error. With Terraform, you define your infrastructure in simple, declarative configuration files and let Terraform figure out how to make the API calls for you. This also lets you treat your infrastructure like your code. You can check your Terraform configuration files into version control and integrate them into your normal software development workflow.
Terraform integrates with many infrastructure providers including Cloudflare. If you'd like to read more about setting up Terraform with Cloudflare, check out Getting started with Terraform and Cloudflare. In this post, I'm going to focus specifically on how to integrate Cloudflare Workers with Terraform.
In this example we're going to create partyparrot.business, and we're going to serve the whole site out of a worker without any origin server. We're starting from scratch here, but if you're already using Cloudflare workers and want to migrate to Continue reading
Norfolk and Richmond, Virginia: Cloudflare’s 152nd and 153rd cities
Virginia has a very important place in Internet history, as well as the history of Cloudflare’s network.
Northern Virginia, in the area around Ashburn VA, has for a long time been core to Internet infrastructure. In the early 1990’s, MAE-East (Metropolitan-Area-Exchange East) , an Internet Exchange Point (IXP) was established. MAE-East and West were some of the earliest IXPs. Internet Exchange Points are crucial interconnection points for ISPs and other Internet Networks to interconnect and exchange traffic. Eco-systems have grown around these through new data center offerings and new Internet platforms. Like many pieces of the Internet, MAE-East had a humble beginning, though not many humble beginnings grew to handle around 50% of Internet traffic exchange.
Cloudflare’s second Data Center, and one that still plays a critical component in our Global Network was Ashburn, Virginia. Similarly across many organizations, the Northern Virginia area has become a Data Center mecca. Many of the largest Clouds have a substantial amount of their footprint in Northern Virginia. Although MAE-East no longer exists, other Internet Exchange Points have come and grown in its place.
Cloudflare’s network has grown beyond what was traditional Interconnection points, like Ashburn/Northern VA, to a new Edge of the Continue reading
Build and Deploy Functions to Cloudflare’s 152+ Data Centers with Serverless
We’re very excited to announce that Cloudflare Workers are now integrated into the Serverless framework as a serverless cloud provider! Serverless’ open source framework has become a must-have for many developers, and we want to make it as simple as possible to harness the power of Cloudflare’s distributed computing network.
Workers have become essential to the way people build applications on the web. The expressiveness of modern JavaScript combined with sub-30 second deploys to Cloudflare’s network of 152+ datacenters means that you can truly build your application into our global compute network. Up until this point, deploying Workers required doing all of your editing through our browser-based IDE or developing one’s own custom tooling on top of our API. But many developers have their own environments and are already familiar with the Serverless framework, so it seemed natural that we build first-class support for Workers into Serverless!
You can now define the entire structure and routing behavior of your Workers scripts in code and deploy them with ease using serverless deploy from your own development environment. Store configuration files in version control alongside your application code. And feel more confident testing your application with serverless invoke, a new way Continue reading
EU Copyright Vote: A Critical Juncture for the Open Internet
Back in June, we blogged about the draft EU copyright proposal which is currently making its way through the legislative process in Brussels. We outlined how under one of the more controversial provisions within the draft Directive, Article 13, certain Internet platforms could be held legally responsible for any copyright content that their users upload and would effectively have to turn to automated filtering solutions to remove infringing content at the point of user upload. Moreover, in order to avoid potential legal liability, it is widely expected that content sharing providers would err on the side of caution and remove excessive amounts of content, resulting in a form of online censorship.
Since that blogpost, the European Parliament Plenary narrowly voted on 5th July to reject the proposal tabled by the Legal Affairs (JURI) Committee and a mandate to negotiate, and now the proposed Directive will undergo a full discussion and rescheduled vote in the next Plenary meeting on 12th September. This was a fantastic outcome, thanks in large part to a groundswell of support from those who value the fundamental right of freedom of expression online. It has presented a window of opportunity to correct the deeply flawed approach to Continue reading
Fixing an old hack – why we are bumping the IPv6 MTU
Back in 2015 we deployed ECMP routing - Equal Cost Multi Path - within our datacenters. This technology allowed us to spread traffic heading to a single IP address across multiple physical servers.
You can think about it as a third layer of load balancing.
- First we split the traffic across multiple IP addresses with DNS.
- Then we split the traffic across multiple datacenters with Anycast.
- Finally, we split the traffic across multiple servers with ECMP.
When deploying ECMP we hit a problem with Path MTU discovery. The ICMP packets destined to our Anycast IP's were being dropped. You can read more about that (and the solution) in the 2015 blog post Path MTU Discovery in practice.
To solve the problem we created a small piece of software, called pmtud (https://github.com/cloudflare/pmtud). Since deploying pmtud, our ECMP setup has been working smoothly.
Hardcoding IPv6 MTU
During that initial ECMP rollout things were broken. To keep services running until pmtud was done, we deployed a quick hack. We reduced the MTU of IPv6 traffic to the minimal possible value: 1280 bytes.
This was done as a tag on a default route. This is Continue reading
Bob Ross, Lorem Ipsum, Heroku and Cloudflare Workers
It may not be immediately obvious how these things are related, but bear with me... It was 4pm Friday and one of the engineers on the Cloudflare Tools team came to me with an emergency. "Steve! The Bob Ross Ipsum generator is down!".
If you've not heard of Lorem Ipsum, it's an extract from a latin poem that designers use as placeholder text when designing the layout of a document. There are generators all over the web that will spit out as much text as you need.
Of course, the web being the web that we all love, there are also endless parodies of Lorem Ipsum. You can generate Hodor Ipsum, Cat Ipsum and Hipster Ipsum. I have a new, undisputed favourite: Bob Ross Ipsum.
Not growing up in the U.S., I hadn't come across the lovable, calm, serene and beautiful human that is Bob Ross. If you haven't spent 30 mins watching him paint a landscape, you should do that now. He built a following as host of the TV show “The Joy of Painting” which ran on the U.S. PBS channel from 1983-1994. He became famous for Continue reading
Introducing Real World Serverless – Practical advice on how to use Cloudflare Workers
We’re getting the best minds on serverless technology from Cloudflare together to lead a series of talks on practical use cases for Cloudflare Workers. Join any of these six global talks for stories of how companies and developers are using serverless in the real world.
San Francisco - London - Austin - Singapore - Sydney - Melbourne
Want a Real World Serverless event in your city? Interested in sharing your stories and experience deploying serverless apps in production? Email [email protected] and let’s put something together.
Check out the event details and register through the Eventbrite links below.
Real World Serverless - San Francisco
Sept 11th, 2018, 6:00pm-9:00pm
In partnership with Serverless Meetup
Location: Heavybit - 325 9th St, San Francisco, CA 94103
View Event Details & Register Here »
Real World Serverless - London
Sept 18th, 2018, 6:00pm-9:00pm
Location: Cloudflare London - 25 Lavington St, Second floor SE1 0NZ London
View Event Details & Register Here »
Real World Serverless - Austin
October 2nd, 2018, 6:00pm-9:00pm
In partnership with ATX Serverless Meetup
Location: Downtown Austin
Website Security Myths
Some conversations are easy; some are difficult. Some are harmonious and some are laborious. But when it comes to website security, the conversation is confusing.
Every organisation agrees, in theory, that their websites need to be secure. But in practice, there is resistance to investing enough time and budget. Reasons for neglecting security include misconceptions surrounding Web Application security.
Below I’ve outlined some of the most common myths and misconceptions that can often put your website at serious security risks.
My website is not the target of an attack because it is small and I run a small business.
An average small business website is attacked 44 times per day. In addition, a low profile website is a nice playground for hackers to try out new tools and techniques. Hackers often use automated tools to find various vulnerable websites and don't discriminate when it comes to the size of the target. Any web application, even if it is not itself a target, may be of interest to attackers. Web applications with lax security are easy pickings for hackers and can be subject to a mass or targeted cyber attack.
The good news is that Continue reading
Welcome, WP Engine!
We’ve had the tremendous pleasure of working with WP Engine for nearly 5 years, starting when both companies employed less than 100 people in total. From the beginning, we noticed striking similarities between our two companies—both were founded in 2010, both are incredibly passionate about their customers’ success, and both strive to make their technology as simple and accessible as possible. Fast forward to 2018: with WP Engine already leveraging Cloudflare for DNS, thousands of mutual WP Engine and Cloudflare customers, and millions of WordPress websites already protected behind Cloudflare, it was a no-brainer to formally partner together.
Today, we are thrilled to announce WP Engine as a Cloudflare partner! The joint offering, Global Edge Security powered by Cloudflare, integrates WP Engine’s platform with Cloudflare’s managed web application firewall (WAF), advanced distributed denial of service mitigation (DDoS), SSL/TLS encryption, and CDN across a global edge network to deliver the world’s most secure and scalable digital experience on WordPress today.
We couldn’t be more excited about our opportunity to collaborate with WP Engine to deploy business-critical security and CDN edge services to Enterprises and SMBs globally.
Protection from Struts Remote Code Execution Vulnerability (S2-057)
On August 22 a new vulnerability in the Apache Struts framework was announced. It allows unauthenticated attackers to perform Remote Code Execution (RCE) on vulnerable hosts.
As security researcher Man Yue Mo explained, the vulnerability has similarities with previous Apache Struts vulnerabilities. The Cloudflare WAF already mitigated these so adjusting our rules to handle the new vulnerability was simple. Within hours of the disclosure we deployed a mitigation with no customer action required.
OGNL, again
Apache Struts RCE payloads often come in the form of Object-Graph Navigation Library (OGNL) expressions. OGNL is a language for interacting with the properties and functions of Java classes and Apache Struts supports it in many contexts.
For example, the snippet below uses OGNL to dynamically insert the value "5" into a webpage by calling a function.
<s:property value="%{getSum(2,3)}" />
OGNL expressions can also be used for more general code execution:
${
#_memberAccess["allowStaticMethodAccess"]=true,
@java.lang.Runtime@getRuntime().exec('calc')
}
Which means if you can find a way to make Apache Struts execute a user supplied OGNL expression, you've found an RCE vulnerability. Security researchers have found a significant number of vulnerabilities where this was the root cause.
What’s different this time?
The major difference between Continue reading
Porting Our Software to ARM64
As we enable more ARM64[1] machines in our network, I want to give some technical insight into the process we went through to reach software parity in our multi-architecture environment.
To give some idea of the scale of this task, it’s necessary to describe the software stack we run on our servers. The foundation is the Linux kernel. Then, we use the Debian distribution as our base operating system. Finally, we install hundreds of packages that we build ourselves. Some packages are based on open-source software, often tailored to better meet our needs. Other packages were written from scratch within Cloudflare.
Industry support for ARM64 is very active, so a lot of open-source software has already been ported. This includes the Linux kernel. Additionally, Debian made ARM64 a first-class release architecture starting with Stretch in 2017. This meant that upon obtaining our ARM64 hardware, a few engineers were able to bring Debian up quickly and smoothly. Our attention then turned to getting all our in-house packages to build and run for ARM64.
Our stack uses a diverse range of programming languages, including C, C++, Go, Lua, Python, and Rust. Different languages have different porting requirements, with some being easier Continue reading
Identifying and alerting on data loss using Cloudflare Workers
Photo by Markus Spiske / Unsplash
You hear about data breaches almost every day in the news these days. New regulations, such as GDPR, require companies to disclose data breaches within 72 hours of becoming aware. Becoming aware of and identifying data breaches as they happen, however, is not an easy task. It is often challenging for companies to become aware of their own data breaches and losses well-before they get picked up by the media.
One symptom of a data breach is data (such as passwords or PII) that should never leave internal systems making its way through an HTTP response into the public Internet. Since Cloudflare Workers sits between your infrastructure and the public for any endpoints exposed to the Internet, Workers can be used as a way of alerting you of canary data leaving.
In the following example, we will be inspecting the content of each response, checking to see if our canary data has leaked out, and if so, returning a static response and calling the PagerDuty API to notify of a potential breach.
Detecting Data Loss
In this example, we’ll be looking for a particular string in the body of the response. This string can Continue reading
Building a Diverse Business Development Team in EMEA
Cloudflare’s mission is to help build a better Internet, and we are also serious about building a diverse workforce where everyone can be themselves, regardless of their gender, sexual orientation, or cultural background.
The Recruiting team sat down with Anil Karavadra, EMEA Head of Business Development, to find out how he has built a diverse team since he joined.
the EMEA team pictured 1st August, some members attending Orientation
Recruiting Team: First question Anil, how did you end up at Cloudflare?
Anil Karavadra: Funny story about this--I was actually approached by my boss! He reached out to me and said “Hey would you be interested in joining Cloudflare?”, and initially I said no because I hadn’t heard of Cloudflare before! Then he suggested “Well how about we share some best practices?”, and I said okay, although I knew he was going to try and pitch Cloudflare to me. After I had a chat with him and he shared his vision to grow the EMEA BDR organisation. This sounded interesting to me so I then went through the interview process and got through. Now that I’m here I love it.
RT: How long have you been Continue reading
Using Edge-Side Includes with Workers for High Availability
Last week, we wrote about implementing ESI with Cloudflare Workers. This is a guest post by Lukas Reider on how to use ESI not only for better performance, but to optimize availability while migrating backends.
In this post, you will learn about how my client Titel Media was able to use Cloudflare Workers to implement simple edge side includes.
The idea is to partially replace the parts of the online magazine highsnobiety.com with a new, and much more refined frontend implementation. In this article, you will get to know the use case, and how I found a powerful application for Cloudflare Workers.
Backstory
My current project, highsnobiety.com is in the process of replacing Wordpress with a dedicated content pipeline and a custom frontend. It is a huge magazine, with tons of contents, hundreds of daily updates and an international team of more than 60 editors, researching and writing exciting stories.
The company behind it, Titel Media GmbH, a publishing house with offices in Berlin, and New York, surely has grown out of Wordpress for hosting their content.
The show must go on
One does not simply rewrite a sophisticated web publishing pipeline like WordPress. Nor does one, simply Continue reading
Using Workers To Make Static Sites Dynamic
The following is a guest post by Paddy Sherry, Lead Developer at Gambling.com Group. They build performance marketing websites and tools, using Cloudflare to serve to their global audience. Paddy is a Web Performance enthusiast with an interest in Serverless Computing.
Choosing technology that is used on a large network of sites is a key architectural decision that must be correct. We build static websites but needed to find a way to make them dynamic to do things like geo targeting, restrict access and A/B testing. This post shares our experiences on what we learned when using Workers to tackle these challenges.
Our Background
At Gambling.com Group, we use Cloudflare on all of our sites so our curiosity level in Workers was higher than most. We are big fans of static websites because nothing is faster than flat HTML. We had been searching for a technology like this for some time and applied to be part of the beta program, so were one of the first to gain access to the functionality.
The reason we were so keen to experiment with Workers is that for anyone running static sites, 99% of the time, the product requirements Continue reading
Introducing ebpf_exporter
This is an adapted transcript of a talk I gave at Promcon 2018. You can find slides with additional information on our Prometheus deployment and presenter notes here. There's also a video.
Tip: you can click on the image to see the original large version.
Here at Cloudflare we use Prometheus to collect operational metrics. We run it on hundreds of servers and ingest millions of metrics per second to get insight into our network and provide the best possible service to our customers.
Prometheus metric format is popular enough, it's now being standardized as OpenMetrics under Cloud Native Computing Foundation. It's exciting to see convergence in long fragmented metrics landscape.
In this blog post we'll talk about how we measure low level metrics and share a tool that can help you to get similar understanding of your systems.
There are two main exporters one can use to get some insight into a Linux system performance.
The first one is node_exporter that gives you information about basics like CPU usage breakdown by type, memory usage, disk IO stats, filesystem and network usage.
The second one is cAdvisor, that gives similar metrics, but drills down to a container level. Instead Continue reading