Last week, we wrote about implementing ESI with Cloudflare Workers. This is a guest post by Lukas Reider on how to use ESI not only for better performance, but to optimize availability while migrating backends.

In this post, you will learn about how my client Titel Media was able to use Cloudflare Workers to implement simple edge side includes.

The idea is to partially replace the parts of the online magazine highsnobiety.com with a new, and much more refined frontend implementation. In this article, you will get to know the use case, and how I found a powerful application for Cloudflare Workers.

Backstory

My current project, highsnobiety.com is in the process of replacing Wordpress with a dedicated content pipeline and a custom frontend. It is a huge magazine, with tons of contents, hundreds of daily updates and an international team of more than 60 editors, researching and writing exciting stories.

The company behind it, Titel Media GmbH, a publishing house with offices in Berlin, and New York, surely has grown out of Wordpress for hosting their content.

The show must go on

One does not simply rewrite a sophisticated web publishing pipeline like WordPress. Nor does one, simply Continue reading

This is an adapted transcript of a talk I gave at Promcon 2018. You can find slides with additional information on our Prometheus deployment and presenter notes here. There's also a video.

Tip: you can click on the image to see the original large version.

Here at Cloudflare we use Prometheus to collect operational metrics. We run it on hundreds of servers and ingest millions of metrics per second to get insight into our network and provide the best possible service to our customers.

Prometheus metric format is popular enough, it's now being standardized as OpenMetrics under Cloud Native Computing Foundation. It's exciting to see convergence in long fragmented metrics landscape.

In this blog post we'll talk about how we measure low level metrics and share a tool that can help you to get similar understanding of your systems.

There are two main exporters one can use to get some insight into a Linux system performance.

The first one is node_exporter that gives you information about basics like CPU usage breakdown by type, memory usage, disk IO stats, filesystem and network usage.

The second one is cAdvisor, that gives similar metrics, but drills down to a container level. Instead Continue reading

You can now refresh 1.1.1.1’s DNS cache for domain names by using the purge cache tool. This is useful for domain owners who have updated their DNS records and want to make sure it is reflected for people who are using 1.1.1.1 as their public DNS resolver.

When a client queries for a domain against 1.1.1.1, the resolver returns the IP address from its cache. The cache TTL for a DNS entry is 3 hours. If the host specifies a cache TTL that is shorter than 3 hours, the resolver respects that. This means, when a domain owner changes the DNS host from one to another, in the worst case, she will have to wait for at least 3 hours before the old IP address expires from 1.1.1.1’s cache. With the help of the purge cache tool, a domain owner can now easily refresh 1.1.1.1’s DNS cache and will not have to wait for the cached entry to expire.

To purge a DNS record, you enter the name of your domain, pick the DNS record type and hit the ‘Purge Cache’ button.

You can Continue reading

I grew up with DOS and Windows 3.1. I remember applications being fast - instant feedback or close to it. Today, native applications like Outlook or Apple Mail still feel fast - click compose and the window is there instantly and it feels snappy. Internet applications do not.

My first Internet experience was paying $30 for a prepaid card with 10 hour access over a 14.4k modem. First, it was bulletin boards and later IRC and the WWW. From my small seaside town in Australia, the Internet was a window into the wider world, but it was slooooooow. In a way, it didn’t matter. The world of opportunities the Internet opened up, from information to music, to socializing and ecommerce, who cared if it was slow? The utility of the Internet and Internet applications meant I would use them regardless of the experience.

Performance improved from the 90s, but in 2008 when I switched from Outlook downloading my Yahoo! email over IMAP to Gmail in the browser, it wasn’t because it was faster - it wasn’t - it was because features like search, backed up mail, and unlimited storage were too good to resist. The cloud computing power Continue reading

The following is a guest post by Xavier Lacot, a developer at redirection.io and founder at JoliCode. He works primarily on Web and mobile projects as a consultant, trainer and technical expert.

What is redirection.io

Redirection.io is a Web traffic redirection manager. It provides a collection of tools for website administrators, SEO agencies, and developers, which help analyze HTTP errors, setup HTTP redirections, customize HTTP responses, and monitor the traffic efficiently.

The main part of a traditional redirection.io setup is the proxy, a software component which parses every request to check if a redirection or another response override is required. This "proxy" can be of several types - we provide libraries in several languages - but this setup can be simplified for Cloudflare clients by taking advantage of Cloudflare Workers.

Here come Cloudflare Workers

Earlier this year, Cloudflare unveiled its Workers product, a smart way of running code on the edge of Cloudflare locations. This computing feature is particularly interesting, as it allows performing several traffic operations without requiring any change on your own platform, code, or infrastructure: just enable Workers, write some code, and let Cloudflare handle the magic ✨

In practical terms, Workers Continue reading

A few days ago, Cloudflare — along with the rest of the world — learned of a "practical" cache poisoning attack. In this post I’ll walk through the attack and explain how Cloudflare mitigated it for our customers. While any web cache is vulnerable to this attack, Cloudflare is uniquely able to take proactive steps to defend millions of customers.

In addition to the steps we’ve taken, we strongly recommend that customers update their origin web servers to mitigate vulnerabilities. Some popular vendors have applied patches that can be installed right away, including Drupal, Symfony, and Zend.

How a shared web cache works

Say a user requests a cacheable file, index.html. We first check if it’s in cache, and if it’s not not, we fetch it from the origin and store it. Subsequent users can request that file from our cache until it expires or gets evicted.

Although contents of a response can vary slightly between requests, customers may want to cache a single version of the file to improve performance:

(See this support page for more info about how to cache HTML with Cloudflare.)

How do we know it’s the same file? We create something Continue reading

At Cloudflare we’re accelerating web assets in a number of different ways. Part of this is caching, by storing the response given by the origin server directly within our 151+ global data centers. This will dramatically improve the delivery of the resources as the visitor will directly get them from the data center closest to them, instead of waiting for us to fetch the request from the origin web server.

The issue with dynamic (but not a lot) pages

The subject we’re gonna cover today is the concept of Edge-Side-Includes. And what’s better than a real use-case to introduce what it is used for? Let’s take a website where all pages are including advertisements at the head and bottom. Could we consider these pages static? We couldn’t as at least part of this page is dynamic. Could we consider caching it? That’s a no again as it would mean the first dynamic part rendered will be cached and served for the other visitors trying to get the page. It would be a catastrophe if the advertisements are user-specific.

So the issue here is that we can’t cache the page. That’s quite a shame as it means that we’ll fetch Continue reading

Looking back at our historical data, we realized how much the Internet and Cloudflare grew. With more than 150 datacenters, 10 percent of web-based applications, customers everywhere around the world, from the tiny islands in the Pacific to the big metropolises, we have an Internet landscape of almost every country and continent.

Cloudflare’s mission is to help build a better Internet. To do that we operate datacenters across the globe. By having datacenters close to end user we provide a fast, secure experience for everyone. Today I’d like to talk about our datacenters in Africa and our plans to serve a population of 1.2 billion people over 58 countries.

Internet penetration in developed countries skyrocketed since the 2000s, Internet usage is growing rapidly across Africa. We are seeing a 4% to 7% increase in traffic month on month. As of July 2018, we have 8 datacenters on the African Continent:

• Angola
• Djibouti
• Egypt
• Kenya
• Continue reading

Recently, Google officially launched Android 9 Pie, which includes a slew of new features around digital well-being, security, and privacy. If you’ve poked around the network settings on your phone while on the beta or after updating, you may have noticed a new Private DNS Mode now supported by Android.

This new feature simplifies the process of configuring a custom secure DNS resolver on Android, meaning parties between your device and the websites you visit won’t be able to snoop on your DNS queries because they’ll be encrypted. The protocol behind this, TLS, is also responsible for the green lock icon you see in your address bar when visiting websites over HTTPS. The same technology is useful for encrypting DNS queries, ensuring they cannot be tampered with and are unintelligible to ISPs, mobile carriers, and any others in the network path between you and your DNS resolver. These new security protocols are called DNS over HTTPS, and DNS over TLS.

Configuring 1.1.1.1

Android Pie only supports DNS over TLS. To enable this on your device:

1. Go to Settings → Network & internet → Advanced → Private DNS.
2. Select the Private DNS provider hostname option.
3. Enter Continue reading

Since leaving beta three weeks ago, Cloudflare Access has become our fastest-growing subscription service. Every day, more teams are using Access to leave their VPN behind and connect to applications quickly and securely from anywhere in the world.

We’ve heard from a number of teams about how they’re using Access. Each team has unique needs to consider as they move away from a VPN and to a zero trust model. In a zero trust framework, each request has to prove that a given application should trust its attempt to reach a secure tool. In this post, we’re highlighting some of the solutions that groups are using to transition to Cloudflare Access.

Solution 1: Collaborate with External Partners

Cloudflare Access integrates with popular identity providers (IdPs) so that your team can reach internal applications without adding more credentials. However, teams rarely work in isolation. They frequently rely on external partners who also need to reach shared tools.

How to grant and manage permissions with external partners poses a security risk. Just because you are working with a third-party doesn’t mean they should have credentials to your IdP. They typically need access to a handful of tools, not all of your internal Continue reading

For the last five years, the Internet Engineering Task Force (IETF), the standards body that defines internet protocols, has been working on standardizing the latest version of one of its most important security protocols: Transport Layer Security (TLS). TLS is used to secure the web (and much more!), providing encryption and ensuring the authenticity of every HTTPS website and API. The latest version of TLS, TLS 1.3 (RFC 8446) was published today. It is the first major overhaul of the protocol, bringing significant security and performance improvements. This article provides a deep dive into the changes introduced in TLS 1.3 and its impact on the future of internet security.

An evolution

One major way Cloudflare provides security is by supporting HTTPS for websites and web services such as APIs. With HTTPS (the “S” stands for secure) the communication between your browser and the server travels over an encrypted and authenticated channel. Serving your content over HTTPS instead of HTTP provides confidence to the visitor that the content they see is presented by the legitimate content owner and that the communication is safe from eavesdropping. This is a big deal in a world where online privacy Continue reading

In February, Troy Hunt unveiled Pwned Passwords v2. Containing over half a billion real world leaked passwords, this database provides a vital tool for correcting the course of how the industry combats modern threats against password security.

In supporting this project; I built a k-Anonymity model to add a layer of security to performed queries. This model allows for enhanced caching by mapping multiple leaked password hashes to a single hash prefix and additionally being performed in a deterministic HTTP-friendly way (which allows caching whereas other implementations of Private Set Intersection require a degree of randomness).

Since launch, PwnedPasswords, using this anonymity model and delivered by Cloudflare, has been implemented in a widespread way across a wide variety of platforms - from site like EVE Online and Kogan to tools like 1Password and Okta's PassProtect. The anonymity model is also used by Firefox Monitor when checking if an email is in a data breach.

Since it has been adopted, Troy has tweeted out about the high cache hit ratio; and people have been asking me about my "secret ways" of gaining such a high cache hit ratio. Over time I touched various pieces of Cloudflare's caching systems; in late 2016 Continue reading

It’s our pleasure to announce the general availability of Cloudflare Stream. Cloudflare Stream is the best way for any founder or developer to deliver an extraordinary video experience to their viewers while cutting development time and costs, and as of today it is available to every Cloudflare customer.

If I had to summarize what we’ve learned as we’ve built Stream it would be: Video streaming is hard, but building a successful video streaming business is even harder. This is why our goal has been to take away the complexity of encoding, storage, and smooth delivery so you can focus on all the other critical parts of your business.

Cloudflare Stream API

You call a single endpoint, Cloudflare Stream delivers a high-quality streaming experience to your visitors. Here’s how it works:

1. Your app calls the /stream endpoint to upload a video. You can submit the contents of the video with the request or you can provide a URL to a video hosted elsewhere.
2. Cloudflare Stream encodes the stream in multiple resolutions to enable multi-bitrate streaming. We also automatically prepare DASH and HLS manifest files.
3. Cloudflare serves your video (in multiple resolutions) from our vast network of 150+ data centers around the Continue reading