Introducing Advanced DDoS Alerts
We’re pleased to introduce Advanced DDoS Alerts. Advanced DDoS Alerts are customizable and provide users the flexibility they need when managing many Internet properties. Users can easily define which alerts they want to receive — for which DDoS attack sizes, protocols and for which Internet properties.
This release includes two types of Advanced DDoS Alerts:
- Advanced HTTP DDoS Attack Alerts - Available to WAF/CDN customers on the Enterprise plan, who have also subscribed to the Advanced DDoS Protection service.
- Advanced L3/4 DDoS Attack Alerts - Available to Magic Transit and Spectrum BYOIP customers on the Enterprise plan.
Standard DDoS Alerts are available to customers on all plans, including the Free plan. Advanced DDoS Alerts are part of Cloudflare’s Advanced DDoS service.
Why alerts?
Distributed Denial of Service attacks are cyber attacks that aim to take down your Internet properties and make them unavailable for your users. As early as 2017, Cloudflare pioneered the Unmetered DDoS Protection to provide all customers with DDoS protection, without limits, to ensure that their Internet properties remain available. We’re able to provide this level of commitment to our customers thanks to our automated DDoS protection systems. But if the systems operate automatically, why even be Continue reading
Introducing Cloudflare Adaptive DDoS Protection – our new traffic profiling system for mitigating DDoS attacks
Every Internet property is unique, with its own traffic behaviors and patterns. For example, a website may only expect user traffic from certain geographies, and a network might only expect to see a limited set of protocols.
Understanding that the traffic patterns of each Internet property are unique is what led us to develop the Adaptive DDoS Protection system. Adaptive DDoS Protection joins our existing suite of automated DDoS defenses and takes it to the next level. The new system learns your unique traffic patterns and adapts to protect against sophisticated DDoS attacks.
Adaptive DDoS Protection is now generally available to Enterprise customers:
- HTTP Adaptive DDoS Protection - available to WAF/CDN customers on the Enterprise plan, who have also subscribed to the Advanced DDoS Protection service.
- L3/4 Adaptive DDoS Protection - available to Magic Transit and Spectrum customers on an Enterprise plan.
Adaptive DDoS Protection learns your traffic patterns
The Adaptive DDoS Protection system creates a traffic profile by looking at a customer’s maximal rates of traffic every day, for the past seven days. The profiles are recalculated every day using the past seven-day history. We then store the maximal traffic rates seen for every predefined dimension value. Every profile Continue reading
Account WAF now available to Enterprise customers
Do you manage more than a single domain? If the answer is yes, now you can manage a single WAF configuration for all your enterprise domains.
Cloudflare has been built around the concept of zone, which is broadly equivalent to a domain. Customers can add multiple domains to a Cloudflare account, and every domain has its own independent security configuration. If you deploy a rule to block bots on example.com, you will need to rewrite the same rule on example.org. You’ll then need to visit the dashboard of every zone when you want to update it. This applies to all WAF products including Managed, Firewall and Rate Limiting rules.
If you have just two domains that’s not a big deal. But if you manage hundreds or thousands of domains like most large organizations do. Dealing with individual domains becomes time-consuming, expensive or outright impractical. Of course, you could build automation relying on our API or Terraform. This will work seamlessly but not all organizations have the capabilities to manage this level of complexity. Furthermore, having a Terraform integration doesn’t fully replicate the experience or give the confidence provided by interacting with a well-designed UI.
Following Cloudflare Continue reading
Improved Access Control: Domain Scoped Roles are now generally available
Starting today, it is possible to scope your users’ access to specific domains with Domain Scoped Roles becoming generally available!
We are making it easier for account owners to manage their team’s access to Cloudflare by allowing user access to be scoped to individual domains. Ensuring users have the least amount of access they need and no more is critical, and Domain Scoped Roles is a major step in this direction. Additionally, with the use of Domain Groups, account owners can grant users access to a group of domains instead of individually. Domains can be added or removed from these groups to automatically update the access of those who have been granted access to the group. This reduces toil in managing user access.
One of the most common uses we have seen for Domain Scoped Roles is to limit access to production domains to a small set of team members, while still allowing development and pre-production domains to be open to the rest of the team. That way, someone can’t make changes to a production domain unless they are given access.
We are doing a rollout of this functionality across all Enterprise Cloudflare accounts, and you will receive an email Continue reading
Cloudforce One is now generally available: empower your security team with threat data, tooling, and access to industry experts
Cloudflare’s threat operations and research team, Cloudforce One, is now open for business and has begun conducting threat briefings. Access to the team is available via an add-on subscription, and includes threat data and briefings, security tools, and the ability to make requests for information (RFIs) to the team.
Fill out this form or contact your account team to learn more.
Subscriptions come in two packages, and are priced based on number of employees: “Premier” includes our full history of threat data, bundled RFIs, and an API quota designed to support integrations with SIEMs. “Core” level includes reduced history and quotas. Both packages include access to all available security tools, including a threat investigation portal and sinkholes-as-a-service.
If you’re an enterprise customer interested in understanding the type of threat briefings that Cloudforce One customers receive, you can register here for “YackingYeti: How a Russian threat group targets Ukraine—and the world”, scheduled for October 12. The briefing will include Q&A with Blake Darché, head of Cloudforce One, and an opportunity to learn more about the team and offering.
Requests for Information (RFIs) and Briefings
The Cloudforce One team is composed of analysts assigned to five subteams: Malware Analysis, Threat Continue reading
Welcome to GA Week
Cloudflare ships a lot of products. Some of those products are shipped as beta, sometimes open, sometimes closed, and our huge customer base gives those betas an incredible workout. Making products work at scale, and in the heterogeneous environment of the real Internet is a challenge. We’re lucky to have so many enthusiastic customers ready to try out our betas.
And when those products exit beta they’re GA or Generally Available. This week you’ll be hearing a lot about products becoming GA.
But it’s not just about making products work and be available, it’s about making the best-of-breed. We ship early and iterate rapidly. We’ve done this over the years for WAF, DDoS mitigation, bot management, API protection, CDN and our developer platform. Today analyst firms such as Gartner, Forrester and IDC recognize us as leaders in all those areas.
That’s one reason we’re trusted by the likes of Broadcom, NCR, DHL Parcel, Panasonic, Canva, Shopify, L'Oréal, DoorDash, Garmin and more.
Over the years we’ve heard criticism that we’re the new kid on the block. The latest iteration of that is Zero Trust vendors seeing us as novices. It sounds all too familiar. It’s what the Continue reading
Using Cloudflare R2 as an apt/yum repository
In this blog post, we’re going to talk about how we use Cloudflare R2 as an apt/yum repository to bring cloudflared (the Cloudflare Tunnel daemon) to your Debian/Ubuntu and CentOS/RHEL systems and how you can do it for your own distributable in a few easy steps!
I work on Cloudflare Tunnel, a product which enables customers to quickly connect their private networks and services through the Cloudflare global network without needing to expose any public IPs or ports through their firewall. Cloudflare Tunnel is managed for users by cloudflared, a tool that runs on the same network as the private services. It proxies traffic for these services via Cloudflare, and users can then access these services securely through the Cloudflare network.
Our connector, cloudflared, was designed to be lightweight and flexible enough to be effectively deployed on a Raspberry Pi, a router, your laptop, or a server running on a data center with applications ranging from IoT control to private networking. Naturally, this means cloudflared comes built for a myriad of operating systems, architectures and package distributions: You could download the appropriate package from our GitHub releases, brew install it or apt/yum install it (https://pkg.cloudflare. Continue reading
How we built Pingora, the proxy that connects Cloudflare to the Internet
Introduction
Today we are excited to talk about Pingora, a new HTTP proxy we’ve built in-house using Rust that serves over 1 trillion requests a day, boosts our performance, and enables many new features for Cloudflare customers, all while requiring only a third of the CPU and memory resources of our previous proxy infrastructure.
As Cloudflare has scaled we’ve outgrown NGINX. It was great for many years, but over time its limitations at our scale meant building something new made sense. We could no longer get the performance we needed nor did NGINX have the features we needed for our very complex environment.
Many Cloudflare customers and users use the Cloudflare global network as a proxy between HTTP clients (such as web browsers, apps, IoT devices and more) and servers. In the past, we’ve talked a lot about how browsers and other user agents connect to our network, and we’ve developed a lot of technology and implemented new protocols (see QUIC and optimizations for http2) to make this leg of the connection more efficient.
Today, we’re focusing on a different part of the equation: the service that proxies traffic between our network and servers on the Internet. This proxy Continue reading
Ethereum Gateway support for Görli + Sepolia Testnets and the Ethereum Merge
Today we are excited to announce support for the Ethereum Merge on the Ethereum network and that our Ethereum gateways now support the Görli and Sepolia test networks (testnets). Sepolia and Görli testnets can be used to test and develop full decentralized applications (dapps) or test upgrades to be deployed on the mainnet Ethereum network. These testnets also use the Ethereum protocol, with the major difference that the Ether transacted on the testnet has no value.
Ethereum is a decentralized blockchain with smart contract functionality which Cloudflare allows you to interact with through an HTTP API. For a quick primer on Ethereum and our gateway, please refer to our previous blog post on the Ethereum Gateway.
As preparation for the merge, the Ethereum Foundation has executed merges on multiple testnets to ensure that the actual mainnet merge will occur with minimal to no disruption. These testnets both successfully upgraded to Proof of Stake and Proof of Authority, respectively. Cloudflare’s Testnet Gateway handled the Görli-Prater merge without issue, ensuring that we will be ready and prepared for the upcoming Ethereum Merge for mainnet. Our testnet gateways are live and ready for use by Cloudflare Ethereum Gateway customers.
In this blog, Continue reading
Cloudflare named a Leader by Gartner
Gartner has recognised Cloudflare as a Leader in the 2022 "Gartner® Magic Quadrant™ for Web Application and API Protection (WAAP)" report that evaluated 11 vendors for their ‘ability to execute’ and ‘completeness of vision’.
You can register for a complimentary copy of the report here.
We believe this achievement highlights our continued commitment and investment in this space as we aim to provide better and more effective security solutions to our users and customers.
Keeping up with application security
With over 36 million HTTP requests per second being processed by the Cloudflare global network we get unprecedented visibility into network patterns and attack vectors. This scale allows us to effectively differentiate clean traffic from malicious, resulting in about 1 in every 10 HTTP requests proxied by Cloudflare being mitigated at the edge by our WAAP portfolio.
Visibility is not enough, and as new use cases and patterns emerge, we invest in research and new product development. For example, API traffic is increasing (55%+ of total traffic) and we don’t expect this trend to slow down. To help customers with these new workloads, our API Gateway builds upon our WAF to provide better visibility and mitigations for well-structured API traffic for Continue reading
Improving the accuracy of our machine learning WAF using data augmentation and sampling
At Cloudflare, we are always looking for ways to make our customers' faster and more secure. A key part of that commitment is our ongoing investment in research and development of new technologies, such as the work on our machine learning based Web Application Firewall (WAF) solution we announced during security week.
In this blog, we’ll be discussing some of the data challenges we encountered during the machine learning development process, and how we addressed them with a combination of data augmentation and generation techniques.
Let’s jump right in!
Introduction
The purpose of a WAF is to analyze the characteristics of a HTTP request and determine whether the request contains any data which may cause damage to destination server systems, or was generated by an entity with malicious intent. A WAF typically protects applications from common attack vectors such as cross-site-scripting (XSS), file inclusion and SQL injection, to name a few. These attacks can result in the loss of sensitive user data and damage to critical software infrastructure, leading to monetary loss and reputation risk, along with direct harm to customers.
How do we use machine learning for the WAF?
The Cloudflare ML solution, at a high level, Continue reading
Blocking Kiwifarms
We have blocked Kiwifarms. Visitors to any of the Kiwifarms sites that use any of Cloudflare's services will see a Cloudflare block page and a link to this post. Kiwifarms may move their sites to other providers and, in doing so, come back online, but we have taken steps to block their content from being accessed through our infrastructure.
This is an extraordinary decision for us to make and, given Cloudflare's role as an Internet infrastructure provider, a dangerous one that we are not comfortable with. However, the rhetoric on the Kiwifarms site and specific, targeted threats have escalated over the last 48 hours to the point that we believe there is an unprecedented emergency and immediate threat to human life unlike we have previously seen from Kiwifarms or any other customer before.
Escalating threats
Kiwifarms has frequently been host to revolting content. Revolting content alone does not create an emergency situation that necessitates the action we are taking today. Beginning approximately two weeks ago, a pressure campaign started with the goal to deplatform Kiwifarms. That pressure campaign targeted Cloudflare as well as other providers utilized by the site.
Cloudflare provides security services to Kiwifarms, protecting them from DDoS and Continue reading
Log analytics using ClickHouse
This is an adapted transcript of a talk we gave at Monitorama 2022. You can find the slides with presenter’s notes here and video here.
When a request at Cloudflare throws an error, information gets logged in our requests_error pipeline. The error logs are used to help troubleshoot customer-specific or network-wide issues.
We, Site Reliability Engineers (SREs), manage the logging platform. We have been running Elasticsearch clusters for many years and during these years, the log volume has increased drastically. With the log volume increase, we started facing a few issues. Slow query performance and high resource consumption to list a few. We aimed to improve the log consumer's experience by improving query performance and providing cost-effective solutions for storing logs. This blog post discusses challenges with logging pipelines and how we designed the new architecture to make it faster and cost-efficient.
Before we dive into challenges in maintaining the logging pipelines, let us look at the characteristics of logs.
Characteristics of logs
Unpredictable - In today's world, where there are tons of microservices, the amount of logs a centralized logging system will receive is very unpredictable. There are various reasons why capacity estimation of log volume is so difficult. Continue reading
Cloudflare’s abuse policies & approach
Cloudflare launched nearly twelve years ago. We’ve grown to operate a network that spans more than 275 cities in over 100 countries. We have millions of customers: from small businesses and individual developers to approximately 30 percent of the Fortune 500. Today, more than 20 percent of the web relies directly on Cloudflare’s services.
Over the time since we launched, our set of services has become much more complicated. With that complexity we have developed policies around how we handle abuse of different Cloudflare features. Just as a broad platform like Google has different abuse policies for search, Gmail, YouTube, and Blogger, Cloudflare has developed different abuse policies as we have introduced new products.
We published our updated approach to abuse last year at:
https://www.cloudflare.com/trust-hub/abuse-approach/
However, as questions have arisen, we thought it made sense to describe those policies in more detail here.
The policies we built reflect ideas and recommendations from human rights experts, activists, academics, and regulators. Our guiding principles require abuse policies to be specific to the service being used. This is to ensure that any actions we take both reflect the ability to address the harm and minimize unintended consequences. We believe that Continue reading
Introducing thresholds in Security Event Alerting: a z-score love story
Today we are excited to announce thresholds for our Security Event Alerts: a new and improved way of detecting anomalous spikes of security events on your Internet properties. Previously, our calculations were based on z-score methodology alone, which was able to determine most of the significant spikes. By introducing a threshold, we are able to make alerts more accurate and only notify you when it truly matters. One can think of it as a romance between the two strategies. This is the story of how they met.
Author’s note: as an intern at Cloudflare I got to work on this project from start to finish from investigation all the way to the final product.
Once upon a time
In the beginning, there were Security Event Alerts. Security Event Alerts are notifications that are sent whenever we detect a threat to your Internet property. As the name suggests, they track the number of security events, which are requests to your application that match security rules. For example, you can configure a security rule that blocks access from certain countries. Every time a user from that country tries to access your Internet property, it will log as a security event. While a Continue reading
Performance isolation in a multi-tenant database environment
Operating at Cloudflare scale means that across the technology stack we spend a great deal of time handling different load conditions. In this blog post we talk about how we solved performance difficulties with our Postgres clusters. These clusters support a large number of tenants and highly variable load conditions leading to the need to isolate activity to prevent tenants taking too much time from others. Welcome to real-world, large database cluster management!
As an intern at Cloudflare I got to work on improving how our database clusters behave under load and open source the resulting code.
Cloudflare operates production Postgres clusters across multiple regions in data centers. Some of our earliest service offerings, such as our DNS Resolver, Firewall, and DDoS Protection, depend on our Postgres clusters' high availability for OLTP workloads. The high availability cluster manager, Stolon, is employed across all clusters to independently control and replicate data across Postgres instances and elect Postgres leaders and failover under high load scenarios.
PgBouncer and HAProxy act as the gateway layer in each cluster. Each tenant acquires client-side connections from PgBouncer instead of Postgres directly. PgBouncer holds a pool of maximum server-side connections to Postgres, allocating those across multiple Continue reading
Open sourcing our fork of PgBouncer
Cloudflare operates highly available Postgres production clusters across multiple data centers, supporting the transactional workloads of our core service offerings such as our DNS Resolver, Firewall, and DDoS Protection.
Multiple PgBouncer instances sit at the front of the gateway layer per each cluster, acting as a TCP proxy that provides Postgres connection pooling. PgBouncer’s pooling enables upstream applications to connect to Postgres, without having to constantly open and close connections (expensive) at the database level, while also reducing the number of Postgres connections used. Each tenant acquires client-side connections from PgBouncer instead of Postgres directly.
PgBouncer will hold a pool of maximum server-side connections to Postgres, allocating those across multiple tenants to prevent Postgres connection starvation. From here, PgBouncer will forward backend queries to HAProxy, which load balances across Postgres primary and read replicas.
As an intern at Cloudflare I got to work on improving how our database clusters behave under load and open source the resulting code.
We run our Postgres infrastructure in non-containerized, bare metal environments which consequently leads to multitenant resource contention between Postgres users. To enforce stricter tenant performance isolation at the database level (CPU time utilized, memory consumption, disk IO operations), we’d like to configure Continue reading
Deep dives & how the Internet works
When August comes, for many, at least in the Northern Hemisphere, it’s time to enjoy summer and/or vacations. Here are some deep dive reading suggestions from our Cloudflare Blog for any time, weather or time of the year. There’s also some reading material on how the Internet works, and a glimpse into our history.
To create the list (that goes beyond 2022), initially we asked inside the company for favorite blog posts. Many explained how a particular blog post made them want to work at Cloudflare (including some of those who have been at the company for many years). And then, we also heard from readers by asking the question on our Twitter account: “What’s your favorite blog post from the Cloudflare Blog and why?”
2022, deep dive & trends odyssey
In early July (thinking of the July 4 US holiday) we did a sum up where some of the more recent blog posts were referenced. We’ve added a few to that list:
- Eliminating CAPTCHAs on iPhones and Macs (✍️)
How it works using open standards. On this topic, you can also read the detailed blog post from our research team, from 2021: Humanity wastes about Continue reading
Cloudflare Support Portal gets an overhaul
The Cloudflare Support team is excited to announce the launch of our brand-new Customer Support Portal. When our customers open support tickets, we understand that they want quick and accurate responses from us. For those of you who have opened a support ticket in the past, we are certain you will notice the improvements we've made! The new Support Portal lives where our ticket submission form has always been, dash.cloudflare.com/support, but that's where the similarities between the old and the new one end.
What can you expect in the new portal?
The new Support Portal will help you solve your problems quickly and effectively, by getting you on the fastest path to resolution. In some cases, the most efficient way to resolve your issue will be to use our self-help resources or our machine learning-trained Support Bot. Other times, the most efficient way to resolve your issue will be by working with one of our Support Engineers via ticket, phone or chat, depending on your plan type. Regardless of how we help you solve your issue, we will have more context about the products you are using and your issue up front, reducing time-consuming back and forth.