Over 700 million events/second: How we make sense of too much data
Cloudflare's network provides an enormous array of services to our customers. We collect and deliver associated data to customers in the form of event logs and aggregated analytics. As of December 2024, our data pipeline is ingesting up to 706M events per second generated by Cloudflare's services, and that represents 100x growth since our 2018 data pipeline blog post.
At peak, we are moving 107 GiB/s of compressed data, either pushing it directly to customers or subjecting it to additional queueing and batching.
All of these data streams power things like Logs, Analytics, and billing, as well as other products, such as training machine learning models for bot detection. This blog post is focused on techniques we use to efficiently and accurately deal with the high volume of data we ingest for our Analytics products. A previous blog post provides a deeper dive into the data pipeline for Logs.
The pipeline can be roughly described by the following diagram.
The data pipeline has multiple stages, and each can and will naturally break or slow down because of hardware failures or misconfiguration. And when that happens, there is just too much data to be able to Continue reading
Record-breaking 5.6 Tbps DDoS attack and global DDoS trends for 2024 Q4
Welcome to the 20th edition of the Cloudflare DDoS Threat Report, marking five years since our first report in 2020.
Published quarterly, this report offers a comprehensive analysis of the evolving threat landscape of Distributed Denial of Service (DDoS) attacks based on data from the Cloudflare network. In this edition, we focus on the fourth quarter of 2024 and look back at the year as a whole.
When we published our first report, Cloudflare’s global network capacity was 35 Terabits per second (Tbps). Since then, our network’s capacity has grown by 817% to 321 Tbps. We also significantly expanded our global presence by 65% from 200 cities in the beginning of 2020 to 330 cities by the end of 2024.
Using this massive network, we now serve and protect nearly 20% of all websites and close to 18,000 unique Cloudflare customer IP networks. This extensive infrastructure and customer base uniquely positions us to provide key insights and trends that benefit the wider Internet community.
In 2024, Cloudflare’s autonomous DDoS defense systems blocked around 21.3 million DDoS attacks, representing a 53% increase compared to 2023. On average, in 2024, Cloudflare blocked 4,870 Continue reading
The fall and rise of TikTok (traffic)
The United States ban on TikTok went into effect on January 19, 2025, and although service began to be restored after just 14 hours, it was only close to the inauguration of Donald Trump as the 47th President of the United States that associated DNS traffic started to recover to closer to previous levels. In this post, we analyze the events of January 19 and 20, and what they meant for TikTok-related DNS traffic, but also other competitors (including their growth outside the US).
For context, we wrote an initial blog post about the TikTok ban on Sunday, January 19, 2025. The ban was part of the "Protecting Americans from Foreign Adversary Controlled Applications Act," proposed in Congress, which ordered ByteDance to divest due to alleged security concerns. The bill was signed into law by Congress and President Biden in April 2024, and was upheld by the Supreme Court on January 17, 2025.
Aggregated data from our 1.1.1.1 DNS resolver shows — as we’ve posted on social media — that the TikTok shutdown in the US began to impact DNS traffic to TikTok-related domains on January 19, just after 03:30 UTC (22:30 ET on January Continue reading
TikTok ban takes hold: data reveals sharp traffic decline and rapid shift to alternatives
The United States ban on TikTok went into effect on January 19, 2025, and our data showed a clear impact starting after 03:30 UTC (10:30 PM ET on January 18, 2025). The ban was part of the "Protecting Americans from Foreign Adversary Controlled Applications Act," proposed in Congress, which ordered ByteDance to divest due to alleged security concerns. The bill was signed into law by Congress and President Biden in April 2024, and was upheld by the Supreme Court.
Aggregated data from our 1.1.1.1 DNS resolver shows — as we’ve posted on X — that the TikTok shutdown in the US began to impact DNS traffic to TikTok-related domains on January 19, just after 03:30 UTC (22:30 ET on January 18). This includes DNS traffic not only for TikTok, but also for other ByteDance-owned platforms, such as the CapCut video editor. Traffic dropped by as much as 85% compared to the previous week and showed signs of further decline in the following hours.
Around that time, a message indicating the TikTok ban began appearing for US users.
Analyzing data from autonomous systems or networks, traffic from TikTok owner ByteDance’s network (AS396986) in the US Continue reading
Demonstrating reduction of vulnerability classes: a key step in CISA’s “Secure by Design” pledge
In today’s rapidly evolving digital landscape, securing software systems has never been more critical. Cyber threats continue to exploit systemic vulnerabilities in widely used technologies, leading to widespread damage and disruption. That said, the United States Cybersecurity and Infrastructure Agency (CISA) helped shape best practices for the technology industry with their Secure-by-Design pledge. Cloudflare signed this pledge on May 8, 2024, reinforcing our commitment to creating resilient systems where security is not just a feature, but a foundational principle.
We’re excited to share an update aligned with one of CISA’s goals in the pledge: To reduce entire classes of vulnerabilities. This goal aligns with the Cloudflare Product Security program’s initiatives to continuously automate proactive detection and vigorously prevent vulnerabilities at scale.
Cloudflare’s commitment to the CISA pledge reflects our dedication to transparency and accountability to our customers. This blog post outlines why we prioritized certain vulnerability classes, the steps we took to further eliminate vulnerabilities, and the measurable outcomes of our work.
Cloudflare’s core security philosophy is to prevent security vulnerabilities from entering production environments. One of the goals for Cloudflare’s Product Security team is to champion this philosophy and ensure Continue reading
Open source all the way down: Upgrading our developer documentation
At Cloudflare, we treat developer content like a product, where we take the user and their feedback into consideration. We are constantly iterating, testing, analyzing, and refining content. Inspired by agile practices, treating developer content like an open source product means we approach our documentation the same way an open source software project is created and maintained. Open source documentation empowers the developer community because it allows anyone, anywhere, to contribute content. By making both the content and the framework of the documentation site publicly accessible, we provide developers with the opportunity to not only improve the material itself but also understand and engage with the processes that govern how the documentation is built, approved, and maintained. This transparency fosters collaboration, learning, and innovation, enabling developers to contribute their expertise and learn from others in a shared, open environment. We also provide feedback to other open source products and plugins, giving back to the same community that supports us.
Great documentation empowers users to be successful with a new product as quickly as possible, showing them how to use the product and describing its benefits. Relevant, timely, and accurate content can save Continue reading
Multi-Path TCP: revolutionizing connectivity, one path at a time
The Internet is designed to provide multiple paths between two endpoints. Attempts to exploit multi-path opportunities are almost as old as the Internet, culminating in RFCs documenting some of the challenges. Still, today, virtually all end-to-end communication uses only one available path at a time. Why? It turns out that in multi-path setups, even the smallest differences between paths can harm the connection quality due to packet reordering and other issues. As a result, Internet devices usually use a single path and let the routers handle the path selection.
There is another way. Enter Multi-Path TCP (MPTCP), which exploits the presence of multiple interfaces on a device, such as a mobile phone that has both Wi-Fi and cellular antennas, to achieve multi-path connectivity.
MPTCP has had a long history — see the Wikipedia article and the spec (RFC 8684) for details. It's a major extension to the TCP protocol, and historically most of the TCP changes failed to gain traction. However, MPTCP is supposed to be mostly an operating system feature, making it easy to enable. Applications should only need minor code changes to support it.
There is a caveat, however: MPTCP is still fairly immature, and while it can Continue reading
Behind the scenes with Stream Live, Cloudflare’s live streaming service
Cloudflare announced Stream Live for open beta in 2021, and in 2022 we went GA. While we talked about the experience of using it and the value it delivers to customers, we didn’t talk about how we built it. So let’s talk about Stream Live’s design, and how it leverages the distributed nature of Cloudflare’s network, rather than centralized locations as many other live services do. Ultimately, our goals are to keep our content ingest as close to broadcasters as possible, our content delivery as close to viewers as possible, and to retain our ability to handle unexpected use cases.
At a high level, Stream Live accepts audio/video content from broadcasters and makes that content available to viewers around the world in real time through the Cloudflare network, which reaches more than 330 cities in over 120 countries. Hence, there are two sides to this: ingesting data from broadcasters and delivering encoded content to viewers. Both sides are built on a combination of internal systems and Cloudflare products, including Cloudflare Workers, Durable Objects, Spectrum, and, of course, Cache.
Let’s start on the ingest side.
Broadcasters generate content in real time, as a Continue reading
The forecast is clear: clouds on e-paper, powered by the cloud
I’ve noticed that many shops are increasingly using e-paper displays. They’re impressive: high contrast, no backlight, and no visible cables. Unlike most electronics, these displays are seamlessly integrated and feel very natural. This got me wondering: is it possible to use such a display for a pet project? I want to experiment with this technology myself.
(source)
My main goal in this project is to understand the hardware and its capabilities. Here, I'll be using an e-paper display to show the current weather, but at its core, I’m simply feeding data from a website to the display. While it sounds straightforward, it actually requires three layers of software to pull off. Still, it’s a fun challenge and a great opportunity to work with both embedded hardware and Cloudflare Workers.
For this project, I'm using components from Waveshare. They offer a variety of e-paper displays, ranging from credit card-sized to A4-sized models. I chose the 7.5-inch, two-color "e-Paper (G)" display. For the controller, I'm using a Waveshare ESP32-based universal board. With just these two components — a display and a controller — I was ready to get started.
When the components arrived, I carefully Continue reading
Open sourcing h3i: a command line tool and library for low-level HTTP/3 testing and debugging
Have you ever built a piece of IKEA furniture, or put together a LEGO set, by following the instructions closely and only at the end realized at some point you didn't quite follow them correctly? The final result might be close to what was intended, but there's a nagging thought that maybe, just maybe, it's not as rock steady or functional as it could have been.
Internet protocol specifications are instructions designed for engineers to build things. Protocol designers take great care to ensure the documents they produce are clear. The standardization process gathers consensus and review from experts in the field, to further ensure document quality. Any reasonably skilled engineer should be able to take a specification and produce a performant, reliable, and secure implementation. The Internet is central to everyone's lives, and we depend on these implementations. Any deviations from the specification can put us at risk. For example, mishandling of malformed requests can allow attacks such as request smuggling.
h3i is a binary command line tool and Rust library designed for low-level testing and debugging of HTTP/3, which runs over QUIC. h3i is free and open source as part of Cloudflare's quiche project. In this post we'll Continue reading
What’s new in Cloudflare: MASQUE now powers 1.1.1.1 & WARP apps, DEX now generally available with Remote Captures
At Cloudflare, we are constantly innovating and launching new features and capabilities across our product portfolio. Today’s roundup blog post shares two exciting updates across our platform: our cross-platform 1.1.1.1 & WARP applications (consumer) and device agents (Zero Trust) now use MASQUE, a cutting-edge HTTP/3-based protocol, to secure your Internet connection. Additionally, DEX is now available for general availability.
We’re excited to announce that as of today, our cross-platform 1.1.1.1 & WARP apps now use MASQUE, a cutting-edge HTTP/3-based protocol, to secure your Internet connection.
As a reminder, our 1.1.1.1 & WARP apps have two main functions: send all DNS queries through 1.1.1.1, our privacy-preserving DNS resolver, and protect your device’s network traffic via WARP by creating a private and encrypted tunnel to the resources you’re accessing, preventing unwanted third parties or public Wi-Fi networks from snooping on your traffic.
There are many ways to encrypt and proxy Internet traffic — you may have heard of a few, such as IPSec, WireGuard, or OpenVPN. There are many tradeoffs Continue reading
Sometimes I cache: implementing lock-free probabilistic caching
HTTP caching is conceptually simple: if the response to a request is in the cache, serve it, and if not, pull it from your origin, put it in the cache, and return it. When the response is old, you repeat the process. If you are worried about too many requests going to your origin at once, you protect it with a cache lock: a small program, possibly distinct from your cache, that indicates if a request is already going to your origin. This is called cache revalidation.
In this blog post, we dive into how cache revalidation works, and present a new approach based on probability. For every request going to the origin, we simulate a die roll. If it’s 6, the request can go to the origin. Otherwise, it stays stale to protect our origin from being overloaded. To see how this is built and optimised, read on.
Let's take the example of an online image library. When a client requests an image, the service first checks its cache to see if the resource is present. If it is, it returns it. If it is not, the image server processes the request, places the response into the Continue reading
Un experimento rápido: translating Cloudflare Stream captions with Workers AI
Cloudflare Stream launched AI-powered automated captions to transcribe English in on-demand videos in March 2024. Customers' immediate next questions were about other languages — both transcribing audio from other languages, and translating captions to make subtitles for other languages. As the Stream Product Manager, I've thought a lot about how we might tackle these, but I wondered…
What if I just translated a generated VTT (caption file)? Can we do that? I hoped to use Workers AI to conduct a quick experiment to learn more about the problem space, challenges we may find, and what platform capabilities we can leverage.
There is a sample translator demo in Workers documentation that uses the “m2m100-1.2b” Many-to-Many multilingual translation model to translate short input strings. I decided to start there and try using it to translate some of the English captions in my Stream library into Spanish.
I started with my short demo video announcing the transcription feature. I wanted a Worker that could read the VTT captions file from Stream, isolate the text content, and run it through the model as-is.
The first step was parsing the input. A VTT file is a text file that Continue reading
When the world logs off: Christmas, New Year’s, and the Internet’s holiday rhythm
As January approaches and the year comes to a close, distinct changes in global Internet usage emerge. Year-end traditions — ranging from Christmas feasts to New Year’s Eve (NYE) countdowns — shape online behavior across continents and cultures. Looking back at Christmas and NYE 2023 offers insights into how these trends may repeat this year, and by January 2025, we’ll be able to directly compare patterns. Examining data from 50 countries and regions reveals how people celebrated in 2023-2024, providing a timely reminder of typical holiday trends.
With Cloudflare’s global reach, we observe planet-wide and local Internet habits during the holiday season. In the past, unintended trends during Christmas and New Year’s Eve have surfaced through our Outage Center, which uses automatic traffic anomaly alerts to detect Internet outages or unusual patterns. In the 2023 overview below, traffic dropped enough on those days to trigger dozens of anomaly alerts (orange and pink bubbles):
While Christmas dominates in many regions, other cultural and religious holidays — like Hanukkah or regional festivities — shape online rhythms in places where Western traditions are less central.
In countries and regions where Christmas is deeply rooted, Internet traffic slows during Christmas Eve dinners, Continue reading
Grinch Bots strike again: defending your holidays from cyber threats
Back in 2021, we covered the antics of Grinch Bots and how the combination of proposed regulation and technology could prevent these malicious programs from stealing holiday cheer.
Fast-forward to 2024 — the Stop Grinch Bots Act of 2021 has not passed, and bots are more active and powerful than ever, leaving businesses to fend off increasingly sophisticated attacks on their own. During Black Friday 2024, Cloudflare observed:
29% of all traffic on Black Friday was Grinch Bots. Humans still accounted for the majority of all traffic, but bot traffic was up 4x from three years ago in absolute terms.
1% of traffic on Black Friday came from AI bots. The majority of it came from Claude, Meta, and Amazon. 71% of this traffic was given the green light to access the content requested.
63% of login attempts across our network on Black Friday were from bots. While this number is high, it was down a few percentage points compared to a month prior, indicating that more humans accessed their accounts and holiday deals.
Human logins on e-commerce sites increased 7-8% compared to the previous month.
These days, holiday shopping Continue reading
Global elections in 2024: Internet traffic and cyber threat trends
Elections define the course of democracies (even as there are several types of democracies), and 2024 was a landmark year, with over 60 countries — plus the European Union — holding national elections, impacting half the world’s population. As highlighted in Pew Research’s global elections report, this was a year of “political disruption,” where the Internet was a relevant stage for both democratic engagement and cyber threats.
At Cloudflare, with our presence in over 330 cities and 120 countries and interconnection with 12,500 networks, we’ve witnessed firsthand the digital impact of these elections. From monitoring Internet traffic patterns to mitigating cyberattacks, we’ve observed trends that reveal how elections increasingly play out online. As detailed in our just-published Cloudflare Impact report, we’ve also worked to protect media outlets, political campaigns, and help elections worldwide.
Here’s the map of countries with national elections that took place in 2024, from our elections report.
We’ve been monitoring 2024 elections worldwide on our blog and in the 2024 Election Insights report available on Cloudflare Radar.
In terms of Internet patterns, we’ve observed how cyber activity in 2024 continues to intersect with real-world events. Online attacks are clearly a significant part of elections, Continue reading
Hi Claude, build an MCP server on Cloudflare Workers
In late November 2024, Anthropic announced a new way to interact with AI, called Model Context Protocol (MCP). Today, we’re excited to show you how to use MCP in combination with Cloudflare to extend the capabilities of Claude to build applications, generate images and more. You’ll learn how to build an MCP server on Cloudflare to make any service accessible through an AI assistant like Claude with just a few lines of code using Cloudflare Workers.
MCP is an open standard that provides a universal way for LLMs to interact with services and applications. As the introduction on the MCP website puts it,
“Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools.”
From an architectural perspective, MCP is comprised of several components:
MCP hosts: Programs or tools (like Claude) where AI models operate and interact with different services
MCP clients: Client within an AI assistant that initiates requests and communicates with MCP servers to Continue reading
Bring multimodal real-time interaction to your AI applications with Cloudflare Calls
OpenAI announced support for WebRTC in their Realtime API on December 17, 2024. Combining their Realtime API with Cloudflare Calls allows you to build experiences that weren’t possible just a few days earlier.
Previously, interactions with audio and video AIs were largely single-player: only one person could be interacting with the AI unless you were in the same physical room. Now, applications built using Cloudflare Calls and OpenAI’s Realtime API can now support multiple users across the globe simultaneously seeing and interacting with a voice or video AI.
Here’s what this means in practice: you can now invite ChatGPT to your next video meeting:
We built this into our Orange Meets demo app to serve as an inspiration for what is possible, but the opportunities are much broader.
In the not-too-distant future, every company could have a 'corporate AI' they invite to their internal meetings that is secure, private and has access to their company data. Imagine this sort of real-time audio and video interactions with your company’s AI:
"Hey ChatGPT, do we have any open Jira tickets about this?"
"Hey Company AI, who are the competitors in the space doing Continue reading
The role of email security in reducing user risk amid rising threats
Phishing remains one of the most dangerous and persistent cyber threats for individuals and organizations. Modern attacks use a growing arsenal of deceptive techniques that bypass traditional secure email gateways (SEGs) and email authentication measures, targeting organizations, employees, and vendors. From business email compromise (BEC) to QR phishing and account takeovers, these threats are designed to exploit weaknesses across multiple communication channels, including email, Slack, Teams, SMS, and cloud drives.
Phishing remains the most popular attack vector for bad actors looking to gain unauthorized access or extract fraudulent payment, and it is estimated that 90% of all attacks start with a phishing email. However, as companies have shifted to using a multitude of apps to support communication and collaboration, attackers too have evolved their approach. Attackers now engage employees across a combination of channels in an attempt to build trust and pivot targeted users to less-secure apps and devices. Cloudflare is uniquely positioned to address this trend thanks to our integrated Zero Trust services, extensive visibility from protecting approximately 20% of all websites, and signals derived from processing billions of email messages a year.
Cloudflare recognizes that combating phishing requires an integrated approach and a more complete view Continue reading
Internationalization and localization: bringing Cloudflare Radar to a global audience
Cloudflare Radar celebrated its fourth birthday in September 2024. As we’ve expanded Radar’s scope over the last four years, the value that it provides as a resource for the global Internet has grown over time, and with Radar data and graphs often appearing in publications and social media around the world, we knew that we needed to make it available in languages beyond English.
Localization is important because most Internet users do not speak English as a first language. According to W3Techs, English usage on the Internet has dropped 8.3 points (57.7% to 49.4%) since January 2023, whereas usage of other languages like Spanish, German, Japanese, Italian, Portuguese and Dutch is steadily increasing. Furthermore, a CSA Research study determined that 65% of Internet users prefer content in their language.
To successfully (and painlessly) localize any product, it must be internationalized first. Internationalization is the process of making a product ready to be translated and adapted into multiple languages and cultures, and it sets the foundation to enable your product to be localized later on at a much faster pace (and at a lower cost, both in time and budget). Below, we review how Cloudflare’s Radar Continue reading