Cloudflare's network provides an enormous array of services to our customers. We collect and deliver associated data to customers in the form of event logs and aggregated analytics. As of December 2024, our data pipeline is ingesting up to 706M events per second generated by Cloudflare's services, and that represents 100x growth since our 2018 data pipeline blog post.
At peak, we are moving 107 GiB/s of compressed data, either pushing it directly to customers or subjecting it to additional queueing and batching.
All of these data streams power things like Logs, Analytics, and billing, as well as other products, such as training machine learning models for bot detection. This blog post is focused on techniques we use to efficiently and accurately deal with the high volume of data we ingest for our Analytics products. A previous blog post provides a deeper dive into the data pipeline for Logs.
The pipeline can be roughly described by the following diagram.
The data pipeline has multiple stages, and each can and will naturally break or slow down because of hardware failures or misconfiguration. And when that happens, there is just too much data to be able to Continue reading
I spent way too much time ironing out the VRRPv3 quirks on the dozen (or so) platforms supported by netlab. This is the second blog post describing some of the ridiculous stuff I had to deal with.
This is how you configure the basic VRRPv3 parameters for IPv4 on a Cisco IOS/XE device:
interface GigabitEthernet0/1
vrrp 217 address-family ipv4
address 172.16.33.42
You would expect something similar for IPv6, right? You’d be right if you were working with Arista EOS:
Hello my friend,
Whenever we develop any network and IT infrastructure automation applications, we need to have some options to provide user input. In previous blog posts in these series we already covered how to provide user input via environment variables and files. Whilst these two approaches can cover majority of your use cases, especially if you develop containerized applications running in autonomy, there are still two options we would like to talk today about.
For many years I was doing network design and operation without automating it (or at least without structured approach to automate it). And there are still loads of such job positions out there. And I see it based on the audience of my blog: majority of people here for networking knowledge, much less are for automation topics. From pure pragmatic standpoint of writing popular blogs, I should stick to network technologies, especially something fancy as SD-WAN and others. However, from the direction of the technologies development, I see that value (including jobs) comes from intersection of domains: networking, compute, storage, software development, data bases, Kubernetes, observability, etc. I’m of a strong opinion that engineers these days must be aware of Continue reading
Most websites we access today use HTTPS, and to fully leverage a Next-Generation Firewall (NGFW) like Palo Alto, inspecting encrypted HTTPS sessions is crucial. Configuring SSL decryption isn't just a set-it-and-forget-it task. It requires careful consideration and ongoing improvements. In this blog post, we'll explore how to configure SSL decryption in Palo Alto firewalls and highlight some pitfalls to be aware of. So, let's get to it.
As always, if you find this post helpful, press the ‘clap’ button on the left. It means a lot to me and helps me know you enjoy this type of content.
As I mentioned earlier, configuring SSL decryption isn’t as simple as flipping a switch. Decryption allows your firewall to inspect the contents of encrypted sessions. Normally, HTTPS traffic is encrypted from your browser to the server, ensuring the sessions are private. However, with SSL decryption, the firewall acts as a man-in-the-middle, inspecting the traffic in plain text. It’s crucial Continue reading
Richard Wexelblat published an article in 1980 titled: “The consequences of one’s first programming language.” We’ve all seen C code written like Python, or Python code written like C, so it’s obvious a coder’s first language has a long lasting effect on their style. What about network engineers? Are there times and places where the first of anything a network engineers encounters has a long lasting impact on the way they think and work? In this roundtable, Tom, Eyvonne, and Russ consider different ways this might apply to network engineering.
When a BGP router cannot fit the whole BGP table into its forwarding table (FIB), we often use inbound filters to limit the amount of information the device keeps in its BGP table. That’s usually a waste of resources:
Wouldn’t it be better for the device with an inbound filter to push that filter to its BGP neighbors?
I just wasted several days trying to figure out how to make the dozen (or so) platforms for which we implemented VRRPv3 in netlab work together. This is the first in a series of blog posts describing the ridiculous stuff we discovered during that journey
The idea was pretty simple:
The believers in the There Be Four Layers religion think everything below IP is just a blob of stuff dealing with physical things:
People steeped in a slightly more nuanced view of the world in which IP is not the centerpiece of the universe might tell you that the blob of stuff we need is two things:
If you follow my blogs, you might know that I recently switched to Proxmox from VMware Workstation Pro for my home lab. I’ve already migrated most of my VMs, including Cisco CML, to Proxmox, and the last piece left was EVE-NG. In this blog post, we’ll go through the steps to install EVE-NG in Proxmox. Let’s get started!
As always, if you find this post helpful, press the ‘clap’ button on the left. It means a lot to me and helps me know you enjoy this type of content.
EVE-NG doesn’t have official documentation for Proxmox, but it works perfectly fine, and I haven’t faced any issues so far. For this example, I’m using
Most of the VM’s settings can be left at their default values, but there are a couple of changes I had to make. Before diving in, let's have a quick look at Nested Virtualization.
Nested virtualization allows you to run virtual machines Continue reading
Welcome to the 20th edition of the Cloudflare DDoS Threat Report, marking five years since our first report in 2020.
Published quarterly, this report offers a comprehensive analysis of the evolving threat landscape of Distributed Denial of Service (DDoS) attacks based on data from the Cloudflare network. In this edition, we focus on the fourth quarter of 2024 and look back at the year as a whole.
When we published our first report, Cloudflare’s global network capacity was 35 Terabits per second (Tbps). Since then, our network’s capacity has grown by 817% to 321 Tbps. We also significantly expanded our global presence by 65% from 200 cities in the beginning of 2020 to 330 cities by the end of 2024.
Using this massive network, we now serve and protect nearly 20% of all websites and close to 18,000 unique Cloudflare customer IP networks. This extensive infrastructure and customer base uniquely positions us to provide key insights and trends that benefit the wider Internet community.
In 2024, Cloudflare’s autonomous DDoS defense systems blocked around 21.3 million DDoS attacks, representing a 53% increase compared to 2023. On average, in 2024, Cloudflare blocked 4,870 Continue reading