Building Jetflow: a framework for flexible, performant data pipelines at Cloudflare
The Cloudflare Business Intelligence team manages a petabyte-scale data lake and ingests thousands of tables every day from many different sources. These include internal databases such as Postgres and ClickHouse, as well as external SaaS applications such as Salesforce. These tasks are often complex and tables may have hundreds of millions or billions of rows of new data each day. They are also business-critical for product decisions, growth plannings, and internal monitoring. In total, about 141 billion rows are ingested every day.
As Cloudflare has grown, the data has become ever larger and more complex. Our existing Extract Load Transform (ELT) solution could no longer meet our technical and business requirements. After evaluating other common ELT solutions, we concluded that their performance generally did not surpass our current system, either.
It became clear that we needed to build our own framework to cope with our unique requirements — and so Jetflow was born.
Over 100x efficiency improvement in GB-s:
Our longest running job with 19 billion rows was taking 48 hours using 300 GB of memory, and now completes in 5.5 hours using 4 GB of memory
We estimate that ingestion of Continue reading
Ultra Ethernet: Reinventing X.25
One should never trust the technical details published by the industry press, but assuming the Tomahawk Ultra puff piece isn’t too far off the mark, the new Broadcom ASIC (supposedly loosely based on emerging Ultra Ethernet specs):
- Uses Optimized Ethernet Header, replacing IP/UDP header with a 10-byte something (let’s call it session identifier)
- Makes Ethernet lossless with hop-by-hop retransmission/error recovery
- Uses credit-based flow control (the receiver continuously updates the sender about the amount of available space)
If you’re ancient enough, you might recognize #3 as part of Fibre Channel, #2 and #3 as part of IEEE 802.1 LLC2 (used by IBM to implement SNA over Token Ring and Ethernet), and all three as the fundamental ideas of X.25 that Broadcom obviously reinvented at 800 Gbps speeds, proving (yet again) RFC 1925 Rule 11.
PP071: SSE Vendor Test Results; Can HPE and Juniper Get Along?
CyberRatings, a non-profit that performs independent testing of security products and services, has released the results of comparative tests it conducted on Secure Service Edge, or SSE, services. Tested vendors include Cisco, Cloudflare, Fortinet, Palo Alto Networks, Skyhigh Security, Versa Networks, and Zscaler. We look at what was tested and how, highlight results, and discuss... Read more »Don’t Let AI Make You Circuit City
I have a little confession. Sometimes I like to go into Best Buy and just listen. I pretend to be shopping or modem bearings or a left handed torque wrench. What I’m really doing is hearing how people sell computers. I remember when 8x CD burners were all the rage. I recall picking one particular machine because it had an integrated Sound Blaster card. Today, I just marvel at how the associates rattle off a long string of impressive sounding nonsense that consumers will either buy hook, line, and sinker or refute based on some Youtube reviewer recommendation. Every once in a while, though, I hear someone that actually does understand the lingo and it is wonderful. They listen and understand the challenges and don’t sell a $3,000 gaming computer to a grandmother just to play Candy Crush and look up grandkid photos on Facebook.
The Experience Matters
What does that story have to do with the title of this post? Well, dear young readers, you may not remember the time when Best Buy Blue was locked in mortal competition with Circuit City Red. In a time before Amazon was ascendant you had to pick between the two giants of Continue reading
Cloudflare protects against critical SharePoint vulnerability, CVE-2025-53770
On July 19, 2025, Microsoft disclosed CVE-2025-53770, a critical zero-day Remote Code Execution (RCE) vulnerability. Assigned a CVSS 3.1 base score of 9.8 (Critical), the vulnerability affects SharePoint Server 2016, 2019, and the Subscription Edition, along with unsupported 2010 and 2013 versions. Cloudflare’s WAF Managed Rules now includes 2 emergency releases that mitigate these vulnerabilities for WAF customers.
The vulnerability's root cause is improper deserialization of untrusted data, which allows a remote, unauthenticated attacker to execute arbitrary code over the network without any user interaction. Moreover, what makes CVE-2025-53770 uniquely threatening is its methodology – the exploit chain, labeled "ToolShell." ToolShell is engineered to play the long-game: attackers are not only gaining temporary access, but also taking the server's cryptographic machine keys, specifically the ValidationKey and DecryptionKey. Possessing these keys allows threat actors to independently forge authentication tokens and __VIEWSTATE payloads, granting them persistent access that can survive standard mitigation strategies such as a server reboot or removing web shells.
In response to the active nature of these attacks, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) added CVE-2025-53770 to its Known Exploited Vulnerabilities (KEV) catalog with an emergency remediation deadline. Continue reading
HS108: Keeping the (IT) House Clean to Avoid the Plague
Whether it’s CNAME records pointing to dead endpoints or abandoned cloud storage buckets still mentioned in the makefile or Chef recipe, seemingly innocuous bits of infrastructure that don’t get cleaned up can turn into serious security threats. (Both of these examples are taken from real-life attacks, BTW). When and how and who within IT should... Read more »
Shutdown season: the Q2 2025 Internet disruption summary
Cloudflare’s network currently spans more than 330 cities in over 125 countries, and we interconnect with over 13,000 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions at both a local and national level, as well as at a network level.
As we have noted in the past, this post is intended as a summary overview of observed and confirmed disruptions, and is not an exhaustive or complete list of issues that have occurred during the quarter. A larger list of detected traffic anomalies is available in the Cloudflare Radar Outage Center. Note that both bytes-based and request-based traffic graphs are used within the post to illustrate the impact of the observed disruptions — the choice of metric was generally made based on which better illustrated the impact of the disruption.
In our Q1 2025 summary post, we noted that we had not observed any government-directed Internet shutdowns during the quarter. Unfortunately, that forward progress was short-lived — in the second quarter of 2025, we Continue reading
NB535: Tomahawk Ultra Chops Congestion; Denmark Invests for Quantum Advantages
Take a Network Break! We begin with a listener question about a paper critiquing Shor’s Algorithm and quantum computing, and touch on a remote code execution vulnerability in Riverbed SteelCentral NetProfiler / NetExpress 10.8.7. We discuss a Cloudflare BGP misconfiguration that caused the Internet to hiccup, Broadcom’s new Tomahawk Ultra ASIC aimed for–you guessed it–AI... Read more »Tech Bytes: How Is SD-WAN Driving SASE Success (Sponsored)
Today on the Tech Bytes podcast, sponsored by Palo Alto Networks, we examine how SASE is being adopted and deployed by enterprise customers. What does this integration of networking and security mean for the teams that need to operate this solution? We talk with Palo Alto Networks about SASE as a transformative technology for networking... Read more »Bell Labs Takes A Topological Approach To Quantum 2.0
Momentum is building for quantum computing and some observers say that a usable, fault-tolerant quantum system could appear in the next few years. …
Bell Labs Takes A Topological Approach To Quantum 2.0 was written by Jeffrey Burt at The Next Platform.
Always Check Your Tests Against Faulty Inputs
A while ago, I published a blog post proudly describing the netlab integration test that should check for incorrect OSPF network types in netlab-generated device configurations. Almost immediately, Erik Auerswald pointed out that my test wouldn’t detect that error (it might detect other errors, though) as the OSPF network adjacency is always established even when the adjacent routers have mismatching OSPF network types.
I made one of the oldest testing mistakes: I checked whether my test would work under the correct conditions but not whether it would detect an incorrect condition.
Immich Setup with Docker & External Library (NFS)
Recently, I started self-hosting most of the apps I use, like Memos for note-taking and Paperless-NGX for document management. The next one on the list was Immich. Immich is a self-hosted photo and video backup solution that supports features like facial recognition and automatic uploads.
Memos - Amazing Open Source, Self-hosted Notes App
That being said, I recently stumbled upon another great self-hosted note-taking app called ‘Memos’ I just couldn’t believe that I didn’t know about this until very recently.
Suresh Vina
Paperless-ngx - Self-Hosted Document Manager
I came across a great self-hosted document manager called ‘Paperless-NGX’. It not only helps with organising documents but also includes OCR functionality
Suresh Vina
In this post, we’ll look at how to set up Immich as a Docker container and also how to add an NFS share as an external library.
But, Why?
I have a lot of pictures on my NAS that I’ve collected over the years. This includes photos of friends, family, and ones from my older phones. I wanted a way to manage and organise them from one place. I also didn’t want to upload all of them to Google or Apple, which would cost quite a bit. Continue reading
How Long Before Half Of TSMC’s Sales Are Driven By AI?
Here at The Next Platform, the quarterly earnings season starts with Taiwan Semiconductor Manufacturing Co. …
How Long Before Half Of TSMC’s Sales Are Driven By AI? was written by Timothy Prickett Morgan at The Next Platform.
TNO035: Network Engineer and Content Creator – A Career Journey with Kevin Nanns
Kevin Nanns is today’s guest on Total Network Operations. Kevin describes his “Wizard of Oz” moment when he discovered the world of networking. He talks about how he came up through the ranks working a help desk and then in a NOC, and his climb up the certification ladder. We also discuss how AI is... Read more »HN788: Behind Megaport’s Network Automation Platform (Sponsored)
We have a network automation discussion for you today from sponsor Megaport. At the AutoCon3 conference earlier this year, Luke Gollan presented on a complex network automation project migrating Megaport’s API-driven software defined network from a legacy VXC overlay to an EVPN framework. This helped improve scalability, but was fraught with practical challenges, not the... Read more »Cisco IOS/XE Hates Redistributed Static IPv6 Routes
Writing tests that check the correctness of network device configurations is hard (overview, more details). It’s also an interesting exercise in getting the timing just right:
- Routing protocols are an eventually-consistent distributed system, and things eventually appear in the right place (if you got the configurations right), but you never know when exactly that will happen.
- You can therefore set some reasonable upper bounds on when things should happen, and declare failure if the timeouts are exceeded. Even then, you’ll get false positives (as in: the test is telling you the configurations are incorrect, when it’s just a device having a bad hair day).
And just when you think you nailed it, you encounter a device that blows your assumptions out of the water.