D2DO252: (Re)Building Cloudflare’s Millions-of-Logs-Per-Second Logging Pipeline
Cloudflare’s transition from SysLog-NG to OpenTelemetry is the topic of discussion on this episode of Day Two DevOps. Guests Colin Douch and Jayson Cena from Cloudflare explain the reasons behind the migration, including the need for better scalability, memory safety, and maintainability. They delve into challenges such as ensuring uninterrupted customer traffic and optimizing performance.... Read more »How Cloudflare auto-mitigated world record 3.8 Tbps DDoS attack
Since early September, Cloudflare's DDoS protection systems have been combating a month-long campaign of hyper-volumetric L3/4 DDoS attacks. Cloudflare’s defenses mitigated over one hundred hyper-volumetric L3/4 DDoS attacks throughout the month, with many exceeding 2 billion packets per second (Bpps) and 3 terabits per second (Tbps). The largest attack peaked 3.8 Tbps — the largest ever disclosed publicly by any organization. Detection and mitigation was fully autonomous. The graphs below represent two separate attack events that targeted the same Cloudflare customer and were mitigated autonomously.
A mitigated 3.8 Terabits per second DDoS attack that lasted 65 seconds
A mitigated 2.14 billion packet per second DDoS attack that lasted 60 seconds
Cloudflare customers are protected
Cloudflare customers using Cloudflare’s HTTP reverse proxy services (e.g. Cloudflare WAF and Cloudflare CDN) are automatically protected.
Cloudflare customers using Spectrum and Magic Transit are also automatically protected. Magic Transit customers can further optimize their protection by deploying Magic Firewall rules to enforce a strict positive and negative security model at the packet layer.
Other Internet properties may not be safe
The scale and frequency of these attacks are unprecedented. Due to their sheer size and bits/packets per second rates, these Continue reading
Patent troll Sable pays up, dedicates all its patents to the public!
Back in February, we celebrated our victory at trial in the U.S. District Court for the Western District of Texas against patent trolls Sable IP and Sable Networks. This was the culmination of nearly three years of litigation against Sable, but it wasn’t the end of the story.
Today we’re pleased to announce that the litigation against Sable has finally concluded on terms that we believe send a strong message to patent trolls everywhere — if you bring meritless patent claims against Cloudflare, we will fight back and we will win.
We’re also pleased to announce additional prizes in Project Jengo, and to make a final call for submissions before we determine the winners of the Final Awards. As a reminder, Project Jengo is Cloudflare’s effort to fight back against patent trolls by flipping the incentive structure that has encouraged the growth of patent trolls who extract settlements out of companies using frivolous lawsuits. We do this by asking the public to help identify prior art that can invalidate any of the patents that a troll holds, not just the ones that are asserted against Cloudflare. We’ve already given out over $125,000 to individuals since the launch Continue reading
How XtendISE Helps with 802.1X Management in ISE?
XtendISE is a simple web application connected to your Cisco ISE, which helps with everyday routine tasks and common challenges related to 802.1X without the need to train everyone in Cisco ISE. XtendISE can help you manage MAC addresses and troubleshoot 802.1X authentications. It also helps with managing the switch's 802.1x configuration or validating the configurations to make sure that they are configured as intended.
All the mentioned features save time for us Network Engineers and help us to do our job efficiently as we do not waste our time on routine tasks. It also increases network security because it makes sure that our network is configured correctly and thus is safe and secured.
What Company Is XtendISE Intended For?
XtendISE is suitable for a company of any size with Cisco ISE and Cisco network devices. However medium or large companies will better use XtendISE features because they are more likely affected by the mentioned problems.
XtendISE helps the Helpdesk or IT Support with
- Easy to use MAC address management and troubleshooting
- They save time and there is no need for extra knowledge on ISE
- They can manage switch port configurations
- They do not Continue reading
HW037: Is Wi-Fi 7 the Game Changer We’ve Been Waiting For?
Wi-Fi 7 brings new features and capabilities to the market, including Multi-Link Operation (MLO) and airtime optimization, which can enhance throughput and reduce latency. In today’s episode of Heavy Wireless, host Keith Parsons and guest Phil Morgan explore the advancements and practical implications of Wi-Fi 7. Phil shares his experiences testing Wi-Fi 7 with various... Read more »PP033: AI and Machine Learning 101 for Cybersecurity
The terms “AI” and “machine learning (ML)” get thrown around pretty regularly in IT and cybersecurity. On today’s Packet Protector we get an introduction to AI and ML to help you ask the right questions when vendors tout their latest AI-infused products. Our guest is Jeff Crume, a distinguished engineer and cybersecurity architect at IBM.... Read more »HS084: Straight Talk from Cisco’s CIO (Sponsored)
Cisco CIO Fletcher Previn visits Heavy Strategy to talk about how to create a technology culture, what it’s like being “customer zero”, and other challenges and tribulations of running IT from inside one of the world’s largest tech companies. Episode Guest: Fletcher Previn, SVP & Chief Information Officer, Cisco Fletcher Previn has over two decades... Read more »
Dell Optiplex 7050 SFF Upgrade for Proxmox
The Dell OptiPlex 7050 SFF is a capable machine for virtualization thanks to its Intel […]
The post Dell Optiplex 7050 SFF Upgrade for Proxmox first appeared on Brezular's Blog.
NB497: Blocking AI Scraper Bots; HPE, Arista Reach for Bigger Slices of the Enterprise
Take a Network Break! This week we discuss significant vulnerabilities in HPE APs and Cisco Catalyst 9K switches, the benefits of bug bounties, how HPE is adding third-party device monitoring to Aruba Central, and whether Arista can succeed at positioning CloudVision as a management platform for enterprise data center, campus, and WAN environments. Alkira’s cloud... Read more »Repost: Layer-3-Only EVPN Behind the Scenes
After publishing the EVPN L3VPN lab-building instructions, I published a deep dive into EVPN and data-plane data structures. You might have missed it, as it was published in mid-August.
Impact of Verizon’s September 30 outage on Internet traffic
On Monday, September 30, customers on Verizon’s mobile network in multiple cities across the United States reported experiencing a loss of connectivity. Impacted phones showed “SOS” instead of the usual bar-based signal strength indicator, and customers complained of an inability to make or receive calls on their mobile devices.
AS6167 (CELLCO) is the autonomous system used by Verizon for its mobile network. To better understand how the outage impacted Internet traffic on Verizon’s network, we took a look at HTTP request volume from AS6167 independent of geography, as well as traffic from AS6167 in various cities that were reported to be the most significantly impacted.
Although initial reports of connectivity problems started around 09:00 ET (13:00 UTC), we didn’t see a noticeable change in request volume at an ASN level until about two hours later. Just before 12:00 ET (16:00 UTC), Verizon published a social media post acknowledging the problem, stating “We are aware of an issue impacting service for some customers. Our engineers are engaged and we are working quickly to identify and solve the issue.”
As the Cloudflare Radar graph below shows, a slight decline (-5%) in HTTP traffic as compared to traffic at the Continue reading
Connection coalescing breaks the Internet
Connection coalescing is the dumbest idea to ever reach RFC status. I can’t believe nobody stopped it before it got this far.
It breaks everything.
Thus starts my latest opinion post.
What is connection coalescing?
It’s specified in the RFC for HTTP/2 as connection reuse, but tl;dr: If the IP address of host A and B overlap, and host A presents a TLS cert that also includes B (via explicit CN/SAN or wildcard cert), then the client is allowed to send HTTP requests directed to B on the connection that was established to A.
Why did they do that?
To save roundtrips and TLS handshakes. It seems like a good idea if you don’t think about it too much.
Why does it break everything?
I’ll resist just yelling “layering violation”, because that’s not helpful. Instead I’ll be more concrete.
Performing connection coalescing is a client side (e.g. browser) decision. But it implicitly mandates a very strict server architecture. It assumes that ALL affected hostnames are configured exactly the same in many regards, and indeed that the HTTP server even has the config for all hostnames.
Concrete things that this breaks:
- The server can’t have a freestanding TLS termination layer, Continue reading
Tech Bytes: Security and Operational Benefits of Fortinet Unified SASE (Sponsored)
Secure Access Service Edge, or SASE, brings together networking and cloud-delivered security services. On today’s Tech Bytes podcast we with talk with sponsor Fortinet about Unified SASE; that is, where a single provider offers both the networking and cloud elements of the service. We discuss essential elements of Unified SASE, Fortinet’s notion of sovereign SASE,... Read more »New Day, New Role
I’m thrilled to announce that I’ve recently stepped into a new role as Solutions Architect at Sweetwater Technology Services! This opportunity marks a significant milestone in my career, allowing me to deepen my expertise and broaden my impact in designing and delivering IT solutions for clients across diverse sectors. What This Means In this role, …
The post New Day, New Role first appeared on StaticNAT.
Leadership vs Management
Leadership and management these two are often used interchangeably but are they interchangeable? Do we consider all managers to also be leaders and are all leaders managers? While the best managers will have traits of a good leader and visa-versa these two roles are not always synonymous. First thing we will want to do is […]AI for Network Engineers: Chapter 2 – Backpropagation Algorithm: Introduction
This chapter introduces the training model of a neural network based on the Backpropagation algorithm. The goal is to provide a clear and solid understanding of the process without delving deeply into the mathematical formulas, while still explaining the fundamental operations of the involved functions. The chapter also briefly explains why, and in which phases the training job generates traffic to the network, and why lossless packet transport is required. The Backpropagation algorithm is composed of two phases: the Forward pass (computation phase) and the Backward pass (adjustment and communication phase).
In the Forward pass, neurons in the first hidden layer calculate the weighted sum of input parameters received from the input layer, which is then passed to the neuron's activation function. Note that neurons in the input layer are not computational units; they simply pass the input variables to the connected neurons in the first hidden layer. The output from the activation function of a neuron is then used as input for the connected neurons in the next layer, whether it is another hidden layer or the output layer. The result of the activation function in the output layer represents the model's prediction, which is compared to the expected Continue reading
Wrapping up another Birthday Week celebration
2024 marks Cloudflare’s 14th birthday. Birthday Week each year is packed with major announcements and the release of innovative new offerings, all focused on giving back to our customers and the broader Internet community. Birthday Week has become a proud tradition at Cloudflare and our culture, to not just stay true to our mission, but to always stay close to our customers. We begin planning for this week of celebration earlier in the year and invite everyone at Cloudflare to participate.
Months before Birthday Week, we invited teams to submit ideas for what to announce. We were flooded with submissions, from proposals for implementing new standards to creating new products for developers. Our biggest challenge is finding space for it all in just one week — there is still so much to build. Good thing we have a birthday to celebrate each year, but we might need an extra day in Birthday Week next year!
In case you missed it, here’s everything we announced during 2024’s Birthday Week:
Monday
|
What |
In a sentence… |
|
Start auditing and controlling the AI models accessing your content |
Understand which AI-related bots and crawlers can access your website, and which content you choose to allow Continue reading |
IS-IS Labs: Configure IS-IS Routing for IPv4
In the first exercise in the IS-IS labs series, you’ll configure IS-IS routing for IPv4. The basic configuration is trivial, but you’ll also have to tweak the defaults that most vendors got wrong (we’ll discuss why those defaults are wrong in the next lab exercises).
I also tried to make the IS-IS labs more than just lab exercises. Each exercise includes a bit of background information or IS-IS theory; this one describes generic OSI addresses (NSAPs) and router addresses (NETs).
Consolidating WAN Backbone with EVPN
Recently I was exploring how to consolidate layer-2 & layer-3 connectivity across WAN backbone network. In this write I have covered (IP-Prefixes handling as EVPN Type 5 Routes) and in subsequent writeup I will cover E-LAN and E-Line use cases as well.
https://github.com/kashif-nawaz/Consolidating_WAN_Backbone_with_EVPN