AI network performance monitoring using containerlab

AI Metrics is available on GitHub. The application provides performance metrics for AI/ML RoCEv2 network traffic, for example, large scale CUDA compute tasks using NVIDIA Collective Communication Library (NCCL) operations for inter-GPU communications: AllReduce, Broadcast, Reduce, AllGather, and ReduceScatter.

The screen capture is from a containerlab topology that emulates a AI compute cluster connected by a leaf and spine network. The metrics include:

  • Total Traffic Total traffic entering fabric
  • Operations Total RoCEv2 operations broken out by type
  • Core Link Traffic Histogram of load on fabric links
  • Edge Link Traffic Histogram of load on access ports
  • RDMA Operations Total RDMA operations
  • RDMA Bytes Average RDMA operation size
  • Credits Average number of credits in RoCEv2 acknowledgements
  • Period Detected period of compute / exchange activity on fabric (in this case just over 0.5 seconds)
  • Congestion Total ECN / CNP congestion messages
  • Errors Total ingress / egress errors
  • Discards Total ingress / egress discards
  • Drop Reasons Packet drop reasons

Note: Clicking on peaks in the charts shows values at that time.

This article gives step-by-step instructions to run the demonstration.

git clone https://github.com/sflow-rt/containerlab.git
Download the sflow-rt/containerlab project from GitHub. 
git clone https://github.com/sflow-rt/containerlab.git
cd containerlab
./run-clab
Run the above commands Continue reading

Finding Source Routing Paths

In the previous blog post, we discussed the generic steps that network devices (or a centralized controller) must take to discover paths across a network. Today, we’ll see how these principles are applied in source routing, one of the three main ways to move packets across a network.

Brief recap: In source routing, the sender has to specify the (loose or strict) path a packet should take across the network. The sender thus needs a mechanism to determine that path, and as always, there are numerous solutions to this challenge. We’ll explore a few of them, using the sample topology shown in the following diagram.

Read more …

Cloudflare service outage June 12, 2025

On June 12, 2025, Cloudflare suffered a significant service outage that affected a large set of our critical services, including Workers KV, WARP, Access, Gateway, Images, Stream, Workers AI, Turnstile and Challenges, AutoRAG, Zaraz, and parts of the Cloudflare Dashboard.

This outage lasted 2 hours and 28 minutes, and globally impacted all Cloudflare customers using the affected services. The cause of this outage was due to a failure in the underlying storage infrastructure used by our Workers KV service, which is a critical dependency for many Cloudflare products and relied upon for configuration, authentication and asset delivery across the affected services. Part of this infrastructure is backed by a third-party cloud provider, which experienced an outage today and directly impacted availability of our KV service.

We’re deeply sorry for this outage: this was a failure on our part, and while the proximate cause (or trigger) for this outage was a third-party vendor failure, we are ultimately responsible for our chosen dependencies and how we choose to architect around them.

This was not the result of an attack or other security event. No data was lost as a result of this incident. Cloudflare Magic Transit and Magic WAN, DNS, cache, proxy, Continue reading

AMD Plots Interception Course With Nvidia GPU And System Roadmaps

To a certain extent, Nvidia and AMD are not really selling GPU compute capacity as much as they are reselling just enough HBM memory capacity and bandwidth to barely balance out the HBM memory they can get their hands on, thereby justifying the ever-embiggening amount of compute their GPU complexes get overstuffed with. …

AMD Plots Interception Course With Nvidia GPU And System Roadmaps was written by Timothy Prickett Morgan at The Next Platform.

It’s Been Three Years, So It’s Time For Another PCI-Express Speed Bump

PCI-SIG, the organization that oversees the roadmap for the critical PCI-Express peripheral attachment specification, is continuing to keep to its three-year drumbeat for releasing the next iteration of the interconnect spec and already has its sights on the one after that, expected to be released in 2028 and appear in devices in 2030 or so. …

It’s Been Three Years, So It’s Time For Another PCI-Express Speed Bump was written by Jeffrey Burt at The Next Platform.

N4N030: Network Shapes and Sizes

What shape is your network? In other words, what is its topology? On today’s episode, we discover the different types of network topologies and designs used in the enterprise, data center, and service provider networks. We cover leaf/spine, hub and spoke, point to point, mesh, and others. We also talk about how topologies affect traffic... Read more »

Cisco’s Hyperscale And Cloud AI Push Will Give It Enterprise Clout

Much of the business that Cisco Systems and others have been doing in the AI infrastructure field since OpenAI lit the generative AI fuse with ChatGPT in November 2022 has been deploying hardware and software with the hyperscalers, a lucrative business that led company executives to promise to sell as much as $1 billion in back-end network technology by the end of its fiscal year and then to blow past that a quarter early. …

Cisco’s Hyperscale And Cloud AI Push Will Give It Enterprise Clout was written by Jeffrey Burt at The Next Platform.

Powering All Ethernet AI Networking

Powering All Ethernet AI Networking

Artificial Intelligence (AI), powered by accelerated processing units (XPUs) like GPUs and TPUs, is transforming industries. The network interconnecting these processors is crucial for efficient and successful AI deployments. AI workloads, involving intensive training and rapid inferencing, require very high bandwidth interconnects with low and consistent latency, and the highest reliability to maximize XPU utilization and reduce AI job completion time (JCT). A best-of-breed network with AI-specific optimizations is critical for delivering AI applications, with any JCT slowdown leading to revenue loss. Typical workloads have fewer, very high-bandwidth, low-entropy flows that run for extended periods, exchanging large messages synchronously, necessitating advanced lossless forwarding and specialized operational tools. They differ from cloud networking traffic as summarized below:

Celebrating 11 years of Project Galileo’s global impact

June 2025 marks the 11th anniversary of Project Galileo, Cloudflare’s initiative to provide free cybersecurity protection to vulnerable organizations working in the public interest around the world. From independent media and human rights groups to community activists, Project Galileo supports those often targeted for their essential work in human rights, civil society, and democracy building.

A lot has changed since we marked the 10th anniversary of Project Galileo. Yet, our commitment remains the same: help ensure that organizations doing critical work in human rights have access to the tools they need to stay online.  We believe that organizations, no matter where they are in the world, deserve reliable, accessible protection to continue their important work without disruption.

For our 11th anniversary, we're excited to share several updates including:

  • An interactive Cloudflare Radar report providing insights into the cyber threats faced by at-risk public interest organizations protected under the project. 

  • An expanded commitment to digital rights in the Asia-Pacific region with two new Project Galileo partners.

  • New stories from organizations protected by Project Galileo working on the frontlines of civil society, human rights, and journalism from around the world.

Tracking and reporting on cyberattacks with the Project Galileo Continue reading

ArubaCX Cannot Count When Dealing with VXLAN

This blog post describes yet another bizarre example of how reliable digital twins are, but don’t worry; they all work great in PowerPoint.

After “fixing” the integration tests to deal with ArubaCX’s notion of VXLAN VNI having 16 bits, the bridging test worked, but the IRB tests kept failing.

In the IRB test, the lab has two layer-3 switches. Each of them should be able to bridge within a VLAN/VXLAN segment and route across the segments.

Read more …

We shipped FinalizationRegistry in Workers: why you should never use it

We’ve recently added support for the FinalizationRegistry API in Cloudflare Workers. This API allows developers to request a callback when a JavaScript object is garbage-collected, a feature that can be particularly relevant for managing external resources, such as memory allocated by WebAssembly (Wasm). However, despite its availability, our general advice is: avoid using it directly in most scenarios.

Our decision to add FinalizationRegistry — while still cautioning against using it — opens up a bigger conversation: how memory management works when JavaScript and WebAssembly share the same runtime. This is becoming more common in high-performance web apps, and getting it wrong can lead to memory leaks, out-of-memory errors, and performance issues, especially in resource-constrained environments like Cloudflare Workers.

In this post, we’ll look at how JavaScript and Wasm handle memory differently, why that difference matters, and what FinalizationRegistry is actually useful for. We’ll also explain its limitations, particularly around timing and predictability, walk through why we decided to support it, and how we’ve made it safer to use. Finally, we’ll talk about how newer JavaScript language features offer a more reliable and structured approach to solving these problems.

Memory management 101

JavaScript

JavaScript relies on automatic memory management through a Continue reading

netlab 2.0: Routers, Hosts, Gateways and Bridges

In a previous blog post, I explained how you can use bridges in a netlab topology to create custom LAN segments. Netlab supports two other node roles (host and router), and we’ll eventually add gateways.

netlab assumes that most network devices are routers (it considers a firewall to be a router in disguise), apart from Linux hosts, but you can always change what a node is with the role node attribute:

Read more …

Secure and Scalable Kubernetes for Multi-Cluster Management

This story is becoming more and more common in the Kubernetes world. What starts as a manageable cluster or two can quickly balloon into a sprawling, multi-cluster architecture spanning public clouds, private data centers, or a bit of both. And with that growth comes a whole new set of headaches. How do you keep tabs on compliance across wildly different configurations? When a service goes down across multiple clusters, how do you pinpoint the cause amidst the chaos? And what about those hard-to-diagnose latency issues that seem to crop up between regions?

The truth is, achieving secure and scalable multi-cluster Kubernetes isn’t about throwing more tools at the problem. It’s about having the right tools and adopting the right best practices. This is where a solution like Calico Cluster Mesh shines, offering those essential capabilities for a seamless multi-cluster experience without the complexity or overhead that you expect with traditional service meshes.

The Multi-Cluster Challenge: When Complexity Takes Over

So, why are so many organizations finding themselves in this multi-cluster maze? Often, it’s driven by solid business reasons:

  • High Availability and Disaster Recovery: Spreading workloads across multiple regions or clusters means that if one goes down, your users shouldn’t notice.
  • Continue reading
1 … 65 66 67 68 69 … 3,859