Ultra Ethernet: Address Resolution with Address Vector Table

Address Vector


Overview

To enable Remote Memory Access (RMA) operations between processes, each endpoint — representing a communication channel much like a TCP socket — must know the destination process’s location within the fabric. This location is represented by the Fabric Address (FA) assigned to a Fabric Endpoint (FEP).

During job initialization, FAs are distributed through a control-plane–like procedure in which the master rank collects FAs from all ranks and then broadcasts the complete Rank-to-FA mapping to every participant (see Chapter 3 for details). Each process stores this Rank–FA mapping locally as a structure, which can then be inserted into the Address Vector (AV) Table.

When FAs from the distributed Rank-to-FA table are inserted into the AV Table, the provider assigns each entry an index number, which is published to the application as an fi_addr_t handle. After an endpoint object is bound to the AV Table, the application uses this handle — rather than the full address — when referencing a destination process. This abstraction hides the underlying address structure from the application and allows fast and efficient lookups during communication.

This mechanism resembles the functionality of a BGP Route Reflector (RR) in IP networks. Each RR client advertises its Continue reading

EVPN Designs: Multi-Pod with IP-Only WAN Routers

In the multi-pod EVPN design, I described a simple way to merge two EVPN fabrics into a single end-to-end fabric. Here are a few highlights of that design:

  • Each fabric is running OSPF and IBGP, with core (spine) devices being route reflectors
  • There’s an EBGP session between the WAN edge routers (sometimes called border leaf switches)
  • Every BGP session carries IPv4 (underlay) and EVPN (overlay) routes.

In that design, the WAN edge routers have to support EVPN (at least in the control plane) and carry all EVPN routes for both fabrics. Today, we’ll change the design to use simpler WAN edge routers that support only IP forwarding.

Read more …

Unpacking Cloudflare Workers CPU Performance Benchmarks

On October 4, independent developer Theo Browne published a series of benchmarks designed to compare server-side JavaScript execution speed between Cloudflare Workers and Vercel, a competing compute platform built on AWS Lambda. The initial results showed Cloudflare Workers performing worse than Node.js on Vercel at a variety of CPU-intensive tasks, by a factor of as much as 3.5x.

We were surprised by the results. The benchmarks were designed to compare JavaScript execution speed in a CPU-intensive workload that never waits on external services. But, Cloudflare Workers and Node.js both use the same underlying JavaScript engine: V8, the open source engine from Google Chrome. Hence, one would expect the benchmarks to be executing essentially identical code in each environment. Physical CPUs can vary in performance, but modern server CPUs do not vary by anywhere near 3.5x.

On investigation, we discovered a wide range of small problems that contributed to the disparity, ranging from some bad tuning in our infrastructure, to differences between the JavaScript libraries used on each platform, to some issues with the test itself. We spent the week working on many of these problems, which means over the past week Workers got better and faster Continue reading

PP082: Building a Workable Mobile Security Strategy In a World of Risky Apps

Today we’re bringing back one of our favorite guests — Akili Akridge. He’s a former Baltimore cop who transitioned to building and leading mobile offense and defense teams for federal agencies and Fortune 100s. These days he’s a straight-talking expert on all things mobile security. We’re digging into mobile threats, why they keep CISOs up... Read more »

Why Can’t We Have Good Documentation

Daniel Dib asked a sad question on LinkedIn:

Where did all the great documentation go?

In more detail:

There was a time when documentation answered almost all questions:

  • What is the thing?
  • What does the thing do?
  • Why would you use the thing?
  • How do you configure the thing?

I’ve seen the same thing happening in training, and here’s my cynical TL&DR answer: because the managers of the documentation/training departments don’t understand the true value of what they’re producing and thus cannot justify a decent budget to make it happen.

Read more …

netlab: Embed Configuration Templates in a Lab Topology File

A few days ago, I described how you can use the new config.inline functionality to apply additional configuration commands to individual devices in a netlab-powered lab.

However, sometimes you have to apply the same set of commands to several devices. Although you could use device groups to do that, netlab release 25.09 offers a much better mechanism: you can embed custom configuration templates in the lab topology file.

Read more …

Ultra Ethernet: Creating Endpoint Object

Endpoint Creation and Operation

[Updated 12-October, 2025: Figure & uet addressing section]

In libfabric and Ultra Ethernet Transport (UET), the endpoint, represented by the object fid_ep, serves as the primary communication interface between a process and the underlying network fabric. Every data exchange, whether it involves message passing, remote memory access (RMA), or atomic operations, ultimately passes through an endpoint. It acts as a software abstraction of the transport hardware, exposing a programmable interface that the application can use to perform high-performance data transfers.

Conceptually, an endpoint resembles a socket in the TCP/IP world. However, while sockets hide much of the underlying network stack behind a simple API, endpoints expose far more detail and control. They allow the process to define which completion queues to use, what capabilities to enable, and how multiple communication contexts are managed concurrently. This design gives applications, especially large distributed training frameworks and HPC workloads, direct control over latency, throughput, and concurrency in ways that traditional sockets cannot provide.

Furthermore, socket-based communication typically relies on the operating system’s networking stack and consumes CPU cycles for data movement and protocol handling. In contrast, endpoint communication paths can interact directly with the NIC, enabling user-space data transfers Continue reading

Netpicker NetBox Plugin and Automation

Netpicker NetBox Plugin and Automation

Netpicker brings together configuration backups, security and compliance checks, and automation in one place. It supports over 150 network vendors, including Cisco, Juniper, Arista, Palo Alto, Fortinet and many more. It also integrates well with other tools like Netbox, Nautobot, and Infrahub for inventory management, and Slurp’it for network discovery. All of these features, including backups, integrations, and automation, are available in the free version. The paid version adds features such as RBAC, approvals, detailed logging, support, and workflows for teams that need more control.

In our previous introductory post, we looked at what Netpicker is, how to set it up, and how it can back up configurations across multiple vendors. In this post, we'll focus on Netpicker Automation and how to use the Netpicker plugin with Netbox, so let's get to it.

SPONSORED

Netpicker has partnered with me for this post, and they also support my blog as a sponsor.

Learn more

Prerequisites

This post assumes you already have a functioning Netpicker and NetBox instances. If you're completely new to Netpicker, check out the introductory post first, where we covered the basics like installation and initial setup. For this example, we’re using NetBox Community version 4.3. Continue reading

Worth Reading 101025


A number of recent changes have helped to push Ethernet forward, advancing its capabilities to better meet the needs of AI.


Cloudflare has confessed to a coding error using a React useEffect hook, notorious for being problematic if not handled carefully, that caused an outage for the platform’s dashboard and many of its APIs.


Working with MikroTik and IP Infusion’s OcNOS to interop EVPN/VxLAN has been on my wish list for a long time.


Have you heard about MRT dumps, but never tried to use them because the bar seems too high? Or are you tired of doing “parse -> grep -> process” every time you touch BGP MRT dumps?


Unfortunately, history is again being rewritten. It is rapidly becoming clear that LLMs are not economical.


As part of our research into post-quantum cryptography (PQC) for DNSSEC, we test PQC as a drop-in replacement for classical algorithms. We explore a transition where both run simultaneously, analysing how resolvers validate records, edge cases, and the feasibility and impact of such a period.


In much of the world, we are in an era that I like to call the “post-gigabit era”. Many users have access to gigabit connections—or at least hundreds of Continue reading

Hedge 283: Technical Planning

We network engineers often find ourselves without a viable plan–our plans always seem to go awry, to the point that many network engineers just give up on planning. Is “giving up” the right solution? Or can we learn to be better planners? Jonathan Adams and Tim McConnaughy join Russ to discussion planning for network engineers.

download

IBM Ships Homegrown “Spyre” Accelerators, Embraces Anthropic For AI Push

Big Blue may have missed the boat on being one of the big AI model builders, but its IBM Research division has built its own enterprise-grade family of models and its server and research divisions have plenty of experience building accelerators and supercomputers. …

IBM Ships Homegrown “Spyre” Accelerators, Embraces Anthropic For AI Push was written by Timothy Prickett Morgan at The Next Platform.

TNO045: IP and Optical Integration: Automation Across Layers

LightRiver has software and service products focused on the automation, optimization, and simplification of multi-layer, multi-vendor, and multi-generation networking.  Today we have a team from LightRiver lead by Jim Brinksma to help explain how LightRiver is advancing automation in optical and bridging the gap between the IP and optical layers. They discuss the challenges, obstacles... Read more »
1 … 39 40 41 42 43 … 3,860