MANRS for Enterprise Customers

In October 2023, I was talking about Internet routing security at the DEEP conference in Zadar, Croatia. After explaining the (obvious) challenges and the initiatives aimed at making Internet routing more secure (MANRS), I made my usual recommendation: vote with your wallet. However, if you’re a company in Croatia (or Slovenia, or a number of other countries), you’re stuck.

While ISPs in Croatia might be doing a great job, none of them is a MANRS participant1, so we don’t know how good they are. The situation is not much better in Slovenia; the only ISPs claiming to serve Slovenia are Anexia (a cloud provider) and Go6 Institute, the small network operated by my good friend (and True Believer in IPv6 and MANRS) Jan Žorž. Moving further north, I was unable to get any useful data for Austria, as its country code (AT) also matches “No Data” string in MANRS table, resulting in over 500 hits.

Read more …

netlab Graphs with Multi-Access Links

A netlab user wanted to create a nice-looking topology graph from a simple topology connecting a few devices to a broadcast (multi-access) link. I don’t have his exact topology, so we’ll use this one (skipping the details like setting device types)

nodes: [ r1, r2, h1, h2 ]

links:
- r1-r2
- interfaces: [ r1, r2, h1, h2 ]

This is what GraphViz generates based on netlab’s description of the lab topology:

Read more …

Worth Reading 031526


As malicious domains continue to pose a threat across the Internet, security professionals are left to consider the measures that can effectively defend their organizations from these threats.

 


Right now, a growing chorus is treating India’s “lock and suspend” approach as a success story and a model for the world.

 


On the last day of December 2016 there was a brief hiccup as the world’s clocks adjusted their time according to the Coordinated Universal Time (UTC) standard by adding an extra second to the last minute of the 31st of December.

 


For the past decade, Docker has provided a robust solution for building, shipping, and sharing applications. But behind its simple “build and run” workflow lie many years of complex technical challenges.

 


AI assistants that produce drawings and other output that looks like evidence reduce questioning without increasing correctness.

Network Device Telemetry Protocols with Dinesh Dutt

Whenever I’m ranting about vendors changing their data models or APIs with every other release, there is inevitably a vendor engineer chiming in, saying, “Life would be so much better if the customers wouldn’t insist on doing screen scraping for the last 50 years.”

While some of that screen scraping is pure inertia, we sometimes have good reasons to do it rather than use protocols like NETCONF, gNMI, or protobufs. In Episode 205 of Software Gone Wild, I’m discussing some of those reasons and exploring the gap between vendor theory and reality with Dinesh Dutt, who is unlucky enough to have become the world’s foremost expert on crappy network telemetry.

Listen to the podcast

From legacy architecture to Cloudflare One

For a network engineer, the cutover weekend is often the most stressful 48 hours of their career. Imagine a 30,000-user organization attempting to flip 1,000+ legacy applications from fragmented VPNs to a new architecture in a single window. The stakes are immense: a single misconfigured firewall rule or a timed-out session can halt essential services and lead to operational gridlock.

This "big bang" migration risk is the single greatest barrier to Zero Trust adoption. Organizations often feel trapped between an aging, vulnerable infrastructure and a migration process that feels too risky to attempt.

Cloudflare and Technology Solutions Provider CDW are changing this narrative. We believe that a successful transition to SASE (Secure Access Service Edge) shouldn't feel like a leap into the dark. By combining Cloudflare’s global Zero Trust platform with CDW’s experience navigating the industry’s most complex deployment failures, we provide the strategic roadmap to de-risk the journey. We don't just move your "plumbing" — we ensure your legacy debt is transformed into a modern, agile security posture without the downtime.

Leveraging partner expertise to avoid migration traps

Traditional migrations often fail because they treat the network as simple plumbing rather than a complex ecosystem of applications. Without a Continue reading

NetFlow/sFlow Monitoring in AI Environments

aadIf you’ve spent time supporting AI infrastructure, whether that’s a GPU training cluster, a fleet of inference nodes, or a multi-tenant model serving platform, you’ve probably noticed something: the network telemetry tools that served you well in a traditional data center feel slightly out of place here. Not useless. Just not quite designed for this.

The traffic patterns are different. The failure modes are different. The things you need to catch early are different. And if you’re running NetFlow or sFlow collection – which you should be – understanding where that data genuinely helps versus where you’re looking at the wrong instrument is the difference between a useful monitoring stack and a false sense of coverage.

Why AI Traffic Is Different

Most of the networking intuition you’ve built over a career was forged on north-south traffic – clients reaching services, users reaching the internet, workloads reaching storage. Even in modern microservices environments with heavy east-west traffic, flows are relatively short-lived, heterogeneous in size, and largely TCP-based with normal congestion dynamics.

AI training breaks most of those assumptions simultaneously.

A distributed training job across a GPU cluster is synchronous in a way that most networked workloads are not. Every GPU in Continue reading

IPv4 ECMP Works on Arista cEOS Release 4.35.2F

When I wrote about the anycast-ECMP-in-MPLS behavior in 2011, I had to use Cisco IOS to prove that ECMP worked, since Arista cEOS (running the Linux kernel for IP forwarding) didn’t install more than one equal-cost path into the Linux forwarding table.

Arista cEOS got better in the meantime; IPv4 ECMP works like a charm on cEOS release 4.35.02F. With the same lab topology I’d used in 2021, I was able to see the traffic spread across multiple nodes:

Read more …

Announcing Cloudflare Account Abuse Protection: prevent fraudulent attacks from bots and humans

Today, Cloudflare is introducing a new suite of fraud prevention capabilities designed to stop account abuse before it starts. We've spent years empowering Cloudflare customers to protect their applications from automated attacks, but the threat landscape has evolved. The industrialization of hybrid automated-and-human abuse presents a complex security challenge to website owners. Consider, for instance, a single account that’s accessed from New York, London, and San Francisco in the same five minutes. The core question in this case is not “Is this automated?” but rather “Is this authentic?” 

Website owners need the tools to stop abuse on their website, no matter who it’s coming from.

During our Birthday Week in 2024, we gifted leaked credentials detection to all customers, including everyone on a Free plan. Since then, we've added account takeover detection IDs as part of our bot management solution to help identify bots attacking your login pages. 

Now, we’re combining these powerful tools with new ones. Disposable email check and email risk help you enforce security preferences for users who sign up with throwaway email addresses, a common tactic for fake account creation and promotion abuse, or whose emails are deemed risky based on email patterns Continue reading

Slashing agent token costs by 98% with RFC 9457-compliant error responses

AI agents are no longer experiments. They are production infrastructure, making billions of HTTP requests per day, navigating the web, calling APIs, and orchestrating complex workflows.

But when these agents hit an error, they still receive the same HTML error pages we built for browsers: hundreds of lines of markup, CSS, and copy designed for human eyes. Those pages give agents clues, not instructions, and waste time and tokens. That gap is the opportunity to give agents instructions, not obstacles.

Starting today, Cloudflare returns RFC 9457-compliant structured Markdown and JSON error payloads to AI agents, replacing heavyweight HTML pages with machine-readable instructions.

That means when an agent sends Accept: text/markdown, Accept: application/json, or Accept: application/problem+json and encounters a Cloudflare error, we return one semantic contract in a structured format instead of HTML. And it comes complete with actionable guidance. (This builds on our recent Markdown for Agents release.)

So instead of being told only "You were blocked," the agent will read: "You were rate-limited — wait 30 seconds and retry with exponential backoff." Instead of just "Access denied," the agent will be instructed: "This block is intentional: do not retry, contact the site owner."

Continue reading

AI Security for Apps is now generally available

Cloudflare’s AI Security for Apps detects and mitigates threats to AI-powered applications. Today, we're announcing that it is generally available.

We’re shipping with new capabilities like detection for custom topics, and we're making AI endpoint discovery free for every Cloudflare customer—including those on Free, Pro, and Business plans—to give everyone visibility into where AI is deployed across their Internet-facing apps.

We're also announcing an expanded collaboration with IBM, which has chosen Cloudflare to deliver AI security to its cloud customers. And we’re partnering with Wiz to give mutual customers a unified view of their AI security posture.

A new kind of attack surface

Traditional web applications have defined operations: check a bank balance, make a transfer. You can write deterministic rules to secure those interactions. 

AI-powered applications and agents are different. They accept natural language and generate unpredictable responses. There's no fixed set of operations to allow or deny, because the inputs and outputs are probabilistic. Attackers can manipulate large language models to take unauthorized actions or leak sensitive data. Prompt injection, sensitive information disclosure, and unbounded consumption are just a few of the risks cataloged in the OWASP Top 10 for LLM Applications.

These risks escalate as AI Continue reading

Dynamic Path MTU Discovery in Cloudflare One Client

Here’s an interesting tidbit from the what took them so long department: Cloudflare One Client continuously measures end-to-end MTU and adjusts the local tunnel interface MTU size accordingly (warning: there’s a fair amount of dubious handwaving over the interesting details), generating ICMP packet-too-big messages as close to the source as possible.

I managed to avoid VPN clients most of my life, so I have no idea whether this is a “finally someone figured that out 🎉” moment or a late catch-up to what other VPN clients have been doing for ages. Feedback (in comments or otherwise) would be most welcome!

Monitoring RoCEv2 with sFlow

The talk Seeing Through the RDMA Fog: Monitoring RoCEv2 with sFlow at the recent North American Network Operator's Group (NANOG) conference describes how leveraging industry standard sFlow telemetry from data center switches provides visibility into RDMA activity in AI / ML networks.

Note: Slides are available from the talk link.

The live SDSC Expanse cluster live AI/ML metrics dashboard described in the talk can be accesses by clicking on the dashboard link. The San Diego Supercomputer Center (SDSC) Expanse cluster specifications: 5 Pflop/s peak; 93,184 CPU cores; 208 NVIDIA GPUs; 220 TB total DRAM; 810 TB total NVMe.

Note: AI Metrics with Prometheus and Grafana shows how to set up the monitoring stack.

More recently, Expanse heatmap provides a publicly accessible real-time visualization live traffic flowing between nodes in the Expanse cluster, see Real-time visualization of AI / ML traffic matrix for more information.

How AI Agents Communicate: Understanding the A2A Protocol for Kubernetes

Since the rise of Large Language Models (LLMs) like GPT-3 and GPT-4, organizations have been rapidly adopting Agentic AI to automate and enhance their workflows. Agentic AI refers to AI systems that act autonomously, perceiving their environment, making decisions, and taking actions based on that information rather than just reacting to direct human input. In many ways, this makes AI agents similar to intelligent digital assistants, but they are capable of performing much more complex tasks over time without needing constant human oversight.

What is an AI Agent

An AI Agent is best thought of as a long-lived, thinking microservice that owns a set of perception, decision-making, and action capabilities rather than simply exposing a single API endpoint. These agents operate continuously, handling tasks over long periods rather than responding to one-time requests.

AI Agents in Kubernetes Environments

In Kubernetes environments, each agent typically runs as a pod or deployment and relies on the cluster network, DNS and possibly a service mesh to talk to tools and other agents.

Frameworks like Kagent help DevOps and platform engineers define and manage AI agents as first-class Kubernetes workloads. This means that instead of using custom, ad-hoc scripts to manually manage AI agents, Continue reading

1 2 3 … 3,853