Workers AI runs inference for some of the best open models in the world on GPUs in Cloudflare data centers close to your users. Two of the most capable, and most demanding, are Moonshot's Kimi K-series and Z.ai's GLM. They are large, long-context, mixture-of-experts models, and they are wonderful to use. They are also very hard to serve efficiently because of memory constraints.
We've written before about how we serve large models on Workers AI and about separating the prefill and decode phases of inference to get more out of each GPU. This post looks at three techniques we layer on top of that to fit these models into memory and keep them fast: quantizing the KV cache, compressing the model weights, and, because both of those pack more requests onto shared hardware, protecting the cache those requests share. These optimizations enable us to support more customers at lower costs, with no change in model accuracy.
All our experiments and production traffic are running and benchmarked with SGLang, an open-source inference serving framework. We found that SGLang offers the best performance in the market, and we work closely with the SGLang team to upstream patches and new Continue reading
Agents Week is about the shift already underway: agents write code, deploy Workers, and provision infrastructure on your behalf. That shift changes what you need to see. If a program is spending money in your Cloudflare account, you need to know what it's spending; throughout the day, per product, in a shape another program can consume. The dashboard is the right answer for humans. It's not the right answer for automation.
So we're launching a new Billable Usage API for self-serve accounts: a single endpoint that returns your account's usage and cost, broken down by product and by service period. It covers every usage-based Cloudflare product on the account, including Workers, R2, D1, Workers AI, Vectorize, Images, and Stream, all with one call. And if you already work in a FinOps toolchain, the column names should look familiar.
You'll get back an HTTP 200 OK with Content-Type: application/json and the usage rows in the response body. Today, usage and cost data are updated daily while we work towards providing more real time data.
Each row in the response is one charge period for one product on your account.
ServiceName and ServiceFamilyName — which product Continue readingAI is changing how people interact with computers, and voice is becoming an increasingly important part of that shift. Real-time assistants, AI-powered dictation, and other voice interfaces need low-latency communication between clients, models, and supporting services. Many developers use gRPC, a Remote Procedure Call (RPC) framework built on HTTP/2 and TCP, for this infrastructure.
Ever since Workers launched in 2017, we’ve been expanding their capabilities, including adding the ability to open outbound TCP connections and a JavaScript-native RPC system built on Cap’n Proto. And so as part of Agents Week, we’re extending Workers in the other direction, supporting inbound TCP connections and adding new ways to run gRPC applications on Cloudflare.
Today, we’re announcing:
We’re introducing this in private beta — Continue reading
In this guide, we install TorBox 0.54 on a Raspberry Pi 3B model and walk […]
The post How to install TorBox on Raspberry Pi 3B first appeared on Brezular's Blog.
This week is Agents Week.
As we started thinking about and planning the week, we wrestled with a broader question of what it means to support this new era of agents and what a purpose-built foundation for agents actually looks like. Which brought us to a simpler framing: what is an Agent Cloud?
We quickly realized however, that our framing was wrong. Not because it’s the wrong question to ask, but because of who we were asking — ourselves, instead of our agents. It’s no longer about us and what we think, but about what agents need.
That, in a nutshell, is what Agents Week is about.
The cloud we have today, and the web it sits on, were built for people. Every layer assumes a human is watching: pages designed to hold your attention, dashboards to click through, interfaces tuned for how we read and decide. But agents don't work that way. They don't get distracted, tired or fatigued… and they have their own needs around speed, structure, and access.
An Agent Cloud has to do two things at once. It has to set us up for an agent-native future, where the primitives are built Continue reading
Sometimes we can be so focused on the problems in front of us that we forget the people who are following in our footsteps. Michael McCollum, a new network engineer, joins Tom and Russ to ask questions about life as a network engineer.
download
Last year, we enabled Media over QUIC (MoQ) on every Cloudflare server and opened the network for anyone to test. It provided a global MoQ endpoint, but not the isolation and access controls needed to run an application.
Today, we’re adding those isolation and access controls. The new MoQ provisioning API lets you create an isolated relay for your application and issue separate credentials for publishers and subscribers. The relays you create are available across Cloudflare’s network within seconds, with no servers to deploy, size, or load balance.
Cloudflare now supports the draft-14 and draft-16 versions of the MoQ Transport protocol with authentication support.
You can create relays through the API and the Cloudflare dashboard. They are completely free to use during beta.
MoQ (originally short for Media over QUIC) is a new open protocol under development at the Internet Engineering Task Force (IETF), the standards body that also standardized HTTP, TLS, and QUIC. It is being developed in the open and will become a free public standard (an RFC) that anyone can implement. No single company owns it.
MoQ is a publish/subscribe system. A publisher sends out streams of data that have names, and Continue reading
As of June 23, 2026, cdnjs, one of the Internet's busiest open-source CDNs, is running exclusively on Cloudflare’s Developer Platform. Along the way, cdnjs surfaced limits in the platform, and the platform grew to meet them.
cdnjs is a free, open-source content delivery network for JavaScript and CSS libraries. Instead of using a bundler or self-hosting jQuery, Bootstrap, or Lodash, you drop a <script> tag pointing to cdnjs.cloudflare.com and the library loads from Cloudflare's edge, instantly, anywhere in the world, with no signup, no API keys, and no rate limits. It's the infrastructure behind a significant portion of “intro to JavaScript” tutorials, CodePen demos, and Stack Overflow answers.
Community-driven, cdnjs is used on roughly 12% of all websites, a 48.3% share of the JavaScript CDN market. It serves an average of 108,000 requests per second, 9 billion per day, across more than 330 Cloudflare data centers, with a 98.6% cache hit rate. Pretty cool, Internet!
In 2011, when bundlers were exotic, npm was barely a year old, and "just drop a <script> tag" was how the web shipped JavaScript, Ryan Kirkman and Thomas Davis built cdnjs as a free, community-run mirror of every popular Continue reading
Cloudflare's Authenticated Origin Pulls and Custom Origin Trust Store now support post-quantum authentication.
Here we’ll explain how you can configure fully post-quantum secure mutually authenticated TLS connections to your origin server, dive into the engineering details of how we built it, make a shameful confession, and finally explain how this work fits into our overall post-quantum migration roadmap.
Our focus for the past several years has been in deploying post-quantum encryption to protect against harvest-now/decrypt-later attacks, where an attacker quietly stockpiles your encrypted data with the hope of decrypting it in the future with a quantum computer.
However, recent breakthroughs in quantum computing and cryptanalysis pulled the timelines for upgrading to post-quantum cryptography forward across industry and government and have caused us to shift our attention to deploying post-quantum authentication, to protect against attackers who will soon be able to use quantum computers to break classical credentials and carry out impersonation attacks.
In a previous post, we announced that Cloudflare is targeting 2029 for full post-quantum security, and laid out several milestones to hit along the way. We have reached the first of those milestones: our Authenticated Origin Pulls and Custom Origin Trust Store products Continue reading