Today we are excited to announce that we are developing APIs and infrastructure to support more TCP, UDP, and QUIC-based protocols in Cloudflare Workers. Once released, these new capabilities will make it possible to use non-HTTP socket connections to and from a Worker or Durable Object as easily as one can use HTTP and WebSockets today.
Out of the box, fetch and WebSocket APIs. With just a few internal changes to make it operational in Workers, we’ve developed an example using an off-the-shelf driver (in this example, a Deno-based Postgres client driver) to communicate with a remote Postgres server via WebSocket over a secure Cloudflare Tunnel.
import { Client } from './driver/postgres/postgres'
export default {
async fetch(request: Request, env, ctx: ExecutionContext) {
try {
const client = new Client({
user: 'postgres',
database: 'postgres',
hostname: 'https://db.example.com',
password: '',
port: 5432,
})
await client.connect()
const result = await client.queryArray('SELECT * FROM users WHERE uuid=1;')
ctx.waitUntil(client.end())
return new Response(JSON.stringify(result.rows[0]))
} catch (e) {
return new Response((e as Error).message)
}
},
}
The example works by replacing the bits of the Postgres client driver that use the Deno-specific TCP socket APIs with standard fetch and WebSockets APIs. We then establish a WebSocket connection with a remote Cloudflare Tunnel daemon running adjacent to the Postgres server, establishing what is effectively TCP-over-WebSockets.
While the fact we were able to build the example and communicate effectively and efficiently with the Postgres server — without making any changes to the Cloudflare Workers runtime — is impressive, there are limitations to the approach. For one, the solution requires additional infrastructure to establish and maintain the WebSocket tunnel — in this case, the instance of the Cloudflare Tunnel daemon running adjacent to the Postgres server. While we are certainly happy to provide that daemon to customers, it would just be better if that component were not required at all. Second, tunneling TCP over WebSockets, which is itself tunneled via HTTP over TCP is a bit suboptimal. It works, but we can do better.
Currently, there is no standard API for socket connections in JavaScript. We want to change that.
If you’ve used Node.js before, then you’re most likely familiar with the net.Socket
and net.TLSSocket
objects. If you use Deno, then you might know that they’ve recently introduced the Deno.connect()
and Deno.connectTLS()
APIs. When you look at those APIs, what should immediately be apparent is how different they are from one another despite doing the exact same thing.
When we decided that we would add the ability to open and use socket connections from within Workers, we also agreed that we really have no interest in developing yet another non-standard, platform-specific API that is unlike the APIs provided by other platforms. Therefore, we are extending an invitation to all JavaScript runtime platforms that need socket capabilities to collaborate on a new (and eventually standardized) API that just works no matter which runtime you choose to develop on.
Here’s a rough example of what we have in mind for opening and reading from a simple TCP client connection:
const socket = new Socket({
remote: { address: '123.123.123.123', port: 1234 },
})
for await (const chunk of socket.readable)
console.log(chunk)
Or, this example, sending a simple “hello world” packet using UDP:
const socket = new Socket({
type: 'udp',
remote: { address: '123.123.123.123', port: 1234 },
});
const enc = new TextEncoder();
const writer = socket.writable.getWriter();
await writer.write(enc.encode('hello world'));
await writer.close();
The API will be designed generically enough to work both client and server-side; for TCP, UDP, and QUIC; with or without TLS, and will not rely on any mechanism specific to any single JavaScript runtime. It will build on existing broadly supported Web Platform standards such as EventTarget, ReadableStream, WritableStream, AbortSignal, and promises. It will be familiar to developers who are already familiar with the fetch()
API, service workers, and promises using async/await.
interface Socket : EventTarget {
constructor(object SocketInit);
Promise<undefined> update(object SocketInit);
readonly attribute ReadableStream readable;
readonly attribute WritableStream writable;
readonly attribute Promise<undefined> ready;
readonly attribute Promise<undefined> closed;
Promise<undefined> abort(optional any reason);
readonly attribute AbortSignal signal;
readonly attribute SocketStats stats;
readonly attribute SocketInfo info;
}
This is just a proposal at this point and the details will very likely change from the examples above by the time the capability is delivered in Workers. It is our hope that other platforms will join us in the effort of developing and supporting this new API so that developers have a consistent foundation upon which to build regardless of where they run their code.
The ability to open socket client connections is only half of the story.
When we first started talking about adding these capabilities an obvious question was asked: What about using non-HTTP protocols to connect to Workers? What if instead of just having the ability to connect a Worker to some other back-end database, we could implement the entire database itself on the edge, inside Workers, and have non-HTTP clients connect to it? For that matter, what if we could implement an SMTP server in Workers? Or an MQTT message queue? Or a full VoIP platform? Or implement packet filters, transformations, inspectors, or protocol transcoders?
Workers are far too powerful to limit to just HTTP and WebSockets, so we will soon introduce Socket Workers -- that is, Workers that can be connected to directly using raw TCP, UDP, or QUIC protocols without using HTTP.
What will this new Workers feature look like? Many of the details are still being worked through, but the idea is to deploy a Worker script that understands and responds to “connect” events in much the same way that “fetch” events work today. Importantly, this would build on the same common socket API being developed for client connections:
addEventListener('connect', (event) => {
const enc = new TextEncoder();
const writer = event.socket.writable.getWriter();
writer.write(enc.encode('Hello World'));
writer.close();
});
The new socket API for JavaScript and Socket Workers are under active development, with focus initially on enabling better and more efficient ways for Workers to connect to databases on the backend — you can sign up here to join the waitlist for access to Database Connectors and Socket Workers. We are excited to work with early users, as well as our technology partners to develop, refine, and test these new capabilities.
Once released, we expect Socket Workers to blow the doors wide open on the types of intelligent distributed applications that can be deployed to the Cloudflare network edge, and we are excited to see what you build with them.
Full Stack Week is all about how developers are embracing the power of Cloudflare’s network to build entire applications that are global by default. The promise of Workers isn’t just improved latency — it’s fundamentally different programming paradigms that make developer’s lives easier and applications more resilient.
Last year, we announced Durable Objects — Cloudflare’s approach to coordinating state across Workers running at Cloudflare’s edge. Durable Objects let developers implement previously complex applications, like collaborative whiteboarding, game servers, or global queues, in just a few lines of code.
Today, we’re announcing that Durable Objects are generally available and production-ready for you to use!
For many traditional applications, state coordination happens through a database. Applications built on Workers present some unique challenges for a database — namely needing to handle global scale out-of-the-box and heavy concurrency that could lead to frequent transaction rollbacks when coordinating on shared keys. Databases themselves are hard to configure and scale, especially at global scale, so developers would need to tweak their database specifically for Workers’ access patterns.
Durable Objects present a simpler paradigm: write a JavaScript class, and your application can create named instances of that class — which Continue reading
We’ve heard a common theme over the past year: developers want to build more of their applications on Workers. With built-in global deployments, insane scalability and the flexibility of JavaScript, more and more applications are choosing to build on our global platform.
To do so, developers need access to data. Our strategy for data on Workers has had three parts:
Today we’re excited to announce that, in addition to our existing partners Fauna and Macrometa, Cloudflare Workers has added support for Prisma and MongoDB Atlas. These data platforms are heavily demanded by developers — Prisma’s modern ORM brings support for Postgres, SQL Server, MySQL via their Prisma client, while MongoDB topped the ranks of integrations most demanded by our users.
Both clients are available from their respective authors, Realm for MongoDB and Prisma for Prisma. You can begin Continue reading
As you read this you are using the Internet. Stop and think about that for a minute. We speak about finding something “on the Internet”; we speak about “using the Internet” to perform a task. We essentially never say something like “I'm going to look for this on a server using the Internet as an intermediary between my computer and the server”.
We speak about and think about the Internet as a single, whole entity that we use and rely on. That’s behind the vision of “The Network is the Computer”. What matters is not the component parts that go into “the Internet” but what they come together to create.
That’s also the vision behind Cloudflare’s network.
We don’t want anyone to think about “caching content on a server in a Cloudflare data center” or “writing code that runs on (something called) the edge”. We want you to simply think of it as a single, global network that provides a CDN, a WAF, DDoS protection, Zero Trust and the ability to write infinitely scalable code and have it just work.
Scaling software is hard, and almost no programmer wants to spend their time worrying what will happen if Continue reading
Earlier this week, Cloudflare automatically detected and mitigated a DDoS attack that peaked just below 2 Tbps — the largest we’ve seen to date. This was a multi-vector attack combining DNS amplification attacks and UDP floods. The entire attack lasted just one minute. The attack was launched from approximately 15,000 bots running a variant of the original Mirai code on IoT devices and unpatched GitLab instances.
Last quarter, we saw multiple terabit-strong DDoS attacks and this attack continues this trend of increased attack intensity. Another key finding from our Q3 DDoS Trends report was that network-layer DDoS attacks actually increased by 44% quarter-over-quarter. While the fourth quarter is not over yet, we have, again, seen multiple terabit-strong attacks that targeted Cloudflare customers.
To begin with, our systems constantly analyze traffic samples “out-of-path” which allows us to asynchronously detect DDoS attacks without causing latency or impacting performance. Once the attack traffic was detected (within sub-seconds), our systems generated a real-time signature that surgically matched against the attack patterns to mitigate the attack without impacting Continue reading
A number of vulnerabilities in Resource Public Key Infrastructure (RPKI) validation software were disclosed in a recent NCSC advisory, discovered by researchers from the University of Twente. These attacks abuse a set of assumptions that are common across multiple RPKI implementations, and some of these issues were discovered within OctoRPKI. More details about the disclosed vulnerabilities can be found in this RIPE labs article written by one of the researchers. In response, we published a new release of OctoRPKI, v1.4.0, to address and remediate these vulnerabilities.
Cloudflare customers do not have to take any action to protect themselves from these newly discovered vulnerabilities, and no Cloudflare customer data was ever at risk.
We have not seen any attempted exploitation of these vulnerabilities described in the advisory. We use OctoRPKI to perform Border Gateway Protocol (BGP) route validation so that our routers know where to direct IP packets at Layer 3 of the TCP/IP stack. TLS provides additional security at the TCP layer to ensure the integrity and confidentiality of customer data going over the Internet in the event of BGP hijacking.
Resource Public Key Infrastructure (RPKI) is a cryptographic method of Continue reading
I joined Cloudflare a few weeks ago, and as someone new to the company, there’s a ton of information to absorb. I have always learned best by doing, so I decided to use Cloudflare like a brand-new user. Cloudflare customers range from individuals with a simple website to companies in the Fortune 100. I’m currently exploring Cloudflare from the perspective of the individual, so I signed up for a free account and logged into the dashboard. Just like getting into a new car, I want to turn all the dials and push all the buttons. I looked for things that would be fun and easy to do and would deliver some immediate value. Now I want to share the best ones with you.
Here are my five ways to get started with Cloudflare. These should be easy for anyone, and they’re free. You’ll likely even save some money and improve your privacy and security in the process. Let’s go!
If you’re like me, you’ve acquired a few (dozen) Internet domains for things like personalizing your email address, a web page for your nature photography hobby, or maybe a side business. You Continue reading
Meris first got our attention due to an exceptionally large 17.2 million requests per second (rps) DDoS attack that it launched against one of our customers. This attack, along with subsequent attacks originated by the Meris botnet, was automatically detected and mitigated by our DDoS protection systems. Cloudflare customers, even ones on the free plan, are protected against Meris attacks.
Over the past months, we’ve been tracking and analyzing the activity of the Meris botnet. Some main highlights include:
View more Meris attack insights and trends in the interactive Radar dashboard.
Meris (Latvian for plague) is the name of an active botnet behind a series of recent DDoS attacks that have targeted thousands of websites around the world. It was originally detected in late June 2021 by QRator in joint research they conducted with Yandex. Their initial research identified 30,000 to 56,000 bots, but they estimated that the numbers Continue reading
I am excited to announce that I have joined Cloudflare as the Head of Southeast Asia and Korea (SEAK) region to help build a better Internet and to expand Cloudflare’s growing customer, partner and local teams across all the countries in SEAK. Cloudflare is at an emergence phase in this region, with immense growth potential, and this is just the beginning. Cloudflare has had a lot of success globally and our charter is to build on that success and momentum to grow our presence locally to address the demands in Singapore, Malaysia, Thailand, Indonesia, Philippines, Indochina and Korea. Customer engagements in each of the countries in SEAK presents a unique, rich and fulfilling engagement each with their own intricacies.
I was born in India (Surat, Gujarat), and at the age of four our family moved to Bahrain where we lived for eight years. We then moved to New Zealand, which is where I completed my senior years of high school and also my Bachelor’s Degree in Information Engineering at Massey University. After graduation, we moved to Melbourne, Australia which is our family home and where my career started.
I love meeting and working with diverse and Continue reading
Quantum computers are a boon and a bane. Originally conceived by Manin and Feyman to simulate nature efficiently, large-scale quantum computers will speed-up innovation in material sciences by orders of magnitude. Consider the technical advances enabled by the discovery of new materials (with bronze, iron, steel and silicon each ascribed their own age!); quantum computers could help to unlock the next age of innovation. Unfortunately, they will also break the majority of the cryptography that’s currently used in TLS to protect our web browsing. They fall in two categories:
A moderately-sized stable quantum computer will easily break the signatures and key exchanges currently used in TLS using Shor’s algorithm. Luckily this can be fixed: over the last two decades, there has been great progress in so-called post-quantum cryptography. “Post quantum”, abbreviated PQ, means secure against quantum computers. Five years ago, the standards institute NIST started a public process to standardise post-quantum signature schemes and key exchanges. The outcome is expected to be announced early 2022.
At Cloudflare, we’re not just following this Continue reading
For many (especially in the Northern Hemisphere, where about 87% of humans live), September is the “get back to school” (or work) month after a summer break and that also reflects changes in the Internet traffic, particularly in mobile usage.
Looking at our data (you can see many of these insights in Cloudflare Radar) there’s a global trend: mobile traffic lost importance (compared with desktop traffic) in September. The next chart shows there was less percentage of Internet traffic from mobile devices after Monday, September 6, 2021, with a difference of -2% in some days, compared with the previous four weeks (August), and in late September it’s more than -3%.
We can also see that the percentage of desktop traffic increased in September compared to August (we compare here to complete weeks between both months because there are significant differences between weekdays and weekends).
A few of weeks ago, we saw there are considerable differences between countries regarding the importance of mobile usage. Getting back to work (or office hours) usually means an increase in desktop traffic. In that blog we highlighted the advantages that mobile devices brought to developing countries — many had their first contact with Continue reading
The third quarter of 2021 was a busy quarter for DDoS attackers. Cloudflare observed and mitigated record-setting HTTP DDoS attacks, terabit-strong network-layer attacks, one of the largest botnets ever deployed (Meris), and more recently, ransom DDoS attacks on voice over IP (VoIP) service providers and their network infrastructure around the world.
Here’s a summary of the trends observed in Q3 ‘21:
Application-layer (L7) DDoS attack trends:
Network-layer (L3/4) DDoS attack trends:
“Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.” — Sherlock Holmes
It’s not every day that you get to debug what may well be a packet of death. It was certainly the first time for me.
What do I mean by “a packet of death”? A software bug where the network stack crashes in reaction to a single received network packet, taking down the whole operating system with it. Like in the well known case of Windows ping of death.
Challenge accepted.
Around a year ago we started seeing kernel crashes in the Linux ipv4 stack. Servers were crashing sporadically, but we learned the hard way to never ignore cases like that — when possible we always trace crashes. We also couldn’t tie it to a particular kernel version, which could indicate a regression which hopefully could be tracked down to a single faulty change in the Linux kernel.
The crashed servers were leaving behind only a crash report, affectionately known as a “kernel oops”. Let’s take a look at it and go over what information we have there.
Parts of the oops, like offsets into Continue reading
Cloudflare provides a broad range of products — ranging from security, to performance and serverless compute — which are used by millions of Internet properties worldwide. Often, these products are built by multiple teams in close collaboration and delivering them can be a complex task. So ever wondered how we do so consistently and safely at scale?
Software delivery consists of all the activities to get working software into the hands of customers. It’s usual to talk about software delivery with reference to a model, or framework. These provide the scaffolding for most modern software delivery models, although in order to minimise operational friction it’s usual for a company to tailor their approach to suit their business context and culture.
For example, a company that designs the autopilot systems for passenger aircraft will require very strict tolerances, as a failure could cost hundreds of lives. They would want a different process to a cutting edge tech startup, who may value time to market over system uptime or stability.
Before outlining the approach we use at Cloudflare it’s worth quickly running through a couple of commonly used delivery models.
Waterfall has its foundations (pun intended) in construction and Continue reading
It's not every day that there is no Internet access in an entire country. In the case of Sudan, it has been five days without Internet after political turmoil that started last Monday, October 25, 2021 (as we described).
The outage continues with almost a flat line and just a trickle of Internet traffic from Sudan. Cloudflare Radar shows that the Internet in Sudan is still almost completely cut off.
There was a blip of traffic on Tuesday at ~14:00 UTC, for about one hour, but it flattened out again, and it continues like that — anyone can track the evolution on the Sudan page of Cloudflare Radar.
Internet disruptions, including shutdowns and social media restrictions, are common occurrences in some countries and Sudan is one where this happens more frequently than most countries according to Human Rights Watch. In our June blog, we talked about Sudan when the country decided to shut down the Internet to prevent cheating in exams, but there were situations in the past more similar to this days-long shutdown — something that usually happens when there’s political unrest.
The country's longest recorded network disruption was back in Continue reading
Forrester’s New Wave for Edge Development Platforms has just been announced. We’re thrilled that they have named Cloudflare a leader (you can download a complimentary copy of the report here).
Since the very beginning, Cloudflare has sought to help developers building on the web, and since the introduction of Workers in 2017, Cloudflare has enabled developers to deploy their applications to the edge itself.
According to the report by Forrester Vice President, Principal Analyst, Jeffrey Hammond, Cloudflare “offers strong compute, data services and web development capabilities. Alongside Workers, Workers KV adds edge data storage. Pages, Stream and Images provide higher level platform services for modern web workloads. Cloudflare has an intuitive developer experience, fast, global deployment of updated code, and minimal cold start times.”
Building on the web has come a long way. The idea that one might have to buy a physical machine in order to build a website seems incomprehensible now. The cloud has played a major role in making it easier for developers to get started. However, since the advent of the cloud, things have stalled — and innovation has become more incremental. That means that while developers Continue reading
Until today, Cloudflare Workers has been a great solution to setting headers, but we wanted to create an even smoother developer experience. Today, we're excited to announce that Pages now natively supports custom headers on your projects! Simply create a _headers
file in the build directory of your project and within it, define the rules you want to apply.
/developer-docs/*
X-Hiring: Looking for a job? We're hiring engineers
(https://www.cloudflare.com/careers/jobs)
Being able to set custom headers is useful for a variety of reasons — let’s explore some of your most popular use cases.
When you create a Pages project, a pages.dev
deployment is created for your project which enables you to get started immediately and easily preview changes as you iterate. However, we realize this poses an issue — publishing multiple copies of your website can harm your rankings in search engine results. One way to solve this is by disabling indexing on all pages.dev
subdomains, but we see many using their pages.dev
subdomain as their primary domain. With today’s announcement you can attach headers such as X-Robots-Tag
to hint to Google and other search Continue reading
Today, October 25, following political turmoil, Sudan woke up without Internet access.
In our June blog, we talked about Sudan when the country decided to shut down the Internet to prevent cheating in exams.
Now, the disruption seems to be for other reasons. AP is reporting that "military forces ... detained at least five senior Sudanese government figures.". This afternoon (UTC) several media outlets confirmed that Sudan's military dissolved the transitional government in a coup that shut down mobile phone networks and Internet access.
Cloudflare Radar allows anyone to track Internet traffic patterns around the world. The dedicated page for Sudan clearly shows that this Monday, when the country was waking up, the Internet traffic went down and continued that trend through the afternoon (16:00 local time, 14:00 UTC).
We dug in a little more on the HTTP traffic data. It usually starts increasing after 06:00 local time (04:00 UTC). But this Monday morning, traffic was flat, and the trend continued in the afternoon (there were no signs of the Internet coming back at 18:00 local time).
When comparing today with the last seven days' pattern, we see that today's drop is abrupt and unusual.
We can see Continue reading
A big part of the job of a technical writer is getting feedback on the content you produce. Writing and maintaining product documentation is a deeply collaborative and cyclical effort — through constant conversation with product managers and engineers, technical writers ensure the content is clear and serves the user in the most effective way. Collaboration with other technical writers is also important to keep the documentation consistent with Cloudflare’s content strategy.
So whether we’re documenting a new feature or overhauling a big portion of existing documentation, sharing our writing with stakeholders before it’s published is quite literally half the work.
In my experience as a technical writer, the feedback I’ve received has been exponentially more impactful when stakeholders could see my changes in context. This is especially true for bigger and more strategic changes. Imagine I’m changing the structure of an entire section of a product’s documentation, or shuffling the order of pages in the navigation bar. It’s hard to guess the impact of those changes just by looking at the markdown files.
We writers check those changes in context by building a development server on our local machines. But sharing what we see locally with our stakeholders has Continue reading
During Developer Week a few months ago, we opened up the Beta for Cloudflare for SaaS: a one-stop shop for SaaS providers looking to provide fast load times, unparalleled redundancy, and the strongest security to their customers.
Since then, we’ve seen numerous developers integrate with our technology, allowing them to spend their time building out their solution instead of focusing on the burdens of running a fast, secure, and scalable infrastructure — after all, that’s what we’re here for.
Today, we are very excited to announce that Cloudflare for SaaS is generally available, so that every customer, big and small, can use Cloudflare for SaaS to continue scaling and building their SaaS business.
If you’re running a SaaS company, you have customers that are fully reliant on you for your service. That means you’re responsible for keeping their domain fast, secure, and protected. But this isn’t simple. There’s a long checklist you need to get through to put a solution in your customers’ hands: