More products, more partners, and a new look for Cloudflare Logs

We are excited to announce a new look and new capabilities for Cloudflare Logs! Customers on our Enterprise plan can now configure Logpush for Firewall Events and Network Error Logs Reports directly from the dashboard. Additionally, it’s easier to send Logs directly to our analytics partners Microsoft Azure Sentinel, Splunk, Sumo Logic, and Datadog. This blog post discusses how customers use Cloudflare Logs, how we’ve made it easier to consume logs, and tours the new user interface.

New data sets for insight into more products

Cloudflare Logs are almost as old as Cloudflare itself, but we have a few big improvements: new datasets and new destinations.

Cloudflare has a large number of products, and nearly all of them can generate Logs in different data sets. We have “HTTP Request” Logs, or one log line for every L7 HTTP request that we handle (whether cached or not). We also provide connection Logs for Spectrum, our proxy for any TCP or UDP based application. Gateway, part of our Cloudflare for Teams suite, can provide Logs for HTTP and DNS traffic.

Today, we are introducing two new data sets:

Firewall Events gives insight into malicious traffic handled by Cloudflare. It provides detailed information Continue reading

Sudan’s exam-related Internet shutdowns

To prevent cheating in exams many countries restrict or even shut down Internet access during critical exam hours. I wrote two weeks ago about Syria having planned Internet shutdowns during June, for exams.

Sudan is doing the same thing and has had four shutdowns so far. Here's the Internet traffic pattern for Sudan over the last seven days. I've circled the shutdowns on Saturday, Sunday, Monday and Tuesday (today, June 22, 2021).

Cloudflare Radar allows anyone to track Internet traffic patterns around the world, and it has country-specific pages. The chart for the last seven days (shown above) came from the dedicated page for Sudan.

The Internet outages start at 0600 UTC (0800 local time) and end three hours later at 0900 UTC (1100 local time). This corresponds to the timings announced by the Sudanese Ministry of Education.

Further shutdowns are likely in Sudan on June 24, 26, 27, 29 and 30 (thanks to Twitter user _adonese for his assistance). Looking deeper into the data, the largest drop in use is for mobile Internet access in Sudan (the message above talks about mobile Internet use being restricted) while some non-mobile access appears to continue.

That can be seen by looking Continue reading

Cloudflare TV: Doing it Live, 1,000 Times and Counting

Last week, Cloudflare TV celebrated its first anniversary the only way it knows how: with a broadcast brimming with live programming spanning everything from the keynotes of Cloudflare Connect, to a day-long virtual career fair, to our flagship game show Silicon Valley Squares.

When our co-founder and CEO Matthew Prince introduced Cloudflare TV to the world last year, he described it as a platform for experimentation. By empowering Cloudflare employees to try whatever they could think up on air — bound only by restraints of common sense — we hoped to unlock aspects of our team’s talent and creativity that otherwise might go untapped in the midst of the pandemic.

The results, as they say, have been extraordinary.

Since launching in June 2020, Cloudflare TV has featured over 1,000 original live episodes covering an incredible array of topics: technical deep dives and tutorials like Hardware at Cloudflare, Leveling up Web Performance with HTTP/3, and Hacker Time. Security expertise from top CISOs and compliance experts. In-depth policy discussions. And of course, updates on Cloudflare’s products with weekly episodes of Latest from Product and Engineering, Estas Semanas en Cloudflare en Español, and launch-day introductions to Magic WAN Continue reading

Announcing WARP for Linux and Proxy Mode

Last October we released WARP for Desktop, bringing a safer and faster way to use the Internet to billions of devices for free. At the same time, we gave our enterprise customers the ability to use WARP with Cloudflare for Teams. By routing all an enterprise's traffic from devices anywhere on the planet through WARP, we’ve been able to seamlessly power advanced capabilities such as Secure Web Gateway and Browser Isolation and, in the future, our Data Loss Prevention platforms.

Today, we are excited to announce Cloudflare WARP for Linux and, across all desktop platforms, the ability to use WARP with single applications instead of your entire device.

What is WARP?

WARP was built on the philosophy that even people who don’t know what “VPN” stands for should be able to still easily get the protection a VPN offers. It was also built for those of us who are unfortunately all too familiar with traditional corporate VPNs, and need an innovative, seamless solution to meet the challenges of an always-connected world.

Enter our own WireGuard implementation called BoringTun.

The WARP application uses BoringTun to encrypt traffic from your device and send it directly to Cloudflare’s edge, ensuring that no Continue reading

Building Waiting Room on Workers and Durable Objects

In January, we announced the Cloudflare Waiting Room, which has been available to select customers through Project Fair Shot to help COVID-19 vaccination web applications handle demand. Back then, we mentioned that our system was built on top of Cloudflare Workers and the then brand new Durable Objects. In the coming days, we are making Waiting Room available to customers on our Business and Enterprise plans. As we are expanding availability, we are taking this opportunity to share how we came up with this design.

What does the Waiting Room do?

You may have seen lines of people queueing in front of stores or other buildings during sales for a new sneaker or phone. That is because stores have restrictions on how many people can be inside at the same time. Every store has its own limit based on the size of the building and other factors. If more people want to get inside than the store can hold, there will be too many people in the store.

The same situation applies to web applications. When you build a web application, you have to budget for the infrastructure to run it. You make that decision according to how many Continue reading

Enable secure access to applications with Cloudflare WAF and Azure Active Directory

Cloudflare and Microsoft Azure Active Directory have partnered to provide an integration specifically for web applications using Azure Active Directory B2C. From today, customers using both services can follow the simple integration steps to protect B2C applications with Cloudflare’s Web Application Firewall (WAF) on any custom domain. Microsoft has detailed this integration as well.

Cloudflare Web Application Firewall

The Web Application Firewall (WAF) is a core component of the Cloudflare platform and is designed to keep any web application safe. It blocks more than 70 billion cyber threats per day. That is 810,000 threats blocked every second.

The WAF is available through an intuitive dashboard or a Terraform integration, and it enables users to build powerful rules. Every request to the WAF is inspected against the rule engine and the threat intelligence built from protecting approximately 25 million internet properties. Suspicious requests can be blocked, challenged or logged as per the needs of the user, while legitimate requests are routed to the destination regardless of where the application lives (i.e., on-premise or in the cloud). Analytics and Cloudflare Logs enable users to view actionable metrics.

The Cloudflare WAF is an intelligent, integrated, and scalable solution to protect business-critical Continue reading

Interconnect Anywhere — Reach Cloudflare’s network from 1,600+ locations

Customers choose Cloudflare for our network performance, privacy and security.  Cloudflare Network Interconnect is the best on-ramp for our customers to utilize our diverse product suite. In the past, we’ve talked about Cloudflare’s physical footprint in over 200+ data centers, and how Cloudflare Network Interconnect enabled companies in those data centers to connect securely to Cloudflare’s network. Today, Cloudflare is excited to announce expanded partnerships that allows customers to connect to Cloudflare from their own Layer 2 service fabric. There are now over 1,600 locations where enterprise security and network professionals have the option to connect to Cloudflare securely and privately from their existing fabric.

Interconnect Anywhere is a journey

Since we launched Cloudflare Network Interconnect (CNI) in August 2020, we’ve been focused on extending the availability of Cloudflare’s network to as many places as possible. The initial launch opened up 150 physical locations alongside 25 global partner locations. During Security Week this year, we grew that availability by adding data center partners to our CNI Partner Program. Today, we are adding even more connectivity options by expanding Cloudflare availability to all of our partners’ locations, as well as welcoming CoreSite Open Cloud Exchange (OCX) and Infiny by Epsilon Continue reading

Introducing Zero Trust Private Networking

Starting today, you can build identity-aware, Zero Trust network policies using Cloudflare for Teams. You can apply these rules to connections bound for the public Internet or for traffic inside a private network running on Cloudflare. These rules are enforced in Cloudflare’s network of data centers in over 200 cities around the world, giving your team comprehensive network filtering and logging, wherever your users work, without slowing them down.

Last week, my teammate Pete’s blog post described the release of network-based policies in Cloudflare for Teams. Your team can now keep users safe from threats by limiting the ports and IPs that devices in your fleet can reach. With that release, security teams can now replace even more security appliances with Cloudflare’s network.

We’re excited to help your team replace that hardware, but we also know that those legacy network firewalls were used to keep private data and applications safe in a castle-and-moat model. You can now use Cloudflare for Teams to upgrade to a Zero Trust networking model instead, with a private network running on Cloudflare and rules based on identity, not IP address.

To learn how, keep reading or watch the demo below.

Deprecating the castle-and-moat model

Private Continue reading

Celebrating 7 Years of Project Galileo

Every June, we celebrate the anniversary of Project Galileo. This year, we are proud to celebrate seven years of protecting the most vulnerable groups on the Internet from cyber attacks. June is a busy month for us at Cloudflare, with the anniversary of Project Galileo and Access Now’s RightsCon, one of the largest events on human rights in the digital age. As we collaborate with civil society on topics from technology, privacy, digital security and public policy, we learn how to better protect critical voices on the Internet but also how to use the Cloudflare network to make positive changes to the Internet ecosystem.

We started Project Galileo in 2014 with the idea that we need to protect voices that are targeted for working in sensitive areas. As such, we give these voices the resources to protect themselves online against powerful opponents. Whether their opponent’s aim is to intimidate, silence, or steal sensitive information, cyber attacks can cause significant damage to organizations that work in areas such as human rights, independent media, education, and social justice. As the world moves online — a factor accelerated by COVID-19 — access to powerful cybersecurity tools is critical for organizations around the world. Continue reading

Modify HTTP request headers with Transform Rules

HTTP headers are central to how the web works. They are used for passing additional information between the client and server, such as which security permissions to apply and information about the client, allowing the correct content to be served.

Today we are announcing the immediate availability of the second action within Transform Rules, “HTTP Request Header Modification”, available for all Cloudflare plans. This new functionality provides Cloudflare administrators with the ability to easily set or remove HTTP request headers as traffic flows through Cloudflare. This allows customers to enrich requests with information such as the Cloudflare Bot Management ‘Bot Score’ prior to being sent to their servers. Previously, HTTP request header modification was done using a Cloudflare Worker. Today we’re introducing an easier way to do this without writing a single line of code.

Luggage tags of the World Wide Web

Think of HTTP headers as the “luggage tag” attached to your bags when you check in at the airport.

Generally, you don't need to know what those numbers and words mean. You just know they are important in getting your suitcase from the boarding desk, to the correct Continue reading

Network-based policies in Cloudflare Gateway

Over the past year, Cloudflare Gateway has grown from a DNS filtering solution to a Secure Web Gateway. That growth has allowed customers to protect their organizations with fine-grained identity-based HTTP policies and malware protection wherever their users are. But what about other Internet-bound, non-HTTP traffic that users generate every day — like SSH?

Today we’re excited to announce the ability for administrators to configure network-based policies in Cloudflare Gateway. Like DNS and HTTP policy enforcement, organizations can use network selectors like IP address and port to control access to any network origin.

Because Cloudflare for Teams integrates with your identity provider, it also gives you the ability to create identity-based network policies. This means you can now control access to non-HTTP resources on a per-user basis regardless of where they are or what device they’re accessing that resource from.

A major goal for Cloudflare One is to expand the number of on-ramps to Cloudflare — just send your traffic to our edge however you wish and we’ll make sure it gets to the destination as quickly and securely as possible. We released Magic WAN and Magic Firewall to let administrators replace MPLS connections, define routing decisions, and apply packet-based Continue reading

Syria’s exam-related Internet shutdowns

To prevent cheating in exams many countries restrict or even shut down Internet access during critical exam hours. For most of June Syria is having planned Internet shutdowns during critical exam periods. The exam schedule is as follows:

I’m grateful to a Twitter user for the translation from the original Arabic and collating the data.

Cloudflare Radar allows anyone to track Internet traffic patterns around the world, and it has country-specific pages. The chart for the last seven days of Internet use in Syria as seen by Cloudflare shows two drops to almost zero corresponding to the first two exams on the schedule.

The Internet outage starts at around 0100 UTC (0400 local time) and ends about four and a half hours later at 0530 UTC (0830 UTC). This covers the period before the exams start apparently to prevent any figuring out the answers.

If you want to follow the other outages for the remaining seven exams you can see live data on the Cloudflare Radar Syria page.

QUIC Version 1 is live on Cloudflare

On May 27 2021, the Internet Engineering Task Force published RFC 9000 - the standardarized version of the QUIC transport protocol. The QUIC Working Group declared themselves done by issuing a Last Call 7 months ago. The i's have been dotted and the t's crossed, RFC 8999 - RFC 9002 are a suite of documents that capture years of engineering design and testing of QUIC. This marks a big occasion.

And today, one day later, we’ve made the standardized version of QUIC available to Cloudflare customers.

Transport protocols have a history of being hard to deploy on the Internet. QUIC overcomes this challenge by basing itself on top of UDP. Compared to TCP, QUIC has security by default, protecting almost all bytes from prying eyes or "helpful" middleboxes that can end up making things worse. It has designed-in features that speed up connection handshakes and mitigate the performance perils that can strike on networks that suffer loss or delays. It is pluggable, providing clear standardised extensions point that will allow smooth, iterative development and deployment of new features or performance enhancements for years to come.

The killer feature of QUIC, however, is that it is deployable in reality. We are Continue reading

Building real-time games using Workers, Durable Objects, and Unity

Durable Objects are an awesome addition to the Workers developer ecosystem, allowing you to address and work inside a specific Worker to provide consistency in your applications. That sounds exciting at a high-level, but if you're like me, you might be wondering "Okay, so what can I build with that?"

There’s nothing like building something real with a technology to truly understand it.

To better understand why Durable Objects matter, and how newer announcements in the Workers ecosystem like WebSockets play with Durable Objects, I turned to a category of software that I've been building in my spare time for a few months now: video games.

The technical aspects of games have changed drastically in the last decade. Many games are online-by-default, and the ubiquity of tools like Unity have made it so anyone can begin experimenting with developing games.

I've heard a lot about the ability of Durable Objects and WebSockets to provide real-time consistency in applications, and to test that use case out, I've built Durable World: a simple 3D multiplayer world that is deployed entirely on our Cloudflare stack: Pages for serving the client-side game, which runs in Unity and WebGL, and Workers as the Continue reading

Cloudflare’s SOC as a Service

When Cloudflare started, sophisticated online security was beyond the reach of all but the largest organizations. If your pockets were deep enough, you could buy the necessary services — and the support that was required to operate them — to keep your online operations secure, fast, and reliable. For everyone else? You were out of luck.

We wanted to change that: to help build a better Internet. To build a set of services that weren’t just technically sophisticated, but easy to use. Accessible. Affordable. Part of this meant that we were always looking to build and equip our customers with all the tools they needed in order to do this for themselves.

Of course, a lot has changed since we started. The Internet has only increased in importance, fast becoming the most important channel for many businesses. Cybersecurity threats have only become more prevalent — and more sophisticated. And the products that Cloudflare offers to keep you safe on the Internet have attracted some of the largest and most recognizable organizations in the world.

Ask some of these larger organizations about cybersecurity, and they’ll tell you a few things: first, they love our products. But, second, that when something happens Continue reading

CDN-Cache-Control: Precision Control for your CDN(s)

Today we are thrilled to announce our support of a new set of HTTP response headers that provide surgical control over our CDN’s caching decisions. CDN-Cache-Control allows customers to directly control how our CDN behaves without affecting the behavior of downstream or upstream caches.

You might be thinking that this sounds a lot like the Cache-Control header we all know and love. And it’s very similar! CDN-Cache-Control has exactly the same directives as the Cache-Control header. The problem CDN-Cache-Control sets out to solve is that with Cache-Control, some directives are targeted at specific classes of caches (like s-maxage for shared caches), while other directives are not targeted at controlling any specific classes of intermediary caches (think stale-while-revalidate). As these non-specific directives are returned to downstream caches, they’re often not applied uniformly. This problem is amplified as the number of intermediary caches grows between an origin and the client.

For example, a website may deploy a caching layer on the origin server itself, there might be a cache on the origin’s network, the site might use one or more CDNs to cache content distributed throughout the Internet, and the visitor’s browser might cache content as well. As the response returns Continue reading

Improving your monitoring setup by integrating Cloudflare’s analytics data into Prometheus and Grafana

The following is a guest post by Martin Hauskrecht, DevOps Engineer at Labyrinth Labs.

Here at Labyrinth Labs, we put great emphasis on monitoring. Having a working monitoring setup is a critical part of the work we do for our clients.

Cloudflare's Analytics dashboard provides a lot of useful information for debugging and analytics purposes for our customer Pixel Federation. However, it doesn’t automatically integrate with existing monitoring tools such as Grafana and Prometheus, which our DevOps engineers use every day to monitor our infrastructure.

Cloudflare provides a Logs API, but the amount of logs we’d need to analyze is so vast, it would be simply inefficient and too pricey to do so. Luckily, Cloudflare already does the hard work of aggregating our thousands of events per second and exposes them in an easy-to-use API.

Having Cloudflare’s data from our zones integrated with other systems’ metrics would give us a better understanding of our systems and the ability to correlate metrics and create more useful alerts, making our Day-2 operations (e.g. debugging incidents or analyzing the usage of our systems) more efficient.

Since our monitoring stack is primarily based on Prometheus and Grafana, we decided to implement our own Continue reading

Multiplayer Doom on Cloudflare Workers

There are halls and corridors in Cloudflare engineering, dangerous places for innocent wanderers, filled with wild project ideas, experiments that we should do, and extremely convincing proponents. A couple of months ago, John Graham-Cumming, our CTO, bumped into me in one of those places and asked: "What if we ported Doom multiplayer to work with our edge network?". He fatally nerd-sniped me.

Aside by John: I nerd-sniped him because I wanted to show how Cloudflare Workers and Durable Objects are a new architectural paradigm where, rather than choosing between two places to write code (the client, the browser or app, and the server, perhaps in a cloud provider availability zone), there’s a third way: put code on the edge.

Writing code that runs on a client (such as JavaScript that runs in a browser or a native app on a phone) has advantages. Because the code runs close to the end-user it can be highly interactive, there’s almost no latency since it’s literally running on the device. But client-side code has security problems: it’s literally in the hands of the end-user and thus can be reverse engineered or modified. And client-side code can be slow to update as it Continue reading

Automating Cloudflare Tunnel with Terraform

Cloudflare Tunnel allows you to connect applications securely and quickly to Cloudflare’s edge. With Cloudflare Tunnel, teams can expose anything to the world, from internal subnets to containers, in a secure and fast way. Thanks to recent developments with our Terraform provider and the advent of Named Tunnels it’s never been easier to spin up.

Classic Tunnels to Named Tunnels

Historically, the biggest limitation to using Cloudflare Tunnel at scale was that the process to create a tunnel was manual. A user needed to download the binary for their OS, install/compile it, and then run the command cloudflared tunnel login. This would open a browser to their Cloudflare account so they could download a cert.pem file to authenticate their tunnel against Cloudflare’s edge with their account.

With the jump to Named Tunnels and a supported API endpoint Cloudflare users can automate this manual process. Named Tunnels also moved to allow a .json file for the origin side tunnel credentials instead of (or with) the cert.pem file. It has been a dream of mine since joining Cloudflare to write a Cloudflare Tunnel as code, along with my instance/application, and deploy it while I go walk my dog. Tooling should Continue reading

Humanity wastes about 500 years per day on CAPTCHAs. It’s time to end this madness

Select all the buses. Click on bikes. Does this photo have traffic lights? As ridiculous as these questions are, you’re almost guaranteed to have seen one recently. They are a way for online services to separate humans from bots, and they’re called CAPTCHAs. CAPTCHAs strengthen the security of online services. But while they do that, there’s a very real cost associated with them.

Based on our data, it takes a user on average 32 seconds to complete a CAPTCHA challenge. There are 4.6 billion global Internet users. We assume a typical Internet user sees approximately one CAPTCHA every 10 days.

This very simple back of the envelope math equates to somewhere in the order of 500 human years wasted every single day — just for us to prove our humanity.

Today, we are launching an experiment to end this madness. We want to get rid of CAPTCHAs completely. The idea is rather simple: a real human should be able to touch or look at their device to prove they are human, without revealing their identity. We want you to be able to prove that you are human without revealing which human you are! You may ask if this is Continue reading