Evolving Machine Learning to stop mobile bots

When we launched Bot Management three years ago, we started with the first version of our ML detection model. We used common bot user agents to train our model to identify bad bots. This model, ML1, was able to detect whether a request is a bot or a human request purely by using the request’s attributes. After this, we introduced a set of heuristics that we could use to quickly and confidently filter out the lowest hanging fruit of unwanted traffic. We have multiple heuristic types and hundreds of specific rules based on certain attributes of the request, many of which are very hard to spoof. But machine learning is a very important part of our bot management toolset.

We started with a static model because we were starting from scratch, and we were able to experiment quickly with aggregated HTTP analytics metadata. After we launched the model, we quickly gathered feedback from early bot management customers to identify where we performed well but also how we could improve. We saw attackers getting smart, and so we generated a new set of model features. Our heuristics were able to accurately identify various types of bad bots giving us much better Continue reading

Congratulations Cloudflare 2021 Partner Award Winners

We’re thrilled to announce the winners of our annual Channel and Alliance Partner Awards for 2021. Throughout a year of continued global disruptions, Cloudflare’s partners kept innovating, expanding their solutions and services capabilities, and accelerated their growth with us and our platform. It is important that we recognize and award the partners of ours who stood out in staying laser-focused on delivering outstanding business outcomes for customers.

With the ongoing shift in 2021 to remote, flexible work forces and the evolving cyber threat landscape, more than ever organizations across every industry and the public sector were looking to Cloudflare, and to work hand in hand with partners who can deliver a modern, Zero Trust approach to security. Seeing this consistent need, we are continuing to build and support new levels of partner-led growth in the year ahead such as with a new partner services program for SASE and Zero Trust which we launched at the start of 2022.

Please join us in congratulating the impressive achievements of our partner award winners over this past year! They enable the further delivery of Internet security, performance, and reliability for organizations of all sizes and types — and we are thrilled to be Continue reading

Protect all network traffic with Cloudflare

Magic Transit protects customers' entire networks—any port/protocol—from DDoS attacks and provides built-in performance and reliability. Today, we’re excited to extend the capabilities of Magic Transit to customers with any size network, from home networks to offices to large cloud properties, by offering Cloudflare-maintained and Magic Transit-protected IP space as a service.

What is Magic Transit?

Magic Transit extends the power of Cloudflare’s global network to customers, absorbing all traffic destined for your network at the location closest to its source. Once traffic lands at the closest Cloudflare location, it flows through a stack of security protections including industry-leading DDoS mitigation and cloud firewall. Detailed Network Analytics, alerts, and reporting give you deep visibility into all your traffic and attack patterns. Clean traffic is forwarded to your network using Anycast GRE or IPsec tunnels or Cloudflare Network Interconnect. Magic Transit includes load balancing and automatic failover across tunnels to steer traffic across the healthiest path possible, from everywhere in the world.

The “Magic” Continue reading

Clientless Web Isolation is now generally available

Today, we’re excited to announce that Clientless Web Isolation is generally available. A new on-ramp for Browser Isolation that natively integrates Zero Trust Network Access (ZTNA) with the zero-day, phishing and data-loss protection benefits of remote browsing for users on any device browsing any website, internal app or SaaS application. All without needing to install any software or configure any certificates on the endpoint device.

Cloudflare’s clientless web isolation simplifies connections to remote browsers through a hyperlink (e.g.: https://<your-auth-domain>.cloudflareaccess.com/browser). We explored use cases in detail in our beta announcement post, but here’s a quick refresher on the use cases that clientless isolated browsing enables:

Share secure browsing across the entire team on any device

Simply navigating to Clientless Web Isolation will land your user such as an analyst, or researcher in a remote browser, ready to securely conduct their research or investigation without exposing their public IP or device to potentially malicious code on the target website.

Deep link into isolated browsing

Suspicious hyperlinks and PDF documents from sensitive applications can be opened in a remote browser by rewriting the link with the clientless endpoint. For example:

https://<authdomain>.cloudflareaccess.com/browser/https://www.example.com/suspiciouslink

This is Continue reading

Packet captures at the edge

Packet captures are a critical tool used by network and security engineers every day. As more network functions migrate from legacy on-prem hardware to cloud-native services, teams risk losing the visibility they used to get by capturing 100% of traffic funneled through a single device in a datacenter rack. We know having easy access to packet captures across all your network traffic is important for troubleshooting problems and deeply understanding traffic patterns, so today, we’re excited to announce the general availability of on-demand packet captures from Cloudflare’s global network.

What are packet captures and how are they used?

A packet capture is a file that contains all packets that were seen by a particular network box, usually a firewall or router, during a specific time frame. Packet captures are a powerful and commonly used tool for debugging network issues or getting better visibility into attack traffic to tighten security (e.g. by adding firewall rules to block a specific attack pattern).

A network engineer might use a pcap file in combination with other tools, like mtr, to troubleshoot problems with reachability to their network. For example, if an end user reports intermittent connectivity to a specific application, an engineer Continue reading

Cloudflare and Aruba partner to deliver a seamless global secure network from the branch to the cloud

Today we are excited to announce that Cloudflare and Aruba are working together to develop a solution that will enable Aruba customers to connect EdgeConnect SD-WAN’s with Cloudflare's global network to further secure their corporate traffic with Cloudflare One. Whether organizations need to secure Internet-bound traffic from branch offices using Cloudflare's Secure Web Gateway & Magic Firewall, or enforce firewall policies for east/west traffic between offices via Magic Firewall, we have them covered. This gives customers peace of mind that they have consistent global security from Cloudflare while retaining granular control of their inter-branch and Internet-bound traffic policies from their Aruba EdgeConnect appliances.

SD-WAN solution

A software-defined WAN (SD-WAN) is an evolution of a WAN (wide area network) that simplifies the underlying architecture. Unlike traditional WAN architecture models where expensive leased, and MPLS links are used, SD-WAN can efficiently use a combination of private lines and the public Internet. It brings together the best of both worlds to provide an integrated solution to network administrators in managing and scaling their network and resources with ease.

Aruba’s EdgeConnect SD-WAN solution

We are proud to announce our first enhanced SD-WAN integration. Aruba’s EdgeConnect solution is an industry leader for WAN edge Continue reading

Cloudflare and CrowdStrike partner to give CISOs secure control across devices, applications, and corporate networks

Today, we are very excited to announce multiple new integrations with CrowdStrike. These integrations combine the power of Cloudflare’s expansive network and Zero Trust suite, with CrowdStrike’s Endpoint Detection and Response (EDR) and incident remediation offerings.

At Cloudflare, we believe in making our solutions easily integrate with the existing technology stack of our customers. Through our partnerships and integrations, we make it easier for our customers to use Cloudflare solutions jointly with that of partners, to further strengthen their security posture and unlock more value. Our partnership with CrowdStrike is an apt example of such efforts.

Together, Cloudflare and CrowdStrike are working to simplify the adoption of Zero Trust for IT and security teams. With this expanded partnership, joint customers can identify, investigate, and remediate threats faster through multiple integrations:

First, by integrating Cloudflare’s Zero Trust services with CrowdStrike Falcon Zero Trust Assessment (ZTA), which provides continuous real-time device posture assessments, our customers can verify users’ device posture before granting them access to internal or external applications.

Second, we joined the CrowdXDR Alliance in December 2021 and are partnering with CrowdStrike to share security telemetry and other insights to make it easier for customers to identify and mitigate threats. Continue reading

Announcing the Cloudflare API Gateway

Over the past decade, the Internet has experienced a tectonic shift. It used to be composed of static websites: with text, images, and the occasional embedded movie. But the Internet has grown enormously. We now rely on API-driven applications to help with almost every aspect of life. Rather than just download files, we are able to engage with apps by exchanging rich data. We track workouts and send the results to the cloud. We use smart locks and all kinds of IoT devices. And we interact with our friends online.

This is all wonderful, but it comes with an explosion of complexity on the back end. Why? Developers need to manage APIs in order to support this functionality. They need to monitor and authenticate every single request. And because these tasks are so difficult, they’re usually outsourced to an API gateway provider.

Unfortunately, today’s gateways leave a lot to be desired. First: they’re not cheap. Then there’s the performance impact. And finally, there’s a data and privacy risk, since more than 50% of traffic reaches APIs (and is presumably sent through a third party gateway). What a mess.

Today we’re announcing the Cloudflare API Gateway. We’re going to completely replace Continue reading

Envoy Media: using Cloudflare’s Bot Management & ML

This is a guest post by Ryan Marlow, CTO, and Michael Taggart, Co-founder of Envoy Media Group.

My name is Ryan Marlow, and I’m the CTO of Envoy Media Group. I’m excited to share a story with you about Envoy, Cloudflare, and how we use Bot Management to monitor automated traffic.

Background

Envoy Media Group is a digital marketing and lead generation company. The aim of our work is simple: we use marketing to connect customers with financial services. For people who are experiencing a particular financial challenge, Envoy provides informative videos, money management tools, and other resources. Along the way, we bring customers through an online experience, so we can better understand their needs and educate them on their options. With that information, we check our database of highly vetted partners to see which programs may be useful, and then match them up with the best company to serve them.

As you can imagine, it’s important for us to responsibly match engaged customers to the right financial services. Envoy develops its own brands that guide customers throughout the process. We spend our own advertising dollars, work purely on a performance basis, and choose partners we know will do right Continue reading

Announcing Friendly Bots

When someone mentions bots on the Internet, what’s your first reaction?

It’s probably negative. Most of us conjure up memories of CAPTCHAs, stolen passwords, or some other pain caused by bad bots.

But the truth is, there are plenty of well-behaved bots on the Internet. These include Google’s search crawler and Stripe’s payment bot. At Cloudflare, we manually “verify” good bots, so they don’t get blocked. Our customers can choose to allowlist any bot that is verified. Unfortunately, new bots are popping up faster than we can verify them. So today we’re announcing a solution: Friendly Bots.

Let’s begin with some background.

How does a bot get verified?

We often find good bots via our public form. Anyone can submit a bot, but we prefer that bot operators complete the form to provide us with the information we need. We ask for some standard bits of information: your bot’s name, its public documentation, and its user agent (or regex). Then, we ask for information that will help us validate your bot. There are four common methods:

IP list
Send us a list of IP addresses used by your bot. This doesn’t have to be a static list — you can Continue reading

Introducing Advanced Rate Limiting

Still relying solely on IP firewalling? It’s time to change that.

While the IP address might still be one of the core technologies allowing networks to function, its value for security is long gone. IPs are rarely static; nowadays, mobile operators use carrier-grade network address translation (CGNAT) to share the same IP amongst thousands of individual devices or users. Bots then carry out distributed attacks with low request volume from different IPs to elude throttling. Furthermore, many countries consider IP addresses to be personal data, and it would be a great advancement for privacy if a replacement could be found for elements of security that currently rely on IP addresses to function. A product that is affected by this trend is rate limiting.

Rate limiting is designed to stop requests from overloading a server. It relies on rules. A rate limiting rule is defined by a filter (which typically is a path, like /login) and the maximum number of requests allowed from each user over a period of time. When this threshold is exceeded, an action is triggered (usually a block) for subsequent requests from the same user for a period of time (known as a timeout). Traditional throttling Continue reading

IDC MarketScape positions Cloudflare as a Leader among worldwide Commercial CDN providers

We are thrilled to announce that Cloudflare has been positioned in the Leaders category in the IDC MarketScape: Worldwide Commercial CDN 2022 Vendor Assessment(doc #US47652821, March 2022).

You can download a complimentary copy here.

The IDC MarketScape evaluated 10 CDN vendors based on their current capabilities and future strategies for delivering Commercial CDN services. Cloudflare is recognized as a Leader.

At Cloudflare, we release products at a dizzying pace. When we talk to our customers, we hear again and again that they appreciate Cloudflare for our relentless innovation. In 2021 alone, over the course of seven Innovation Weeks, we launched a diverse set of products and services that made our customers’ experiences on the Internet even faster, more secure, more reliable, and more private.

We leverage economies of scale and network effects to innovate at a fast pace. Of course, there’s more to our secret sauce than our pace of innovation. In the report, IDC notes that Cloudflare is “a highly innovative vendor and continues to invest in its competencies to support advanced technologies such as virtualization, serverless, AI/ML, IoT, HTTP3, 5G and (mobile) edge computing.” In addition, IDC also recognizes Cloudflare for its “integrated SASE offering (that) Continue reading

WAF for everyone: protecting the web from high severity vulnerabilities

At Cloudflare, we like disruptive ideas. Pair that with our core belief that security is something that should be accessible to everyone and the outcome is a better and safer Internet for all.

This isn’t idle talk. For example, back in 2014, we announced Universal SSL. Overnight, we provided SSL/TLS encryption to over one million Internet properties without anyone having to pay a dime, or configure a certificate. This was good not only for our customers, but also for everyone using the web.

In 2017, we announced unmetered DDoS mitigation. We’ve never asked customers to pay for DDoS bandwidth as it never felt right, but it took us some time to reach the network size where we could offer completely unmetered mitigation for everyone, paying customer or not.

Still, I often get the question: how do we do this? It’s simple really. We do it by building great, efficient technology that scales well—and this allows us to keep costs low.

Today, we’re doing it again, by providing a Cloudflare WAF (Web Application Firewall) Managed Ruleset to all Cloudflare plans, free of charge.

Why are we doing this?

High profile vulnerabilities have a major impact across the Internet affecting organizations of Continue reading

Cloudflare Zaraz supports CSP

Cloudflare Zaraz can be used to manage and load third-party tools on the cloud, achieving significant speed, privacy and security improvements. Content Security Policy (CSP) configuration prevents malicious content from being run on your website.

If you have Cloudflare Zaraz enabled on your website, you don’t have to ask yourself twice if you should enable CSP because there’s no harmful collision between CSP & Cloudflare Zaraz.

Why would Cloudflare Zaraz collide with CSP?

Cloudflare Zaraz, at its core, injects a <script> block on every page where it runs. If the website enforces CSP rules, the injected script can be automatically blocked if inline scripts are not allowed. To prevent this, at the moment of script injection, Cloudflare Zaraz adds a nonce to the script-src policy in order for everything to work smoothly.

Cloudflare Zaraz supports CSP enabled by using both Content-Security-Policy headers or Content-Security-Policy <meta> blocks.

What is CSP?

Content Security Policy (CSP) is a security standard meant to protect websites from Cross-site scripting (XSS) or Clickjacking by providing the means to list approved origins for scripts, styles, images or other web resources.

Although CSP is a reasonably mature technology with most modern browsers already implementing the standard, less Continue reading

Security for SaaS providers

Some of the largest Software-as-a-Service (SaaS) providers use Cloudflare as the underlying infrastructure to provide their customers with fast loading times, unparalleled redundancy, and the strongest security — all through our Cloudflare for SaaS product. Today, we’re excited to give our SaaS providers new tools that will help them enhance the security of their customers’ applications.

For our Enterprise customers, we’re bringing WAF for SaaS — the ability for SaaS providers to easily create and deploy different sets of WAF rules for their customers. This gives SaaS providers the ability to segment customers into different groups based on their security requirements.

For developers who are getting their application off the ground, we’re thrilled to announce a Free tier of Cloudflare for SaaS for the Free, Pro, and Biz plans, giving our customers 100 custom hostnames free of charge to provision and test across their account. In addition to that, we want to make it easier for developers to scale their applications, so we’re happy to announce that we are lowering our custom hostname price from \$2 to \$0.10 a month.

But that’s not all! At Cloudflare, we believe security should be available for all. That’s why we’re extending a Continue reading

Improving the WAF with Machine Learning

Cloudflare handles 32 million HTTP requests per second and is used by more than 22% of all the websites whose web server is known by W3Techs. Cloudflare is in the unique position of protecting traffic for 1 out of 5 Internet properties which allows it to identify threats as they arise and track how these evolve and mutate.

The Web Application Firewall (WAF) sits at the core of Cloudflare's security toolbox and  Managed Rules are a key feature of the WAF. They are a collection of rules created by Cloudflare’s analyst team that block requests when they show patterns of known attacks. These managed rules work extremely well for patterns of established attack vectors, as they have been extensively tested to minimize both false negatives (missing an attack) and false positives (finding an attack when there isn’t one). On the downside, managed rules often miss attack variations (also known as bypasses) as static regex-based rules are intrinsically sensitive to signature variations introduced, for example, by fuzzing techniques.

We witnessed this issue when we released protections for log4j. For a few days, after the vulnerability was made public, we had to constantly update the rules to match variations and mutations as Continue reading

A new WAF experience

Around three years ago, we brought multiple features into the Firewall tab in our dashboard navigation, with the motivation “to make our products and services intuitive.” With our hard work in expanding capabilities offerings in the past three years, we want to take another opportunity to evaluate the intuitiveness of Cloudflare WAF (Web Application Firewall).

Our customers lead the way to new WAF

The security landscape is moving fast; types of web applications are growing rapidly; and within the industry there are various approaches to what a WAF includes and can offer. Cloudflare not only proxies enterprise applications, but also millions of personal blogs, community sites, and small businesses stores. The diversity of use cases are covered by various products we offer; however, these products are currently scattered and that makes visibility of active protection rules unclear. This pushes us to reflect on how we can best support our customers in getting the most value out of WAF by providing a clearer offering that meets expectations.

A few months ago, we reached out to our customers to answer a simple question: what do you consider to be part of WAF? We employed a range of user research methods including Continue reading

Democratizing email security: protecting individuals and businesses of all sizes from phishing and malware attacks

Since our founding, Cloudflare has been on a mission to take expensive, complex security solutions typically only available to the largest companies and make them easy to use and accessible to everyone. In 2011 and 2015 we did this for the web application firewall and SSL/TLS markets, simplifying the process of protecting websites from application vulnerabilities and encrypting HTTP requests down to single clicks; in 2020, during the start of the COVID-19 pandemic, we made our Zero Trust suite available to everyone; and today—in the face of heightened phishing attacks—we’re doing the same for the email security market.

Once the acquisition of Area 1 closes, as we expect early in the second quarter of 2022, we plan to give all paid self-serve plans access to their email security technology at no additional charge. Control, customization, and visibility via analytics will vary with plan level, and the highest flexibility and support levels will be available to Enterprise customers for purchase.

All self-serve users will also get access to a more feature-packed version of the Zero Trust solution we made available to everyone in 2020. Zero Trust services are incomplete without an email security solution, and CISA’s recent report makes that clearer Continue reading

Investigating threats using the Cloudflare Security Center

Cloudflare blocks a lot of diverse security threats, with some of the more interesting attacks targeting the “long tail” of the millions of Internet properties we protect. The data we glean from these attacks trains our machine learning models and improves the efficacy of our network and application security products, but historically hasn’t been available to query directly. This week, we’re changing that.

All customers will soon be granted access to our new threat investigations portal, Investigate, in the Cloudflare Security Center (first launched in December 2021). Additionally, we’ll be annotating threats across our analytics platform with this intelligence to streamline security workflows and tighten feedback loops.

What sorts of data might you want to look up here? Let’s say you’re seeing an IP address in your logs and want to learn which hostnames have pointed to it via DNS, or you’re seeing a cluster of attacks come from an autonomous system (AS) you’re not familiar with. Or maybe you want to investigate a domain name to see how it’s been categorized from a threat perspective. Simply enter any of those items into the omni search box, and we’ll tell you everything we know.

IPs and hostnames will be Continue reading

Get full observability into your Cloudflare logs with New Relic

Building a great customer experience is at the heart of any business. Building resilient products is half the battle — teams also need observability into their applications and services that are running across their stack.

Cloudflare provides analytics and logs for our products in order to give our customers visibility to extract insights. Many of our customers use Cloudflare along with other applications and network services and want to be able to correlate data through all of their systems.

Understanding normal traffic patterns, causes of latency and errors can be used to improve performance and ultimately the customer experience. For example, for websites behind Cloudflare, analyzing application logs and origin server logs along with Cloudflare’s HTTP request logs give our customers an end-to-end visibility about the journey of a request.

We’re excited to have partnered with New Relic to create a direct integration that provides this visibility. The direct integration with our logging product, Logpush, means customers no longer need to pay for middleware to get their Cloudflare data into New Relic. The result is a faster log delivery and fewer costs for our mutual customers!

We’ve invited the New Relic team to dig into how New Relic One can Continue reading