Bot Attack trends for Jan-Jul 2020

Now that we’re a long way through 2020, let’s take a look at automated traffic, which makes up almost 40% of total Internet traffic.

This blog post is a high-level overview of bot traffic on Cloudflare’s network. Cloudflare offers a comprehensive Bot Management tool for Enterprise customers, along with an effective free tool called Bot Fight Mode. Because of the tremendous amount of traffic that flows through our network each day, Cloudflare is in a unique position to analyze global bot trends.

In this post, we will cover the basics of bot traffic and distinguish between automated requests and other human requests (What Is A Bot?). Then, we’ll move on to a global overview of bot traffic around the world (A RoboBird’s Eye View, A Bot Day and Bots All Over The World), and dive into North American traffic (A Look into North American Traffic).  Lastly, we’ll finish with an overview of how the coronavirus pandemic affected global traffic, and we’ll take a deeper look at European traffic (Bots During COVID-19 In Europe).

On average, Cloudflare processes 18 million HTTP requests every second. This is a great opportunity to understand how bots shape the Continue reading

How a Customer’s Trust in Cloudflare Led to a Big Win against Bots

Key Points

• Anyone with public-facing web properties is likely to have bot traffic on their website.
• One type of bot that commonly targets eCommerce and online portals is a ‘scraper bot’.
• Some scraper bots are good (such as those used by search engines to assess your website’s content to inform search results, or price comparison sites to help inform consumer decisions), however many are malicious, and will work to scrape not only images but also pricing data from your site for use by a competitor.
• Many Bot Management providers will need to divert your traffic to a dedicated data centre to analyse your traffic and ‘scrub’ it clean from malicious bot traffic before sending it on to your site. While effective this will almost certainly add latency to the traffic’s ‘journey’ resulting in degraded user experience. Look for a technology partner with an expansive network who can scan your traffic in real time as it passes through any data centre on their network.
• Good and bad scraper bots behave in largely the same way, making it difficult for bot protection systems to differentiate between the two. A common challenge with Bot Management solutions is that they can return a high Continue reading

One more (Zero Trust) thing: Cloudflare Intrusion Detection System

Today, we’re very excited to announce our plans for Cloudflare Intrusion Detection System, a new product that monitors your network and alerts when an attack is suspected. With deep integration into Cloudflare One, Cloudflare Intrusion Detection System gives you a bird’s eye view of your entire global network and inspects all traffic for bad behavior, regardless of whether it came from outside or inside your network.

Analyze your network without doing the legwork

Enterprises build firewall rules to keep their networks safe from external and internal threats. When bad actors try to attack a network, those firewalls check if the attack matches a rule pattern. If it does, the firewall steps in and blocks the attack.

Teams used to configure those rules across physical firewall appliances, frequently of different makes and models, deployed to physical locations. Yesterday, we announced Magic Firewall, Cloudflare’s network-level firewall delivered in our data centers around the world. Your team can write a firewall rule once, deploy it to Cloudflare, and our global network will protect your offices and data centers without the need for on-premises hardware.

This is great if you know where attacks are coming from. If you don’t have that level Continue reading

How our network powers Cloudflare One

Earlier this week, we announced Cloudflare One™, a unified approach to solving problems in enterprise networking and security. With Cloudflare One, your organization’s data centers, offices, and devices can all be protected and managed in a single control plane. Cloudflare’s network is central to the value of all of our products, and today I want to dive deeper into how our network powers Cloudflare One.

Over the past ten years, Cloudflare has encountered the same challenges that face every organization trying to grow and protect a global network: we need to protect our infrastructure and devices from attackers and malicious outsiders, but traditional solutions aren’t built for distributed networks and teams. And we need visibility into the activity across our network and applications, but stitching together logging and analytics tools across multiple solutions is painful and creates information gaps.

We’ve architected our network to meet these challenges, and with Cloudflare One, we’re extending the advantages of these decisions to your company’s network to help you solve them too.

Distribution

Enterprises and some small organizations alike have team members around the world. Legacy models of networking forced traffic back through central choke points, slowing down users and constraining network scale. Continue reading

Introducing Magic Firewall

Today we’re excited to announce Magic Firewall™, a network-level firewall delivered through Cloudflare to secure your enterprise. Magic Firewall covers your remote users, branch offices, data centers and cloud infrastructure. Best of all, it’s deeply integrated with Cloudflare One™, giving you a one-stop overview of everything that’s happening on your network.

Cloudflare Magic Transit™ secures IP subnets with the same DDoS protection technology that we built to keep our own global network secure. That helps ensure your network is safe from attack and available and it replaces physical appliances that have limits with Cloudflare’s network.

That still leaves some hardware onsite, though, for a different function: firewalls. Networks don’t just need protection from DDoS attacks; administrators need a way to set policies for all traffic entering and leaving the network. With Magic Firewall, we want to help your team deprecate those network firewall appliances and move that burden to the Cloudflare global network.

Firewall boxes are miserable to manage

Network firewalls have always been clunky. Not only are they expensive, they are bound by their own hardware constraints. If you need more CPU or memory, you have to buy more boxes. If you lack capacity, the entire network suffers, directly Continue reading

How small businesses can start using Cloudflare One today

Earlier this week, we announced Cloudflare One™, our comprehensive, cloud-based network-as-a-service solution. Cloudflare One improves network performance and security while reducing cost and complexity for companies of all sizes.

Cloudflare One is built to handle the scale and complexity of the largest corporate networks. But when it comes to network security and performance, the industry has focused all too often on the largest of customers with significant budgets and technology teams. At Cloudflare, we think it’s our opportunity and responsibility to serve everyone, and help companies of all sizes benefit from a better Internet.

This is Zero Trust Week at Cloudflare, and we’ve already talked about our mantra of Zero Trust for Everyone. As a quick refresher, Zero Trust is a security framework that assumes all networks, devices, and Internet destinations are inherently compromised and therefore should not be trusted. Cloudflare One facilitates Zero Trust security by securing how your users connect to corporate applications and the Internet at large.

As a small business network administrator, there are fundamentally three things you need to protect: devices, applications, and the network itself. Below, I’ll outline how you can secure devices whether they are in your office (DNS Filtering) or remote (WARP+ Continue reading

Introducing Cloudflare Browser Isolation beta

Reimagining the Browser

A web browser, the same application that connects users to the entire Internet, also connects you to all of the potentially harmful parts of the Internet. It’s an open door to nearly every connected system on the planet, which is powerful and terrifying.

We also rely on browsers more than ever. Most applications that we use live in a browser and that will continue to increase. For more and more organizations, a corporate laptop is just a managed web browser machine.

To keep those devices safe, and the data they hold or access, enterprises have started to deploy “browser isolation” services where the browser itself doesn’t run on the machine. Instead, the browser runs on a virtual machine in a cloud provider somewhere. By running away from the device, threats from the browser stay on that virtual machine somewhere in the cloud.

However, most isolation solutions take one of two approaches that both ruin the convenience and flexibility of a web browser:

• Record the isolated browser and send a live stream of it to the user, which is slow and makes it difficult to do basic things like input text to a form.
• Unpack the webpage, inspect Continue reading

Introducing Cloudflare One Intel

Earlier this week, we announced Cloudflare One, a single platform for networking and security management. Cloudflare One extends the speed, reliability, and security we’ve brought to Internet properties and applications over the last decade to make the Internet the new enterprise WAN.

Underpinning Cloudflare One is Cloudflare’s global network - today, our network spans more than 200 cities worldwide and is within milliseconds of nearly everyone connected to the Internet. Our network handles, on average, 18 million HTTP requests and 6 million DNS requests per second. With 1 billion unique IP addresses connecting to the Cloudflare network each day, we have one of the broadest views on Internet activity worldwide.

We see a large diversity of Internet traffic across our entire product suite. Every day, we block 72 billion cyberthreats. This visibility provides us with a unique position to understand and mitigate Internet threats, and enables us to see new threats and malware before anyone else.

At the beginning of this month, as part of our 10th Birthday Week, we launched Cloudflare Radar, which shares high-level trends with the general public based on our network’s aggregate data. The same data that powers that view of the Internet also Continue reading

Introducing WARP for Desktop and Cloudflare for Teams

Cloudflare launched ten years ago to keep web-facing properties safe from attack and fast for visitors. Cloudflare customers owned Internet properties that they placed on our network. Visitors to those sites and applications enjoyed a faster experience, but that speed was not consistent for accessing Internet properties outside the Cloudflare network.

Over the last few years, we began building products that could help deliver a faster and safer Internet to everyone, not just visitors to sites on our network. We started with the first step to visiting any website, a DNS query, and released the world’s fastest public DNS resolver, 1.1.1.1. Any Internet user could improve the speed to connect to any website simply by changing their resolver.

While making the Internet faster for users, we also focused on making it more private. We built 1.1.1.1 to accelerate the last mile of connections, from user to our edge or other destinations on the Internet. Unlike other providers, we did not build it to sell ads.

Last year we went one step further to make the entire connection from a device both faster and safer when we launched Cloudflare WARP. With the push of a Continue reading

Cloudflare Gateway now protects teams, wherever they are

In January 2020, we launched Cloudflare for Teams—a new way to protect organizations and their employees globally, without sacrificing performance. Cloudflare for Teams centers around two core products - Cloudflare Access and Cloudflare Gateway.

In March 2020, Cloudflare launched the first feature of Cloudflare Gateway, a secure DNS filtering solution powered by the world’s fastest DNS resolver. Gateway’s DNS filtering feature kept users safe by blocking DNS queries to potentially harmful destinations associated with threats like malware, phishing, or ransomware. Organizations could change the router settings in their office and, in about five minutes, keep the entire team safe.

Shortly after that launch, entire companies began leaving their offices. Users connected from initially makeshift home offices that have become permanent in the last several months. Protecting users and data has now shifted from a single office-level setting to user and device management in hundreds or thousands of locations.

Security threats on the Internet have also evolved. Phishing campaigns and malware attacks have increased in the last six months. Detecting those types of attacks requires looking deeper than just the DNS query.

Starting today, we’re excited to announce two features in Cloudflare Gateway that solve those new challenges. First, Continue reading

Argo Tunnels that live forever

Cloudflare secures your origin servers by proxying requests to your DNS records through our anycast network and to the external IP of your origin. However, external IP addresses can provide attackers with a path around Cloudflare security if they discover those destinations.

We launched Argo Tunnel as a secure way to connect your origin to Cloudflare without a publicly routable IP address. With Tunnel, you don’t send traffic to an external IP. Instead, a lightweight daemon runs in your infrastructure and creates outbound-only connections to Cloudflare’s edge. With Argo Tunnel, you can quickly deploy infrastructure in a Zero Trust model by ensuring all requests to your resources pass through Cloudflare’s security filters.

Originally, your Argo Tunnel connection corresponded to a DNS record in your account. Requests to that hostname hit Cloudflare’s network first and our edge sends those requests over the Argo Tunnel to your origin. Since these connections are outbound-only, you no longer need to poke holes in your infrastructure’s firewall. Your origins can serve traffic through Cloudflare without being vulnerable to attacks that bypass Cloudflare.

However, fitting an outbound-only connection into a reverse proxy creates some ergonomic and stability hurdles. The original Argo Tunnel architecture attempted to both Continue reading

Zero Trust For Everyone

We launched Cloudflare for Teams to make Zero Trust security accessible for all organizations, regardless of size, scale, or resources. Starting today, we are excited to take another step on this journey by announcing our new Teams plans, and more specifically, our Cloudflare for Teams Free plan, which protects up to 50 users at no cost. To get started, sign up today.

If you’re interested in how and why we’re doing this, keep scrolling.

Our Approach to Zero Trust

Cloudflare Access is one-half of Cloudflare for Teams - a Zero Trust solution that secures inbound connections to your protected applications. Cloudflare Access works like a bouncer, checking identity at the door to all of your applications.

The other half of Cloudflare for Teams is Cloudflare Gateway which, as our clever name implies, is a Secure Web Gateway protecting all of your users’ outbound connections to the Internet. To continue with this analogy, Cloudflare Gateway is your organization’s bodyguard, securing your users as they navigate the Internet.

Together, these two solutions provide a powerful, single dashboard to protect your users, networks, and applications from malicious actors.

A Mission-Driven Solution

At Cloudflare, our mission is to help build a better Internet. That Continue reading

Cloudflare Access: now for SaaS apps, too

We built Cloudflare Access™ as a tool to solve a problem we had inside of Cloudflare. We rely on a set of applications to manage and monitor our network. Some of these are popular products that we self-host, like the Atlassian suite, and others are tools we built ourselves. We deployed those applications on a private network. To reach them, you had to either connect through a secure WiFi network in a Cloudflare office, or use a VPN.

That VPN added friction to how we work. We had to dedicate part of Cloudflare’s onboarding just to teaching users how to connect. If someone received a PagerDuty alert, they had to rush to their laptop and sit and wait while the VPN connected. Team members struggled to work while mobile. New offices had to backhaul their traffic. In 2017 and early 2018, our IT team triaged hundreds of help desk tickets with titles like these:

While our IT team wrestled with usability issues, our Security team decided that poking holes in our private network was too much of a risk to maintain. Once on the VPN, users almost always had too much access. We had limited visibility into what happened on Continue reading

Introducing Cloudflare One

Today we’re announcing Cloudflare One™. It is the culmination of engineering and technical development guided by conversations with thousands of customers about the future of the corporate network. It provides secure, fast, reliable, cost-effective network services, integrated with leading identity management and endpoint security providers.

Over the course of this week, we'll be rolling out the components that enable Cloudflare One, including our WARP Gateway Clients for desktop and mobile, our Access for SaaS solution, our browser isolation product, and our next generation network firewall and intrusion detection system.

The old model of the corporate network has been made obsolete by mobile, SaaS, and the public cloud. The events of 2020 have only accelerated the need for a new model. Zero Trust networking is the future and we are proud to be enabling that future. Having worked on the components of what is Cloudflare One for the last two years, we’re excited to unveil today how they’ve come together into a robust SASE solution and share how customers are already using it to deliver the more secure and productive future of the corporate network.

What Is Cloudflare One? Secure, Optimized Global Networking

Cloudflare One is a comprehensive, cloud-based network-as-a-service solution Continue reading

What is Cloudflare One?

Running a secure enterprise network is really difficult. Employees spread all over the world work from home. Applications are run from data centers, hosted in public cloud, and delivered as services. Persistent and motivated attackers exploit any vulnerability.

Enterprises used to build networks that resembled a castle-and-moat. The walls and moat kept attackers out and data in. Team members entered over a drawbridge and tended to stay inside the walls. Trust folks on the inside of the castle to do the right thing, and deploy whatever you need in the relative tranquility of your secure network perimeter.

The Internet, SaaS, and “the cloud” threw a wrench in that plan. Today, more of the workloads in a modern enterprise run outside the castle than inside. So why are enterprises still spending money building more complicated and more ineffective moats?

Today, we’re excited to share Cloudflare One™, our vision to tackle the intractable job of corporate security and networking.

Cloudflare One combines networking products that enable employees to do their best work, no matter where they are, with consistent security controls deployed globally.

Starting today, you can begin replacing traffic backhauls to security appliances with Cloudflare WARP and Gateway to filter Continue reading

Exploring WebAssembly AI Services on Cloudflare Workers

This is a guest post by Videet Parekh, Abelardo Lopez-Lagunas, Sek Chai at Latent AI.

Edge networks present a significant opportunity for Artificial Intelligence (AI) performance and applicability. AI technologies already make it possible to run compelling applications like object and voice recognition, navigation, and recommendations.

AI at the edge presents a host of benefits. One is scalability—it is simply impractical to send all data to a centralized cloud. In fact, one study has predicted a global scope of 90 zettabytes generated by billions of IoT devices by 2025. Another is privacy—many users are reluctant to move their personal data to the cloud, whereas data processed at the edge are more ephemeral.

When AI services are distributed away from centralized data centers and closer to the service edge, it becomes possible to enhance the overall application speed without moving data unnecessarily.  However, there are still challenges to make AI from the deep-cloud run efficiently on edge hardware. Here, we use the term deep-cloud to refer to highly centralized, massively-sized data centers. Deploying edge AI services can be hard because AI is both computational and memory bandwidth intensive. We need to tune the AI models so the computational latency and bandwidth Continue reading

Let’s build a Cloudflare Worker with WebAssembly and Haskell

This is a guest post by Cristhian Motoche of Stack Builders.

At Stack Builders, we believe that Haskell’s system of expressive static types offers many benefits to the software industry and the world-wide community that depends on our services. In order to fully realize these benefits, it is necessary to have proper training and access to an ecosystem that allows for reliable deployment of services. In exploring the tools that help us run our systems based on Haskell, our developer Cristhian Motoche has created a tutorial that shows how to compile Haskell to WebAssembly using Asterius for deployment on Cloudflare.

What is a Cloudflare Worker?

Cloudflare Workers is a serverless platform that allows us to run our code on the edge of the Cloudflare infrastructure. It's built on Google V8, so it’s possible to write functionalities in JavaScript or any other language that targets WebAssembly.

WebAssembly is a portable binary instruction format that can be executed fast in a memory-safe sandboxed environment. For this reason, it’s especially useful for tasks that need to perform resource-demanding and self-contained operations.

Why use Haskell to target WebAssembly?

Haskell is a pure functional languages that can target WebAssembly. As such, It helps developers Continue reading

Know When You’ve Been DDoS’d

Today we’re announcing the availability of DDoS attack alerts. The alerts are available for free for all Cloudflare’s customers on paid plans.

Unmetered DDoS protection

Last week we celebrated Cloudflare’s 10th birthday in what we call Birthday Week. Every year, on each day of Birthday Week, we announce a new product with the goal of helping make the Internet a better place -- one that is safer and faster. To do that, over the years we’ve democratized many products that were previously only available to large enterprises by making them available for free (or at very low cost) to all. For example, on Cloudflare’s 7th birthday in 2017, we announced free unmetered DDoS protection as part of every Cloudflare product and every plan, including the free plan.

DDoS attacks aim to take down websites or online services and make them unavailable to the public. We wanted to make sure that every organization and every website is available and accessible, regardless if they can or can’t afford enterprise-grade DDoS protection. This has been a core part of our mission. We’ve been heavily investing in our DDoS protection capabilities over the last 10 years, and we will continue to do so in Continue reading

Birthday week: Cloudflare turns 10

2020 marks a major milestone for Cloudflare: it’s our 10th birthday.

We’ve always used birthdays as an opportunity to give back to the Internet. But this year — a year in which the Internet has been so central to giving us all some degree of connectedness and normalcy — it feels like giving back to the Internet has been more important than ever.

And while we couldn’t celebrate in person, we were humbled by some of the incredible minds that joined us online to talk about how the Internet has changed over the last ten years — and what we might see over the next ten.

With that, let’s recap the key announcements from Birthday Week 2020.

Day 1, Monday: Workers

During Birthday Week in 2017, Cloudflare announced Workers — a serverless platform that represented a completely new way to build applications: by writing your code directly onto our network edge. On Monday of this year’s Birthday Week, we announced Durable Objects and Cron Triggers — both of which continue to expand the use cases that Workers can address.

Many folks associate the serverless paradigm with functions as a service — which, at its core, is stateless. Workers KV started Continue reading

Introducing support for the AVIF image format

We've added support for the new AVIF image format in Image Resizing. It compresses images significantly better than older-generation formats such as WebP and JPEG. It's supported in Chrome desktop today, and support is coming to other Chromium-based browsers, as well as Firefox.

What’s the benefit?

More than a half of an average website's bandwidth is spent on images. Improved image compression can save bandwidth and improve overall performance of the web. The compression in AVIF is so good that images can reduce to half the size of JPEG and WebP

What is AVIF?

AVIF is a combination of the HEIF ISO standard, and a royalty-free AV1 codec by Mozilla, Xiph, Google, Cisco, and many others.

Currently JPEG is the most popular image format on the Web. It's doing remarkably well for its age, and it will likely remain popular for years to come thanks to its excellent compatibility. There have been many previous attempts at replacing JPEG, such as JPEG 2000, JPEG XR and WebP. However, these formats offered only modest compression improvements, and didn't always beat JPEG on image quality. Compression and image quality in AVIF is better than in all of them, and by a wide margin.

Continue reading