Announcing the Cloudflare API Gateway

Over the past decade, the Internet has experienced a tectonic shift. It used to be composed of static websites: with text, images, and the occasional embedded movie. But the Internet has grown enormously. We now rely on API-driven applications to help with almost every aspect of life. Rather than just download files, we are able to engage with apps by exchanging rich data. We track workouts and send the results to the cloud. We use smart locks and all kinds of IoT devices. And we interact with our friends online.

This is all wonderful, but it comes with an explosion of complexity on the back end. Why? Developers need to manage APIs in order to support this functionality. They need to monitor and authenticate every single request. And because these tasks are so difficult, they’re usually outsourced to an API gateway provider.

Unfortunately, today’s gateways leave a lot to be desired. First: they’re not cheap. Then there’s the performance impact. And finally, there’s a data and privacy risk, since more than 50% of traffic reaches APIs (and is presumably sent through a third party gateway). What a mess.

Today we’re announcing the Cloudflare API Gateway. We’re going to completely replace Continue reading

Announcing Friendly Bots

When someone mentions bots on the Internet, what’s your first reaction?

It’s probably negative. Most of us conjure up memories of CAPTCHAs, stolen passwords, or some other pain caused by bad bots.

But the truth is, there are plenty of well-behaved bots on the Internet. These include Google’s search crawler and Stripe’s payment bot. At Cloudflare, we manually “verify” good bots, so they don’t get blocked. Our customers can choose to allowlist any bot that is verified. Unfortunately, new bots are popping up faster than we can verify them. So today we’re announcing a solution: Friendly Bots.

Let’s begin with some background.

How does a bot get verified?

We often find good bots via our public form. Anyone can submit a bot, but we prefer that bot operators complete the form to provide us with the information we need. We ask for some standard bits of information: your bot’s name, its public documentation, and its user agent (or regex). Then, we ask for information that will help us validate your bot. There are four common methods:

IP list
Send us a list of IP addresses used by your bot. This doesn’t have to be a static list — you can Continue reading

The Grinch Bot is Stealing Christmas!

This week, a group of US lawmakers introduced the Stopping Grinch Bots Act — new legislation that could stop holiday hoarders on the Internet. This inspired us to put a spin on a Dr. Seuss classic:

Each person on the Internet liked Christmas a lot
But the Grinch Bot, built by the scalper did not!
The Grinch Bot hated Christmas! The whole Christmas season!
Now, please don’t ask why. No one quite knows the reason.

Cloudflare stops billions of bad bots every day. As you might have guessed, we see all types of attacks, but none is more painful than a Grinch Bot attack. Join us as we take a closer look at this notorious holiday villain...

25 days seconds of Christmas

What is the Grinch Bot? Technically speaking, it’s just a program running on a computer, making automated requests that reach different websites. We’ve come to refer to these requests as “bots” on the Internet. Bots move quickly, leveraging the efficiency of computers to carry out tasks at scale. The Grinch Bot is a very special type that satisfies two conditions:

1. It only pursues online inventory, attempting to purchase items before humans can complete their orders.
2. It only operates Continue reading

Introducing Super Bot Fight Mode

Almost half of the Internet’s traffic is powered by bots. Bots have scoured the net for years, relentlessly hacking into bank accounts, scooping up Bruno Mars tickets, and scraping websites for data. The problem is so widespread that we launched Bot Fight Mode in 2019 to fight back. Since then, over 150,000 individuals and small businesses have used the product, and we’ve received countless requests for more functionality. More analytics, more detections, and more controls.

Introducing Super Bot Fight Mode.

Beginning immediately, any Cloudflare user with a Pro or Business site can take new action against bots. We’ve added advanced features in the dashboard and some exciting updates to analytics. Free customers will retain all the benefits they've enjoyed with Bot Fight Mode, and our Enterprise Bot Management product will continue to push the needle on innovation.

In the Dashboard

Our bot solutions have a new home. The features we discuss in this blog post go beyond a single toggle, so we created a hub for bot protection. Head to the Firewall app and select the “Bots” subtab to get started.

The new hub is live for all users, including those with Enterprise Bot Management.

Pro Plan Features

First up: Continue reading

Announcing API Abuse Detection

APIs are incredibly important. Throughout the 2000s, they formed the backbone of popular web services, helping the Internet become more useful and accessible. In the 2010s, APIs played a larger role in our lives, allowing personal devices to communicate with the digital world. Many of our daily activities, like using rideshare services and paying for lattes, are dependent on this form of modern communication. Now we are approaching a post-pandemic world in which APIs will be more important than ever.

Unfortunately, as any technology grows, so does its surface area for abuse. APIs are no exception. Competing rideshare services might monitor each other’s prices via API, spawning a price war and a waste of digital resources. Or a coffee drinker might manipulate an API for a latte discount. Some companies have thousands of APIs — including ones that they don’t even know about. Cloudflare can help solve these problems.

Today, we are announcing early access to API Discovery and API Abuse Detection.

Background

Before going further, it’s important to explain why we need a solution for APIs. Traditional security tools, including Rate Limiting and DDoS Protection, can be wonderfully useful. But these approaches were not built to act Continue reading

Introducing Bot Analytics

Bots — both good and bad — are everywhere on the Internet. Roughly 40% of Internet traffic is automated. Fortunately, Cloudflare offers a tool that can detect and block unwanted bots: we call it Bot Management. This is the most recent platform in our long history of detecting bots for our customers. In fact, Cloudflare has always offered some form of bot detection. Over the past two years, our team has focused on building advanced detection engines, innovating as bots become more sophisticated, and creating new features.

Today, we are releasing Bot Analytics to help you visualize your automated traffic.

Background

It’s worth including some background for those who are new to bots.

Many websites expect human behavior. When I shop online, I behave as anyone else would: I might search for a few items, read reviews when I find something interesting, and eventually complete an order. This is expected. It is a standard use of the Internet.

Unfortunately, without protection these sites can be ripe for exploitation. Those shoes I was looking at? They are limited edition sneakers that resell for five times the price. Sneaker hoarders clamor at the chance to buy a pair (or fifty). Or perhaps Continue reading

Introducing Certificate Transparency Monitoring

Today we’re launching Certificate Transparency Monitoring (my summer project as an intern!) to help customers spot malicious certificates. If you opt into CT Monitoring, we’ll send you an email whenever a certificate is issued for one of your domains. We crawl all public logs to find these certificates quickly. CT Monitoring is available now in public beta and can be enabled in the Crypto Tab of the Cloudflare dashboard.

Background

Most web browsers include a lock icon in the address bar. This icon is actually a button — if you’re a security advocate or a compulsive clicker (I’m both), you’ve probably clicked it before! Here’s what happens when you do just that in Google Chrome:

This seems like good news. The Cloudflare blog has presented a valid certificate, your data is private, and everything is secure. But what does this actually mean?

Certificates

Your browser is performing some behind-the-scenes work to keep you safe. When you request a website (say, cloudflare.com), the website should present a certificate that proves its identity. This certificate is like a stamp of approval: it says that your connection is secure. In other words, the certificate proves that content was not intercepted or Continue reading