Category Archives for "CloudFlare"

Do hackers eat turkey? And other Thanksgiving Internet trends

Do hackers eat turkey? And other Thanksgiving Internet trends

Thanksgiving is a tradition celebrated by millions of Americans across six time zones and 50 states, usually involving travel and bringing families together. This year, it was celebrated yesterday, on November 23, 2023. With the Internet so deeply enmeshed into our daily lives, anything that changes how so many people behave is going to also have an impact on online traffic. But how big an impact, exactly?

At a high level: a 10% daily decrease in Internet traffic in the US (compared to the previous week). That happens to be the exact same percentage decrease we observed in 2022. So, Thanksgiving in the US, at least in the realm of Internet traffic, seems consistent with last year.

Let’s dig into more details about how people deal with cooking (or online ordering!) and whether family gatherings are less online, according to our Cloudflare Radar data. We’ll also touch on whether hackers stop for turkey, too.

The Thanksgiving hour: around 15:00 (local time)

While we can see a 10% overall daily drop in US traffic due to Thanksgiving, the drop is even more noticeable when examining traffic on an hour-by-hour basis. Internet activity began to decrease significantly after 12:00 EST, persisting Continue reading

Workers AI Update: Stable Diffusion, Code Llama + Workers AI in 100 cities

Workers AI Update: Stable Diffusion, Code Llama + Workers AI in 100 cities

Thanksgiving might be a US holiday (and one of our favorites — we have many things to be thankful for!). Many people get excited about the food or deals, but for me as a developer, it’s also always been a nice quiet holiday to hack around and play with new tech. So in that spirit, we're thrilled to announce that Stable Diffusion and Code Llama are now available as part of Workers AI, running in over 100 cities across Cloudflare’s global network.

As many AI fans are aware, Stable Diffusion is the groundbreaking image-generation model that can conjure images based on text input. Code Llama is a powerful language model optimized for generating programming code.

For more of the fun details, read on, or head over to the developer docs to get started!

Generated by Stable Diffusion - “Happy llama in an orange cloud celebrating thanksgiving”

Generating images with Stable Diffusion

Stability AI launched Stable Diffusion XL 1.0 (SDXL) this past summer. You can read more about it here, but we’ll briefly mention some really cool aspects.

First off, “Distinct images can be prompted without having any particular ‘feel’ imparted by the model, ensuring absolute freedom of Continue reading

Workers AI Update: Hello Mistral 7B

This post is also available in Deutsch.

Workers AI Update: Hello Mistral 7B

Today we’re excited to announce that we’ve added the Mistral-7B-v0.1-instruct to Workers AI. Mistral 7B is a 7.3 billion parameter language model with a number of unique advantages. With some help from the founders of Mistral AI, we’ll look at some of the highlights of the Mistral 7B model, and use the opportunity to dive deeper into “attention” and its variations such as multi-query attention and grouped-query attention.

Mistral 7B tl;dr:

Mistral 7B is a 7.3 billion parameter model that puts up impressive numbers on benchmarks. The model:

  • Outperforms Llama 2 13B on all benchmarks
  • Outperforms Llama 1 34B on many benchmarks,
  • Approaches CodeLlama 7B performance on code, while remaining good at English tasks, and
  • The chat fine-tuned version we’ve deployed outperforms Llama 2 13B chat in the benchmarks provided by Mistral.

Here’s an example of using streaming with the REST API:

curl -X POST \
“{account-id}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1” \
-H “Authorization: Bearer {api-token}” \
-H “Content-Type:application/json” \
-d '{ “prompt”: “What is grouped query attention”, “stream”: true }'

API Response: { response: “Grouped query attention is a technique used in natural language processing  (NLP) and machine learning  Continue reading

2024, the year of elections

2024, the year of elections

2024 is a year of elections, with more than 70 elections scheduled in 40 countries around the world. One of the key pillars of democracy is trust. To that end, ensuring that the Internet is trusted, secure, reliable, and accessible for the public and those working in the election space is critical to any free and fair election.

Cloudflare has considerable experience in gearing up for elections and identifying how our cyber security tools can be used to help vulnerable groups in the election space. In December 2022, we expanded our product set to include Zero Trust products to assist these groups against new and emerging threats. Over the last few years, we’ve reported on our work in protecting a range of election entities and as we prepare for the 2024 elections, we want to provide insight into attack trends we’ve seen against these groups to understand what to expect in the next year.

For this blog post, we identified cyber attack trends for a variety of groups in the elections space based in the United States, as many of our Cloudflare Impact projects provide services to these groups. These include U.S. state and local government websites protected under Continue reading

How to execute an object file: Part 4, AArch64 edition

How to execute an object file: Part 4, AArch64 edition

Translating source code written in a high-level programming language into an executable binary typically involves a series of steps, namely compiling and assembling the code into object files, and then linking those object files into the final executable. However, there are certain scenarios where it can be useful to apply an alternate approach that involves executing object files directly, bypassing the linker. For example, we might use it for malware analysis or when part of the code requires an incompatible compiler. We’ll be focusing on the latter scenario: when one of our libraries needed to be compiled differently from the rest of the code. Learning how to execute an object file directly will give you a much better sense of how code is compiled and linked together.

To demonstrate how this was done, we have previously published a series of posts on executing an object file:

The initial posts are dedicated to the x86 architecture. Since then the fleet of our working machines has expanded to include a large and growing number of ARM CPUs. This Continue reading

Introducing advanced session audit capabilities in Cloudflare One

The basis of Zero Trust is defining granular controls and authorization policies per application, user, and device. Having a system with a sufficient level of granularity to do this is crucial to meet both regulatory and security requirements. But there is a potential downside to so many controls: in order to troubleshoot user issues, an administrator has to consider a complex combination of variables across applications, user identity, and device information, which may require painstakingly sifting through logs.

We think there’s a better way — which is why, starting today, administrators can easily audit all active user sessions and associated data used by their Cloudflare One policies. This enables the best of both worlds: extremely granular controls, while maintaining an improved ability to troubleshoot and diagnose Zero Trust deployments in a single, simple control panel. Information that previously lived in a user’s browser or changed dynamically is now available to administrators without the need to bother an end user or dig into logs.

A quick primer on application authentication and authorization

Authentication and Authorization are the two components that a Zero Trust policy evaluates before allowing a user access to a resource.

Authentication is the process of verifying the identity Continue reading

Introducing hostname and ASN lists to simplify WAF rules creation

Introducing hostname and ASN lists to simplify WAF rules creation

If you’re responsible for creating a Web Application Firewall (WAF) rule, you’ll almost certainly need to reference a large list of potential values that each field can have. And having to manually manage and enter all those fields, for numerous WAF rules, would be a guaranteed headache.

That’s why we introduced IP lists. Having a separate list of values that can be referenced, reused, and managed independently of the actual rule makes for a better WAF user experience. You can create a new list, such as $organization_ips, and then use it in a rule like “allow requests where source IP is in $organization_ips”. If you need to add or remove IPs, you do that in the list, without touching each of the rules that reference the list. You can even add a descriptive name to help track its content. It’s easy, clean, and organized.

Which led us, and our customers, to ask the next natural question: why stop at IPs?

Cloudflare’s WAF is highly configurable and allows you to write rules evaluating a set of hostnames, Autonomous System Numbers (ASNs), countries, header values, or values of JSON fields. But to do so, you’ve to input a list of Continue reading

Streaming and longer context lengths for LLMs on Workers AI

Streaming LLMs and longer context lengths available in Workers AI

Workers AI is our serverless GPU-powered inference platform running on top of Cloudflare’s global network. It provides a growing catalog of off-the-shelf models that run seamlessly with Workers and enable developers to build powerful and scalable AI applications in minutes. We’ve already seen developers doing amazing things with Workers AI, and we can’t wait to see what they do as we continue to expand the platform. To that end, today we’re excited to announce some of our most-requested new features: streaming responses for all Large Language Models (LLMs) on Workers AI, larger context and sequence windows, and a full-precision Llama-2 model variant.

If you’ve used ChatGPT before, then you’re familiar with the benefits of response streaming, where responses flow in token by token. LLMs work internally by generating responses sequentially using a process of repeated inference — the full output of a LLM model is essentially a sequence of hundreds or thousands of individual prediction tasks. For this reason, while it only takes a few milliseconds to generate a single token, generating the full response takes longer, on the order of seconds. The good news is we can start displaying the response as soon as the first tokens are generated, Continue reading

Post Mortem on Cloudflare Control Plane and Analytics Outage

Beginning on Thursday, November 2, 2023 at 11:43 UTC Cloudflare's control plane and analytics services experienced an outage. The control plane of Cloudflare consists primarily of the customer-facing interface for all of our services including our website and APIs. Our analytics services include logging and analytics reporting.

The incident lasted from November 2 at 11:44 UTC until November 4 at 04:25 UTC. We were able to restore most of our control plane at our disaster recovery facility as of November 2 at 17:57 UTC. Many customers would not have experienced issues with most of our products after the disaster recovery facility came online. However, other services took longer to restore and customers that used them may have seen issues until we fully resolved the incident. Our raw log services were unavailable for most customers for the duration of the incident.

Services have now been restored for all customers. Throughout the incident, Cloudflare's network and security services continued to work as expected. While there were periods where customers were unable to make changes to those services, traffic through our network was not impacted.

This post outlines the events that caused this incident, the architecture we had in place to prevent issues Continue reading

Cloudflare incident on October 30, 2023

Multiple Cloudflare services were unavailable for 37 minutes on October 30, 2023. This was due to the misconfiguration of a deployment tool used by Workers KV. This was a frustrating incident, made more difficult by Cloudflare’s reliance on our own suite of products. We are deeply sorry for the impact it had on customers. What follows is a discussion of what went wrong, how the incident was resolved, and the work we are undertaking to ensure it does not happen again.

Workers KV is our globally distributed key-value store. It is used by both customers and Cloudflare teams alike to manage configuration data, routing lookups, static asset bundles, authentication tokens, and other data that needs low-latency access.

During this incident, KV returned what it believed was a valid HTTP 401 (Unauthorized) status code instead of the requested key-value pair(s) due to a bug in a new deployment tool used by KV.

These errors manifested differently for each product depending on how KV is used by each service, with their impact detailed below.

What was impacted

A number of Cloudflare services depend on Workers KV for distributing configuration, routing information, static asset serving, and authentication state globally. These services instead received Continue reading

Introducing notifications for HTTP Traffic Anomalies

Introducing notifications for HTTP Traffic Anomalies

When it comes to managing Internet properties, the difference between a small technical hiccup and major incident is often a matter of speed. Proactive alerting plays a crucial role, which is why we were excited when we released HTTP Error Rate notifications — giving administrators visibility into when end users are experiencing errors.

But what if there are issues that don't show up as errors, like a sudden drop in traffic, or a spike?

Today, we're excited to announce Traffic Anomalies notifications, available to enterprise customers. These notifications trigger when Cloudflare detects unexpected changes in traffic, giving another valuable perspective into the health of your systems.

Unexpected changes in traffic could be indicative of many things. If you run an ecommerce site and see a spike in traffic that could be great news — maybe customers are flocking to your sale, or you just had an ad run on a popular TV show. However, it could also mean that something is going wrong: maybe someone accidentally turned off a firewall rule, and now you’re seeing more malicious traffic. Either way, you might want to know that something has changed.

Similarly, a sudden drop in traffic could mean many things. Perhaps Continue reading

Introducing HAR Sanitizer: secure HAR sharing

Introducing HAR Sanitizer: secure HAR sharing

On Wednesday, October 18th, 2023, Cloudflare’s Security Incident Response Team (SIRT) discovered an attack on our systems that originated from an authentication token stolen from one of Okta’s support systems. No Cloudflare customer information or systems were impacted by the incident, thanks to the real-time detection and rapid action of our Security Incident Response Team (SIRT) in tandem with our Zero Trust security posture and use of hardware keys. With that said, we’d rather not repeat the experience — and so we have built a new security tool that can help organizations render this type of attack obsolete for good.

The bad actor in the Okta breach compromised user sessions by capturing session tokens from administrators at Cloudflare and other impacted organizations. They did this by infiltrating Okta’s customer support system and stealing one of the most common mechanisms for troubleshooting — an HTTP Response Archive (HAR) file.

HAR files contain a record of a user’s browser session, a kind of step-by-step audit, that a user can share with someone like a help desk agent to diagnose an issue. However, the file can also contain sensitive information that can be used to launch an attack.

As a follow-up to the Continue reading

Email Routing subdomain support, new APIs and security protocols

Email Routing subdomain support, new APIs and security protocols

It's been two years since we announced Email Routing, our solution to create custom email addresses for your domains and route incoming emails to your preferred mailbox. Since then, the team has worked hard to evolve the product and add more powerful features to meet our users' expectations. Examples include Route to Workers, which allows you to process your Emails programmatically using Workers scripts, Public APIs, Audit Logs, or DMARC Management.

We also made significant progress in supporting more email security extensions and protocols, protecting our customers from unwanted traffic, and keeping our IP space reputation for email egress impeccable to maximize our deliverability rates to whatever inbox upstream provider you chose.

Since leaving beta, Email Routing has grown into one of our most popular products; it’s used by more than one million different customer zones globally, and we forward around 20 million messages daily to every major email platform out there. Our product is mature, robust enough for general usage, and suitable for any production environment. And it keeps evolving: today, we announce three new features that will help make Email Routing more secure, flexible, and powerful than ever.

New security protocols

The SMTP email protocol Continue reading

DDoS threat report for 2023 Q3

DDoS threat report for 2023 Q3

Welcome to the third DDoS threat report of 2023. DDoS attacks, or distributed denial-of-service attacks, are a type of cyber attack that aims to disrupt websites (and other types of Internet properties) to make them unavailable for legitimate users by overwhelming them with more traffic than they can handle — similar to a driver stuck in a traffic jam on the way to the grocery store.

We see a lot of DDoS attacks of all types and sizes, and our network is one of the largest in the world spanning more than 300 cities in over 100 countries. Through this network we serve over 64 million HTTP requests per second at peak and about 2.3 billion DNS queries every day. On average, we mitigate 140 billion cyber threats each day. This colossal amount of data gives us a unique vantage point to understand the threat landscape and provide the community access to insightful and actionable DDoS trends.

In recent weeks, we've also observed a surge in DDoS attacks and other cyber attacks against Israeli newspaper and media websites, as well as financial institutions and government websites. Palestinian websites have also seen a significant increase in DDoS attacks. View Continue reading

Q3 2023 Internet disruption summary

This post is also available in 简体中文, 繁體中文, 한국어, Deutsch, Français and Español.

Q3 2023 Internet disruption summary

Cloudflare operates in more than 300 cities in over 100 countries, where we interconnect with over 12,500 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions.

We have been publishing these summaries since the first quarter of 2022, and over that time, the charts on Cloudflare Radar have evolved. Many of the traffic graphs in early editions of this summary were screenshots from the relevant traffic pages on Radar. Late last year, we launched the ability to download graphs, and earlier this year, to embed dynamic graphs, and these summaries have taken advantage of those capabilities where possible. Sharp-eyed readers may notice an additional evolution in some of the graphs below: yellow highlighting indicating an observed “traffic anomaly”. Identification of such anomalies, along with the ability to be notified about them, as well as a timeline enhancement (embedded below) to the Cloudflare Radar Outage Center, were launched as Continue reading

Cache Reserve goes GA: enhanced control to minimize egress costs

Cache Reserve goes GA: enhanced control to minimize egress costs

Everyone is chasing the highest cache ratio possible. Serving more content from Cloudflare’s cache means it loads faster for visitors, saves website operators money on egress fees from origins, and provides multiple layers of resiliency and protection to make sure that content is available to be served and websites scale effortlessly. A year ago we introduced Cache Reserve to help customer’s serve as much content as possible from Cloudflare’s cache.

Today, we are thrilled to announce the graduation of Cache Reserve from beta to General Availability (GA), accompanied by the introduction of several exciting new features. These new features include adding Cache Reserve into the analytics shown on the Cache overview section of the Cloudflare dashboard, giving customers the ability to see how they are using Cache Reserve over time. We have also added the ability for customers to delete all data in Cache Reserve without losing content in the edge cache. This is useful for customers who are no longer using Cache Reserve storage.

We’re also introducing new tools that give organizations more granular control over which files are saved to Cache Reserve, based on valuable feedback we received during the beta. The default configuration of Cache Reserve Continue reading

Cache Rules go GA: precision control over every part of your cache

Cache Rules go GA: precision control over every part of your cache

One year ago we introduced Cache Rules, a new way to customize cache settings on Cloudflare. Cache Rules provide greater flexibility for how users cache content, offering precise controls, a user-friendly API, and seamless Terraform integrations. Since it was released in late September 2022, over 100,000 websites have used Cache Rules to fine-tune their cache settings.

Today, we're thrilled to announce that Cache Rules, along with several other Rules products, are generally available (GA). But that’s not all — we're also introducing new configuration options for Cache Rules that provide even more options to customize how you cache on Cloudflare. These include functionality to define what resources are eligible for Cache Reserve, what timeout values should be respected when receiving data from your origin server, which custom ports we should use when we cache content, and whether we should bypass Cloudflare’s cache in the absence of a cache-control header.

Cache Rules give users full control and the ability to tailor their content delivery strategy for almost any use case, without needing to write code. As Cache Rules go GA, we are incredibly excited to see how fast customers can achieve their perfect cache strategy.

History of Customizing Cache Continue reading

Cyber attacks in the Israel-Hamas war

Cyber attacks in the Israel-Hamas war

On October 7, 2023, at 03:30 GMT (06:30 AM local time), Hamas attacked Israeli cities and fired thousands of rockets toward populous locations in southern and central Israel, including Tel Aviv and Jerusalem. Air raid sirens began sounding, instructing civilians to take cover.

Approximately twelve minutes later, Cloudflare systems automatically detected and mitigated DDoS attacks that targeted websites that provide critical information and alerts to civilians on rocket attacks. The initial attack peaked at 100k requests per second (rps) and lasted ten minutes. Forty-five minutes later, a second much larger attack struck and peaked at 1M rps. It lasted six minutes. Additional smaller DDoS attacks continued hitting the websites in the next hours.

DDoS attacks against Israeli websites that provide civilians information and alerts on rocket attacks
DDoS attacks against Israeli websites that provide civilians information and alerts on rocket attacks

Not just DDoS attacks

Multiple Israeli websites and mobile apps have become targets of various pro-Palestinian hacktivist groups. According to Cybernews, one of those groups, AnonGhost, exploited a vulnerability in a mobile app that alerts Israeli civilians of incoming rockets, “Red Alert: Israel”. The exploit allowed them to intercept requests, expose servers and APIs, and send fake alerts to some app users, including a message that a “nuclear bomb is coming Continue reading

How Cloudflare mitigated yet another Okta compromise

On Wednesday, October 18, 2023, we discovered attacks on our system that we were able to trace back to Okta – threat actors were able to leverage an authentication token compromised at Okta to pivot into Cloudflare’s Okta instance. While this was a troubling security incident, our Security Incident Response Team’s (SIRT) real-time detection and prompt response enabled containment and minimized the impact to Cloudflare systems and data. We have verified that no Cloudflare customer information or systems were impacted by this event because of our rapid response. Okta has now released a public statement about this incident.

This is the second time Cloudflare has been impacted by a breach of Okta’s systems. In March 2022, we blogged about our investigation on how a breach of Okta affected Cloudflare. In that incident, we concluded that there was no access from the threat actor to any of our systems or data – Cloudflare’s use of hard keys for multi-factor authentication stopped this attack.  

The key to mitigating this week’s incident was our team’s early detection and immediate response. In fact, we contacted Okta about the breach of their systems before they had notified us. The attacker used an open Continue reading

Empowering our partners with the new Tenant Platform dashboard

Itching to get started? Apply to the Self Serve Partner Beta or Enterprise partner programs now.

Empowering our partners with the new Tenant Platform dashboard

Cloudflare has always worked closely with partners to help build a better Internet. From our earliest Hosting Partners, to our latest Cloudflare One program and Authorized Service Delivery partners, we are dedicated to supporting our peers across the networking and cybersecurity ecosystem to secure Enterprise networks, mission-critical applications, and remote employees. As part of that commitment, we are proud to announce the general availability of our first dashboard for our Tenant Platform, providing an intuitive user interface for agencies and partners to manage their client accounts.

Tenant Platform introduction

The first version of the Tenant Platform was created in 2018 to support one of our large integration partners, IBM Cloud. They needed a secure way to independently provision accounts for their clients, spin up custom subscriptions, invite service users within each new account, and begin to configure the service. This platform, although API only, worked extremely well with our OEM and integration partners that were including our solution within their current platform to support their customers.

Multi-Tenant Structure

User interface overview

As Cloudflare has expanded the type of partners and customers it works Continue reading