Archive

Category Archives for "CloudFlare"

Getting started with Terraform and Cloudflare (Part 1 of 2)

Getting started with Terraform and Cloudflare (Part 1 of 2)

As a Product Manager at Cloudflare, I spend quite a bit of my time talking to customers. One of the most common topics I'm asked about is configuration management. Developers want to know how they can write code to manage their Cloudflare config, without interacting with our APIs or UI directly.

Following best practices in software development, they want to store configuration in their own source code repository (be it GitHub or otherwise), institute a change management process that includes code review, and be able to track their configuration versions and history over time. Additionally, they want the ability to quickly and easily roll back changes when required.

When I first spoke with our engineering teams about these requirements, they gave me the best answer a Product Manager could hope to hear: there's already an open source tool out there that does all of that (and more), with a strong community and plugin system to boot—it's called Terraform.

This blog post is about getting started using Terraform with Cloudflare and the new version 1.0 of our Terraform provider. A "provider" is simply a plugin that knows how to talk to a specific set of APIs—in this case, Cloudflare, but Continue reading

Copenhagen & London developers, join us for five events this May

Copenhagen & London developers, join us for five events this May

Copenhagen & London developers, join us for five events this May
Photo by Nick Karvounis / Unsplash

Are you based in Copenhagen or London? Drop by one or all of these five events.

Ross Guarino and Terin Stock, both Systems Engineers at Cloudflare are traveling to Europe to lead Go and Kubernetes talks in Copenhagen. They'll then join Junade Ali and lead talks on their use of Go, Kubernetes, and Cloudflare’s Mobile SDK at Cloudflare's London office.

My Developer Relations teammates and I are visiting these cities over the next two weeks to produce these events with Ross, Terin, and Junade. We’d love to meet you and invite you along.

Our trip will begin with two meetups and a conference talk in Copenhagen.

Event #1 (Copenhagen): 6 Cloud Native Talks, 1 Evening: Special KubeCon + CloudNativeCon EU Meetup

Copenhagen & London developers, join us for five events this May

Tuesday, 1 May: 17:00-21:00

Location: Trifork Copenhagen - Borgergade 24B, 1300 København K

How to extend your Kubernetes cluster

A brief introduction to controllers, webhooks and CRDs. Ross and Terin will talk about how Cloudflare’s internal platform builds on Kubernetes.

Speakers: Ross Guarino and Terin Stock

View Event Details & Register Here »

Event #2 (Copenhagen): Gopher Meetup At Falcon.io: Building Go With Bazel & Internationalization in Go

Copenhagen & London developers, join us for five events this May

Continue reading

BGP leaks and cryptocurrencies

BGP leaks and cryptocurrencies

Over the few last hours, a dozen news stories have broken about how an attacker attempted (and perhaps managed) to steal cryptocurrencies using a BGP leak.

BGP leaks and cryptocurrencies
CC BY 2.0 image by elhombredenegro

What is BGP?

The Internet is composed of routes. For our DNS resolver 1.1.1.1 , we tell the world that all the IPs in the range 1.1.1.0 to 1.1.1.255 can be accessed at any Cloudflare PoP.

For the people who do not have a direct link to our routers, they receive the route via transit providers, who will deliver packets to those addresses as they are connected to Cloudflare and the rest of the Internet.

This is the normal way the Internet operates.

There are authorities (Regional Internet Registries, or RIRs) in charge of distributing IP addresses in order to avoid people using the same address space. Those are IANA, RIPE, ARIN, LACNIC, APNIC and AFRINIC.

What is a BGP leak?

BGP leaks and cryptocurrencies
CC BY 2.0 image by Magnus D

The broad definition of a BGP leak would be IP space that is announced by somebody not allowed by the owner of the Continue reading

Now You Can Setup Centrify, OneLogin, Ping and Other Identity Providers with Cloudflare Access

Now You Can Setup Centrify, OneLogin, Ping and Other Identity Providers with Cloudflare Access

We use Cloudflare Access to secure our own internal tools instead of a VPN. As someone that does a lot of work on the train, I can attest this is awesome (though I might be biased). You can see it in action below. Instead of having to connect to a VPN to reach our internal jira, we just login with our Google account and we are good to go:

Now You Can Setup Centrify, OneLogin, Ping and Other Identity Providers with Cloudflare Access
Before today, you could setup Access if you used GSuite, Okta or Azure AD to manage your employee accounts. Today we would like to announce support for two more Identity Providers with Cloudflare Access: Centrify and OneLogin.

We launched Cloudflare Access earlier this year and have been overwhelmed by the response from our customers and community. Customers tell us they love the simplicity of setting up Access to secure applications and integrate with their existing identity provider solution. Access helps customers implement a holistic solution for both corporate and remote employees without having to use a VPN.

If you are using Centrify or OneLogin as your identity provider you can now easily integrate them with Cloudflare Access and have your team members login with their accounts to securely reach your internal Continue reading

A Carbon Neutral North America

A Carbon Neutral North America

A Carbon Neutral North America
Photo by Karsten Würth (@inf1783) / Unsplash

Cloudflare's mission is to help build a better Internet. While working toward our goals, we want to make sure our processes are conducted in a sustainable manner.

In an effort to do so, we’ve reduced Cloudflare’s environmental impact by contracting to purchase regional renewable energy certificates, or “RECs,” to match 100% of the electricity used in our North American data centers as well as our U.S. offices. Cloudflare now has servers in 154 unique cities around the world, with 38 located in North America. Cloudflare has opted to support geographically diverse projects in proximity to our office and data center electricity usage. This renewable energy initiative reduces our electricity-based carbon footprint by 5,561 tons of CO2 which has a positive environmental impact. The impact can be compared to growing 144,132 trees seedlings for 10 years, or taking 1,191 cars off the road for one year.

A Carbon Neutral North America

How does buying a REC help reduce Cloudflare's carbon footprint you may ask? When 1MWh of electricity is produced from a renewable generator, such as a wind turbine, there are two products: the energy, which is delivered to the grid and mixes with other forms of energy, Continue reading

Keeping Drupal sites safe with Cloudflare’s WAF

Keeping Drupal sites safe with Cloudflare's WAF

Cloudflare’s team of security analysts monitor for upcoming threats and vulnerabilities and where possible put protection in place for upcoming threats before they compromise our customers. This post examines how we protected people against a new major vulnerability in the Drupal CMS, nicknamed Drupalgeddon 2.

Two weeks after adding protection with WAF rule ID D0003 which mitigates the critical remote code execution Drupal exploit (SA-CORE-2018-002/CVE-2018-7600), we have seen significant spikes of attack attempts. Since the 13th of April the Drupal security team has been aware of automated attack attempts and it significantly increased the security risk score of the vulnerability. It makes sense to go back and analyse what happened in the last seven days in Cloudflare’s WAF environment.

What is Drupalgeddon 2

The vulnerability potentially allows attackers to exploit multiple attack vectors on a Drupal site, which could make a site completely compromised.

Drupal introduced renderable arrays, which are a key-value structure, with keys starting with a ‘#’ symbol, that allows you to alter data during form rendering. These arrays however, did not have enough input validation. This means that an attacker could inject a custom renderable array on one of these keys in the form structure.

Continue reading

1 year and 3 months working at Cloudflare: How is it going so far?

1 year and 3 months working at Cloudflare: How is it going so far?

This post is inspired by a very good blog post from one of my colleague in the US, which I really appreciated as I was a newcomer to the company. It was great to see what it is like working for Cloudflare after one year and to learn from the lessons she had learnt.

I'll try to do the same in three parts. Beginning with how my on-boarding went, my first customer experiences and finally what is my day-to-day life at Cloudflare. These writings only reflect my personal feelings and thoughts. The experience is different for each and every newcomer to Cloudflare.

Chapter 1 - On-boarding, being impressed and filling the (big) knowledge gaps.


Before I joined Cloudflare, I was working as a Security Consultant in Paris, France. I never had the opportunity to move abroad to speak English (me.englishLevel = 0), I never had any reason to live outside of France and was at the same time looking for another Job. Perfect then!

When I saw the job posting, I immediately applied as I knew the company well, the mindset and the products Cloudflare provided. It took me 6 months to get the offer probably because Continue reading

mmproxy – Creative Linux routing to preserve client IP addresses in L7 proxies

mmproxy - Creative Linux routing to preserve client IP addresses in L7 proxies

In previous blog post we discussed how we use the TPROXY iptables module to power Cloudflare Spectrum. With TPROXY we solved a major technical issue on the server side, and we thought we might find another use for it on the client side of our product.

mmproxy - Creative Linux routing to preserve client IP addresses in L7 proxies
This is Addressograph. Source Wikipedia

When building an application level proxy, the first consideration is always about retaining real client source IP addresses. Some protocols make it easy, e.g. HTTP has a defined X-Forwarded-For header[1], but there isn't a similar thing for generic TCP tunnels.

Others have faced this problem before us, and have devised three general solutions:

(1) Ignore the client IP

mmproxy - Creative Linux routing to preserve client IP addresses in L7 proxies

For certain applications it may be okay to ignore the real client IP address. For example, sometimes the client needs to identify itself with a username and password anyway, so the source IP doesn't really matter. In general, it's not a good practice because...

(2) Nonstandard TCP header

A second method was developed by Akamai: the client IP is saved inside a custom option in the TCP header in the SYN packet. Early implementations of this method weren't conforming to any standards, e.g. using option field 28 Continue reading

NEON is the new black: fast JPEG optimization on ARM server

NEON is the new black: fast JPEG optimization on ARM server

As engineers at Cloudflare quickly adapt our software stack to run on ARM, a few parts of our software stack have not been performing as well on ARM processors as they currently do on our Xeon® Silver 4116 CPUs. For the most part this is a matter of Intel specific optimizations some of which utilize SIMD or other special instructions.

One such example is the venerable jpegtran, one of the workhorses behind our Polish image optimization service.

A while ago I optimized our version of jpegtran for Intel processors. So when I ran a comparison on my test image, I was expecting that the Xeon would outperform ARM:

vlad@xeon:~$ time  ./jpegtran -outfile /dev/null -progressive -optimise -copy none test.jpg

real    0m2.305s
user    0m2.059s
sys     0m0.252s
vlad@arm:~$ time ./jpegtran -outfile /dev/null -progressive -optimise -copy none test.jpg

real    0m8.654s
user    0m8.433s
sys     0m0.225s

Ideally we want to have the ARM performing at or above 50% of the Xeon performance per core. This would make sure we have no performance regressions, and net performance gain, since the ARM CPUs have double the core count as our current 2 socket setup.

In this case, however, I Continue reading

Cloudflare launches 1.1.1.1 DNS service with privacy, TLS and more

There was an important development this month with the launch of Cloudflare’s new 1.1.1.1 DNS resolver service. This is a significant development for several reasons, but in particular it supports the new DNS-over-TLS and DNS-over-HTTPS protocols that allow for confidential DNS querying and response.

Why 1.1.1.1?

Before we get to that though, Cloudflare joins Google’s Public DNS that uses 8.8.8.8 and Quad9 DNS that uses 9.9.9.9, by implementing 1.1.1.1 as a memorable IP address for accessing its new DNS service. IP addresses are generally not as memorable as domain names, but you need access to a DNS server before you can resolve domain names to IP addresses, so configuring numbers is a necessity. And whilst a memorable IP address might be cool, it’s also proved important recently when DNS resolvers have been blocked or taken down, requiring devices to be pointed elsewhere.

The 1.1.1.1 address is part of the 1.1.1.0 – 1.1.1.255 public IP address range actually allocated to APNIC, one of the five Regional Internet Registries, but it has been randomly used as an address for Continue reading

Introducing Spectrum: Extending Cloudflare To 65,533 More Ports

Introducing Spectrum: Extending Cloudflare To 65,533 More Ports

Today we are introducing Spectrum, which brings Cloudflare’s security and acceleration to the whole spectrum of TCP ports and protocols for our Enterprise customers. It’s DDoS protection for any box, container or VM that connects to the internet; whether it runs email, file transfer or a custom protocol, it can now get the full benefits of Cloudflare. If you want to skip ahead and see it in action, you can scroll to the video demo at the bottom.

Introducing Spectrum: Extending Cloudflare To 65,533 More Ports

DDoS Protection

The core functionality of Spectrum is its ability to block large DDoS attacks. Spectrum benefits from Cloudflare’s existing DDoS mitigation (which this week blocked a 900 Gbps flood). Spectrum’s DDoS protection has already been battle tested. Just soon as we opened up Spectrum for beta, Spectrum received its first SYN flood.

One of Spectrum's earliest deployments was in front of Hypixel’s infrastructure. Hypixel runs the largest minecraft server, and because gamers can be - uh, passionate - they were one of the earliest targets of the terabit-per-second Mirai botnet. “Hypixel was one of the first subjects of the Mirai botnet DDoS attacks and frequently receives large attacks. Before Spectrum, we had to rely on unstable services & techniques Continue reading

Abusing Linux’s firewall: the hack that allowed us to build Spectrum

Abusing Linux's firewall: the hack that allowed us to build Spectrum

Today we are introducing Spectrum: a new Cloudflare feature that brings DDoS protection, load balancing, and content acceleration to any TCP-based protocol.

Abusing Linux's firewall: the hack that allowed us to build Spectrum
CC BY-SA 2.0 image by Staffan Vilcans

Soon after we started building Spectrum, we hit a major technical obstacle: Spectrum requires us to accept connections on any valid TCP port, from 1 to 65535. On our Linux edge servers it's impossible to "accept inbound connections on any port number". This is not a Linux-specific limitation: it's a characteristic of the BSD sockets API, the basis for network applications on most operating systems. Under the hood there are two overlapping problems that we needed to solve in order to deliver Spectrum:

  • how to accept TCP connections on all port numbers from 1 to 65535
  • how to configure a single Linux server to accept connections on a very large number of IP addresses (we have many thousands of IP addresses in our anycast ranges)

Assigning millions of IPs to a server

Cloudflare’s edge servers have an almost identical configuration. In our early days, we used to assign specific /32 (and /128) IP addresses to the loopback network interface[1]. This worked well when we had dozens of IP Continue reading

리눅스 방화벽을 남용하기: Spectrum 을 만들 수 있었던 ​해킹​

리눅스 방화벽을 남용하기: Spectrum 을 만들 수 있었던 ​해킹​

This is a Korean translation of a prior post by Marek Majkowski.


얼마전 우리는 Spectrum을 발표하였습니다: 어떤 TCP 기반의 프로토콜이라도 DDoS 방어, 로드밸런싱 그리고 컨텐츠 가속을 할 수 있는 새로운 Cloudflare의 기능입니다.

리눅스 방화벽을 남용하기: Spectrum 을 만들 수 있었던 ​해킹​
CC BY-SA 2.0 image by Staffan Vilcans

Spectrum을 만들기 시작하고 얼마 되지 않아서 중요한 기술적 난관에 부딛히게 되었습니다: Spectrum은 1부터 65535 사이의 어떤 유효한 TCP 포트라도 접속을 허용해야 합니다. 우리의 리눅스 엣지 서버에서는 "임의의 포트 번호에 인바운드 연결을 허용"은 불가능합니다. 이것은 리눅스만의 제한은 아닙니다: 이것은 대부분 운영 체제의 네트워크 어플리케이션의 기반인 BSD 소켓 API의 특성입니다. 내부적으로 Spectrum을 완성하기 위해서 풀어야 하는 서로 겹치는 문제가 둘 있었습니다:

  • 1에서 65535 사이의 모든 포트 번호에 TCP 연결을 어떻게 받아들일 것인가
  • 매우 많은 수의 IP 주소로 오는 연결을 받아들이도록 단일 리눅스 서버를 어떻게 설정할 것인가 (우리는 애니캐스트 대역에 수많은 IP주소를 갖고 있습니다)

서버에 수백만의 IP를 할당

Cloudflare의 엣지 서버는 거의 동일한 구성을 갖고 있습니다. 초창기에는 루프백 네트워크 인터페이스에 특정한 /32 (그리고 /128) IP 주소를 할당하였습니다[1]. 이것은 수십개의 IP주소만 갖고 있었을 때에는 잘 동작 하였지만 더 성장함에 따라 확대 적용하는 것에는 실패하였습니다.

그때 "AnyIP" 트릭이 등장하였습니다. AnyIP는 단일 주소가 아니라 전체 IP 프리픽스 (서브넷)을 루프백 인터페이스에 할당하도록 해 줍니다. 사실 AnyIP를 많이 사용하고 있습니다: 여러분 컴퓨터에는 루브백 인터페이스에 Continue reading

Fixing reachability to 1.1.1.1, GLOBALLY!

Fixing reachability to 1.1.1.1, GLOBALLY!

Recently we announced our fast, privacy-centric DNS resolver 1.1.1.1, supported by our global network. As you can see 1.1.1.1 is very easy to remember, which is both a blessing and a curse. In the time leading up to the announcement of the resolver service we began testing reachability to 1.1.1.1, primarily using the RIPE Atlas probes. The RIPE Atlas project is an extensive collection of small monitoring devices hosted by the public around the world. Currently there are over 10,000 active probes hosted in over 3,000 networks, giving great vantage points for testing. We found large numbers of probes unable to query 1.1.1.1, but successfully able to query 1.0.0.1 in almost all cases. 1.0.0.1 is the secondary address we have assigned for the resolver, to allow clients who are unable to reach 1.1.1.1 to be able to make DNS queries.

This blog focuses on IPv4. We provide four IPs (two for each IP address family) in order to provide a path toward the DNS resolver independent of IPv4 or IPv6 reachability.

1.0.0.0/8 was assigned in Continue reading

Privacy-Protecting Portable Router: Adding DNS-Over-TLS support to OpenWRT (LEDE) with Unbound

Privacy-Protecting Portable Router: Adding DNS-Over-TLS support to OpenWRT (LEDE) with Unbound

If you want to skip ahead to instructions, scroll to the next section. But I, like a TLS handshake, am very verbose so please enjoy this opener.

Imagine this scenario - I'm at a restaurant and need to have a private phone conversation but unfortunately my phone's battery is drained. To get around this problem, I borrow my friend's phone and dial the number - to protect my privacy I walk outside. When I'm done with the call, I come back inside and return the phone.

Whilst the phone itself doesn't store the coversation I've had, it does have a log of the recently dialed number, if the friend from whom I borrowed the phone wanted to, they could easily see who I actually called - even if they don't specifically know the topic of conversation.

Sometimes, the data about who you've spoken to can tell an aweful lot about the conversation - if someone was to call an emotional support hotline or a debt collector, you could probably infer a lot about the conversation from the caller ID.

When we browse the internet, we use encryption to try and protect the conversations we have. When you connect to a Continue reading

Cloudflare Argo Tunnel with Rust+Raspberry Pi

Cloudflare Argo Tunnel with Rust+Raspberry Pi

Yesterday Cloudflare launched Argo Tunnel. In the words of the product team:

Argo Tunnel exposes applications running on your local web server, on any network with an Internet connection, without adding DNS records or configuring a firewall or router. It just works.

Once I grokked this, the first thing that came to mind was that I could actually use one of my Raspberry Pi's sitting around to serve a website, without:

  • A flaky DDNS running on my router
  • Exposing my home network to the world
  • A cloud VM

Ooooh... so exciting.

The Rig

I'll assume you already have a Raspberry Pi with Raspbian on it.

Cloudflare Argo Tunnel with Rust+Raspberry Pi

Plug the Pi into your router. It should now have an IP address. Look that up in your router’s admin UI:

Cloudflare Argo Tunnel with Rust+Raspberry Pi

OK, that's promising. Let's connect to that IP using the default pi/raspberry credentials:

$ ssh 192.168.8.26 -l pi
[email protected]'s password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sun Mar 18 23:24:11 2018 from  Continue reading

Reclaiming 1.1.1.1 For The Internet

Hopefully by now you’ve seen the announcement that CloudFlare has opened a new DNS service at the address of 1.1.1.1. We covered a bit of it on this week’s episode of the Gestalt IT Rundown. Next to Gmail, it’s probably the best April Fool’s announcement I’ve seen. However, it would seem that the Internet isn’t quite ready for a DNS resolver service that’s easy to remember. And that’s thanks in part to the accumulation of bad address hygiene.

Not So Random Numbers

The address range of 1/8 is owned by APNIC. They’ve had it for many years now but have never announced it publicly. Nor have they ever made any assignments of addresses in that space to clients or customers. In a world where IPv4 space is at a premium, why would a RIR choose to lose 16 million addresses?

As it turns out, 1/8 is a pretty bad address space for two reasons. 1.1.1.1 and 1.2.3.4. These two addresses are responsible for most of the inadvertent announcements in the entire 1/8 space. 1.2.3.4 is easy to figure out. It’s the most common example IP address Continue reading

Argo Tunnel: A Private Link to the Public Internet

Argo Tunnel: A Private Link to the Public Internet

Argo Tunnel: A Private Link to the Public Internet
Photo from Wikimedia Commons

Today we’re introducing Argo Tunnel, a private connection between your web server and Cloudflare. Tunnel makes it so that only traffic that routes through Cloudflare can reach your server.

You can think of Argo Tunnel as a virtual P.O. box. It lets someone send you packets without knowing your real address. In other words, it’s a private link. Only Cloudflare can see the server and communicate with it, and for the rest of the internet, it’s unroutable, as if the server is not even there.

How this used to be done

This type of private deployment used to be accomplished with GRE tunnels. But GRE tunnels are expensive and slow, they don’t really make sense in a 2018 internet.

GRE is a tunneling protocol for sending data between two servers by simulating a physical link. Configuring a GRE tunnel requires coordination between network administrators from both sides of the connection. It is an expensive service that is usually only available for large corporations with dedicated budgets. The GRE protocol encapsulates packets inside other packets, which means that you will have to either lower the MTU of your origin servers, or have your router do Continue reading

Extend your security view from the data center to the edge

Extend your security view from the data center to the edge

How great would it be to have a dashboard with a holistic view of threats, malicious server activity, vulnerabilities, sensitive data access levels and a daily scan of resources across all of your applications and services? Now you can.

Cloudflare is thrilled to announce its integration with Cloud Security Command Center (Cloud SCC) for Google Cloud Platform: A security and data risk platform helping enterprises gather data, identify threats, and act on them before they result in business damage or loss.

The advantage of the Cloud SCC solution is that it surfaces insights from both the Google Cloud Platform, as well as Cloudflare’s edge, in a unified dashboard.

What Cloudflare data is visible within the Cloud SCC dashboard?

Through Cloudflare’s API endpoints, data is pushed to Google’s Cloud SCC dashboard and domain name information mapped to the appropriate Google Cloud asset. Cloudflare’s branded card in the Cloud SCC dashboard is automatically populated with a summary of top theat origins, top types of threats, and latest Web Application Firewall (WAF) events.

Extend your security view from the data center to the edge

To view a full list of Cloudflare events, click on the Cloudflare card in Cloud SCC and it will take you to a “Cloudflare Findings” page. From there, you can Continue reading

Announcing 1.1.1.1: the fastest, privacy-first consumer DNS service

Announcing 1.1.1.1: the fastest, privacy-first consumer DNS service

Cloudflare's mission is to help build a better Internet. We're excited today to take another step toward that mission with the launch of 1.1.1.1 — the Internet's fastest, privacy-first consumer DNS service. This post will talk a little about what that is and a lot about why we decided to do it. (If you're interested in the technical details on how we built the service, check out Ólafur Guðmundsson's accompanying post.)

Quick Primer On DNS

DNS is the directory of the Internet. Whenever you click on a link, send an email, open a mobile app, often one of the first things that has to happen is your device needs to look up the address of a domain. There are two sides of the DNS network: Authoritative (the content side) and Resolver (the consumer side).

Every domain needs to have an Authoritative DNS provider. Cloudflare, since our launch in September 2010, has run an extremely fast and widely-used Authoritative DNS service. 1.1.1.1 doesn't (directly) change anything about Cloudflare's Authoritative DNS service.

On the other side of the DNS system are resolvers. Every device that connects to the Internet needs a DNS resolver. By default, Continue reading