How to Build a Highly Productive Remote Team (or Team of Contractors) with Cloudflare for Teams
Much of IT has been built on two outdated assumptions about how work is done. First, that employees all sit in the same building or branch offices. Second, that those employees will work full-time at the same company for years.
Both of these assumptions are no longer true.
Employees now work from anywhere. In the course of writing this blog post, I opened review tickets in our internal JIRA from my dining table at home. I reviewed internal wiki pages on my phone during my commute on the train. And I spent time reviewing some marketing materials in staging in our CMS.
In a past job, I would have suffered trying to connect to these tools through a VPN. That would have slowed down my work on a laptop and made it nearly impossible to use a phone to catch up on my commute.
The second challenge is ramp-up. I joined Cloudflare a few months ago. As a member of the marketing team, I work closely with our product organization and there are several dozen tools that I need to do that.
I’m hardly alone. The rise of SaaS and custom internal applications means that employees need access to all Continue reading
Cloudflare for Teams Free for Small Businesses During Coronavirus Emergency
There are a lot of people and businesses worldwide that are currently suffering, so I don't want to waste any time in getting to the point.
Beginning today, we are making our Cloudflare for Teams products free to small businesses around the world. Teams enables remote workers to operate securely and easily. We will continue this policy for at least the next 6 months. We're doing this to help ensure that small businesses that implement work from home policies in order to combat the spread of the virus can ensure business continuity. You can learn more and apply at: https://www.cloudflare.com/smallbusiness
We've also helped launch an online hub where small businesses can see technology services available to them for free or a substantial discount from multiple companies, during the Coronavirus Emergency: https://openforbusiness.org
To understand more about why we're doing this, read on.
The IT Strain of WFH
We have a team at Cloudflare carefully monitoring the spread of the SARS-Coronavirus-2, which is responsible for the COVID-19 respiratory disease. Like at many other companies, we have heeded the advice of medical professionals and government agencies and are increasingly allowing employees to work from home in impacted regions in order Continue reading
Why the 100th Anniversary of Women’s Right to Vote in the U.S. is Important to Celebrate on International Women’s Day
Seven months ago, I joined Cloudflare to work on the Public Policy Team focusing on our democracy projects such as Project Galileo, Athenian Project and Cloudflare for Campaigns. Since I joined the team, I have learned a lot about how important cybersecurity protections are for organizations that are the target of sophisticated cyberattacks, while also learning about the complex election security environment in the United States and abroad.
It seems fitting that on International Women’s Day, a day people throughout the world are celebrating the achievements of women, we also celebrate the Centennial Anniversary of the Women’s Suffrage Movement which was the tipping point that gave many women voting rights in the United States.
Since I have been working on Cloudflare’s election security projects, this day means something extra special to me and many of my colleagues who believe that voting is the cornerstone of democracy and that having access to information regarding voting and elections is essential.
Here are five reflections that I want to share on International Women’s Day and the Centennial Anniversary of the Nineteenth Amendment which granted women the right to vote in the United States:
1. The Women’s Suffrage Movement in the United States was Continue reading
How Cloudflare keeps employees productive from any location
Cloudflare employs more than 1,200 people in 13 different offices and maintains a network that operates in 200 cities. To do that, we used to suffer through a traditional corporate VPN that backhauled traffic through a physical VPN appliance. It was, frankly, horrible to work with as a user or IT person.
With today’s mix of on-prem, public cloud and SaaS and a workforce that needs to work from anywhere, be it a coffee shop or home, that model is no longer sustainable. As we grew in headcount, we were spending too much time resolving VPN helpdesk tickets. As offices around the world opened, we could not ask our workforce to sit as every connection had to go back through a central location.
We also had to be ready to scale. Some organizations are currently scrambling to load test their own VPN in the event that their entire workforce needs to work remotely during the COVID-19 outbreak. We could not let a single physical appliance constrain our ability to deliver 26M Internet properties to audiences around the world.
To run a network like Cloudflare, we needed to use Cloudflare’s network to stay fast and secure.
We built Cloudflare Access, part Continue reading
International Women’s Day 2020: Building a Modern Security Team
When we started at Cloudflare in the summer of 2018, we joined a small security team intent on helping it grow quickly. Cloudflare was already a successful “unicorn” startup and its profile was changing fast, providing cyber security protection for millions of Internet-facing properties and moving towards becoming a public company. We were excited to help build the team that would ensure the security of Cloudflare’s systems and the sensitive customer data that flows through them.
Competing for security talent in the tech industry - where every company is investing heavily on security - isn't easy. But, in 18 months, we have grown our team 400% from under 10 people to almost 50 (and still hiring). We are proud that 40% of our team are women and 25% are from an under-represented minority. We believe from experience, and the research shows, that more diverse teams drive better business results and can be a better place to work.
In honor of International Women’s Day this Sunday, we wanted to share some of our lessons learned on how to build a diverse team and inclusive culture on a modern security team.
Lessons Learned Building a Diverse Team
- Our effort to Continue reading
The History of the URL
This issue was occuring now because the ARPANET was on the verge of switching from its original NCP protocol, to the TCP/IP protocol which powers what we now call the Internet. With that switch suddenly there would be a multitude of interconnected networks (an ‘Inter... net’) requiring a more ‘hierarchical’ domain system where ARPANET could resolve its own domains while the other networks resolved theirs.
Other networks at the time had great names like “COMSAT”, “CHAOSNET”, “UCLNET” and “INTELPOSTNET” and were maintained by groups of universities and companies all around the US who wanted to be able to communicate, and could afford to lease 56k lines from the phone company and buy the requisite PDP-11s to handle routing.
In the original ARPANET design, a central Network Information Center Continue reading
Pwned Passwords Padding (ft. Lava Lamps and Workers)
The Pwned Passwords API (part of Troy Hunt’s Have I Been Pwned service) is used tens of millions of times each day, to alert users if their credentials are breached in a variety of online services, browser extensions and applications. Using Cloudflare, the API cached around 99% of requests, making it very efficient to run.
From today, we are offering a new security advancement in the Pwned Passwords API - API clients can receive responses padded with random data. This exists to effectively protect from any potential attack vectors which seek to use passive analysis of the size of API responses to identify which anonymised bucket a user is querying. I am hugely grateful to security researcher Matt Weir who I met at PasswordsCon in Stockholm and has explored proof-of-concept analysis of unpadded API responses in Pwned Passwords and has driven some of the work to consider the addition of padded responses.
Now, by passing a header of “Add-Padding” with a value of “true”, Pwned Passwords API users are able to request padded API responses (to a minimum of 800 entries with additional padding of a further 0-200 entries). The padding consists of randomly generated hash suffixes with the usage Continue reading
Why I’m Excited to Join Cloudflare as its First CIO
I am delighted to share that I have joined Cloudflare as its first Chief Information Officer to help scale the company in this new phase of its business. It’s an incredibly exciting time to be joining Cloudflare, and I am grateful for the opportunity to do my part to help build a better Internet.
At one of my previous companies, I made a bet on Cloudflare to equip us with security and performance solutions across a very decentralized global set of products and services. This is something that would have been very difficult without a cloud solution like Cloudflare’s. Since then I’ve been watching Cloudflare grow, and have always been very impressed by the speed of innovation and transparency, but also how Cloudflare operates: doing the right thing, with integrity, and above all building trust with customers and partners. The “do the right thing, even if it’s hard” mentality that I saw from Cloudflare since I started doing business with them as a customer, was key for me. When I heard that Cloudflare was looking for its first CIO I was excited to have a discussion to see if I could help.
During the interview process I got a sense Continue reading
RPKI and the RTR protocol
Today’s Internet requires stronger protection within its core routing system and as we have already said: it's high time to stop BGP route leaks and hijacks by deploying operationally-excellent RPKI!
Luckily, over the last year plus a lot of good work has happened in this arena. If you’ve been following the growth of RPKI’s validation data, then you’ll know that more and more networks are signing their routes and creating ROA’s or Route Origin Authorizations. These are cryptographically-signed assertions of the validity of an announced IP block and contribute to the further securing of the global routing table that makes for a safer Internet.
The protocol that we have not written much about is RTR. The Resource Public Key Infrastructure (RPKI) to Router Protocol - or RTR Protocol for short. Today we’re fixing that.
RPKI rewind
We have written a few times about RPKI (here and here). We have written about how Cloudflare both signs its announced routes and filters its routing inbound from other networks (both transits and peers) using RPKI data. We also added our efforts in the open-source software space with the release of the Cloudflare RPKI Toolkit.
The primary part of the RPKI (Resource Continue reading
Addressing the Web’s Client-Side Security Challenge
Modern web architecture relies heavily on JavaScript and enabling third-party code to make client-side network requests. These innovations are built on client-heavy frameworks such as Angular, Ember, React, and Backbone that leverage the processing power of the browser to enable the execution of code directly on the client interface/web browser. These third-party integrations provide richness (chat tools, images, fonts) or extract analytics (Google Analytics). Today, up to 70% of the code executing and rendering on your customer’s browser comes from these integrations. All of these software integrations provide avenues for potential vulnerabilities.
Unfortunately, these unmanaged, unmonitored integrations operate without security consideration, providing an expansive attack surface that attackers have routinely exploited to compromise websites. Today, only 2% of the Alexa 1000 global websites were found to deploy client-side security measures to protect websites and web applications against attacks such as Magecart, XSS, credit card skimming, session redirects and website defacement.
Improving website security and ensuring performance with Cloudflare Workers
In this post, we focus on how Cloudflare Workers can be used to improve security and ensure the high performance of web applications. Tala has joined Cloudflare’s marketplace to further our common goals of ensuring website security, preserving data privacy and Continue reading
When Bloom filters don’t bloom
I've known about Bloom filters (named after Burton Bloom) since university, but I haven't had an opportunity to use them in anger. Last month this changed - I became fascinated with the promise of this data structure, but I quickly realized it had some drawbacks. This blog post is the tale of my brief love affair with Bloom filters.
While doing research about IP spoofing, I needed to examine whether the source IP addresses extracted from packets reaching our servers were legitimate, depending on the geographical location of our data centers. For example, source IPs belonging to a legitimate Italian ISP should not arrive in a Brazilian datacenter. This problem might sound simple, but in the ever-evolving landscape of the internet this is far from easy. Suffice it to say I ended up with many large text files with data like this:
This reads as: the IP 192.0.2.1 was recorded reaching Cloudflare data center number 107 with a legitimate request. This data came from many sources, including our active and passive probes, logs of certain domains we own (like cloudflare.com), public sources (like BGP table), etc. The same line would usually be repeated across multiple Continue reading
Technical Details of Why Cloudflare Chose AMD EPYC for Gen X Servers
From the very beginning Cloudflare used Intel CPU-based servers (and, also, Intel components for things like NICs and SSDs). But we're always interested in optimizing the cost of running our service so that we can provide products at a low cost and high gross margin.
We're also mindful of events like the Spectre and Meltdown vulnerabilities and have been working with outside parties on research into mitigation and exploitation which we hope to publish later this year.
We looked very seriously at ARM-based CPUs and continue to keep our software up to date for the ARM architecture so that we can use ARM-based CPUs when the requests per watt is interesting to us.
In the meantime, we've deployed AMD's EPYC processors as part of Gen X server platform and for the first time are not using any Intel components at all. This week, we announced details of this tenth generation of servers. Below is a recap of why we're excited about the design, specifications, and performance of our newest hardware.
Servers for an Accelerated Future
Every server can run every service. This architectural decision has helped us achieve higher efficiency across the Cloudflare network. It has also given us more Continue reading
Going Beyond Black History Month
Around this time of year in the United States, African-Americans are often tasked with explaining why we spend 28 (or in the case of a leap year 29) days celebrating the contributions our ancestors made to this country. It may come in the form of responding to ignorant questions posed in learning environments or expressed in well-crafted articles lauding the relevancy of Black history in our modern time.
Black history is not only relevant, it is how we ensure that our heroes are not forgotten and that we have a viable future in our respective industries. As Carter G. Woodson famously said, “If a race has no history, if it has no worthwhile tradition, it becomes a negligible factor in the thought of the world, and it stands in danger of being exterminated.”
As the US leaders of Afroflare, Cloudflare’s employee resource group (ERG) for employees of African descent, we made a personal commitment this month and beyond to effectively represent, build, and grow at Cloudflare and in the tech industry.
To honor that commitment, we decided to tackle some commonly asked questions about the state of African-Americans in tech.
How many African-Americans work in tech?
The latest Continue reading
Securing Memory at EPYC Scale
Security is a serious business, one that we do not take lightly at Cloudflare. We have invested a lot of effort into ensuring that our services, both external and internal, are protected by meeting or exceeding industry best practices. Encryption is a huge part of our strategy as it is embedded in nearly every process we have. At Cloudflare, we encrypt data both in transit (on the network) and at rest (on the disk). Both practices address some of the most common vectors used to exfiltrate information and these measures serve to protect sensitive data from attackers but, what about data currently in use?
Can encryption or any technology eliminate all threats? No, but as Infrastructure Security, it’s our job to consider worst-case scenarios. For example, what if someone were to steal a server from one of our data centers? How can we leverage the most reliable, cutting edge, innovative technology to secure all data on that host if it were in the wrong hands? Would it be protected? And, in particular, what about the server’s RAM?
Data in random access memory (RAM) is usually stored in the clear. This can leave data vulnerable to software or hardware probing by Continue reading
Gen X Performance Tuning
We are using AMD 2nd Gen EPYC 7642 for our tenth generation “Gen X” servers. We found many aspects of this processor compelling such as its increase in performance due to its frequency bump and cache-to-core ratio. We have partnered with AMD to get the best performance out of this processor and today, we are highlighting our tuning efforts that led to an additional 6% performance.
Thermal Design Power & Dynamic Power
Thermal design power (TDP) and dynamic power, amongst others, play a critical role when tuning a system. Many share a common belief that thermal design power is the maximum or average power drawn by the processor. The 48-core AMD EPYC 7642 has a TDP rating of 225W which is just as high as the 64-core AMD EPYC 7742. It comes to mind that fewer cores should translate into lower power consumption, so why is the AMD EPYC 7642 expected to draw just as much power as the AMD EPYC 7742?
Let’s take a step back and understand that TDP does not always mean the maximum or average power that the processor will draw. At a glance, Continue reading
Introducing Secrets and Environment Variables to Cloudflare Workers
The Workers team here at Cloudflare has been hard at work shipping a bunch of new features in the last year and we’ve seen some amazing things built with the tools we’ve provided. However, as my uncle once said, with great serverless platform growth comes great responsibility.
One of the ways we can help is by ensuring that deploying and maintaining your Workers scripts is a low risk endeavor. Rotating a set of API keys shouldn’t require risking downtime through code edits and redeployments and in some cases it may not make sense for the developer writing the script to know the actual API key value at all. To help tackle this problem, we’re releasing Secrets and Environment Variables to the Wrangler CLI and Workers Dashboard.
Supporting secrets
As we started to design support for secrets in Workers we had a sense that this was already a big concern for a lot of our users but we wanted to learn about all of the use cases to ensure we were building the right thing. We headed to the community forums, twitter, and the inbox of Louis Grace, business development representative extraordinaire, for some anecdotes about Secrets usage. We also sent Continue reading
Impact of Cache Locality
In the past, we didn't have the opportunity to evaluate as many CPUs as we do today. The hardware ecosystem was simple – Intel had consistently delivered industry leading processors. Other vendors could not compete with them on both performance and cost. Recently it all changed: AMD has been challenging the status quo with their 2nd Gen EPYC processors.
This is not the first time that Intel has been challenged; previously there was Qualcomm, and we worked with AMD and considered their 1st Gen EPYC processors and based on the original Zen architecture, but ultimately, Intel prevailed. AMD did not give up and unveiled their 2nd Gen EPYC processors codenamed Rome based on the latest Zen 2 architecture.
Playing with some new fun kit. #epyc pic.twitter.com/1No8Cmfzwl
— Matthew Prince ? (@eastdakota) November 8, 2019
This made many improvements over its predecessors. Improvements include a die shrink from 14nm to 7nm, a doubling of the top end core count from 32 to 64, and a larger L3 cache size. Let’s emphasize again on the size of that L3 cache, which is 32 MiB L3 cache per Core Complex Die (CCD).
This time around, we have taken steps to Continue reading
An EPYC trip to Rome: AMD is Cloudflare’s 10th-generation Edge server CPU
More than 1 billion unique IP addresses pass through the Cloudflare Network each day, serving on average 11 million HTTP requests per second and operating within 100ms of 95% of the Internet-connected population globally. Our network spans 200 cities in more than 90 countries, and our engineering teams have built an extremely fast and reliable infrastructure.
We’re extremely proud of our work and are determined to help make the Internet a better and more secure place. Cloudflare engineers who are involved with hardware get down to servers and their components to understand and select the best hardware to maximize the performance of our stack.
Our software stack is compute intensive and is very much CPU bound, driving our engineers to work continuously at optimizing Cloudflare’s performance and reliability at all layers of our stack. With the server, a straightforward solution for increasing computing power is to have more CPU cores. The more cores we can include in a server, the more output we can expect. This is important for us since the diversity of our products and customers has grown over time with increasing demand that requires our servers to do more. To help us drive compute performance, we needed Continue reading
Seamless remote work with Cloudflare Access
The novel coronavirus is actively changing how organizations work in real-time. According to Fortune, the virus has led to the “world’s largest work-from-home experiment.” As the epidemic crosses borders, employees are staying home and putting new stress on how companies manage remote work.
This is only accelerating an existing trend, however. Remote work has gained real traction in the last decade and Gartner projects that it will only continue. However, teams which are moving to a distributed model tend to do so slowly. When those timelines are accelerated, IT and security administrators need to be able to help their workforce respond without disrupting their team members.
Cloudflare Access can help teams migrate to a model that makes it seamless for users to work from any location, or any device, without the need for lengthy migrations or onboarding sessions. Cloudflare Access can be deployed in less than one hour and bring SaaS-like convenience and speed to the self-hosted applications that previously lived behind a VPN.
Leaving the castle-and-moat
When users share a physical space, working on a private network is easy. Users do not need clunky VPN clients to connect to the resources they need. Team members physically sit close Continue reading
Cloudflare’s Gen X: Servers for an Accelerated Future
“Every server can run every service.”
We designed and built Cloudflare’s network to be able to grow capacity quickly and inexpensively; to allow every server, in every city, to run every service; and to allow us to shift customers and traffic across our network efficiently. We deploy standard, commodity hardware, and our product developers and customers do not need to worry about the underlying servers. Our software automatically manages the deployment and execution of our developers’ code and our customers’ code across our network. Since we manage the execution and prioritization of code running across our network, we are both able to optimize the performance of our highest tier customers and effectively leverage idle capacity across our network.
An alternative approach might have been to run several fragmented networks with specialized servers designed to run specific features, such as the Firewall, DDoS protection or Workers. However, we believe that approach would have resulted in wasted idle resources and given us less flexibility to build new software or adopt the newest available hardware. And a single optimization target means we can provide security and performance at the same time.
We use Anycast to route a web request to the Continue reading