Join us for 5 serverless events in SF Bay Area this week
Developer Week Bay Area is happening this week and Cloudflare engineers and developer relations team members are delivering several talks around the Bay. Join us in San Francisco and Oakland for the following talks. We’ll hope to see you soon.
WebAssembly on the Server, npm & genomics tools @ Cloudflare
We've partnered with the WebAssembly SF meetup group to curate three talks from Zack Bloom of Cloudflare, Laurie Voss of npm, and Robert Aboukhalil of Invitae.
Event Type: Meetup
Location: Cloudflare HQ, San Francisco, CA
Date: February 20, 2019
View Event Details & Register Here »
Serverless: An Inside Look
Cloudflare engineers are delivering three serverless talks in downtown Oakland: How Workers Work, Security: the Serverless Future, and Building a Serverless World (Map) with IoT and Workers.
Event Type: Meetup
Location: At My Sphere, Oakland, CA
Date: February 21, 2019
View Event Details & Register Here »
Developer Week Bay Area
Cloudflare will be at Developer Week Bay Area. Be sure to check out Single-Process Serverless, Building an Iot World (Map) with Serverless, and Make Your Existing Application Serverless talks.
Event Type: Conference
Location: Oakland Convention Center, Oakland, CA
Date: February 20-24, 2019
Stop the Bots: Practical Lessons in Machine Learning
Bot-powered credential stuffing is a scourge on the modern Internet. These attacks attempt to log into and take over a user’s account by assaulting password forms with a barrage of dictionary words and previously stolen account credentials, with the aim of performing fraudulent transactions, stealing sensitive data, and compromising personal information.
At Cloudflare we’ve built a suite of technologies to combat bots, many of them grounded in Machine Learning. ML is a hot topic these days, but the literature tends to focus on improving the core technology — and not how these learning machines are incorporated into real-world organizations.
Given how much experience we have with ML (which we employ for many security and performance products, in addition to bot management), we wanted to share some lessons learned with regard to how this technology manifests in actual products.
There tend to be three stages every company goes through in the life cycle of infusing machine learning into their DNA. They are:
- Business Intelligence
- Standalone Machine Learning
- Machine Learning Productization
These concepts are a little abstract — so let’s walk through how they might apply to a tangible field we all know and love: dental insurance.
Business Intelligence
Many companies already Continue reading
Announcing workers.dev
We are working really hard to allow you to deploy Workers without having a Cloudflare domain. You will soon be able to deploy your Cloudflare Workers to a subdomain-of-your-choice.workers.dev, which you can go claim now on workers.dev!
Why are we doing this?
You may have read the announcement blog post for Workers (or one of the many tutorials and guest posts), and thought “let me give this a try!”. If you’re an existing Cloudflare customer, you logged into the dashboard, and found a new icon called “Workers”, paid $5 and were on your way. If you’re not, you clicked “Sign Up”, but instead of getting to create and deploy a Worker, we asked you for your domain (if you didn’t have one, we had you register one), and move your nameservers.
Since launch, we have had tons of people who wanted to build a new serverless project from scratch or just try Workers out, but found it difficult to get started. We want to make it easier for anyone to get started building and deploying serverless applications.
How did we get here?
The way you get started on Workers today reflects our journey as a company. Continue reading
SOCKMAP – TCP splicing of the future
Recently we stumbled upon the holy grail for reverse proxies - a TCP socket splicing API. This caught our attention because, as you may know, we run a global network of reverse proxy services. Proper TCP socket splicing reduces the load on userspace processes and enables more efficient data forwarding. We realized that Linux Kernel's SOCKMAP infrastructure can be reused for this purpose. SOCKMAP is a very promising API and is likely to cause a tectonic shift in the architecture of data-heavy applications like software proxies.
But let’s rewind a bit.
Birthing pains of L7 proxies
Transmitting large amounts of data from userspace is inefficient. Linux provides a couple of specialized syscalls that aim to address this problem. For example, the sendfile(2) syscall (which Linus doesn't like) can be used to speed up transferring large files from disk to a socket. Then there is splice(2) which traditional proxies use to forward data between two TCP sockets. Finally, vmsplice can be used to stick memory buffer into a pipe without copying, but is very hard to use correctly.
Sadly, sendfile, splice and vmsplice are very specialized, synchronous and solve only one part Continue reading
Introducing Cf-Terraforming
Ever since we implemented support for configuring Cloudflare via Terraform, we’ve been steadily expanding the set of features and services you can manage via this popular open-source tool.
If you're unfamiliar with how Terraform works with Cloudflare, check out our developer docs.
We are Terraform users ourselves, and we believe in the stability and reproducibility that can be achieved by defining your infrastructure as code.
What is Terraform?
Terraform is an open-source tool that allows you to describe your infrastructure and cloud services (think virtual machines, servers, databases, network configurations, Cloudflare API resources, and more) as human-readable configurations.
Once you’ve done this, you can run the Terraform command-line tool and it will figure out the difference between your desired state and your current state, and make the API calls in the background necessary to reconcile the two.
Unlike other solutions, Terraform does not require you to run software on your hosts, and instead of spending time manually configuring machines, creating DNS records, and specifying Page Rules, you can simply run:
terraform apply
and the state described in your configuration files will be built for you.
Enter Cloudflare Terraforming
Terraform is a tremendous time-saver once you have your configuration files Continue reading
SEO Best Practices with Cloudflare Workers, Part 2: Implementing Subdomains
Recap
In Part 1, the merits and tradeoffs of subdirectories and subdomains were discussed. The subdirectory strategy is typically superior to subdomains because subdomains suffer from keyword and backlink dilution. The subdirectory strategy more effectively boosts a site's search rankings by ensuring that every keyword is attributed to the root domain instead of diluting across subdomains.
Subdirectory Strategy without the NGINX
In the first part, our friend Bob set up a hosted Ghost blog at bobtopia.coolghosthost.com that he connected to blog.bobtopia.com using a CNAME DNS record. But what if he wanted his blog to live at bobtopia.com/blog to gain the SEO advantages of subdirectories?
A reverse proxy like NGINX is normally needed to route traffic from subdirectories to remotely hosted services. We'll demonstrate how to implement the subdirectory strategy with Cloudflare Workers and eliminate our dependency on NGINX. (Cloudflare Workers are serverless functions that run on the Cloudflare global network.)
Back to Bobtopia
Let's write a Worker that proxies traffic from a subdirectory – bobtopia.com/blog – to a remotely hosted platform – bobtopia.coolghosthost.com. This means that if I go to bobtopia.com/blog, I should see the content of Continue reading
SEO Best Practices with Cloudflare Workers, Part 1: Subdomain vs. Subdirectory
Subdomain vs. Subdirectory: 2 Different SEO Strategies
Alice and Bob are budding blogger buddies who met up at a meetup and purchased some root domains to start writing. Alice bought aliceblogs.com and Bob scooped up bobtopia.com.
Alice and Bob decided against WordPress because its what their parents use and purchased subscriptions to a popular cloud-based Ghost blogging platform instead.
Bob decides his blog should live at at blog.bobtopia.com – a subdomain of bobtopia.com. Alice keeps it old school and builds hers at aliceblogs.com/blog – a subdirectory of aliceblogs.com.
Subdomains and subdirectories are different strategies for instrumenting root domains with new features (think a blog or a storefront). Alice and Bob chose their strategies on a whim, but which strategy is technically better? The short answer is, it depends. But the long answer can actually improve your SEO. In this article, we'll review the merits and tradeoffs of each. In Part 2, we'll show you how to convert subdomains to subdirectories using Cloudflare Workers.
Setting Up Subdomains and Subdirectories
Setting up subdirectories is trivial on basic websites. A web server treats its subdirectories (aka subfolders) the same as regular old folders in a Continue reading
Solving Problems with Serverless – The Cloudflare LED Data Center Board, Part I
You know you have a cool job when your first project lets you bring your hobby into the office.
That’s what happened to me just a few short weeks ago when I joined Cloudflare. The task: to create a light-up version of our Data Center map – we’re talking more than a hundred LEDs tied to the deployment state of each and every Cloudflare data center. This map will be a part of our booths, so it has to be able to travel; meaning we have to consider physical shipping and the ability to update the data when the map is away from the office. And the fun part – we are debuting it at SF Developer Week in late February (I even get to give a talk about it!) That gave me one week of software time in our San Francisco office, and a little over two and a half in the Austin office with the physical materials.
So what does this have to do with Serverless? Well, let’s think about where and how this map will need to operate: This will be going to Continue reading
Winning the Blackbird Battle
Frequent readers of the Cloudflare blog are aware of the efforts we’ve undertaken in response to our first encounter with a patent troll. We’re happy to report that on Wednesday, the U.S. Court of Appeals for the Federal Circuit issued an opinion affirming a lower court decision dismissing the case brought by Blackbird Tech. This is the last step in the process 1, we’ve won.
In addition to vigorously opposing this case in court, we created and sponsored Project Jengo to push back against the incentives that empower patent trolls like Blackbird Tech. Now that the case is over, we will be wrapping up Project Jengo and will report back with a summary of the Project’s successes in the near future.
But before we move on from the litigation, I want to share a few reflections on this case.
We noted from the very beginning: “The infringement claim is not a close one … if the ‘335 patent is read broadly enough to cover our system (which shouldn’t happen), it would also cover any system where electronic communications are examined and redacted or modified.”
Our initial observation, which we Continue reading
The Curious Case of the Garden State Imposter
Dealing with abuse complaints isn’t easy, for any Internet company. The variety of subject matters at issue, the various legal and regulatory requirements, and the uncertain intentions of complaining parties combine to create a ridiculously complex situation. We often suggest to those who propose easy answers to this challenge that they spend a few hours tracking the terminal of a member of our Trust & Safety team to get a feel for how difficult it can be. Yet even we were a bit surprised by an unusual abuse report we’ve been dealing with recently.
Last week, we received what looked like a notable law enforcement request: a complaint from an entity that identified itself as the “New Jersey Office of the Attorney General” and claimed to be a notice Cloudflare was “serving files consisting of 3D printable firearms in violation of NJ Stat. Ann. § 2C:39-9 3(I)(2).” The complaint further asked us to “delete all files described within 24 hours” and threatened “to press charges in order to preserve the safety of the citizens of New Jersey.”
Because we are generally not the host of information, and are unable to remove content from the Internet that we don’t Continue reading
Introducing The Serverlist: Cloudflare’s New Serverless Newsletter
At Cloudflare, we've been digging our heels into serverless, so we created The Serverlist newsletter, enabling us to share interesting content in the serverless space with the developer community. The Serverlist newsletter highlights all things serverless, with content that covers news from the serverless world, tutorials to learn how to get involved yourself, and different events you can attend.
Check out our first edition of The Serverlist below and sign up here to have this newsletter delivered to your inbox.
IBM Cloud Internet Services protects any cloud – now with Cloudflare Spectrum and Workers
At Cloudflare, we have an ambitious mission of helping to build a better Internet. Partnerships are a core part of how we achieve this mission. Last year we joined forces with IBM. Their expertise and deep relationships with the world's largest organizations are highly complementary with Cloudflare's cloud-native, API-first architecture that provides superior security, performance, and availability for Internet-facing workloads. Our shared goal of enabling and supporting a hybrid and multi-cloud world is becoming a greater component of our combined message to the market.
As we prepare for the IBM Think customer conference in San Francisco this week, the Cloudflare team is excited about the opportunities ahead. We closed 2018 with momentum, bringing several of the world’s leading brands onto the Cloud Internet Services (CIS) platform in 2018. Customers have used CIS for several purposes, including:
- The CIS Global Load Balancer provides high availability across IBM Cloud regions for customers in Europe, North America, and Latin America
- CIS caching capabilities have ensured availability and performance for world spectator events with high traffic spikes
- The CIS authoritative DNS delivers greater availability and performance for Internet-facing workloads supporting thousands of developers
At Think, please visit Cloudflare at our booth (#602). In addition, Continue reading
Give your automated services credentials with Access service tokens
Cloudflare Access secures your internal sites by adding authentication. When a request is made to a site behind Access, Cloudflare asks the visitor to login with your identity provider. With service tokens, you can now extend that same level of access control by giving credentials to automated tools, scripts, and bots.
Authenticating users and bots alike
When users attempt to reach a site behind Access, Cloudflare looks for a JSON Web Token (a JWT) to determine if that visitor is allowed to reach that URL. If user does not have a JWT, we redirect them to the identity provider configured for your account. When they login successfully, we generate the JWT.
When you create an Access service token, Cloudflare generates a unique Client ID and Secret scoped to that service. When your bot sends a request with those credentials as headers, we validate them ourselves instead of redirecting to your identity provider. Access creates a JWT for that service and the bot can use that to reach your application.
Getting started
Within the Access tab of the Cloudflare dashboard, you’ll find a new section: Service Tokens. To get started, select “Generate a New Service Token.”
You’ll be asked to Continue reading
Better business results from faster web applications – Cloudflare is the fastest
Web performance encompasses a lot of things: page load time, responsiveness for web and mobile applications. But overall, the key element is response times. How quickly can the origin server or the cache fulfill a user request? How quickly can a DNS response reach the client device?
The Cloudflare mission is to help build a better Internet to everyone, and we offer multiple services for boosting the speed and performance of our customers and users. Cloudflare is faster than the competition when it comes to accelerating performance.
How site speed impacts the bottom line
There is a lot of research out there that confirms what many businesses and web developers already know: speed affects revenue. Better web performance means better user engagement and more conversions. Better performance also results in better SEO, driving up overall traffic numbers and increasing lead generation, sales, or ad revenue.
One study by Google and Bing concluded that on average, a two-second delay in a website's page rendering led to a 4.3% loss in revenue per visitor. Another independent study has shown that 1 additional second of load time reduces conversions by 7%.
How does using Cloudflare affect performance?
According to testing from Continue reading
Helping To Build Cloudflare, Part 6: What does Cloudflare’s CTO do?
This is the final part of a six part series based on a talk I gave in Trento, Italy. To start from the beginning go here.
If you are still awake there’s really one final question that you might want to know the answer to: What does the CTO do? The reality is that it means different things in different companies. But I can tell you a little about what I do.
The longest temporary job
I didn’t join Cloudflare as CTO. My original job title was Programmer and for the first couple of years I did just that. I wrote a piece of technology called Railgun (a differential compression program used to speed up the connection between Cloudflare and origin web servers) and then I went on to write our WAF. After that I worked on our Go-based DNS server and other parts of the stack.
At some point Lee Holloway decided he didn’t want to manage Cloudflare’s growing staff and Michelle Zatlyn (one of Cloudflare’s founders) asked me if I would ‘temporarily’ manage engineering. This is now the longest temporary job I’ve ever had!
Initially a lot of what I did was manage the team and help interview Continue reading
Cloudflare Support for Azure Customers
Cloudflare seeks to help its end customers use whichever public and private clouds best suit their needs. Towards that goal, we have been working to make sure our solutions work well with various public cloud providers including Microsoft’s Azure platform.
If you are an Azure customer, or thinking about becoming one, here are three ways we have made Cloudflare’s performance and security services work well with Azure.
1) The development of an Azure application for Cloudflare Argo Tunnel.
We are proud to announce an application for Cloudflare Argo Tunnel within the Azure marketplace. As a quick reminder, Argo Tunnel establishes an encrypted connection between the origin and the Cloudflare edge. The small tunnel daemon establishes outbound connections to the two nearest Cloudflare PoPs, and the origin is only accessible via the tunnel between Cloudflare and origin.
Because these are outbound connections, there is likely no need to modify firewall rules, configure DNS records, etc. You can even go so far as to block all IPs on the origin and allow traffic only to flow through the tunnel. You can learn more here. The only prerequisite for using Argo Tunnel is to have Argo enabled on your Cloudflare zone. You can Continue reading
Helping To Build Cloudflare, Part 5: People: Finding, Nurturing and Learning to Let Go
This is part 5 of a six part series based on a talk I gave in Trento, Italy. To start from the beginning go here.
So, let me talk a bit about people. Software is made by people. Sometimes individuals but more likely by teams. I’ve talked earlier about some aspects of our architecture and our frequent rewrites but it’s people that make all that work.
And, honestly, people can be an utter joy and a total pain. Finding, keeping, nurturing people and teams is the single most important thing you can do in a company. No doubt.
Finding People
Finding people is really hard. Firstly, the technology industry is booming, and so engineers have a lot of choices. Countries create special visas just for them. Politicians line up to create mini-Silicon Valleys in their countries. Life is good!
But the really hard thing is interviewing. How do you find good people from an interview? I don’t know the answer to that. We put people through on average 8 interviews and a pair programming exercise. We look at open source contributions. Sometimes we look at people’s degrees.
We tend to look for potential. An old boss used to say, “Don’t Continue reading
The Black Elephant in the Room
When I come to work at Cloudflare, I understand and believe in this main purpose of why we exist: Helping to Build a Better Internet.
The reason why we feel like we can help build a better internet is simply because we believe in values that instill a nature of freedom, privacy, and empowerment in the tool that helps individuals broaden their intellectual and cultural perspective on the daily.
Knowing all of this, our own great company needs to be able to build itself daily into a better company. And that starts with having those conversations which are always uncomfortable. And let me be clear in saying this, being uncomfortable is a good thing because that makes one grow and not be stagnant. Saying all that, here we go...
The Afrocultural community at Cloudflare should take pride in being diverse and inclusive for all just as we all work together to help build a better internet for all.
And one of the many ways we can build upon this effort is to do more than just belong in a work place and eventually build off of that, feeling normal over time. When I mean belong, it’s more than the "Impostor Continue reading
Helping To Build Cloudflare, Part 4: Public Engagement
This is part 4 of a six part series based on a talk I gave in Trento, Italy. To start from the beginning go here.
We don’t believe that any of our software, not a single line of code, provides us with a long-term advantage. We could, today, open source every single line of code at Cloudflare and we don’t believe we’d be hurt by it.
How we think about Open Source
Why don’t we? We actually do open source a lot of code, but we try to be thoughtful about it. Firstly, a lot of our code is so Cloudflare-specific, full of logic about how our service works, that it’s not generic enough for someone else to pick up and use for their service. So, for example, open sourcing the code that runs our web front end would be largely useless.
But other bits of software are generic. There’s currently a debate going on internally about a piece of software called Quicksilver. I mentioned before that Cloudflare used a distributed key-value store to send configuration to machines across the world. We used to use an open source project called Kyoto Tycoon. It was pretty cool.
But Continue reading
Helping To Build Cloudflare, Part 2: The Most Difficult Two Weeks
This is part 2 of a six part series based on a talk I gave in Trento, Italy. Part 1 is here.
It’s always best to speak plainly and honestly about the situation you are in. Or as Matthew Prince likes to put it “Panic Early”. Long ago I started a company in Silicon Valley which had the most beautiful code. We could have taught a computer science course from the code base. But we had hardly any customers and we failed to “Panic Early” and not face up to the fact that our market was too small.
Ironically, the CEO of that company used to tell people “Get bad news out fast”. This is a good maxim to live by, if you have bad news then deliver it quickly and clearly. If you don’t the bad news won’t go away, and the situation will likely get worse.
Cloudbleed
Cloudflare had a very, very serious security problem back in 2017. This problem became known as Cloudbleed. We had, without knowing it, been leaking memory from inside our machines into responses returned to web browsers. And because our machines are shared across millions of web sites, that meant that HTTP requests Continue reading