The Technical Challenges of Building Cloudflare WARP

The Technical Challenges of Building Cloudflare WARP

If you have seen our other post you know that we released WARP to the last members of our waiting list today. With WARP our goal was to secure and improve the connection between your mobile devices and the Internet. Along the way we ran into problems with phone and operating system versions, diverse networks, and our own infrastructure, all while working to meet the pent up demand of a waiting list nearly two million people long.

To understand all these problems and how we solved them we first need to give you some background on how the Cloudflare network works:

How Our Network Works

The Cloudflare network is composed of data centers located in 194 cities and in more than 90 countries. Every Cloudflare data center is composed of many servers that receive a continual flood of requests and has to distribute those requests between the servers that handle them. We use a set of routers to perform that operation:

The Technical Challenges of Building Cloudflare WARP

Our routers listen on Anycast IP addresses which are advertised over the public Internet. If you have a site on Cloudflare, your site is available via two of these addresses. In this case, I am doing a DNS query Continue reading

WARP is here (sorry it took so long)

WARP is here (sorry it took so long)
WARP is here (sorry it took so long)

Today, after a longer than expected wait, we're opening WARP and WARP Plus to the general public. If you haven’t heard about it yet, WARP is a mobile app designed for everyone which uses our global network to secure all of your phone’s Internet traffic.

We announced WARP on April 1 of this year and expected to roll it out over the next few months at a fairly steady clip and get it released to everyone who wanted to use it by July. That didn’t happen. It turned out that building a next generation service to secure consumer mobile connections without slowing them down or burning battery was… harder than we originally thought.

Before today, there were approximately two million people on the waitlist to try WARP. That demand blew us away. It also embarrassed us. The common refrain is consumers don’t care about their security and privacy, but the attention WARP got proved to us how wrong that assumption actually is.

This post is an explanation of why releasing WARP took so long, what we've learned along the way, and an apology for those who have been eagerly waiting. It also talks briefly about the rationale for why we Continue reading

Testing Open vSwitch-DPDK

In a previous tutorial we have built DPDK 18.11.2 and Open vSwitch 2.11.1 on Linux Debian 10.1 VM (buster). We have created a L3 Open vSwitch QEMU appliance that we will use to compare performance of Open vSwitch with enabled/disabled DPDK. A network topology is depicted on the Picture 1.

The lab is created as GNS3 project which consists of a single Debian Open vSwitch appliance. Two Linux Core 6.3 hosts PC1 and PC2 are connected to the switchports Ethernet0 and Ethernet1, respectively. Firstly, we are going to bind the both ports to DPDK-compatible driver and create a bridge with the DPDK ports attached to the bridge. As a last step, we will measure maximum achievable bandwidth with iPerf3 tool. The host PC1 will be be acting as a iPerf3 client connecting to the iPerf3 server running on the host PC2.

Note: Open vSwitch 2.11.1 appliance built on Debian 10 buster and DPDK 18.11.2 is available in Download-> Appliance Section. The Linux Core 6.3 image is available there, as well.

Picture 1 - Debian Open vSwitch/DPSK Appliance with Connected Host

1. GNS3 Configuration

Check GNS3 settings for Open Continue reading

Debian Open vSwitch Appliance with DPDK

Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license.  It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, 802.1ag).

DPDK is the Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures.

Note: Open vSwitch images are customized with my after install script  and they are ready for use in GNS3.

Username is debian with the password debian.

Debian Linux 10 (buster) VMDK Appliance with DPDK 18.11.2 and Open vSwitch 2.11.1 [997,2MB]
https://drive.google.com/file/d/1ZuVVP2POUnFjySt0YpFwPeSG5Rtw_6Gj/view?usp=sharing

 

The 10 most powerful companies in IoT

The Internet of Things is still very much a growth industry. As a technology area whose development is dictated by the needs of the operational side of any given business, it’s a new challenge for traditional IT companies – and one that gives them an unusual array of competitors. But there are always going to be a few companies that set the tone, and we’ve collected what we think are the 10 most powerful players in the IoT sector right now.A word on methodology. We began by looking at about 25 prominent corporate names in IoT, comparing them based on how innovative their technology is, their market share and solution depth and breadth.To read this article in full, please click here

10 most powerful companies in IoT

The Internet of Things is still very much a growth industry. As a technology area whose development is dictated by the needs of the operational side of any given business, it’s a new challenge for traditional IT companies – and one that gives them an unusual array of competitors. But there are always going to be a few companies that set the tone, and we’ve collected what we think are the 10 most powerful players in the IoT sector right now.A word on methodology. We began by looking at about 25 prominent corporate names in IoT, comparing them based on how innovative their technology is, their market share and solution depth and breadth.To read this article in full, please click here

Most enterprise networks can’t handle big data loads

Another week, another survey that finds IT cannot keep up with the ever-expanding data overload. This time the problem surrounds network bandwidth and overall performance.A survey of 300 IT professionals conducted by management consultant firm Accenture found the majority feel their enterprise networks are not up to the task of handling big data and internet of things (IoT) deployments. Only 43% of those companies polled said their networks are ready to support the cloud, IoT, and other digital technologies.To read this article in full, please click here

Automation Solution: Deploy BGP Routing with YANG Data Models

A while ago Ruben Tripiana tried to configure BGP on Cisco IOS using IETF YANG data models… and failed. In Spring 2019 Building Network Automation Solutions online course Chris Crook decided to deploy BGP routing on multiple platforms using YANG data models instead of configuration templates. Not only did he succeed, he also documented his work and the tools he used, and published the solution so you can replicate his efforts.

You can find many more network automation solutions created by the attendees of our automation course in solutions showcase.

50 ways to leak your data: an exploration of apps’ circumvention of the Android permissions system

50 ways to leak your data: an exploration of apps’ circumvention of the Android permissions system Reardon et al., USENIX Security Symposium 2019

The problem is all inside your app, she said to me / The answer is easy if you take it logically / I’d like to help data in its struggle to be free / There must be fifty ways to leak their data.

You just slip it out the back, Jack / Make a new plan, Stan / You don’t need to be coy, Roy / Just get the data free.

Hop it on the bus, Gus / You don’t need to discuss much / Just drop off the key, Lee / And get the data free…

— Lyrics adapted from “50 ways to leave your lover” by Paul Simon (fabulous song btw., you should definitely check it out if you don’t already know it!).

This paper is a study of Android apps in the wild that leak permission protected data (identifiers which can be used for tracking, and location information), where those apps should not have been able to see such data due to a lack of granted permissions. By detecting Continue reading

Controlling Information Flow: NATS Subjects and Queues

Publish/subscribe messaging platforms like NATS allow us to build highly event-driven software systems, where we can build software to constantly listen for relevant messages and take action accordingly. This is what makes EDI projects like StackStorm (a project I’ve worked on and written about before) - and others like it- so powerful. Another advantage of pub/sub systems is that publishers don’t have to know or care if anyone is listening.

Controlling Information Flow: NATS Subjects and Queues

Publish/subscribe messaging platforms like NATS allow us to build highly event-driven software systems, where we can build software to constantly listen for relevant messages and take action accordingly. This is what makes EDI projects like StackStorm (a project I’ve worked on and written about before) - and others like it- so powerful. Another advantage of pub/sub systems is that publishers don’t have to know or care if anyone is listening.

Citrix Meshes Istio Service Mesh Into Its ADC Platform

The company is offering Istio in two ways: as an ingress gateway for north-south traffic into the...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Kubernetes Kills Containership’s Cruise

The company's founder and CTO explained that it was unable to adequately monetize its operations in...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Rubrik, NetApp Catapult Data Management to the Cloud

The partnership aims to simplify data management on cloud-scale architectures through policy-based...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Full Stack Journey 034: Choosing Your Own Adventure

If you've ever thought about working for yourself, catch this episode of the Full Stack Journey. Guest Ned Bellavance discusses his decision to leave a job as an IT consultant to become an independent content creator of tech training material. Ned shares real-world insights into his experience along with practical takeaways.

Full Stack Journey 034: Choosing Your Own Adventure

If you've ever thought about working for yourself, catch this episode of the Full Stack Journey. Guest Ned Bellavance discusses his decision to leave a job as an IT consultant to become an independent content creator of tech training material. Ned shares real-world insights into his experience along with practical takeaways.

The post Full Stack Journey 034: Choosing Your Own Adventure appeared first on Packet Pushers.

NSX-T 2.5 What is new for Kubernetes

NSX-T 2.5 was released last week and now we also have the new NSX Container Plugin 2.5 on the market…. VMworld US 2019 is also over with announcements of huge innovations like NSX Intelligence, enhanced security, and improved operability and observability. You can find more information here! I would like to take this opportunity to specifically highlight the new containers capabilities we had been working on for this release.

What is new in NSX Container plugin (NCP)

Simplified UI (Policy API) support

With the introduction of the new intent-based Policy API and corresponding Simplified UI, administrators have seen that objects created by NCP for Kubernetes are not visible in the new UI. Instead they had to go to the Advanced Networking & Security tab that corresponds to the old imperative APIs. Furthermore, all related objects like T0, IP Block, IP Pool had to be created using the Advanced Networking & Security tab. I am happy to announce that from NCP 2.5+ we have a parameter in the NCP configmap (ncp.ini) policy_nsxapi that can be set to True. In that case, NCP will start calling the new intent-based API and all objects will be visible in the simplified UI. Continue reading

3 quick tips for working with Linux files

Linux provides a wide variety of commands for working with files — commands that can save you time and make your work a lot less tedious.Finding files When you're looking for files, the find command is probably going to be the first command to come to mind, but sometimes a well-crafted ls command works even better. Want to remember what you called that script you were working on last night before you fled the office and drove home? Easy! Use an ls command with the -ltr options. The last files listed will be the ones most recently created or updated.$ ls -ltr ~/bin | tail -3 -rwx------ 1 shs shs 229 Sep 22 19:37 checkCPU -rwx------ 1 shs shs 285 Sep 22 19:37 ff -rwxrw-r-- 1 shs shs 1629 Sep 22 19:37 test2 A command like this one will list only the files that were updated today:To read this article in full, please click here

1 … 1,211 1,212 1,213 1,214 1,215 … 3,912