Cloudflare outage caused by bad software deploy (updated)

This is a short placeholder blog and will be replaced with a full post-mortem and disclosure of what happened today.

For about 30 minutes today, visitors to Cloudflare sites received 502 errors caused by a massive spike in CPU utilization on our network. This CPU spike was caused by a bad software deploy that was rolled back. Once rolled back the service returned to normal operation and all domains using Cloudflare returned to normal traffic levels.

This was not an attack (as some have speculated) and we are incredibly sorry that this incident occurred. Internal teams are meeting as I write performing a full post-mortem to understand how this occurred and how we prevent this from ever occurring again.

Update at 2009 UTC:

Starting at 1342 UTC today we experienced a global outage across our network that resulted in visitors to Cloudflare-proxied domains being shown 502 errors (“Bad Gateway”). The cause of this outage was deployment of a single misconfigured rule within the Cloudflare Web Application Firewall (WAF) during a routine deployment of new Cloudflare WAF Managed rules.

The intent of these new rules was to improve the blocking of inline JavaScript that is used in attacks. These rules were Continue reading

Transparency, Fairness, and Respect: The Policy Brief on Responsible Data Handling

It’s been a little over a year since the European Union’s General Data Protection Regulation (GDPR) was implemented, but almost immediately, people noticed its impact. First, there was the flurry of emails seeking users’ consent to the collection and use of their data. Since then, there’s also been an increase in the number of sites that invite the user to consent to tracking by clicking “Yes to everything,” or to reject them by going through a laborious process of clicking “No” for each individual category. (Though some non-EU sites simply broadcast “if we think you’re visiting from the EU, we can’t let you access our content.”) There was also the headline-grabbing €50 million fine imposed on Google by the French supervisory authority.

In its summary of the year, the EU Data Protection Board (EDPB) reported an increase in the number of complaints received under GDPR, compared to the previous year, and a “perceived rise in awareness about data protection rights among individuals.” Users are more informed and want more control over the collection and use of their personal data.

They’re probably irritated by the current crop of consent panels, and either ignore, bypass, or click through them Continue reading

BrandPost: “Shift Left” to Push Customer Support into Overdrive

The “Shift Left” concept is all about efficiency and quality. In software development, shifting left means performing testing early and often in the project lifecycle instead of waiting until the end. By discovering and addressing errors and bugs earlier, teams can ultimately deliver a higher quality product, one that is better aligned with addressing customers’ needs.In support, it means shifting requests as close to the customer as possible – which includes offering the ability self-serve. Moving solutions closer to the operational frontline and to the point of the first issue allows customers get answers quicker and organizations to close tickets faster. There are 3 big benefits of taking this approach:To read this article in full, please click here

BrandPost: SD-WAN Buyers Should Think Application Performance as well as Resiliency

As an industry analyst, not since the days of WAN Optimization have I seen a technology gain as much interest as I am seeing with SD-WANs today. Although full deployments are still limited, nearly every network manager, and often many IT leaders I talk to, are interested in it. The reason for this is two-fold – the WAN has grown in importance for cloud-first enterprises and is badly in need of an overhaul. This hasn’t gone unnoticed by the vendor community as there has been an explosion of companies bringing a broad range of SD-WAN offerings to market. The great news for buyers is that there is no shortage of choices. The bad news is there are too many choices and making the right decision difficult.To read this article in full, please click here

Light Board Video Series: VMware NSX Cloud

Over the last decade there has been a gradual, continuous shift of enterprise software applications away from the data center and towards one or multiple public clouds. As more and more applications are built natively in public clouds like AWS or Azure, the management of networking and security for those workloads becomes more complex: each cloud has its own set of unique constructs that must be managed independently of those in the data center.

What if there was a way to unify all of those workloads under one consistent networking fabric that can manage one standard set of networking and security policies across both on-premises and public clouds? This is where VMware NSX Cloud comes in.

What is NSX Cloud?

Designed specifically for public-cloud-native workloads, NSX Cloud extends VMware NSX software-defined networking and security from the data center to multiple public clouds, enabling consistent policy management from a single NSX interface.

To explain what NSX Cloud is and how it can deliver consistent hybrid networking and security for you, we asked our product manager Shiva Somasundaram to recored a three-part lightboard video series.

Part 1: NSX Cloud Overview

Shiva gives a high-level overview of what NSX Cloud is and how Continue reading

The Week in Internet News: Small Routing Error Has Big Consequences

Bad route: A small routing error led to Internet outages in the Northeastern United States on June 24, Inc.com reports. Small network services provider DQE Communications shared inaccurate routing information with Verizon, which then passed it along to the wider network. Internet services were flaky for about two hours, with Verizon Fios phone and Internet services in Virginia, Massachusetts, New York, New Jersey, Pennsylvania, and other states affected, the Washington Post said. Server issues also affected Reddit, Twitch, and video gaming service Discord.

Attacking encryption? U.S. President Donald Trump’s National Security Council recently discussed ways to prohibit companies from offering customers unbreakable encryption, Politico reports. Officials debated whether to ask Congress to effectively outlaw end-to-end encryption, according to anonymous sources.

Embrace the dark side: Government entities looking to improve Internet speeds in their areas should consider dark fiber when it’s available, advises AmericanCityandCounty.com. Switching to dark fiber can offer both performance improvement and cost savings, but the transition can demand a major overhaul.

Service restored, for one guy: Sudan’s three-week Internet shutdown keeps going, except for one lawyer, who won a lawsuit against telecom operator Zain Sudan over the blackout ordered by the country’s military rulers, the Continue reading

BrandPost: How does David battle Goliath? With great strategy and the technology to implement it

The classic story of David battling Goliath resonates with any successful entrepreneur. At some point, small companies must confront large, entrenched rivals. Those big companies possess clear advantages: brand recognition, economies of scale, financial leverage and many others. Customers need a compelling reason to switch providers.How do would-be Davids compete? They need to develop their own modernized slingshot. Technology provides virtually endless possibilities for competitive advantage. Like David, though, you need to size up your opponent and adopt the right strategy before choosing your weapon.In the United Kingdom, a company called Ocado did just that in the exotic, sophisticated market of … grocery stores.To read this article in full, please click here

Network Break 241: Extreme Buys Aerohive; Sloppy BGP Plumbing Causes Route Leak

Extreme Networks spends approx. $227 million to buy Aerohive Networks to add a cloud-managed WLAN to its portfolio, a route leak resulted in cascading failures on June 24th, Oracle will retire Dyn managed DNS services, Mist Systems rolls out a new 11ax AP, and more tech news on today's Network Break podcast.

The post Network Break 241: Extreme Buys Aerohive; Sloppy BGP Plumbing Causes Route Leak appeared first on Packet Pushers.

Leaving Comments in Code Expressed Artefacts

Week of 24th June 2019 was interesting. We had #ferrogate which made a lot of network engineers very unhappy and also an ongoing social media thread on code comments. For this discussion, I’m going with the title of "leaving comments in code expressed artefacts" because code represents more than writing software. I feel quite passionately about this having been on the raw end of no code comments and also being guilty of leaving plenty of crappy and unhelpful comments too.

The Mystic Arts

Let’s set a scene. You’ve had a long day and you’re buckled in for what can only be described as a mentally exhausting night. The system architecture is clearly formed in your head and you’re beginning to see issues ahead of time. You can’t quite justify any premature optimisation, but you know this current design has a ceiling. You also know there are system wide intricacies that are not obvious at the component level.

Normality in these scenarios is to insert context based comments, which make perfect sense at 2am, but next day 9am exhausted you may be confused as to what on earth happened in the early hours. We’ve all been there.

There are multiple trains Continue reading

BrandPost: Three Support Trends to Fuel Business Advantage

Who hasn’t had an infuriating support experience as a customer? We’ve all been there. Waiting on the phone. Repeating details. Being passed like a hot potato from one agent to the next. Has no one in the universe ever had or dealt with our same issue before? It can be frustrating, to say the least. And we’re shell-shocked the next time we need to reach out for customer support.There’s a light at the end of the tunnel, though. Before we get there, let’s first consider why this is happening. If we look at the support landscape, we can see that it’s changing: The technology we use is changing. Think about how technology has changed over the last few years. Ten years ago, we didn’t have iPads. Now we have universal connectivity. And behind simple interfaces, underlying technologies are increasingly complex. The people we support are changing. ​End users are more mobile. People can work from anywhere, and they take their digital lives with them. ​And with nearly unlimited – and searchable – data, people are consuming information in a variety of ways. Customer expectations are changing. The more immediate and available our technology becomes, the more Continue reading

Datanauts 167: Patch Now Or Later? The Delicate Art Of Vulnerability Management

Security expert Zoë Rose beams aboard the Datanauts podcast to discuss the intricacies of vulnerability management, including how to asses risks, when and what to patch, the importance of input from multiple stakeholders, compensating controls, and more.

The post Datanauts 167: Patch Now Or Later? The Delicate Art Of Vulnerability Management appeared first on Packet Pushers.

Creating VyOS ISO Image

There are several ways to get an VyOS ISO image. Firstly, you can buy subscription, so you will have an access to LTS VyOS ISO images. The LTS images are also available for VyOS contributors or evangelists with perpetual 1-year access. The third option involves building ISO image itself. Building involves cloning VyOS repository with git, taking care of required dependencies and finally compiling from sources. Either you need Debian as a base and manage dependencies manually or you can compile using the docker method and Debian is not needed.

Using the Dockerfile you create your own Docker container that is used to build a VyOS ISO image or other required VyOS packages. The Dockerfile contains some of the most used packages needed to build a VyOS ISO, a qemu image, and several of the submodules.

1. Install Docker CE

We are going to install Docker CE on Ubuntu 18.04.2 LTS (bionic).

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates curl software-properties-common git
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo apt-key fingerprint 0EBFCD88

$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce
Continue reading

Announcing an Executive Director for the Internet Society Foundation

Photo of Sarah Armstrong

Last year we established the Internet Society Foundation, with a plan to make clearer the Internet Society’s grant-making activities, and distinguish them from Internet Society programmes. We announced that we would develop the Foundation over the course of the following year.

Since then, we have introduced the Internet Society Foundation’s new website and opened up the process for applications from ISOC Chapters and Special Interest Groups (SIGs) for the Beyond the Net Grants Programme, which is now housed within the Foundation. This now includes the full range of Beyond the Net Small, Medium, and Large Grants.

In parallel with moving the Beyond the Net Programme, we have been searching for a leader for the Foundation. I am pleased to announce our selection. 

Sarah Armstrong starts in her new role as the Foundation’s Executive Director today, July 1. She brings a wealth of experience to us, having built a career in non-profit, humanitarian, and international development work over many years.

Please join me in extending a warm Internet Society welcome to Sarah. I am sure she will play a key role in ensuring that our financial support for others’ activities is focused and effective. I look forward to a Continue reading

Tempered Networks simplifies secure network connectivity and microsegmentation

The TCP/IP protocol is the foundation of the internet and pretty much every single network out there. The protocol was designed 45 years ago and was originally only created for connectivity. There’s nothing in the protocol for security, mobility, or trusted authentication.The fundamental problem with TCP/IP is that the IP address within the protocol represents both the device location and the device identity on a network. This dual functionality of the address lacks the basic mechanisms for security and mobility of devices on a network.This is one of the reasons networks are so complicated today. To connect to things on a network or over the internet, you need VPNs, firewalls, routers, cell modems, etc. and you have all the configurations that come with ACLs, VLANs, certificates, and so on. The nightmare grows exponentially when you factor in internet of things (IoT) device connectivity and security. It’s all unsustainable at scale.To read this article in full, please click here

1 … 1,185 1,186 1,187 1,188 1,189 … 3,832