Stuff The Internet Says On Scalability For September 28th, 2018

Hey, it's HighScalability time:

 

@danielbryantuk: "A LAMP stack is a good thing. Never inflict a distributed system on yourself unless you have too..." @mipsytipsy #CloudNativeLondon

 

Do you like this sort of Stuff? Please support me on Patreon and you'll get 100 free cloud credits in heaven. Know anyone looking for a simple book explaining the cloud? Then please recommend my well reviewed book: Explain the Cloud Like I'm 10. They'll love it and you'll be their hero forever.

 

  • $2 billion: Pokémon GO revenue since launch; 10: say happy birthday to StackOverflow; $148 million: Uber data breach fine; 75%: streaming music industry revenue in the US;  5.2 TB: Fastly peak per second traffic; 10 billion: Ethereum requests per day; 01%: DNS resolution issues when the KSK rolls; 15B: projected gaming community views on Reddit; £4.1bn: saved by UK Government's Digital Transformation Journey; 10X: Core ML model runs  faster on the A12 processor; 4 million: cores managed by Open Stack at Yahoo; 1PB: Azure's data box; 21 million: US Apple music subscribers; .675: Curry's league leading true shooting percentage;  $3 trillion: taxes collected Continue reading

Weekend Reads 092818

Windows and Linux users need to beware, as an all-in-one, destructive malware strain has been discovered in the wild that features multiple malware capabilities including ransomware, cryptocurrency miner, botnet, and self-propagating worm targeting Linux and Windows systems. —Swati Khandelwal @The Hacker News

According to IDC Research’s recent US DDoS Prevention Survey, more than 50% of IT security decision makers said that their organization had been the victim of a distributed denial-of-service (DDoS) attack as many as 10 times in the past year. For those who experienced an attack, more than 40% lasted longer than 10 hours. This statistic correlates with our ATLAS findings, which show there were 7.5 million DDoS attacks in 2017 — a rate, says Cisco, that is increasing at roughly the same rate as Internet traffic. —Carlos Morales @Dark Reading

As the topic of hacking back continues to resurface among elected officials, those of us in the cybersecurity community are scratching our heads over why this concept refuses to die. After digging deeper, one can see that there are many misperceptions regarding what the terms “hacking back” and “active cyber defense” (ACD) actually mean. General frustration and misinformation are driving the interest, but the mixing of Continue reading

Make your Ansible Playbooks flexible, maintainable, and scalable

Extending-Ansible-PlaybookSince starting my journey using Ansible in 2013, I've built Ansible Playbooks to automate many things: SaaS products, a cluster of Raspberry Pi's, a home automation system, even my own computers!

In the years since, I've learned a lot of tricks to help ease the maintenance burden for my work. It's important to me to have maintainable projects, because many of my projects—like Hosted Apache Solr—have been in operation for over a decade! If it's hard to maintain the project or it's hard to make major architecture changes, then I can lose customers to more nimble competitors, I can lose money, and—most importantly—I can lose my sanity!

I'm presenting a session at AnsibleFest Austin this year, Make your Ansible Playbooks flexible, maintainable, and scalable, and I thought I'd summarize some of the major themes here.

Stay Organized

45982928-455cdb80-c020-11e8-96e4-833efbac87f4

I love photography and automation, and so I spend a lot of time building electronics projects that involve Raspberry Pis and cameras. Without the organization system I use (part of it pictured above), it would be very frustrating putting together the right components for my project.

Similarly, in Ansible, I like to have my tasks organized so I can compose them more Continue reading

Outing Your Outages

How are you supposed to handle outages? What happens when everything around you goes upside down in an instant? How much communication is “too much”? Or “not enough”? And is all of this written down now instead of being figured out when the world is on fire?

Team Players

You might have noticed this week that Webex Teams spent most of the week down. Hard. Well, you might have noticed if you used Microsoft Teams, Slack, or any other messaging service that wasn’t offline. Webex Teams went offline about 8:00pm EDT Monday night. At first, most people just thought it was a momentary outage and things would be back up. However, as the hours wore on and Cisco started updating the incident page with more info it soon became apparent that Teams was not coming back soon. In fact, it took until Thursday for most of the functions to be restored from whatever knocked them offline.

What happened? Well, most companies don’t like to admit what exactly went wrong. For every CloudFlare or provider that has full disclosures on their site of outages, there are many more companies that will eventually release a statement with the least amount of technical Continue reading

Push for Greater Control Over the Internet Coming Back Around

A group of countries will likely try to resurrect old battles on international control of Internet in the coming months, during upcoming meetings related to Internet Governance, some experts say.

The effort to relitigate unresolved debates on government control over the Internet will likely come up during the International Telecommunication Union’s Plenipotentiary Conference starting Oct. 29 in Dubai, said Robert Morgus, senior policy director focused on cybersecurity at U.S. think tank New America.

Morgus expects Russia, China, and other countries to renew their push for new internationally sanctioned controls over the Internet during the ITU meeting, he said Thursday at an Internet governance discussion hosted by New America and co-sponsored by the Internet Society’s Washington Chapter.

While the ITU has traditionally stayed away from Internet policy decisions, the group of authoritarian countries will likely push for a new World Conference on International Telecommunications (WCIT) meeting, Morgus said, where Internet control and governance issues have been hot topics.

The last WCIT meeting, in December 2012, ended with the United States, the U.K., Japan, and a handful of other countries declining to sign an agreement supported by 89 nations that called for international cooperation in fighting security problems Continue reading

Building With Workers KV, a Fast Distributed Key-Value Store

Building With Workers KV, a Fast Distributed Key-Value Store
Building With Workers KV, a Fast Distributed Key-Value Store

Your Workers now have access to a low-latency key-value data store which lives inside our network all around the world!

For those who don’t know, Cloudflare Workers is a new type of compute platform, built on top of our global network of 152+ data centers around the world. It allows you to write serverless code which runs in the fabric of the Internet itself, allowing you to engage with your users faster than other platforms can even get a packet to where your code is running. It’s built on a new architecture which eliminates cold starts and dramatically reduces the memory overhead of keeping your code running when compared to a platform like Amazon Lambda.

As powerful as this is, compute is just one component of what you need to build an application, you also need the ability to store data. We evaluated many of the available open source data stores on the market, but ultimately nothing was designed for a world with quite as many distributed nodes as our network. Instead, we have begun releasing our own vision for distributed storage, beginning today.

The Workers KV is a highly distributed, eventually-consistent, key value store. It will allow you to Continue reading

Introducing Workers KV

Introducing Workers KV

In 1864 British computer pioneer Charles Babbage described the first key-value store. It was meant to be part of his Analytical Engine. Sadly, the Analytical Engine, which would have been the first programmable computer, was never built. But Babbage lays out clearly the design for his key-value store in his autobiography. He imagined a read-only store implemented as punched cards. He referred to these as Tables:

I explained that the Tables to be used must, of course, be computed and punched on cards by the machine, in which case they would undoubtedly be correct. I then added that when the machine wanted a tabular number, say the logarithm of a given number, that it would ring a bell and then stop itself. On this, the attendant would look at a certain part of the machine, and find that it wanted the logarithm of a given number, say of 2303. The attendant would then go to the drawer containing the pasteboard cards representing its table of logarithms. From amongst these he would take the required logarithmic card, and place it in the machine.

Introducing Workers KV

Punched card illustration from Babbage’s autobiography showing an integer key (2303) and value representing the decimal part of Continue reading

Setting up the Kubernetes AWS Cloud Provider

The AWS cloud provider for Kubernetes enables a couple of key integration points for Kubernetes running on AWS; namely, dynamic provisioning of Elastic Block Store (EBS) volumes and dynamic provisioning/configuration of Elastic Load Balancers (ELBs) for exposing Kubernetes Service objects. Unfortunately, the documentation surrounding how to set up the AWS cloud provider with Kubernetes is woefully inadequate. This article is an attempt to help address that shortcoming.

More details are provided below, but at a high-level here’s what you’ll need to make the AWS cloud provider in Kubernetes work:

  • The hostname of each node must match EC2’s private DNS entry for that node
  • An IAM role and policy that EC2 instances can assume as an instance profile
  • Kubernetes-specific tags applied to the AWS resources used by the cluster
  • Particular command-line flags added to the Kubernetes API server, Kubernetes controller manager, and the Kubelet

Let’s dig into these requirements in a bit more detail.

Node Hostname

It’s important that the name of the Node object in Kubernetes matches the private DNS entry for the instance in EC2. You can use hostnamectl or a confiugration management tool (take your pick) to set the instance’s hostname to the FQDN that matches the EC2 Continue reading

Back By Popular Demand – Docker Pals Is Coming To Barcelona!

At DockerCon Copenhagen we launched the Docker Pals program in order to connect attendees and help them make the most out of their trip to DockerCon. Attending a conference by yourself can be intimidating and we don’t want anyone to feel that way at DockerCon! Pals get matched with a few others who are new (the “Pals”), and someone who knows their way around (the “Guide”) so that you will know someone before you arrive at the conference. So, DockerCon veterans, please consider signing up to be a Guide and help welcome those newer to DockerCon to the amazing Docker community. Participating gives you the opportunity to learn even more, grow an even bigger network, and have even more fun!

Here’s what Pals had to say:

“Docker Pals was an excellent opportunity to meet new Docker Captains and Community Leaders who are open to engaging with container enthusiasts of all skill levels, specialities and backgrounds. I would certainly take advantage of the program again, and volunteer to be a Guide next year.” – Jackie Liu

“I was able to learn and understand how Docker is used in real time and in production with my fellow Docker Pal.” – Continue reading

Columnstore and B+ tree – are hybrid physical designs important?

Columnstore and B+ tree – are hybrid physical designs important? Dziedzic et al., SIGMOD’18

Earlier this week we looked at the design of column stores and their advantages for analytic workloads. What should you do though if you have a mixed workload including transaction processing, decision support, and operational analytics? Microsoft SQL Server supports hybrid physical design combining both column store and B+ tree indexes in the same database.

It is generally understood that columnstores are crucial to achieving high performance for analytic queries and that B+ tree indexes are key to supporting transactional workloads efficiently. However, it is not well understood whether hybrid physical designs – both columnstore and B+ tree indices on the same database and potentially the same table – are important for any of the above workloads.

Through a series of benchmarks the authors show that hybrid physical designs can result in more than an order of magnitude lower execution costs for many workloads when compared to alternatives using B+ tree-only or columnstore-only. The Database Engine Tuning Advisor (DTA) for SQL Server is extended to analyze and recommend the appropriate indices for a given workload. Support for columnstore indices and the new DTA functionality was Continue reading

BrandPost: It’s Time to Think Outside the Router

“Turn Me Loose” I finally delivered four large boxes of CDs to my local library – my collection of nearly 1,000 titles, some as old as 33 years, like my Loverboy CD, when I acquired them in back in 1985. And back in 1985, or maybe 1986, I did a similar uncluttering with my cassettes and vinyl (except for a few select albums like an original release of Dark Side of the Moon).Music has gone from 12” vinyl to more compact cassette tapes to even more compact CDs and now streaming services that connect listeners to music anywhere, anytime. Just as music technology has changed through the years to provide more flexibility on how music is consumed, the application consumption model has changed, too. The “job of the WAN” has always been to connect users to applications, but wide area networking (WAN) technology based on conventional branch routers has not kept up to provide the optimal means of connecting to them.To read this article in full, please click here

1 … 1,404 1,405 1,406 1,407 1,408 … 3,794