Sharding the shards: managing datastore locality at scale with Akkio

Sharding the shards: managing datastore locality at scale with Akkio Annamalai et al., OSDI’18

In Harry Potter, the Accio Summoning Charm summons an object to the caster of the spell, sometimes transporting it over a significant distance. In Facebook, Akkio summons data to a datacenter with the goal of improving data access locality for clients. Central to Akkio is the notion of microshards (μ-shards), units of data much smaller than a typical shard. μ-shards are defined by the client application, and should exhibit strong access locality (i.e., the application tends to read/write the data in a μ-shard together in a small window of time). Sitting as a layer between client applications and underlying datastores, Akkio has been in production at Facebook since 2014, where it manages around 100PB of data.

Measurements from our production environment show that Akkio reduces latencies by up to 50%, cross-datacenter traffic by up to 50%, and storage footprint by up to 40% compared to reasonable alternatives.

Akkio can support trillions of μ-shards and many 10s of millions of data access requests per second.

Motivation

Our work in this area was initially motivated by our aim to reduce service response times and resource Continue reading

Working with distance sensor – solving overhead water tank problem

This is not a networking post.

Schematic , sensor code and spec  – https://www.linuxnorth.org/raspi-sump

My code – https://github.com/yukthr/auts/blob/master/random_programs/water_sensor.py

1x Breadboard

1x Raspberry pi zero w

1xhcsr04 ultrasonic sensor

2x1kohm resistors

 

Just as a side note i do not have any intro into resistors nor electronics, but what all i did was to follow some posts written by people who already did it, its not hard believe me, if i could do it any one should easily be able to do it as am very far away from electronics and programming, so let these things not overwhelm you.

 

Problem – Am not sure in other parts of the world, but place I live has an over head water Tank which stores water. So every day you technically turn on a water motor which sucks water from a reserve under the ground and pumps it to all the the way to a three store high building

So what’s the issue – The issue is that we have no clue what’s the current water level in the tank nor how long would it take to fill the water tank. There are two tribal ways by which we Continue reading

Brian Kemp is bad on cybersecurity

I'd prefer a Republican governor, but as a cybersecurity expert, I have to point out how bad Brian Kemp (candidate for Georgia governor) is on cybersecurity. When notified about vulnerabilities in election systems, his response has been to shoot the messenger rather than fix the vulnerabilities. This was the premise behind the cybercrime bill earlier this year that was ultimately vetoed by the current governor after vocal opposition from cybersecurity companies. More recently, he just announced that he's investigating the Georgia State Democratic Party for a "failed hacking attempt".


According to news stories, state elections websites are full of common vulnerabilities, those documented by the OWASP Top 10, such as "direct object references" that would allow any election registration information to be read or changed, as allowing a hacker to cancel registrations of those of the other party.

Testing for such weaknesses is not a crime. Indeed, it's desirable that people can test for security weaknesses. Systems that aren't open to test are insecure. This concept is the basis for many policy initiatives at the federal level, to not only protect researchers probing for weaknesses from prosecution, but to even provide bounties encouraging them to do so. Continue reading

IETF 103, Day 1: IPv6, TLS, DNS Privacy & Other Crypto

The Working Group sessions start tomorrow at IETF 103 in Bangkok, Thailand, and we’re bringing you daily blog posts highlighting the topics of interest to us in the ISOC Internet Technology Team. Only four days have been scheduled for the working groups this time around, which means there’s a lot of pack into each day; with Monday being no exception.

V6OPS is a key group and will be meeting on Monday morning starting at 09.00 UTC+7. It’s published four RFCs since its last meeting, including Happy Eyeballs v2, and this time will kick-off with a presentation on the CERNET2 network which is an IPv6-only research and education in China.

There’s also four drafts to be discussed, including three new ones. IPv6-Ready DNS/DNSSSEC Infrastructure recommends how DNS64 should be deployed as it modifies DNS records which in some circumstances can break DNSSEC. IPv6 Address Assignment to End-Sites obsoletes RFC 6177 with best current operational practice from RIPE-690 that makes recommendations on IPv6 prefix assignments, and reiterates that assignment policy and guidelines belong to the RIR community. Pros and Cons of IPv6 Transition Technologies for IPv4aaS discusses different use case scenarios for the five most prominent IPv4-as-a-service (IPv4aaS) transitional technologies, Continue reading

Ansible and Infoblox: Roles Deep Dive

Ansible_and_Infoblox

As Sean Cavanaugh mentioned in his earlier Infoblox blog post, the release of Ansible 2.5 introduced a lookup plugin, a dynamic inventory script, and five modules that allow for Infoblox automation. A combination of these modules and lookups in a role provides a powerful DNS automation framework.

Summary

Today we are going to demonstrate how automating Infoblox Core Network Services using Ansible can help make managing IP addresses and routing traffic across your network easy, quick, and reliable. Your network systems for virtualization and cloud require rapid provisioning life cycles; Infoblox helps you manage those lifecycles. When paired with Infoblox, Ansible lets you automate that work. Ansible’s integration with Infoblox is flexible and powerful: you can automate Infoblox tasks with modules or with direct calls to the Infoblox WAPI REST API.

This post will walk you through six real-world scenarios where Ansible and Infoblox can streamline your network tasks:

  1. Creating a provider in one place that is reusable across a collection of roles.
  2. Expanding your network by creating a new subnet with a forward DNS zone. Ansible modules for Infoblox make this common two-part task simple.
  3. Creating a reverse DNS zone, for example, to flag email from any Continue reading

Is Oracle’s silence on its on-premises servers cause for concern?

When Oracle consumed Sun Microsystems in January 2010, founder Larry Ellison promised new hiring and new investment in the hardware line, plus a plan to offer fully integrated, turnkey systems.By and large, he kept that promise. Oracle dispensed with the commodity server market in favor of high-end, decked-out servers such as Exadata and Exalogic fully loaded with Oracle software, which included Java.Earlier this year, word leaked that the company had gutted its Solaris Unix and Sparc processor development, but after eight years of spinning its wheels, no one could say Oracle had been impatient. It had invested rather heavily in Sparc for a long time, but the writing was on the wall.To read this article in full, please click here

Is Oracle’s silence on its on-premises servers cause for concern?

When Oracle consumed Sun Microsystems in January 2010, founder Larry Ellison promised new hiring and new investment in the hardware line, plus a plan to offer fully integrated, turnkey systems.By and large, he kept that promise. Oracle dispensed with the commodity server market in favor of high-end, decked-out servers such as Exadata and Exalogic fully loaded with Oracle software, which included Java.Earlier this year, word leaked that the company had gutted its Solaris Unix and Sparc processor development, but after eight years of spinning its wheels, no one could say Oracle had been impatient. It had invested rather heavily in Sparc for a long time, but the writing was on the wall.To read this article in full, please click here

BrandPost: What Will 802.11ax Bring To Your Airspace?

Aruba Tom Hollingsworth, Blog Contributor The industry is on the cusp of a new wireless protocol. It's been almost 10 years since 802.11ac was proposed, and five years since final ratification. 802.11ac has been built upon to deliver speeds past 1 Gpbs and has become the preferred method of wireless connectivity for computers and mobile devices alike.To read this article in full, please click here

Stuff The Internet Says On Scalability For November 2nd, 2018

Wake up! It's HighScalability time:

 

"The future is already here – it's just not evenly distributed." — William Gibson 

 

Do you like this sort of Stuff? Please support me on Patreon. I'd really appreciate it. Know anyone looking for a simple book explaining the cloud? Then please recommend my well reviewed (30 reviews on Amazon and 72 on Goodreads!) book: Explain the Cloud Like I'm 10. They'll love it and you'll be their hero forever.

 

  • $10 billion: Apple services revenue; 1.49B: Facebook daily active users; 34: cache sites for iOS rollout; 87M: paying Spotify users; 20k: Facebook's new large-scale dataset for video description as a new challenge for multi-sentence video description; 6 million: online court case dataset; 125 quadrillion: Sierra supercomputer calculations each second; 1500: per day automated chaos experiments run at Netflix; 12: neurons needed to park a car; 2025: boots on Mars; 600: free online courses; 94%: Continue reading

Docker Certified Logging Containers and Plugins from Partners

 

 

The Docker Certified Technology Program is designed for ecosystem partners and customers to recognize Containers and Plugins that excel in quality, collaborative support and compliance. Docker Certification gives organizations enterprises an easy way to run trusted software and components in containers on the Docker Enterprise container platform with support from both Docker and the publisher.  

In this review, we’re looking at Docker Logging Containers and Plugins. Docker Enterprise provides built-in logging drivers to help users get information from docker nodes, running containers and services. The Docker Engine also exposes a Docker Logging Plugin API for use by Partner Docker logging plugins. The user’s needs are solved by innovations from the extensive Docker ecosystem that extend Docker’s logging capabilities which provide complete log management solutions that include searching, visualizing, monitoring, and alerting.

These solutions are validated by both Docker and the partner company and integrated into a seamless support pipeline that provide customers the world class support they have become accustomed to when working with Docker.

Check out the latest certified Docker Logging Containers and Plugins that are now available from our partners on Docker Store:

1 … 1,389 1,390 1,391 1,392 1,393 … 3,820