Design continuums and the path toward self-designing key-value stores that know and learn

Design continuums and the path toward self-designing key-value stores that know and learn Idreos et al., CIDR’19

We’ve seen systems that help to select the best data structure from a pre-defined set of choices (e.g. ‘Darwinian data structure selection’), systems that synthesise data structure implementations given an abstract specification (‘Generalized data structure synthesis’), systems that use learning to tune configurations given a pre-defined set of configuration options (e.g, ‘BOAT: Building auto-tuners with structured Bayesian optimization’), systems that use learned models as key inputs to algorithms (e.g. ‘SageDB: a learned database system’), and systems that use reinforcement learning to discover fit-for-workload policies (‘Towards a hands-free query optimizer through deep learning’). Today’s paper choice jumps right into the middle of this mix, showing how families of data structures can be organised into design continuums with well-understood design parameters and cost models informing selection of a given data-structure design from within the space. As well as helping us to understand and reason about data structures within the space, the constrained feature space with quick-to-evaluate cost models opens the way to practical machine learning guided data structure synthesis and selection. Continue reading

New Year, New Home

We have left the Bay Area, and headed North. We have moved to the Greater Seattle area - specifically the Eastside, between Bellevue and Redmond. We’ve given up the old apartment in San Francisco for a larger, nicer house…for a lot less in rent. A lot fewer bars & restaurants, a lot more trees, parks and lakes.

But Why?

The typical Bay Areas response is: “But why??? It rains all the time in the Pacific Northwest!!!!”

A few things:

1. Yes, it rains more here than San Francisco, but not as much as people think. It’s not even in the top 10 cities in the US for annual rainfall. Boston, New York, Washington DC all receive more.

2. Rain is OK. In fact rain is good. You don’t get lush forests through irrigation. You also don’t get clean streets just from street sweepers. 

The main attractions for us are:

  • Much better lifestyle for us. It’s easy to go mountain biking, running, hiking, skiing here.

  • Much lower rent. Yes, rents have gone up a lot here, but it’s still much better value than San Francisco. I pay much less rent here, but I get a nice place, and the Continue reading

New Year, New Home

We have left the Bay Area, and headed North. We have moved to the Greater Seattle area - specifically the Eastside, between Bellevue and Redmond. We’ve given up the old apartment in San Francisco for a larger, nicer house…for a lot less in rent. A lot fewer bars & restaurants, a lot more trees, parks and lakes.

But Why?

The typical Bay Areas response is: “But why??? It rains all the time in the Pacific Northwest!!!!”

A few things:

1. Yes, it rains more here than San Francisco, but not as much as people think. It’s not even in the top 10 cities in the US for annual rainfall. Boston, New York, Washington DC all receive more.

2. Rain is OK. In fact rain is good. You don’t get lush forests through irrigation. You also don’t get clean streets just from street sweepers. 

The main attractions for us are:

  • Much better lifestyle for us. It’s easy to go mountain biking, running, hiking, skiing here.

  • Much lower rent. Yes, rents have gone up a lot here, but it’s still much better value than San Francisco. I pay much less rent here, but I get a nice place, and the Continue reading

New Year, New Home

We have left the Bay Area, and headed North. We have moved to the Greater Seattle area - specifically the Eastside, between Bellevue and Redmond. We’ve given up the old apartment in San Francisco for a larger, nicer house…for a lot less in rent. A lot fewer bars & restaurants, a lot more trees, parks and lakes.

But Why?

The typical Bay Areas response is: “But why??? It rains all the time in the Pacific Northwest!!!!”

A few things:

1. Yes, it rains more here than San Francisco, but not as much as people think. It’s not even in the top 10 cities in the US for annual rainfall. Boston, New York, Washington DC all receive more.

2. Rain is OK. In fact rain is good. You don’t get lush forests through irrigation. You also don’t get clean streets just from street sweepers. 

The main attractions for us are:

  • Much better lifestyle for us. It’s easy to go mountain biking, running, hiking, skiing here.

  • Much lower rent. Yes, rents have gone up a lot here, but it’s still much better value than San Francisco. I pay much less rent here, but I get a nice place, and the Continue reading

Docker and Netdata – Awesome for Monitoring our home servers !

I don’t belong to any DevOps space but I keep hearing things like Docker / kubernetes and what not. I Quickly wanted to see what these are capable of and if I can use them to my advantage so that later I can see the use case for networking.

Docker so far seems to be far more capable and am enjoying it. Thanks to one of my friends who suggested this wonderful Repo, monitoring my server has been more granular.

I will have a detailed post on Juniper MX image via docker but for now, I have used it for something out of networking space to serve a small purpose.

https://github.com/netdata/netdata

This what Netdata UI looks like while monitoring my server, this is really wonderful as for the long time  I was trying to implement many Monitoring systems and most of them required some dedicated hardware or at least a Vmware spin-off instance, which is fine for me I suppose but I wanted something small and yet effective.

Netdata operates on port 19999 of localhost and should be reachable via any web-browser.

I made a small cronjob which starts this instance on every reboot.

Installation is pretty straight forward, Continue reading

SUSE releases enterprise Linux for all major ARM processors

SUSE has released its enterprise Linux distribution, SUSE Linux Enterprise Server (SLES), for all major ARM server processors. It also announced the general availability of SUSE Manager Lifecycle.SUSE is on par with the other major enterprise Linux distributions — Red Hat and Ubuntu — in the x86 space, but it has lagged in its ARM support. It’s not like SLES for ARM is only now coming to market for the first time, either. It has been available for several years, but on a limited basis. [ Two-Minute Linux Tips: Learn how to master a host of Linux commands in these 2-minute video tutorials ] "Previously, SUSE subscriptions for the ARM hardware platforms were only available to SUSE Partners due to the relative immaturity of the ARM server platform," Jay Kruemcke, a senior product manager at SUSE, wrote in a blog post announcing the availability.To read this article in full, please click here

Linux systems: Scraping up information about apt updates

When you use the apt command to install new packages or run routine upgrades on your Debian-based Linux system, you might wonder where information about your activities are being recorded. For one, your history file probably retains information on the commands that you use, though history files like ~/.bash_history will only keep the most recent commands that you've run — depending on your $HISTSIZE setting — and generally will not include dates and times. There is, however, another place to find information about apt commands and that place is /var/log/apt.The /var/log/apt directory contains a number of log files — the history.log file, plus a series of older versions of the file named history.log.1.gz, history.log.2.gz, history.log.3,gz and so on. Each of these logs will contain information on apt commands that have been run within a particular timeframe.To read this article in full, please click here

Huawei introduces AI-driven data center switch

Chinese telecom giant Huawei introduced a new data center switch powered by an artificial intelligence (AI) chip designed to improve performance and reduce latency to near zero. The new switch follows the announcement of a 64-core ARM server processor.The CloudEngine 16800 series of data center switches use AI to improve network operations and also provide an underlying network foundation for companies to build new apps that utilize AI for network performance.Huawei claims the CloudEngine 16800 is the first data center switch use an embedded AI chip, using the iLossless algorithm to implement auto-sensing and auto-optimization of the traffic model, thereby lowering latency and providing higher throughput based on zero packet loss.To read this article in full, please click here

Huawei introduces AI-driven data center switch

Chinese telecom giant Huawei introduced a new data center switch powered by an artificial intelligence (AI) chip designed to improve performance and reduce latency to near zero. The new switch follows the announcement of a 64-core ARM server processor.The CloudEngine 16800 series of data center switches use AI to improve network operations and also provide an underlying network foundation for companies to build new apps that utilize AI for network performance.Huawei claims the CloudEngine 16800 is the first data center switch use an embedded AI chip, using the iLossless algorithm to implement auto-sensing and auto-optimization of the traffic model, thereby lowering latency and providing higher throughput based on zero packet loss.To read this article in full, please click here

Weekend Reads 011819

Optimism around 5G may be strong for now, but service providers are finding it difficult to convey a feasible business case for the technology as average consumers remain ill-informed of its true benefits. 61% of respondents were hopeful about the overall availability of 5G coming in the next two years, however, 40% don’t believe consumers will be willing to pay more for the privilege. —Sigal Biran-Nagar

DNS security from the client perspective is a one-hour online course examining both technical and managerial topics associated with the security of the Domain Name System. Participants will learn about the process of a domain name and DNS request, from registration to name resolution; the security risks of each component; and the mitigation options currently available. —Lisa Corness

Your job search can feel like a movie you’ve seen over and over. The same thing seems to happen every time. You get motivated to finally start looking for a new job. You hunt around the internet and make a list of intriguing jobs. You start imagining yourself getting out of your current job. You start applying, there’s progress, you get some call backs, maybe you even go on-site to interview a few times. There’s a Continue reading

Understanding the JunOS routing table

I was just about to finish another blog post on MPLS when I got a question from a colleague about Junos routing tables. He was confused as to how to interpret the output of a basic Juniper routing table. I spent some time trying to find some resource to point him at – and was amazed at how hard it was to find anything that answered his questions specifically. Sure, there are lots of blogs and articles that explain RIB/FIB separation, but I couldn’t find any that backed it up with examples and the level of detail he was looking for. So while this is not meant to be exhaustive – I hope it might provide you some details about how to interpret the output of some of the more popular show commands. This might be especially relevant for those of you who might be coming from more of a Cisco background (like myself a number of years ago) as there are significant differences between the two vendors in this area.

Let’s start with a basic lab that looks a lot like the one I’ve been using in the previous MPLS posts…

For the sake of focusing on the real Continue reading

The SamKnows Cloudflare Platform

The SamKnows Cloudflare Platform

This is a guest post by Jamie Mason, who is the Head of Test Servers at SamKnows. This post originally appears on the SamKnows Megablog.

The SamKnows Cloudflare Platform

We leveraged Cloudflare Workers to expand the SamKnows measurement infrastructure.

At SamKnows, we run lots of tests to measure internet performance. Actually, that’s an understatement. Our software is embedded on tens of millions of devices, and that number grows daily.

The SamKnows Cloudflare Platform

We measure performance between the user’s home and the internet, across dozens of metrics. Some of these metrics measure the performance of major video-streaming services, popular games, or large websites. Others focus on the more traditional ‘quality of service’ metrics: speed, latency, and packet loss.

In order to measure speed, latency, and packet loss, SamKnows needs test servers to carry out the measurements against. These servers should be relatively near to the user’s home - this ensures that we’re measuring solely the user’s internet connection (i.e. what their Internet Service Provider sells them) and not some external factor.

As a result, we manage high-capacity test servers all over the world. Some are donated by research groups, some we host ourselves in major data centers, and still others are run inside ISPs’ own networks.

Customers Continue reading

Consumer Electronics Show: Everything’s Connected, But What About Security and Privacy?

We spent last week at the Consumer Electronics Show (aka CES) in Las Vegas, with over 180,000 of our closest friends. And with 4,500 exhibitors present, you’d have less than 30 seconds at each booth if you wanted to talk to all of them. Many articles have covered the cool new things, so in this blogpost we are going to discuss our overall impressions as they relate to our work on consumer IoT security and privacy.

Not surprisingly, there were many interesting conference sessions and a wide variety of innovative products on display, including some that seemed to push the bounds of credibility in their claims. Integration of devices with voice-driven and other platforms was everywhere – Amazon Alexa, Google Assistant, Apple HomeKit, and Samsung SmartThings being the most widely adopted to date. 5G was a hot topic, especially for its improved speeds and flexibility, though specifics about its availability are still hard to pin down.

Everything these days is getting connected to the Internet – from cat toys to sports simulators to home automation. One area that seems to be gaining more traction because it has gone beyond the “gadget” stage and is solving real problems is health and Continue reading

Stuff The Internet Says On Scalability For January 18th, 2019

Sorry, Stuff The Internet Says On Scalability has been called on the account of wind, rain, power outages and general mayhem. We're all safe, but it's hard to write a post using stone knives and bear skins. See you next week.

 

 

1 … 1,361 1,362 1,363 1,364 1,365 … 3,860