Characterizing, modeling, and benchmarking RocksDB key-value workloads at Facebook

Characterizing, modeling, and benchmarking RocksDB key-value workloads at Facebook, Cao et al., FAST’20

You get good at what you practice. Or in the case of key-value stores, what you benchmark. So if you want to design a system that will offer good real-world performance, it’s really useful to have benchmarks that accurately represent real-world workloads. In this paper, Facebook analyse three of their real-world RocksDB workloads and find (surprise!) that they all look quite different. More interesting, is that there are important differences between the way these real-world workloads behave, and workloads generated by the venerable YCSB benchmark.

Therefore, using the benchmarking results of YCSB as guidance for production might cause some misleading results… To address this issue, we propose a key-range based modelling and develop a benchmark that can better emulate the workloads of real-world key-value stores. This benchmark can synthetically generate more precise key-value queries that represent the reads and writes of key-value stores to the underlying storage system.

The tracing, replay, and analysis tools developed for this work are released in open source as part of the latest RocksDB release, and the new benchmark is now part of the db_bench benchmarking tool.

TL;DR Continue reading

Data-center power consumption holds steady

A predicted explosion in power consumption by data centers has not manifested thanks to advances in power efficiency and, ironically enough, the move to the cloud, according to a new report.The study, published in the journal Science last week, notes that while there has been an increase in global data-center energy consumption over the past decade, this growth is negligible compared with the rise of workloads and deployed hardware during that time.Data centers accounted for about 205 terawatt-hours of electricity usage in 2018, which is roughly 1% of all electricity consumption worldwide, according to the report. (That's well below the often-cited stat that data centers consume 2% of the world's electricity). The 205 terawatt-hours represent a 6% increase in total power consumption since 2010, but global data center compute instances rose by 550% over that same time period.To read this article in full, please click here

Replicating Elastic File System With AWS DataSync

I recently used AWS DataSync as part of a lab I was building. These are my notes for using DataSync to replicate an Amazon Elastic File System (EFS) share from one region to another.

AWS DataSync is a managed service that enables replication of data between AWS services and from on-prem to AWS. It automates the scheduling of transfer activities, validates copied data, and uses a purpose-built network protocol and multi-threaded architecture to achieve very high efficiency on the wire.

The use case I needed to tackle was replicating an Amazon EFS share in one region to an EFS share in a different region (a one-way replication). (DataSync can also connect to Amazon S3 and Amazon FSx for Windows File Server)

Daily Roundup: Cisco Dreams of 5G WAN

Cisco dreamed of 5G WAN; VMware's modern application strategy spun Kubernetes Goldilocks tale; and...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Docker Refocus Targets DevOps Challenges

The vendor has a unique opportunity to help make the code to cloud space incredibly efficient,...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

VMware Tanzu Service Mesh, built on VMware NSX is Now Available!

VMware marked its entry into the service mesh space with the announcement of VMware NSX Service Mesh. Today, we have some exciting developments to share. First, VMware NSX Service Mesh is now VMware Tanzu Service Mesh. This new brand aligns with the VMware Tanzu Portfolio for modern applications that we launched today. Second, and more importantly, we are announcing that Tanzu Service Mesh, built on VMware NSX is now available for purchase.

Tanzu Service Mesh provides consistent connectivity and security for microservices – across all your Kubernetes clusters and clouds – in the most demanding multi-cluster and multi-cloud environments. Tanzu Service Mesh can be installed in Tanzu Kubernetes Grid (TKG) clusters and third-party Kubernetes-conformant clusters, and used with clusters managed by Tanzu Mission Control (i.e., Tanzu-managed clusters) or clusters managed by other Kubernetes platforms and managed services.

 

Beyond its multi-cloud focus, one of the other differentiating characteristics of Tanzu Service Mesh is its ability to support cross-cluster and cross-cloud use cases via Global Namespaces (GNS). A GNS abstracts an application from the underlying Kubernetes cluster namespaces and networking, allowing you to transcend infrastructure limitations and boundaries, and securely stretch applications across clusters and clouds. Global Namespaces allow Continue reading

Rakuten Mobile Plays the (Vendor) Field

A series of deals indicates that incumbent vendors like Nokia still have an important, and...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Docker testbed

The sFlow-RT real-time analytics platform receives a continuous telemetry stream from sFlow Agents embedded in network devices, hosts and applications and converts the raw measurements into actionable metrics, accessible through open APIs, see Writing Applications.

Application development is greatly simplified if you can emulate the infrastructure you want to monitor on your development machine. Mininet flow analytics, Mininet dashboard, and Mininet weathermap describe how to use the open source Mininet network emulator to simulate networks and generate a live stream of standard sFlow telemetry data.

This article describes how to use Docker containers as a development platform. Docker Desktop provides a convenient method of running Docker on Mac and Windows desktops. These instructions assume you have already installed Docker.

Start a Host sFlow agent using the pre-built sflow/host-sflow image: 
docker run --rm -d -e "COLLECTOR=host.docker.internal" -e "SAMPLING=10" \
--net=host -v /var/run/docker.sock:/var/run/docker.sock:ro \
--name=host-sflow sflow/host-sflow
Note: Host, Docker, Swarm and Kubernetes monitoring describes how to deploy Host sFlow agents to monitor large scale container environments.

Start an iperf3 server using the pre-built sflow/iperf3 image: 
docker run --rm -d -p 5201:5201 --name iperf3 sflow/iperf3 -s
In a separate terminal window, run the following command to start sFlow-RT: 
 Continue reading

Stateless Taps Intel’s Barefoot Tofino for Interconnection

Tofino’s P4 programmability is what makes it important to Stateless, a startup targeting...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Now Available: Calico for Windows on Red Hat OpenShift Container Platform

Approximately one year ago, Kubernetes 1.14 made support of Windows containers running on Microsoft Windows Server nodes generally available. This was a declaration that Windows node support was stable, well-tested, and ready for adoption, meaning the vast ecosystem of Windows-based applications could be deployed on the platform.

Collaborating with Microsoft, Tigera leveraged the new Windows platform capabilities to create Calico for Windows, the industry’s first cross-platform Kubernetes solution to manage networking and network policy for Kubernetes deployments on Windows and Linux.

We are excited to announce that Calico for Windows now supports the latest Windows Dev Preview on the Red Hat OpenShift Container Platform (OCP). Built on Red Hat Enterprise Linux and Kubernetes, OCP v4 provides developers and IT organizations with a hybrid and multi-cloud application platform for deploying both new and existing applications on scalable resources, with minimal configuration and management overhead. OCP enables organizations to meet security, privacy, compliance, and governance requirements.

Calico for Windows is the only Kubernetes networking solution for teams using Windows on OpenShift. The combination of Calico for Windows and Red Hat OpenShift Container Platform represents a major leap forward in productivity for organizations that are deploying Windows on Kubernetes. DevOps teams Continue reading

COVID-19 Trade Show Impact

SDxCentral will be keeping a running list of changes to trade shows and events tied to the COVID-19...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Helping You and Your Development Team Build and Ship Faster

I remember the first time one of my co-workers told me about Docker. There is a longer story behind it, but it ended with “it was so easy and saved me so much time.” That compelled me to install Docker and try it for myself. Yup, she was right. Easy, simple, efficient. Sometime later, at a conference, while catching up with some friends who are developers, I asked them “how are things going?” The conversation eventually led to the topic of where things are going in the container space. I asked, “what’s the biggest issue you are having right now?” I expected the response to be something Kubernetes related. I was surprised the answer was “managing all the tech that gets my code deployed and running.” 

The above sentiment is echoed by our CEO, Scott Johnston, in this post. Millions of you use Docker today (check out the Docker Index for the latest usage stats), and we are so very thankful for the vibrant Docker Community. We heard from you that easily going from code to cloud is a problem, and Scott outlined the complexities. There are many choices across packaging, inner loop, packaging, registry, Continue reading

Tech Field Day #TFD21

We are expecting another action packed day and of course it will be streamed live from this blog. Don’t worry if the timings for the live event don’t work for you. We’ll record each session and embed here for easy Ondemand viewing.

The theme for the day is Modernize, Connect and Manage your Network. With representatives from across VMware including Cloud Foundation, vSAN, NSX and vRealize Network Insight (vRNI) we have all our bases covered.

Here is the latest agenda:

 

 

 

 

 

‘

 

 

 

 

 

 

 

 

 

 

 

 

The post Tech Field Day #TFD21 appeared first on Network Virtualization.

Cisco Live US 2020: To CLUS or not to CLUS? [updated]

CLUS-2020

Update March 16, 2020: The in-person event is “relaunched” as an online event. Here is a copy of the information email I just received: The health and well-being of our customers, partners, employees and communities is of utmost importance to us. As a result, during this unprecedented time of the COVID-19 pandemic, Cisco Live, our premier in-person customer and partner experience of the year, is being relaunched as a complimentary, full-scale digital event, enabling remote participation from anywhere in the world. We’re dedicated to making sure that the experience at…

The post Cisco Live US 2020: To CLUS or not to CLUS? [updated] appeared first on AboutNetworks.net.

I Hate Excellent Questions

I was listening to a recent episode of the Packet Pushers Podcast about SD-WAN and some other stuff. At one point, my good friend Greg Ferro (@EtherealMind) asked the guest something, and the guest replied with, “That’s an excellent question!” Greg replied with, “Of course it was. I only ask excellent questions.” I was walking and laughed out loud harder than I’ve laughed in a long time.

This was also a common theme during Networking Field Day. Everyone was asking “great” or “excellent” questions. I chuckled and told the delegates that it was a canned response that most presenters give today. But then I wondered why all our questions are excellent. And why I hated that response so much.

Can You Define “Excellent”?

The first reason why I think people tend to counter with “excellent” praise is because they are stalling for an answer. It’s a time-honored tradition from spelling bees when you don’t know how to spell the word and you need a few more seconds to figure out if this is one of those “i before e” words or not. I get the purpose of defining something of non-native speaker origin. But defining a Continue reading

How Replicated Developers Develop Remotely

How Replicated Developers Develop Remotely

This is a guest post by Marc Campbell and Grant Miller, co-founders of Replicated.

How Replicated Developers Develop Remotely

Replicated is a 5-year old infrastructure software company working to make it easy for businesses to install and operate third party software. We don’t want you to have to send your data to a multi-tenant SaaS provider just to use their services. Our team is made up of twenty-two people distributed throughout the US. One thing that’s different about Replicated is our developers don’t actually store or execute code on their laptops; all of our development happens on remote instances in the cloud.

Our product, KOTS, runs in Kubernetes and manages the lifecycle of 3rd-party applications in the Kubernetes cluster. Building and validating the product requires a developer to have access to a cluster. But as we started to hire more and more engineers it became ridiculous to ask everyone to run their own local Kubernetes cluster. We needed to both simplify and secure our setup to allow every engineer to run their environment in the cloud, and we needed to do it in a way which was seamless and secure.

Previous Dev Environments with Docker for Mac

We started with each developer building Continue reading

The Stupidity of Trying to Be Like Google

Someone recommended me a fantastic book on corporate stupidity. Here’s just one of the million small gems it contains:

For instance, many companies conclude that they need to be more innovative. To increase their rates of innovation, they look at firms well known for being innovative, such as Google, then dispatch their executives to Silicon Valley to visit tech companies’ corporate campuses in the hope that they will learn something.

Not surprisingly, the book authors observed the same behavior in those companies as I did a while ago when I was still teaching SDN workshops:

They often ignore the fact that Google is an entirely different sector to them, and the lessons in view probably of limited value. They also overlook that even if they do learn something, actually implementing it within their organization is likely to be difficult, if not impossible.

Finally a warning: that book will make you laugh or cry hysterically (or both), so take it in small daily doses.

1 … 1,060 1,061 1,062 1,063 1,064 … 3,895