Medea: scheduling of long running applications in shared production clusters

Medea: scheduling of long running applications in shared production clusters Garefalakis et al., EuroSys’18

(If you don’t have ACM Digital Library access, the paper can be accessed either by following the link above directly from The Morning Paper blog site).

We’re sticking with schedulers today, and a really interesting system called Medea which is designed to support the common real world use case of mixed long running applications (LRAs) and shorter duration tasks within the same cluster. The work is grounded in production cluster workloads at Microsoft and is now part of the Apache Hadoop 3.1 release. In the evaluation, when compared to the Kubernetes’ scheduling algorithm Medea reduces median runtimes by up to 32%, and by 2.1x compared to the previous generation YARN scheduler.

…a substantial portion of production clusters today is dedicated to LRAs…. placing LRAs, along with batch jobs, in shared clusters is appealing to reduce cluster operational costs, avoid unnecessary data movement, and enable pipelines involving both classes of applications. Despite these observations, support for LRAs in existing schedulers is rudimentary.

The challenges of scheduling long-running applications

Example uses of long running application containers include streaming systems, interactive data-intensive applications (maintaining Continue reading

Cumulus content roundup: June

It’s officially summer time, so we’re bringing you the HOTTEST new content from Cumulus Networks in this month’s content roundup! Whether you want to layer on the sunscreen and enjoy our content while basking in the sun, or stay safe and cool indoors with your laptop and AC, you’re bound to enjoy what we’ve got in store for you. We’ve got new videos and white papers, and even a brand new official Cumulus Networks podcast for you to check out!

New from Cumulus:

Kernel of Truth Episode 01 – Networking Automation: “Kernel of Truth” is a Cumulus Networks podcast dedicated to bringing the best of open networking thought leadership straight to your ears. Listen to our very first episode where we discuss network automation and its impact on the industry!

5 Network automation tips and tricks for NetOps: In this white paper, we’ll give you five tips and tricks to get clarity around your automation decisions and reduce any friction that may be inhibiting (further) adoption of network automation. Check it out!

Joint solution overview: OpenStack and Cumulus Networks: By combining with Cumulus Linux, you can unify the entire stack on Linux, bringing together the OpenStack servers Continue reading

IDG Contributor Network: Is the IT help desk model broken?

There’s no doubt about it – today’s workers have fully embraced the trend toward remote working. In fact, according to last year’s Gallup “State of the American Workplace” survey, roughly 43 percent of employees report they have worked remotely.  It would seem that the genie is out of the bottle, and it’s not likely to go back in without a fight.This mass migration off premises changes the dynamic between users and IT help desk teams. An operator can no longer run down the hall to ask a user “Can you show me what the problem is with your computer?” More importantly, without having total visibility in the cloud, the operator may be completely unable to ‘see’ any problems that users are experiencing as they work remotely.To read this article in full, please click here

Arista Runs Barefoot With Tofino Programmable Switch Chips

If Andy Bechtolsheim, the chief technology officer at datacenter switching upstart Arista Networks, wanted to design ASICs to try to take a bigger piece of the switch pie – or more precisely, thought that this was a good idea at all – rest assured, Arista would be spending money engineering its own chips and fighting for capacity at the four remaining foundries that have advanced processes. …

Arista Runs Barefoot With Tofino Programmable Switch Chips was written by Timothy Prickett Morgan at .

Short Take: Practical Career Advice

One problem I’ve heard in the past is that much of the career advice given in the networking world is not practical. In this short take, I take this problem on, explaining why it might be more practical than it initially seems.

Shell Scripts to Ansible

Shell-Scripts-to-Ansible

During a recent client visit, we were asked to help migrate the following script for deploying a centralized sudoers file to RHEL and AIX servers. This is a common scenario which can provide some good examples of leveraging advanced Ansible features. Additionally, we can consider the shift in approach from a script that does a task to describing and enforcing the state of an item idempotently.

Here is the script:

#!/bin/sh
# Desc: Distribute unified copy of /etc/sudoers
#
# $Id: $
#set -x

export ODMDIR=/etc/repos

#
# perform any cleanup actions we need to do, and then exit with the
# passed status/return code
#
clean_exit()
{
cd /
test -f "$tmpfile" && rm $tmpfile
exit $1
}

#Set variables
PROG=`basename $0`
PLAT=`uname -s|awk '{print $1}'`
HOSTNAME=`uname -n | awk -F. '{print $1}'`
HOSTPFX=$(echo $HOSTNAME |cut -c 1-2)
NFSserver="nfs-server"
NFSdir="/NFS/AIXSOFT_NFS"
MOUNTPT="/mnt.$$"
MAILTO="[email protected]"
DSTRING=$(date +%Y%m%d%H%M)
LOGFILE="/tmp/${PROG}.dist_sudoers.${DSTRING}.log"
BKUPFILE=/etc/sudoers.${DSTRING}
SRCFILE=${MOUNTPT}/skel/sudoers-uni
MD5FILE="/.sudoers.md5"

echo "Starting ${PROG} on ${HOSTNAME}" >> ${LOGFILE} 2>&1

# Make sure we run as root
runas=`id | awk -F'(' '{print $1}' | awk -F'=' '{print $2}'`
if [ $runas -ne 0 ] ; then
echo "$PROG: you must be root to run  Continue reading
1 … 1,566 1,567 1,568 1,569 1,570 … 3,836