Skyway: connecting managed heaps in distributed big data systems

Skyway: connecting managed heaps in distributed big data systems Nguyen et al., ASPLOS’18

Yesterday we saw how to make Java objects persistent using NVM-backed heaps with Espresso. One of the drawbacks of using that as a persistence mechanism is that they’re only stored in the memory of a single node. If only there was some way to create a cluster of JVMs, and efficiently copy objects across remote heaps in the cluster… Meet Skyway!

Skyway is aimed at JVM-based big data systems (think Spark, Flink) that end up spending a lot of their time serializing and deserializing objects to move them around the cluster (e.g., to and from workers – see ‘Making sense of performance in data analytics frameworks’). Java comes with a default serialization mechanism, and there are also many third party libraries. Kyro is the recommended library for use with Spark.

Consider a small Spark cluster (3 worker nodes each with a 20 GB heap) running a triangle counting algorithm over the LiveJournal graph (about 1.2GB). With both the standard Java serializers and Kyro, serialization and deserialization combined account for a significant portion of the overall execution time (more than 30%).

Where Continue reading

We’ve Added a New AWS Big Data Course to Our Video Library!

Cloud service providers like Amazon Web Services (AWS) are leading the way to cloud adoption across all industries, changing the way organizations interact with their data. Board rooms already know they need to move to the cloud so it’s important to understand the how’s and the why’s of your cloud journey. AWS has built and made available powerful tools and services to support the most demanding big data workloads. You no longer need to be a data scientist to take advantage of complex data analytics tools, as AWS has made these tools within reach both from a technical and cost perspective to the masses.


Why You Should Watch:

If you’d like to learn how to minimize the complexity of big data pipelines, this course is for you.
Growing your skills in big data and analytics positions you to help fill the massive skills gap that exist for this high wage in-demand job sector.


What You’ll Learn:

The AWS Certified Big Data Specialty course is designed to give students a solid foundation of AWS Services related to a big data pipeline. The hands-on exercises in this course demonstrates the speed of innovation that can be gained using the AWS Cloud services. Continue reading

Recruiting The Puppet Masters Of Infrastructure

All kinds of convergence is going on in infrastructure these days, with the mashing up of servers and storage or servers and networking, or sometimes all three at once. This convergence is not just occurring at the system hardware or basic system software level. It is also happening up and down the software stack, with a lot of codebases branching out from various starting points and building platforms of one kind or another.

Some platforms stay down at the server hardware level – think of the Cisco Systems UCS blade server, which mashes up servers and networking – while others …

Recruiting The Puppet Masters Of Infrastructure was written by Timothy Prickett Morgan at The Next Platform.

No, Ray Ozzie hasn’t solved crypto backdoors

According to this Wired article, Ray Ozzie may have a solution to the crypto backdoor problem. No, he hasn't. He's only solving the part we already know how to solve. He's deliberately ignoring the stuff we don't know how to solve. We know how to make backdoors, we just don't know how to secure them.


The vault doesn't scale

Yes, Apple has a vault where they've successfully protected important keys. No, it doesn't mean this vault scales. The more people and the more often you have to touch the vault, the less secure it becomes. We are talking thousands of requests per day from 100,000 different law enforcement agencies around the world. We are unlikely to protect this against incompetence and mistakes. We are definitely unable to secure this against deliberate attack.

A good analogy to Ozzie's solution is LetsEncrypt for getting SSL certificates for your website, which is fairly scalable, using a private key locked in a vault for signing hundreds of thousands of certificates. That this scales seems to validate Ozzie's proposal.

But at the same time, LetsEncrypt is easily subverted. LetsEncrypt uses DNS to verify your identity. But spoofing DNS is easy, as was recently shown in Continue reading

Next Market Transition ? Cheaper Buying, Less Selling

This article was first published in Human Infrastructure Magazine at PacketPushers.net. The cost selling products to the Enterprise is large. How large ? As a rough guide, 20% of a typical IT companies cost is sales and marketing. The number varies, more when a company has a push to grow quickly (Palo Alto Networks had […]

IDG Contributor Network: What networks and the Internet of Things are doing for health care

With budgets and resources becoming ever tighter, the health care industry, in common with many others, is undergoing a significant digital transformation. Advances in digital technology are proving to be a great benefit, enabling health care providers to do more with less.The ability to securely access a patient’s Electronic Medical Records (EMR) and their diagnostic test results in real time, from virtually any device, regardless of location, is now largely taken for granted. Not only does it improve a health care provider’s operational efficiency, but it can also allow for more accurate diagnoses and inform a patient’s ongoing treatment plan. Likewise, the introduction of e-prescriptions, and almost ubiquitous Wi-Fi connectivity throughout hospitals and doctors’ surgeries, have reduced the volume of administrative tasks, allowing frontline operatives to concentrate more on their primary task of delivering high-quality health care to their patients.To read this article in full, please click here

IDG Contributor Network: What networks and the Internet of Things are doing for health care

With budgets and resources becoming ever tighter, the health care industry, in common with many others, is undergoing a significant digital transformation. Advances in digital technology are proving to be a great benefit, enabling health care providers to do more with less.The ability to securely access a patient’s Electronic Medical Records (EMR) and their diagnostic test results in real time, from virtually any device, regardless of location, is now largely taken for granted. Not only does it improve a health care provider’s operational efficiency, but it can also allow for more accurate diagnoses and inform a patient’s ongoing treatment plan. Likewise, the introduction of e-prescriptions, and almost ubiquitous Wi-Fi connectivity throughout hospitals and doctors’ surgeries, have reduced the volume of administrative tasks, allowing frontline operatives to concentrate more on their primary task of delivering high-quality health care to their patients.To read this article in full, please click here

Another BGP Hijacking Event Highlights the Importance of MANRS and Routing Security

Another BGP hijacking event is in the news today. This time, the event is affecting the Ethereum cryptocurrency. (Read more about it here, or here.) Users were faced with an insecure SSL certificate. Clicking through that, like so many users do without reading, they were redirected to a server in Russia, which proceeded to empty the user’s wallet. DNSSEC is important to us, so please check out the Deploy360 DNSSEC resources to make sure your domain names are protected. In this post, though, we’ll focus on the BGP hijacking part of this attack.

What happened?

First, here’s a rundown of routing attacks on cryptocurrency in general – https://btc-hijack.ethz.ch/.

In this case specifically, the culprit re-routed DNS traffic using a man in the middle attack using a server at an Equinix data center in Chicago. Cloudflare has put up a blog post that explains the technical details. From that post:

“This [hijacked] IP space is allocated to Amazon(AS16509). But the ASN that announced it was eNet Inc(AS10297) to their peers and forwarded to Hurricane Electric(AS6939).

“Those IPs are for Route53 Amazon DNS servers. When you query for one of their client zones, those servers Continue reading

Update: What will programming look like in the future?

 

Update: Ólafur Arnalds built a robotic music system to accompany him on the piano. He calls his system of two semi generative, self playing pianos—STRATUS. You can hear his most recent song re:member. There's more explanation in The Player Pianos pt. II and a short Facebook live session. His software reacts to his playing in real-time. Then he has to react to the robots because he's not sure what they're going to do. He's improvising like a jazz band would do, but it's with robots. It's his own little orchestra. The result is beautiful. The result is also unexpected. Ólafur makes the fascinating point that usually your own improvisation is limited by your own muscle memory. But with the randomness of the robots you are forced to respond in different ways. He says you get a "pure unrestricted creativity." And it's fun he says with a big smile on his face.

 

 

Maybe programming will look something like the above video. Humans and AIs working together to produce software better than either can separately.

The computer as a creative agent, working in tandem with a human partner, to produce software, in a beautiful act of Continue reading

This Girls in ICT Day, Let’s Increase the Visibility of Women on Wikipedia

At the Women’s Special Interest Group of the Internet Society we are committed to promoting the participation of women in the Internet ecosystem. We also consider it important to increase the content created by and about women on the Internet to give voice and make visible the actions, work, and participation of women in Information and Communication Technologies (ICT).

Just 17% of Wikipedia biographies are of women. This happens because of the invisibility on the Internet of their work, which makes it difficult to create their biographies, no matter how valuable their work is.

April 26 is the International Day of Girls in ICT, promoted by the ITU. It aims to reduce the digital gender gap and encourage and motivate girls to participate in tech careers. With the support of the Wikimedia Foundation and Internet Society Chapters, we are going to commemorate the Girls in ICT Day with an editathon marathon in Wikipedia, to include all the women who are working to build an open Internet, free, safe, transparent, and affordable for everyone.

The important thing about this Global Editathon “Girls in ICT” is that the content will be created by women in their native language or in the Continue reading

1 … 1,631 1,632 1,633 1,634 1,635 … 3,841