Smoke: fine-grained lineage at interactive speed

Smoke: fine-grained lineage at interactive speed Psallidas et al., VLDB’18

Data lineage connects the input and output data items of a computation. Given a set of output records, a backward lineage query selects a subset of the output records and asks “which input records contributed to these results?” A forward lineage query selects a subset of the input records and asks, “which output records depend on these inputs?”. Lineage-enabled systems capture record-level relationships throughout a workflow and support lineage queries.

Data lineage is useful in lots of different applications; this paper uses as its main example interactive visualisation systems. This domain requires fast answers to queries and is typically dominated by hand-written implementations. Consider the two views in the figure below. When the user selects a set of marks in , marks derived from the same records are highlighted in (linked brushing).

A typical visualisation system implements this manually, but it can equally be viewed as a backward lineage query from the selection points in , followed by a forward lineage query from the resulting input records to .

(See ‘Explaining outputs in modern data analytics’ which we looked at last year for an introduction Continue reading

AWS Cloud – Part 1

Here we will go through the basics Amazon Virtual Private Cloud or VPC :Its virtually isolated networks ,they cannot communicate to each other ,to external world,internet ,to a VPN without explicitly granting that ability.we create VPC per account per region basis.Lets first understand about the few terms related to AWS  .

Amazon EC2 : :Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) cloud. Using Amazon EC2 eliminates your need to invest in hardware up front, so you can develop and deploy applications Continue reading

From VNC to reverse shell

From VNC to reverse shell

Personal websites are weird. We are mostly past the era of having them, as things like twitter and hosted blog services like Medium have taken them over, but I’m a hold out. I run both my own blog, and have a landing page

Cloudflare Turns 8 — here’s what we mean by a “better Internet”

I have always loved birthdays. It is a chance to get together with loved ones, a chance to have fun and a chance to reflect on anything you want to keep doing or change in the upcoming year. At Cloudflare, we’ve embraced celebrating our birthday as well.

This week, Cloudflare turns 8 years old. It feels like just yesterday that Matthew, Lee, Matthieu, Ian, Sri, Chris, Damon and I stepped on stage at Techcrunch Disrupt to launch Cloudflare to the world. Since then, we have celebrated our birthday every year by giving a gift back to our customers and the Internet. This year, we plan to celebrate each day with a new product benefiting our community. Or in other words, it is a weeklong birthday celebration. Like I said, I love birthdays!

The Cloudflare team when we launched the service at Techcrunch Disrupt during September 27 to 29, 2010 – Matthieu, Chris, Sri, Ian, Lee, Matthew, Michelle and Damon.

While I can’t share exactly what we’re releasing every day — after all who doesn’t like a surprise? — I wanted to share some thoughts on how we decide what to release birthday week.

Our mission at Cloudflare is to help Continue reading

Weekend Reads 092118

Five of the largest U.S. technology companies pledged support this year for a dangerous law that makes our emails, chat logs, online videos and photos vulnerable to warrantless collection by foreign governments. Now, one of those companies has voiced a meaningful pivot, instead pledging support for its users and their privacy. EFF appreciates this commitment, and urges other companies to do the same. —David Ruiz @EFF

Quantum computing is a new way of computing — one that could allow humankind to perform computations that are simply impossible using today’s computing technologies. It allows for very fast searching, something that would break some of the encryption algorithms we use today. And it allows us to easily factor large numbers, something that would break the RSA cryptosystem for any key length. @Schneier on Security

If you manage Internet number resources in the APNIC Whois Database, you are requested to provide contact information so that people can contact you for network abuse or troubleshooting. You and your colleagues might have created person objects for this purpose. However, from time to time a person performing a role may change. If you have a lot of resource contacts to manage, updating person contacts can Continue reading

CCIE – Should I Renew?

It is 6 years since I passed the CCIE Lab Exam. The dreaded email has arrived:

CCIE: Your CCIE status is ‘suspended’ and you need to recertify in twelve months.

Time to re-evaluate what the CCIE means to me. Should renew it? Should people start out on the CCIE track now? My opinions have shifted over the years.

Should I Renew?

I’ve been through this cycle a few times now. I’m getting closer to Emeritus, but it’s still a few years away.

My career has shifted over the last few years. I work for a Network Vendor, but networking is only part of what I do. I am a Product Manager, focused on automation. I spend very little time looking at network devices, or CLI. I spend my time talking to customers, updating roadmaps, writing Python, reviewing Pull Requests.

My future will be working with technologies like Serverless Computing, IoT, and Edge.

CCIE R&S doesn’t cover any of that.

It is unlikely that I will ever work as a traditional hands-on network engineer again. Not impossible, but unlikely. I doubt that any future employer will care about whether I have a current CCIE certification. At this point my experience Continue reading

CCIE – Should I Renew?

It is 6 years since I passed the CCIE Lab Exam. The dreaded email has arrived:

CCIE: Your CCIE status is ‘suspended’ and you need to recertify in twelve months.

Time to re-evaluate what the CCIE means to me. Should renew it? Should people start out on the CCIE track now? My opinions have shifted over the years.

Should I Renew?

I’ve been through this cycle a few times now. I’m getting closer to Emeritus, but it’s still a few years away.

My career has shifted over the last few years. I work for a Network Vendor, but networking is only part of what I do. I am a Product Manager, focused on automation. I spend very little time looking at network devices, or CLI. I spend my time talking to customers, updating roadmaps, writing Python, reviewing Pull Requests.

My future will be working with technologies like Serverless Computing, IoT, and Edge.

CCIE R&S doesn’t cover any of that.

It is unlikely that I will ever work as a traditional hands-on network engineer again. Not impossible, but unlikely. I doubt that any future employer will care about whether I have a current CCIE certification. At this point my experience Continue reading

Ansible Tower Advanced Smart Inventory Usage

Background

Smart Inventory is a feature that was added to Red Hat Ansible Tower 3.2. The feature allows you to generate a new Inventory that is made of up hosts existing in other Inventory in Ansible Tower. This inventory is always-up-to-date and is populated using what we call a host filter. The host filter is a domain specific query language that is a mix of Django Rest Framework GET query language with a JSON query syntax added in. Effectively, this allows you create an Inventory of Hosts and their relational fields as well as related JSON structures. 

The ansible_facts field is a related field on a Host that is populated by Job Template runs (Jobs) that have fact caching enabled. Ansible Tower bolts on an Ansible fact cache plugin with Job Template that have fact caching enabled. Job Templates of this kind that run playbooks that invoke Ansible gather_facts will result in those facts being saved to the Ansible Tower database when the Job finishes.

A limitation of the Smart Inventory filter is that it only allows equality matching on ansible_fact JSON data. In this blog post I will show you how to overcome this limitation and add Continue reading

Cleared JNCIE-DC

After close to a year of study and after one failed attempt I cleared it in the second attempt. Here is my experience in short and tips to prepare for the exam

Reading Resources 

-> Juniper Dayone – Anything and everything related to DC

-> QFX Series Book

https://www.safaribooksonline.com/library/view/juniper-qfx5100-series/9781491949566/app03.html

-> JNCIP – ADCX/TDCX/DCX

-> Datacenter Network / EVPN – Overview

https://www.safaribooksonline.com/library/view/evpn-in-the/9781492029045/ch04.html

Lab Resources 

-> If you are into any serious preparation you need to consider the below git resource, its awesome and you can practice pretty much everything even on a laptop and also in your flights/travel.

https://github.com/Juniper/vqfx10k-vagrant

You need to know a bit of vagrant and need to have VirtualBox and ansible installed, not hard by any means, all it takes is a days dedication to make your laptop ready for these, let me know if you want me to write a blog post for the setup.

-> I had Dell R810 Server, https://r2079.wordpress.com/2018/01/05/my-dc-virtual-lab-setup-insights/ , I did most of my practice on this one.

-> I also had the privilege of using hardware resources and examined the ideal configuration for a production network and learned few things from them.

Continue reading

Portworx Stuffs Stateful Container Storage Updates Into Enterprise Platform

The PX-Enterprise 1.6 updates take into account the distributed storage nature of modern applications running across different clouds and container environments.

SDxCentral Weekly Wrap: Sept. 21

Nokia Slashes 500 Jobs; Oracle Cloud Exec on Extended Leave; SK Telecom Picks 5G Vendors Nokia will cut 500 jobs in Illinois by year-end as part of a restructuring plan. Oracle executives declined to elaborate about the company’s cloud chief taking an extended leave from work. SK Telecom ignored Chinese vendor Huawei and picked Nokia,... Read more →

Exfo Service Assurance VNFs Give ‘Eyes’ to ONAP With Amdocs NFV Integration

Exfo is onboarding its service assurance VNFs into the Amdocs NFV software and services portfolio, which is powered by ONAP.

Writing Is Hard

Writing isn’t the easiest thing in the world to do. There are a lot of times that people sit down to pour out their thoughts onto virtual paper and nothing happens. Or they spend hours and hours researching a topic only to put something together that falls apart because of assumptions about a key point that aren’t true.

The world is becoming more and more enamored with other forms of media. We like listening to podcasts instead of reading. We prefer short videos instead of long articles. Visual aids beat a wall of text any day. Even though each of these content types has a script it still feels better having a conversation. Informal chat beats formal prose every day.

Written Wringers

I got into blogging because my typing fingers are way more eloquent than the thoughts running through my brain. I had tons of ideas that I needed to put down on paper and the best way to do that was to build a simple blog and get to it. It’s been eight years of posting and I still feel like I have a ton to say. But it’s not easy to make the words flow all the time.

Continue reading

Linux community acts after years of complaints like Sarah Sharp’s

Update: On Sept. 16, 2018, after being questioned by The New Yorker about his abusive behavior, Linus Torvalds apologized for his conduct and announced he was stepping back from kernel development to get help understanding people's emotions and how to respond properly. In addition, for the first time, the Linux community will be adopting a Code of Conduct to create a welcome and opening environment. -----------------------------------------------A prominent Linux kernel developer announced today in a blog post that she would step down from her direct work in the kernel community, saying that the community values blunt honesty, often containing profane and personal attacks above “basic human decency.”To read this article in full, please click here

Linux kernel dev Sarah Sharp quits, citing ‘brutal’ communications style

Update: On Sept. 16, 2018, after being questioned by The New Yorker about his abusive behavior, Linus Torvalds apologized for his conduct and announced he was stepping back from kernel development to get help understanding people's emotions and how to respond properly. In addition, for the first time, the Linux community will be adopting a Code of Conduct to create a welcome and opening environment. -----------------------------------------------A prominent Linux kernel developer announced today in a blog post that she would step down from her direct work in the kernel community, saying that the community values blunt honesty, often containing profane and personal attacks above “basic human decency.”To read this article in full, please click here

There’s a Group Working on an Open Source Silicon Instruction Set

The RISC-V initiative is creating a free and open instruction set architecture for the next generation of chipsets.

Research Brief: Cutting through the SD-WAN Confusion: A Handy Guide for Enterprises

SDxCentral's latest research brief is aimed at providing enterprises contemplating purchasing SD-WAN solutions with a concrete five-step process and a set of core considerations critical to success.

What’s Ahead at the 2018 Plenipotentiary Conference

Today the Internet Society published a matrix of issues that will be discussed at the International Telecommunication Union’s (ITU) Plenipotentiary treaty conference (PP-18) in Dubai next month.  The matrix reflects common proposals adopted recently at some of the ITU’s regional preparatory meetings. It is intended to aid our community in preparations and serve as a useful guide on where governments stand on some of the issues that are important to the Internet Society community. Note that the matrix will be updated periodically as individual country proposals are submitted closer to the conference date. Based on the input from governments so far, Internet Governance, emerging technologies such as the Internet of Things (IoT), Artificial Intelligence (AI), and Over-the-Top (OTT) applications and services will rank high on the agenda at the Plenipotentiary.

While the Plenipotentiary happens every four years, it comes at a time when Internet Governance stakes are particularly high, as governments’ response to the borderless nature of Internet issues such as cybersecurity and data privacy is intensifying, and support for multilateral solutions to deal with them grows. Those that favor a multilateral governance approach might view the ITU’s international cooperation framework for global telecommunications as the natural vector into Continue reading

Cyber Threat Alliance Reports 459% Spike in Cryptomining Malware

“If you’ve got mining happening on your network, you probably have other bad stuff happening on your network, and it’s probably much worse than mining,” said CTA’s Neil Jenkins.

GitLab Gets $100M Series D, Hits $1B Valuation

The software repository used the news to take a swipe at rival GitHub, which Microsoft purchased for $7.5 billion.