Towards a hands-free query optimizer through deep learning

Towards a hands-free query optimizer through deep learning Marcus & Papaemmanouil, CIDR’19

Where the SageDB paper stopped— at the exploration of learned models to assist in query optimisation— today’s paper choice picks up, looking exclusively at the potential to apply learning (in this case deep reinforcement learning) to build a better optimiser.

Why reinforcement learning?

Query optimisers are traditionally composed of carefully tuned and complex heuristics based on years of experience. Feedback from the actual execution of query plans can be used to update cardinality estimates. Database cracking, adaptive indexing, and adaptive query processing all incorporate elements of feedback as well.

In this vision paper, we argue that recent advances in deep reinforcement learning (DRL) can be applied to query optimization, resulting in a “hands-free” optimizer that (1) can tune itself for a particular database automatically without requiring intervention from expert DBAs, and (2) tightly incorporates feedback from past query optimizations and executions in order to improve the performance of query execution plans generated in the future.

If we view query optimisation as a DRL problem, then in reinforcement learning terminology the optimiser is the agent, the current query plan is the state, and each available action Continue reading

AnsibleFest is heading to Atlanta!

AnsibleFest-ATL-2019-Social-Image

We are excited to announce the dates and location for AnsibleFest 2019. We’ve selected a location that not only provides the ease of use (or access in this case) that users expect from all things Ansible, but also the enjoyment folks expect after automating their way through complex problems.

ATLANTA! Home of the Braves (MLB), Falcons (NFL), Hawks (NBA), the largest aquarium in the world, the busiest airport in the world, and great restaurants like The Varsity and Old Lady Gang. Soon it will also be the home of AnsibleFest 2019!

Join us at the Hilton Atlanta Downtown, September 24-26, 2019. We will follow the same format as last year with a Welcome Party on September 23, two days of content on September 24-25, and some add-on options, like workshops, on September 26. There will also be a Contributor Summit again (details to follow at a later date). We’ll be bringing back the high quality experiences attendees have come to expect, including Ask an Expert and the Getting Started Hub. But, most importantly, we’ll have sessions from folks across the Ansible community.

Here’s what to expect between now and AnsibleFest Atlanta 2019:

Hook Up Your Business with Award-Winning Video Conferencing For As Little As $19.99/mo

If you’re still using a business landline, you’re likely not doing much business in 2018. Video conferencing is as commonplace as sending an email, and remote team members are increasing by the minute, making internet-based communication systems essential for any rising company. So if you’re finally looking to get your office set up with a state-of-the-art communications system, look no further than RingCentral, an award-winning VoIP (voice over internet provider). RingCentral offers comprehensive, affordable packages for any size organization, and you can set it up in minutes for as low as $19.99 per month. To read this article in full, please click here

Internet Society Delhi Chapter and CCAOI Organize Webinar on India’s Draft Intermediary Rules

On 10 January, the Internet Society Delhi Chapter and CCAOI jointly organised an interactive webinar on the draft Information Technology [Intermediary Guidelines (Amendment) Rules] 2018 (“the draft Intermediary Rules”) to improve understanding of it and to encourage members and other Indian stakeholders to submit their comments to the Ministry of Electronics and Information Technology (MeitY) during their public comment period. The draft Intermediary Rules seeks to modify Section 79(2)(c) of the Information Technology Act, 2000 (the IT Act). Section 79 of the IT Act introduces obligations for intermediaries to meet to gain exemption from liability over the third-party information that they “receive, store, transmit, or provide any service with respect to.” These proposed changes were developed by MeitY to try to address misinformation and harmful content on social media, which have been connected with lynching and other recent violent acts of vigilantism.

The session was moderated by Subhashish Panigrahi, chapter development manager for Asia-Pacific at the Internet Society, and Amrita Choudhury, treasurer of the Internet Society Delhi Chapter and director of the CCAOI.

The changes to the IT Act proposed in the draft Intermediary Rules would require intermediaries to provide monthly notification to users on content they should not share; ensure that the originator Continue reading

Quantum-embedded chips could secure IoT

Microprocessors that are unique to each Internet of Things (IoT) device is the way forward in the ongoing and tricky quest to secure the IoT, says Crypto Quantique. One idea is that by making each chip one of a kind and unclonable, an application would become almost impossible to hack.The U.K.-based startup says it has introduced “the world's most advanced security product for IoT devices.” The microprocessor-based solution uses quantum physics, combined with cryptography, all embedded in silicon, it explained in a press release last October.To read this article in full, please click here

Quantum-embedded chips could secure IoT

Microprocessors that are unique to each Internet of Things (IoT) device is the way forward in the ongoing and tricky quest to secure the IoT, says Crypto Quantique. One idea is that by making each chip one of a kind and unclonable, an application would become almost impossible to hack.The U.K.-based startup says it has introduced “the world's most advanced security product for IoT devices.” The microprocessor-based solution uses quantum physics, combined with cryptography, all embedded in silicon, it explained in a press release last October.To read this article in full, please click here

NAE: Some Help Dealing with Brain Block

For years, thanks to the gift of misaligned perception, I’ve been mentally blocked. I’ve avoided things like Machine Learning because my perceived skill with mathematics is weak, avoided programming languages like C# because the perceived uphill hike to get familiar is high and avoided front end web development because of the perceived browser nightmares.

Technology has come a long way since I last touched C# and web development and there are some great ML libraries out there which minimize the requirement for hardcore mathematical skill sets. My perceived problems have remained yet the actual blockers have moved and morphed. I’ve lived on old ideas without re-grouping and forming a refreshed attack. More on my foolish ways later.

For many people and organizations, it pains me to admit that perception of network automation is also misplaced. It spans from “Ansible is the answer, sorry, what were you asking?” to “Python will save the day”, following “The automation is the design!”.

Ivan Pepelnjak as usual has wrote some great content on topic as per usual. Read this post for a rather targeted view on expert beginners. TL;DR: “I got hello-world working for one tool, me now expert”.

Currently I also Continue reading

Five Stages of Automation Grief

As I’m doing occasional consulting for large enterprises redesigning their data centers, I encounter a wide range of network automation readiness, from “we don’t need that” to “how could we automate as much as possible”.

Based on the pervasiveness of “we don’t need that” responses it looks like many enterprise network engineers still have to go through the five stages of automation grief.

Read more ...

Containers are here to stay, who has the right skill set?

Who controls containers: developers, or operations teams? While this might seem like something of an academic discussion, the question has very serious implications for the future of IT in any organization. IT infrastructure is not made up of islands; each component interacts with, and depends on, others. Tying all components of all infrastructures together is the network.

If operations teams control containers, they can carefully review the impact that the creation of those containers will have on all the rest of an organization’s infrastructure. They can carefully plan for the consequences of new workloads, assign and/or reserve resources, map out lifecycle, and plan for the retirement of the workload, including the return of those resources.

If developers control containers, they don’t have the training to see how one small piece fits into the wider puzzle, and almost certainly don’t have the administrative access to all the other pieces of the puzzle to gain that insight. Given the above, it might seem like a no-brainer to let operations teams control containers, yet in most organizations deploying containers, developers are responsible for the creation and destruction of containers, which they do as they see fit.

This is not as irrational as it Continue reading

1 2 3 2,432