Alexa , AWS Lambda & AWS IOT MQTT and you can interact with anything

I hear a lot on IOT but don’t have a clue on underlying protocols. My interest is only to understand how it might help a business or more than that my personal interests. So continuing the server power on/off series I wanted to do it with Amazon echo voice command. Now, this is not a smart power switch where you can power-on with a command on Echo but you actually have to send a message to IDRAC, we already covered this in a previous post.

Well, the main goal isn’t to power-on a server that can be done manually as it sits beside me, the main goal is to extend this to any business / personal ideas which might get the benefit.

Summary – Develop a small interactive model to understand Alexa voice service / AWS lambda and MQTT so that we can get a feel of what can be achieved with this.

I will not go much into any tech explanations or bore you with English, I will put here two screenshots and code to git, hopefully, you should be able to give it a try.

Its illustrated in 6 steps

1. Voice command to echo (I have made Continue reading

Python Decorators – From a Network Engineers Perspective

As going through learning some basic programming, I encountered Decorators. I should be very honest if any of you are trying to figure out or learn what decorators in python do from my blog post you are dangerously in trouble.

So what this post about if not learning, well its mostly on what the functionality is so that you can learn the concept from better programming resources.

Let’s examine the below code

The output will be something like below

What’s in this code:

The first thing you have to realize is that some representation with ‘@’ symbol. If you have noticed get_reinfo and get_modelinfo functions, they have one thing in common which is to connect to the device and get output before they parse the required fields, that what a Decorator is helping us to do here, we extend that wrapping functionality around new functions without having to write everything or globalize everything.

At least that is what I understood.  So, next time when you are writing some code try to think if you can incorporate decorators into them.

-Rakesh

Emulating Juniper Devices – Various options

Hi,

I have got a lot of requests for writing up a blog post on various Methods of emulating Juniper devices for practice.

Note : For 2/3 methods to work, you need to have official Junos software (vmx-vcp and vmx-vfp)

Method 1 – Gns3

Most popular and Familiar Method – Install via gns3

After installing Gns3, download the

Vmx-vfp Appliance – https://docs.gns3.com/appliances/juniper-vmx-vfp.html

Vmx-vcp Appliance – https://docs.gns3.com/appliances/juniper-vmx-vcp.html

Documentation is straight forward, all you need to do is double click on the appliance and upload the image when requested, as easy as it can get.

Method2 – Vagrant

Use Juniper uploaded Images via vagrant.

Most of Juniper Vqfx / JNCIE-DC practice came up from a vagrant, You don’t need to have any official images or access to Juniper software downloads. The downside is that it only supports VQFX and generic SRX, but for most of the Routing protocol and MPLS learning this should be good.

https://app.vagrantup.com/boxes/search?utf8=%E2%9C%93&sort=downloads&provider=&q=juniper

https://app.vagrantup.com/juniper/boxes/vqfx10k-re

https://app.vagrantup.com/juniper/boxes/vqfx10k-pfe

Again, following Github link, will auto setup the topology without you worrying about much details to vagrant.

https://github.com/Juniper/vqfx10k-vagrant -> Go into a specific folder and say vagrant up, that should take Continue reading

Docker and Netdata – Awesome for Monitoring our home servers !

I don’t belong to any DevOps space but I keep hearing things like Docker / kubernetes and what not. I Quickly wanted to see what these are capable of and if I can use them to my advantage so that later I can see the use case for networking.

Docker so far seems to be far more capable and am enjoying it. Thanks to one of my friends who suggested this wonderful Repo, monitoring my server has been more granular.

I will have a detailed post on Juniper MX image via docker but for now, I have used it for something out of networking space to serve a small purpose.

https://github.com/netdata/netdata

This what Netdata UI looks like while monitoring my server, this is really wonderful as for the long time  I was trying to implement many Monitoring systems and most of them required some dedicated hardware or at least a Vmware spin-off instance, which is fine for me I suppose but I wanted something small and yet effective.

Netdata operates on port 19999 of localhost and should be reachable via any web-browser.

I made a small cronjob which starts this instance on every reboot.

Installation is pretty straight forward, Continue reading

My Ignorance for a Year and Dell IDRAC

Hi,

I have a Dell R810 and it makes a lot of noise. So, I have put it somewhere remote in the home where it’s completely inhabitable for human beings. Now, the problem was always to go there and manually power-up the system.  This has been the scenario for years. So, technically if am away from my home I need to take help from my wife. All this is going well and I was always wondered should it not have a better way to do things.

Was talking to my friend and he had some paid solution for the same thing, he doesn’t even remember and that’s for his cisco gear, now for Cisco gear as long you supply power they will be powered (if the Power button is always on), for servers however just like our Personal CPU’s and Laptops you have to manually press the power button.

Then yesterday all of a sudden I took this somewhat seriously and explored options, then I understood it was sitting right inside the server, Dell has something called Integrated DRAC system which helps you do this thing, all you need to have is a proper Lan connection, the message was Continue reading

Multithreading – Whats in it ?

Hi,

I never took threading seriously, partly because I was barely getting started with programming let alone running and improving a Programs performance. I have all the time in the universe to wait for the program to run instead of making it run much faster

Problem statement: Has 3 sites to ping, make 3 parallel pings (like opening 3 terminal and issuing a parallel ping statement) instead of one by one approach and has to be done with python

Threading Module comes to Rescue

https://github.com/yukthr/auts/blob/master/random_programs/threading_program.py

Basically, in the below output if you see two blocks, one block gets executed sequentially one ping after the other while the other one includes threading and everything is taken parallel and hence we see the output differences as well. I think am yet to figure on calculating time aspect of the threading but it is definitely faster.

Again not going much into GIL or (IO / CPU threading differences), all I can say is we can use this to enhance some day to day activities.

This may sound a little strange, macosx was not giving me any results for threading while any Linux / Unix (raspberry pi in this Continue reading

Listing TOP 5 Processes – Top command

Continuing some exploration of Pandas, I realized in networking we often has to deal with Toptalkers, I dont have any Networking Realted Top Talker IP Data as such but wanted to see if this can checked on my Laptop’s current Processes comsuming CPU and Top processes which are repeated often.

Without dragging the topic

-> Took the text file it was delimited with space (TOP Command will generally Delimit)

-> Converted to Pandas Read FWF and then converted the file to CSV

-> Used CSV to read into specific %CPU coloumn and implemented SORT function in descending order.

-> Finally Took Counter from Collections Module and implemented it on the list.

output looksl something like this

Pandas are effective and easy, I will continue to explore some functions so as to build some scripts in day to day activities.

-Rakesh

Python Pandas – Reading text files got a lot better.

Problem Statement – Have a list of VMWare Instances and wanted to quickly iterate and see what VMs are powered on, this has nothing to do with Vmware or ESXI, look at the below image, programmatically I want to pick two specific rows and perform dictionary operations on them.

Git – https://github.com/yukthr/auts/blob/master/random_programs/pandas_vmware_esxi.py

for someone like me who is partly into programming and mostly into networking, anything effective is easy, I could have gone with the other way of doing this but  since Pandas are effective i would text parsing with them.

I only want two sections which are of interest to me, in a normal way maybe I should have pasted this in excel and Do a text to the column and do a manual data extraction, but wanted to do it with Pandas and they are powerful.

First things first

• Read with Pandas
• Read the text
• If possible convert into CSV (as I have some experience with csv file operations)
• Then split the Columns and make the dictionary for further use

Let’s see how this proceeds

Pandas make it so powerful to just pick Columns out of the fly, in this case, I wanted Continue reading

Working with distance sensor – solving overhead water tank problem

This is not a networking post.

Schematic , sensor code and spec  – https://www.linuxnorth.org/raspi-sump

My code – https://github.com/yukthr/auts/blob/master/random_programs/water_sensor.py

1x Breadboard

1x Raspberry pi zero w

1xhcsr04 ultrasonic sensor

2x1kohm resistors

Just as a side note i do not have any intro into resistors nor electronics, but what all i did was to follow some posts written by people who already did it, its not hard believe me, if i could do it any one should easily be able to do it as am very far away from electronics and programming, so let these things not overwhelm you.

Problem – Am not sure in other parts of the world, but place I live has an over head water Tank which stores water. So every day you technically turn on a water motor which sucks water from a reserve under the ground and pumps it to all the the way to a three store high building

So what’s the issue – The issue is that we have no clue what’s the current water level in the tank nor how long would it take to fill the water tank. There are two tribal ways by which we Continue reading

Sonoff Flashing and steps towards Smart home

This post is no way related to Networking !

It’s always my fascination to implement a specific tech towards home automation so as to have more time in hands. The first step towards this was to use a Tp-link smart switch which came with Alexa, now problem with that is that Tplink app was not properly responding to Alexa and TP-Link smart switch in itself was costly.

I wanted something low-cost, something which I can experiment yet be cost effective, After some re-search I came across Sonoff-Basic model.

https://sonoff.itead.cc/en/products/sonoff/sonoff-basic

I integrated with Amazon echo but again there were app issues (default app is ewelink), though it was working fine something was missing, I couldn’t tinker it to my wish

I found then there is a open-source version of a similar firmware called TASMOTA and we have to flash this sonoff with the new firm-ware of Tasmota, I have to tell you here there is every possibility you would brick your device and make it useless in the process.

https://github.com/arendst/Sonoff-Tasmota/wiki/Prerequisite

Everything is so well documented, there is no need to re-iterate things here, but just to show case another successful implementation.

Need-less to say you are playing with AC/DC Continue reading

Cleared JNCIE-DC

After close to a year of study and after one failed attempt I cleared it in the second attempt. Here is my experience in short and tips to prepare for the exam

Reading Resources 

-> Juniper Dayone – Anything and everything related to DC

-> QFX Series Book

https://www.safaribooksonline.com/library/view/juniper-qfx5100-series/9781491949566/app03.html

-> JNCIP – ADCX/TDCX/DCX

-> Datacenter Network / EVPN – Overview

https://www.safaribooksonline.com/library/view/evpn-in-the/9781492029045/ch04.html

Lab Resources 

-> If you are into any serious preparation you need to consider the below git resource, its awesome and you can practice pretty much everything even on a laptop and also in your flights/travel.

https://github.com/Juniper/vqfx10k-vagrant

You need to know a bit of vagrant and need to have VirtualBox and ansible installed, not hard by any means, all it takes is a days dedication to make your laptop ready for these, let me know if you want me to write a blog post for the setup.

-> I had Dell R810 Server, https://r2079.wordpress.com/2018/01/05/my-dc-virtual-lab-setup-insights/ , I did most of my practice on this one.

-> I also had the privilege of using hardware resources and examined the ideal configuration for a production network and learned few things from them.

Continue reading

PrettyTable – Printing anything in Table

Hi,

As the scripting and programming deals in logging into the device and fetching data, there will be a time where presentability of Data matters. PrettyTable is one such package which greatly helps in reading things

A simple example, re-visiting the code to get the list of routes from the Device

PrettyTable will help in tabulating the Data, the installation and usage can be found here

Pip Package  – https://pypi.org/project/PrettyTable/

Usage – http://zetcode.com/python/prettytable/

Once we have the code, let take a look at how the program looks

The Table form looks something like this

Hope this help for anyone who gets started with presentability of Data, honestly, there was one time I got crazy with the print statement just to make the data presentable.

Fetching Pcap Files and running Wireshark – Junos Pyez

As a part of some content development and revision on MPLS, I was trying to capture some MPLS Pcap Files from the Vmx Router. The problem was that every time I capture a file, I had to first scp or sftp into the device and then after successful import, I had to run Wireshark on it.

The Aim of the post is to fetch the pcap file and run Wireshark on it. Am not a decent programmer by any stretch of the imagination, anyone can see that from my code but the point that matters is that it works and you should give it a try too.

https://github.com/yukthr/auts/blob/master/junos_pyez/getfile_executewireshark.py– Is the file

When I run this program to fetch a non-existent Pcap file

And when I query for an existent file

-Rakesh

Raspberry Pi and AWS IOT – First steps

Hi All,

I have slightly changed this to networking, but the intention and my current use is to measure the water level of a sump, since that deviates from the network blog writing, i have extended the same to a Router.

Purpose – Have a Router and also a Syslog Server which monitors my internal network (This can easily be extended to a Car / Moisture Sensor or a Temperature/Humidity Sensor), what we want to do is to make sure if any anomaly is seen in Log Messages, it logs to IOT service. We can then take this up as a Part-2 writing to perform a specific action / automated on what action can be taken to mitigate

Discussion about configuring a Linux device is out of scope, so lets think that we all have that setup. What happens next ?

Lets quickly see our python script, which parses for a anomaly, in this lets say when someone runs a ping command, well its not a anomaly but will do for our use-case.

logparse.py is our program, so i have imported it into the readily available sample program provided by AWSIOT Kit, so you dont have to know Continue reading

Revisiting – Why IGP sync with LDP is required ?

Hi All,

I was preparing some content on MPLS for a training session  and as a part of it, was going through LDP. The interesting aspect is very obvious

-> LDP is dependent on IGP

-> What ever Draw-backs IGP has will be inherited by LDP

-> LDP has to be enabled on the Interface to exchange Labels, else it wont consider the exit-interface from IGP and hence there will be no LSP’s

So far so good and makes sense as well

I will not be boring with command line outputs in this case

-> I have disabled the interface between R3/R4 so if R3 Has to reach R1, it will use R3-R2-R1 path

All good, Am going to just tweak the metric of the interface on R3 -> R2 before I enable back the R3 – R4

Now let me enabled the interface between R3-R4

-> It has a Better cost

-> It has not been enabled for LDP

If we go back to R3, to examine the result

This is dangerously familiar for me , There is a LDP neighbor, but No routes are present in Inet.3 (neither for R1 or R2) as Routes Continue reading

Book Review – How SRE Relates to Devops

Link – https://www.safaribooksonline.com/library/view/how-sre-relates/9781492030645

Quick Read – Few Pages
Other Books which are related to SRE
-> Site Reliability Engineering: How Google Runs Production Systems

Few Points that I liked

– Quick read , hardly an hour
– Intro on how Devops got introduced
– what needs to be improved – Key Idea “No More Silos”
– stress on non-localization of knowledge, lack of collaboration
– New Job Role called SRE – Site Reliability Engineering
– Operations is a Software problem and Work to minimize Toil are some best examples of productivity and how we should view
– Key Idea – “Automate This Year’s Job Away” and “It’s Better to fix-it over-selves than blame someone else”

Good Read to understand the over picture of SRE Role and some of the
work Discipline Ideas.

-Rakesh

Analyzing data with Pandas Package – An Intro to Pandas

Hi,

Title may sound extremely Hitech for someone who never heard about pandas ;), but what I have written is a simple hello world equivalent  program, which I guess should start to help my day to day analysis, as always the aim is to let anyone know the advantage of something than hammering with some theory !

I was going through various python packages available to analyze data and came across pandas package along with numpy package. These are not there by default in Python installation and if you like them to be on your system, you should install them via PIP, I have them installed already hence you can see that it complains in the below image.

Note :

Understand why you need to have something like Pandas / Numpy even if you have never heard of them, that’s the point of this tiny program

Imagine, how you would solve this if you never knew Pandas/Numpy and you will see the power of these

packages, again you don’t have to know these to realize their full power.

Now coming to the requirement, here is a sample spreadsheet that I have below, its a CSV Sheet which contains certain Continue reading

Cleared JNCIS-Devops

Last week I went to write JNCIS-Devops exam, I was under an impression that I may not be able to clear it but good did happen!

First and Foremost

-> I had the official training for JAUTcourse – The course is extremetly helpful as it provides the precise material and also the structured lab environment for you to explore and study, nothing beats a class-room study and training environment

But, after appearing I can tell you that you dont really require the offiicial training (if that is the only thing stopping you to think about the exam), the exam will test you for your understanding of automation philosophy and also how Juniper Implements it.

Topics of Interest

– Juniper  pyez – understand how everything helps in Pyez

Dayone Books Helps – https://www.juniper.net/uk/en/training/jnbooks/day-one/automation-series/junos-pyez-cookbook/

– Juniper ansible – https://www.juniper.net/uk/en/training/jnbooks/day-one/automation-series/junos-pyez-cookbook/

-Book – Network Programmability and Automation

https://www.safaribooksonline.com/library/view/network-programmability-and/9781491931240/

— Jsnapy – https://www.juniper.net/uk/en/training/jnbooks/day-one/automation-series/using-jsnap-automate-network-verifications/

All you need to have are couple of VMX devices a Linux machine and you should be able to deploy all of the automation efforts discussed in above books.

You dont have to know the code in your head or how to write a Continue reading

Plotting the interface flap – That’s some analysis

Hi,

What started to be a exploration project is now turning out to be pretty useful for me in day to day analysis. Back in days when I worked in support, there was nothing to predict or really worry about historical events for any future work, just grep for logs and you are done with the last flap and analysis.

Customers / Networks now look for more data, while there are systems which do the telemetry and prediction, from an analysis point of view, as an engineer I want to know if the device or a circuit over an interface is stable over a period of time or even if it flaps what is the likely time and day it flaps in a week for a smoother migration.

Requirement : Plot a simple graph analyzing the interface flaps over a period of one week for a  specific interface and decide the actions next from the log messages.[in this case i used a junos device]

Well grepping the logs is not something new for a seasoned engineer but having visual data will prove to be useful for a cutover or migration.

There are systems which can do this work on Continue reading

Poweron and Poweroff Esxi instance from CLI using a Python script

Hi,

I have to agree that to start a esxi node i was depending heavily on a windows VM and then was using a VSphere client to connect to a Esxi 5.5.

In a typical day all of my VM’s are hosted on Esxi and am not any advanced user of esxi by any stretch of Imagination.

It came down to a point where i had to manually click close to 8 VMS in order to boot up and all this was sort of irriatating for me, so i wrote a very basic script which can do this for me. Most of the experienced VM admins have been doing this for very long, for someone like me or anyone who is new to Esxi this is going to help.

Here is the code for the script, all you need to do is to copy to your lab esxi, obviously if any one using production esxi they already know how to manage this.

https://github.com/yukthr/auts/tree/master/vmware_scripts

Requirement – I have 5 Vm-machines and i would like  to start them via script and also power them off.

First things, list the Vm-instances

Now that we have it, let explore the Continue reading