Python Pandas – Reading text files got a lot better.
Problem Statement – Have a list of VMWare Instances and wanted to quickly iterate and see what VMs are powered on, this has nothing to do with Vmware or ESXI, look at the below image, programmatically I want to pick two specific rows and perform dictionary operations on them.
Git – https://github.com/yukthr/auts/blob/master/random_programs/pandas_vmware_esxi.py
for someone like me who is partly into programming and mostly into networking, anything effective is easy, I could have gone with the other way of doing this but since Pandas are effective i would text parsing with them.
I only want two sections which are of interest to me, in a normal way maybe I should have pasted this in excel and Do a text to the column and do a manual data extraction, but wanted to do it with Pandas and they are powerful.
First things first
- Read with Pandas
- Read the text
- If possible convert into CSV (as I have some experience with csv file operations)
- Then split the Columns and make the dictionary for further use
Let’s see how this proceeds
Pandas make it so powerful to just pick Columns out of the fly, in this case, I wanted Continue reading
Working with distance sensor – solving overhead water tank problem
This is not a networking post.
Schematic , sensor code and spec – https://www.linuxnorth.org/raspi-sump
My code – https://github.com/yukthr/auts/blob/master/random_programs/water_sensor.py
1x Breadboard
1x Raspberry pi zero w
1xhcsr04 ultrasonic sensor
2x1kohm resistors
Just as a side note i do not have any intro into resistors nor electronics, but what all i did was to follow some posts written by people who already did it, its not hard believe me, if i could do it any one should easily be able to do it as am very far away from electronics and programming, so let these things not overwhelm you.
Problem – Am not sure in other parts of the world, but place I live has an over head water Tank which stores water. So every day you technically turn on a water motor which sucks water from a reserve under the ground and pumps it to all the the way to a three store high building
So what’s the issue – The issue is that we have no clue what’s the current water level in the tank nor how long would it take to fill the water tank. There are two tribal ways by which we Continue reading
Sonoff Flashing and steps towards Smart home
This post is no way related to Networking ! 
It’s always my fascination to implement a specific tech towards home automation so as to have more time in hands. The first step towards this was to use a Tp-link smart switch which came with Alexa, now problem with that is that Tplink app was not properly responding to Alexa and TP-Link smart switch in itself was costly.
I wanted something low-cost, something which I can experiment yet be cost effective, After some re-search I came across Sonoff-Basic model.
https://sonoff.itead.cc/en/products/sonoff/sonoff-basic
I integrated with Amazon echo but again there were app issues (default app is ewelink), though it was working fine something was missing, I couldn’t tinker it to my wish
I found then there is a open-source version of a similar firmware called TASMOTA and we have to flash this sonoff with the new firm-ware of Tasmota, I have to tell you here there is every possibility you would brick your device and make it useless in the process.
https://github.com/arendst/Sonoff-Tasmota/wiki/Prerequisite
Everything is so well documented, there is no need to re-iterate things here, but just to show case another successful implementation.
Need-less to say you are playing with AC/DC Continue reading
Cleared JNCIE-DC
After close to a year of study and after one failed attempt I cleared it in the second attempt. Here is my experience in short and tips to prepare for the exam
Reading Resources
-> Juniper Dayone – Anything and everything related to DC
-> QFX Series Book
https://www.safaribooksonline.com/library/view/juniper-qfx5100-series/9781491949566/app03.html
-> JNCIP – ADCX/TDCX/DCX
-> Datacenter Network / EVPN – Overview
https://www.safaribooksonline.com/library/view/evpn-in-the/9781492029045/ch04.html
Lab Resources
-> If you are into any serious preparation you need to consider the below git resource, its awesome and you can practice pretty much everything even on a laptop and also in your flights/travel.
https://github.com/Juniper/vqfx10k-vagrant
You need to know a bit of vagrant and need to have VirtualBox and ansible installed, not hard by any means, all it takes is a days dedication to make your laptop ready for these, let me know if you want me to write a blog post for the setup.
-> I had Dell R810 Server, https://r2079.wordpress.com/2018/01/05/my-dc-virtual-lab-setup-insights/ , I did most of my practice on this one.
-> I also had the privilege of using hardware resources and examined the ideal configuration for a production network and learned few things from them.
PrettyTable – Printing anything in Table
Hi,
As the scripting and programming deals in logging into the device and fetching data, there will be a time where presentability of Data matters. PrettyTable is one such package which greatly helps in reading things
A simple example, re-visiting the code to get the list of routes from the Device
PrettyTable will help in tabulating the Data, the installation and usage can be found here
Pip Package – https://pypi.org/project/PrettyTable/
Usage – http://zetcode.com/python/prettytable/
Once we have the code, let take a look at how the program looks
The Table form looks something like this
Hope this help for anyone who gets started with presentability of Data, honestly, there was one time I got crazy with the print statement just to make the data presentable.
Fetching Pcap Files and running Wireshark – Junos Pyez
As a part of some content development and revision on MPLS, I was trying to capture some MPLS Pcap Files from the Vmx Router. The problem was that every time I capture a file, I had to first scp or sftp into the device and then after successful import, I had to run Wireshark on it.
The Aim of the post is to fetch the pcap file and run Wireshark on it. Am not a decent programmer by any stretch of the imagination, anyone can see that from my code but the point that matters is that it works and you should give it a try too.
https://github.com/yukthr/auts/blob/master/junos_pyez/getfile_executewireshark.py– Is the file
When I run this program to fetch a non-existent Pcap file
And when I query for an existent file
-Rakesh
Raspberry Pi and AWS IOT – First steps
Hi All,
I have slightly changed this to networking, but the intention and my current use is to measure the water level of a sump, since that deviates from the network blog writing, i have extended the same to a Router.
Purpose – Have a Router and also a Syslog Server which monitors my internal network (This can easily be extended to a Car / Moisture Sensor or a Temperature/Humidity Sensor), what we want to do is to make sure if any anomaly is seen in Log Messages, it logs to IOT service. We can then take this up as a Part-2 writing to perform a specific action / automated on what action can be taken to mitigate
Discussion about configuring a Linux device is out of scope, so lets think that we all have that setup. What happens next ?
Lets quickly see our python script, which parses for a anomaly, in this lets say when someone runs a ping command, well its not a anomaly but will do for our use-case.
logparse.py is our program, so i have imported it into the readily available sample program provided by AWSIOT Kit, so you dont have to know Continue reading
Revisiting – Why IGP sync with LDP is required ?
Hi All,
I was preparing some content on MPLS for a training session and as a part of it, was going through LDP. The interesting aspect is very obvious
-> LDP is dependent on IGP
-> What ever Draw-backs IGP has will be inherited by LDP
-> LDP has to be enabled on the Interface to exchange Labels, else it wont consider the exit-interface from IGP and hence there will be no LSP’s
So far so good and makes sense as well
I will not be boring with command line outputs in this case
-> I have disabled the interface between R3/R4 so if R3 Has to reach R1, it will use R3-R2-R1 path
All good, Am going to just tweak the metric of the interface on R3 -> R2 before I enable back the R3 – R4
Now let me enabled the interface between R3-R4
-> It has a Better cost
-> It has not been enabled for LDP
If we go back to R3, to examine the result
This is dangerously familiar for me 
, There is a LDP neighbor, but No routes are present in Inet.3 (neither for R1 or R2) as Routes Continue reading
Book Review – How SRE Relates to Devops
Link – https://www.safaribooksonline.com/library/view/how-sre-relates/9781492030645
Quick Read – Few Pages
Other Books which are related to SRE
-> Site Reliability Engineering: How Google Runs Production Systems
Few Points that I liked
– Quick read , hardly an hour
– Intro on how Devops got introduced
– what needs to be improved – Key Idea “No More Silos”
– stress on non-localization of knowledge, lack of collaboration
– New Job Role called SRE – Site Reliability Engineering
– Operations is a Software problem and Work to minimize Toil are some best examples of productivity and how we should view
– Key Idea – “Automate This Year’s Job Away” and “It’s Better to fix-it over-selves than blame someone else”
Good Read to understand the over picture of SRE Role and some of the
work Discipline Ideas.
-Rakesh
Analyzing data with Pandas Package – An Intro to Pandas
Hi,
Title may sound extremely Hitech for someone who never heard about pandas ;), but what I have written is a simple hello world equivalent program, which I guess should start to help my day to day analysis, as always the aim is to let anyone know the advantage of something than hammering with some theory !
I was going through various python packages available to analyze data and came across pandas package along with numpy package. These are not there by default in Python installation and if you like them to be on your system, you should install them via PIP, I have them installed already hence you can see that it complains in the below image.
Note :
Understand why you need to have something like Pandas / Numpy even if you have never heard of them, that’s the point of this tiny program
Imagine, how you would solve this if you never knew Pandas/Numpy and you will see the power of these
packages, again you don’t have to know these to realize their full power.
Now coming to the requirement, here is a sample spreadsheet that I have below, its a CSV Sheet which contains certain Continue reading
Cleared JNCIS-Devops
Last week I went to write JNCIS-Devops exam, I was under an impression that I may not be able to clear it but good did happen!
First and Foremost
-> I had the official training for JAUTcourse – The course is extremetly helpful as it provides the precise material and also the structured lab environment for you to explore and study, nothing beats a class-room study and training environment
But, after appearing I can tell you that you dont really require the offiicial training (if that is the only thing stopping you to think about the exam), the exam will test you for your understanding of automation philosophy and also how Juniper Implements it.
Topics of Interest
– Juniper pyez – understand how everything helps in Pyez
Dayone Books Helps – https://www.juniper.net/uk/en/training/jnbooks/day-one/automation-series/junos-pyez-cookbook/
– Juniper ansible – https://www.juniper.net/uk/en/training/jnbooks/day-one/automation-series/junos-pyez-cookbook/
-Book – Network Programmability and Automation
https://www.safaribooksonline.com/library/view/network-programmability-and/9781491931240/
All you need to have are couple of VMX devices a Linux machine and you should be able to deploy all of the automation efforts discussed in above books.
You dont have to know the code in your head or how to write a Continue reading
Plotting the interface flap – That’s some analysis
Hi,
What started to be a exploration project is now turning out to be pretty useful for me in day to day analysis. Back in days when I worked in support, there was nothing to predict or really worry about historical events for any future work, just grep for logs and you are done with the last flap and analysis.
Customers / Networks now look for more data, while there are systems which do the telemetry and prediction, from an analysis point of view, as an engineer I want to know if the device or a circuit over an interface is stable over a period of time or even if it flaps what is the likely time and day it flaps in a week for a smoother migration.
Requirement : Plot a simple graph analyzing the interface flaps over a period of one week for a specific interface and decide the actions next from the log messages.[in this case i used a junos device]
Well grepping the logs is not something new for a seasoned engineer but having visual data will prove to be useful for a cutover or migration.
There are systems which can do this work on Continue reading
Poweron and Poweroff Esxi instance from CLI using a Python script
Hi,
I have to agree that to start a esxi node i was depending heavily on a windows VM and then was using a VSphere client to connect to a Esxi 5.5.
In a typical day all of my VM’s are hosted on Esxi and am not any advanced user of esxi by any stretch of Imagination.
It came down to a point where i had to manually click close to 8 VMS in order to boot up and all this was sort of irriatating for me, so i wrote a very basic script which can do this for me. Most of the experienced VM admins have been doing this for very long, for someone like me or anyone who is new to Esxi this is going to help.
Here is the code for the script, all you need to do is to copy to your lab esxi, obviously if any one using production esxi they already know how to manage this.
https://github.com/yukthr/auts/tree/master/vmware_scripts
Requirement – I have 5 Vm-machines and i would like to start them via script and also power them off.
First things, list the Vm-instances
Now that we have it, let explore the Continue reading
Integrating the configuration build – Next steps
Hi, The last post link below, I got introduced to a CI System and basics of it. https://r2079.wordpress.com/2018/04/03/using-travis-ci-continuous- integration-with-github/ This post goes further in actually using the CI system. All the code is hosted here https://github.com/yukthr/auts.git -> Requirement is very simple This is a very basic program which introduces anyone to Jinja2 and yaml syntaxing Problem - Have two interfaces ge-0/0/0 and ge-0/0/1, we have to use Yaml / Jinja2 and Pyez to develop the configurational syntax for this and later on a CI system need to validate the build. The code hosted in Github above. intf.yml - will have all the interfaces template.j2 - will have the appropriate Jinja2 template.py - will have the python program combining these two So, we write the codeFinally build the CI file, but here we also buld the dependencies because when CI starts to validate it needs to have all the appropriate software installed. It amuses me to the point, it spins up the VM and then install the dependencies and then it validates our code. I have come a long way from manual verificaitons / lab testing / CI testing now This is how Continue reading
Using Travis CI (Continuous Integration) with GitHub
Hi , Am Planning to write a in detail usage of how we can leverage Aws cloud - ansible - github - travis-(ci/cd) with in our networking deployment space. As of now, I will quickly author how you can leverage the usage of Travis CI in our experimental space. You can find more about Travis CI - Here - .org of travis will help to run Opensource Projects https://travis-ci.org/ I am using AWS cloud desktop to do the changes to the code, get it pushed to git-hub and then integrate everything if Travis CI passes the checks To let you know the workflow in a very simpler way -> You write any code or config related to networks on AWS cloud desktop -> push the code into git-hub in a branch later to be integrated into Master Branch -> Setup Travis to automatically run some pre-defined tests -> If all successful, we will merge the code into our master branch -> Lets write a very basic code in a branch and push to git-hub
The github page has been integrated with Travis-CI
Travis CI peforms the required checks, here it just checks for syntax, obvious this can be exetended Continue reading
Filtering EVPN Routes with PyEZ
Hi,
From the previous EVPN Blog post, the next logical thing was to do is to filter out EVPN routes from the device and have them analyzed for a task.
I have made the program to analyze Type 2 and Type 3 routes
Things to take-away:
-> Understand what module has what advantages – For example OP here has route-table level calls which can extract routing information from the device, while Device has the specific usecase of opening a connection to the device.
-> Understand there are many more efficient ways of doing the same thing, but goal here is to make the script work, there is no impact on the Device as the script parses the data offline once we get it, so even if there was a better way, just do the things the way you want.
Below is the sample program which is written to analyze the routes from vQFX series
Am not a Programmer by any stretch of imagination as you can see my code ;), but this helps and gets my tasks done, if any one of you want to download this here is the below github link
https://github.com/r2079/JDC/blob/master/evpn_pyez.py
Regards
VxLan – Short Story Lab
Hi,
Note: Its perfectly possible to do VXLAN/EVPN on VQFX and VMX, all you have to do is to setup a good lab over ESXI or if you want you can do it over Eve-ng emulator. I personally did it via ESXI.
Am not covering the petty BGP configuration of Full-Mesh and Evpn-BGP configuration, its very simple, this post mainly Aims at show-casing the quick and short way of setting up EVPN/VXLAN in Vqfx and over vMX
On the way to some DC Lab Practise, I wanted to quickly show you guys how to lab up Vxlan on Vqfx and Vmx.
Intention – I was reading on VxLan and as most of my learning comes around seeing things first and understanding the later, I felt uncomfortable too soon reading at the Documentation, I wanted to learn it by doing.
Here is the topology
Goal – Build Vxlan / Evpn with a very small set-up to under the workings.
First things first
-> In order to build any VxLan, you need to have some underlay and some overlay. Our underlay is BGP (It can be anything you see, as long as it can exchange Loopback Space and establish IP Continue reading
MC-LAG
MC-Lag
Everyone mostly know what MC-Lag does, for the benefit its a variant of LAG where the Down stream Devices share LAG interface on two Physical devices instead of One, I know its confusing let see a sample topology
Vqfx1 will see the upstream as 1 ae1 instead of two different VMX devices, this has its own advantage and the entire discussion is something out of scope of this blog post.
Blog Post Goal – Demonstrate MC-Lag on VMX and Quickly highlight the options Common and different in MC-LAG, a ready reference for someone going for an exam or a implementation.
Take-Away’s
-> ICCP is the protocol between the nodes
- Uses TCP/IP to replicates control plane and forwarding traffic between peers
- One stand-by and one active (active responsible status control)
- ICCP messages exchange configuration information to ensure both peers use correct LACP Parameters
- ICL-PL (interface between Mx1 and Mx2 ex) supports Multiple MC-Lag Between the peers so its recommended to be a AE.
-> ICL-PL HA
- A Keep-alive message is exchanged between MC-LAG peers which is recommended to be the management connection
- If ICL-PL fails, keep alive is still through the management Continue reading
My DC Virtual Lab Setup – Insights
Hi,
I have been getting a lot of requests offline and online on the lab setup I use, I have to say I have tested many things and finally settled with Vmware ESXi 5.5. Eve-ng was good but not good enough when spawned with multiple instances of qemu, maybe because my host operating system might be slow enough for it.
I will cover the connectivity for two VMX devices but the logic Remains same for any device connectivity.
What I have Already
-> Vmware Esxi5.5
-> Dell R810
-> Insane amount of time to waste :), I hope you won’t fall into the same path.
You need to have the OVA files, all settings are straightforward, don’t even worry about Memory allocations yet, there is time for that and also the networking part, import the OVA
There will be two OVA images
Vfp – forwarding plane
VCP – Control Plane
General import – No Rocket science – Don’t worry about any settings as of now
This is how my VCP looks like – Again don’t worry about any networking here, catch here – VCP has only two networking Adapters – One for Fxp0 which is the Continue reading
PYEZ Script – Commit the configuration or Indicate the Diff
Requirement – Connect to a MX device to commit a configuration on the Device, if there is any un-committed configuration, Script should hold and display the un-committed configuration.
Basically,
JNPR.JUNOS – Device – helps us to connect to device
JNPR.JUNOS – Util.Config – helps to issue config related (Rollback/Config) etc
\033 – Helps the print statement to display in colored Format, [91m – Red , [1m – Bold , [0m indicates to end the color format
I have some uncommitted configuration on the device and hence we expect the script to indicate us the uncommitted configuration
Once i Fix the config, on the device, lets see if the configuration from the script gets fixed
This is an Intro to how we can start deploying or to check any devices which has any UnCommitted configuration on the Devices and Proceed Accordingly.
-Rakesh