AWS DeepLens and Telegram saving bird’s food and plants from my Dog!
Weekend project after a while!. Not Long ago, I did mention that my cute Dog is responsible for destructing most of the wall and paper, when enquired pro’s said its part of puppy growing up and it still needs time.Along with Dog, I personally would love to feed and enjoy petting other species to be honest out of which birds and plants are major, there are earthworms and sea life but Dog is not a threat to them based on the placement.
Links
Git: https://github.com/yukthr/auts/tree/master/aws_deeplens
Credits: Some of the code block is picked up from : https://aws.amazon.com/blogs/machine-learning/building-a-smart-garage-door-opener-with-aws-deeplens-and-amazon-rekognition/
Tripod stand: https://www.amazon.co.uk/gp/product/B08FGSV9CZ/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8&psc=1
AWS DeepLens
https://aws.amazon.com/deeplens/ – Is ML/AI enabled camera which can be used at edge inferencing. Slipping down to my usual choice of words, this piece of equipment has an onboard camera , a Gpu , a Cpu and ubuntu host OS. Easily integrates with AWS and runs a local Lambda for inferencing, hence decisions are made on the device instead of reaching out to AWS to do the logic.
What is the problem!
There is a small garden at the rear of the Continue reading
A Near 2 year Part Time Project Done – A sustaining Model!
Ever since I got interested in plants getting some sort of metrics has been a part time obsession.
Iteration 1 – No wireless and no outdoor model with always on usb power.
Iteration 2 – Learnt about ESP8266 microcontroller and deep sleep feature
Iteration 3 – Saving battery through deep sleep and battery power instead of usb mains, Adding ESP32 Microcontroller.
Iteration 4 – Study about Lithium Ion batteries
Iteration 5 – Making model wireless and usb free power, running on batteries
Iteration 6 – Containerising the entire software and integration with AWS and Telnyx
Iteration 7 – Making the model sustaining on itself through solar power and making it weather resistant
This completes an End to End IOT Model with a micro controller , a moisture sensor and two lithium ION batteries which get charged based on a small solar panel. Am going to extend this to LoRa Wan and will try to achieve ultra low power long distance.
The idea is that there is an allotment 6 kms from the place I live and I will see if AWS and LoRa Wan Supports me for protocol needs.
Docker containers associated with this project
DeepLearning – Intro Fun
I am enjoying insights into deep learning and computer vision and how machine learning is applied on images. While Deep Learning and Nueral networks are subjects for experts in field, I wanted simply to be an end user to do some experiments on my dog and nothing more. I have to mention that I had internal resources to get training on image augmentation techniques, DLL Nueral network frameworks pretrained to be used readily but am still trying to find some way into what this technology is all about and how to implement in my AWS Deeplens.
https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf
This is a good book to get introduced into pytorch and deep learning terminology. There is a very interesting examples of doing image manipulation of a horse to zebra, and how you can transfer that learning to anything.
Below is how I used on my Dog and results, I had fun starting this and hope you will have fun too, you dont need any great programming or math to just access some of the Neural networks and do some analysis
Normal Image
I wrote this post to let anyone interested to know that I dint had to write any complicated coding Continue reading
Keep up with Tech ? How Far ?
One of the hardest things to do is to keep up with tech, now thats some thing every once in a while we hear from our colleagues and we might say it to ourselves as well. There are two things which needs some discussion here
- New certifications in Networking scope
- New paths like programming and cloud services
After working for a while, am at a point where I have been exposed to so many technologies within networking, there is Service provider, datacenter, enterprise and other aspects like programming and cloud services.
The choice to take both paths is not efficient if I give it a good thought, but on the other hand its always good to know other aspects of the job as well, am not trying to be diplomatic but rather speaking from experience here.
One path is to be an expert in certain aspect of the field, the other side is to invest time to learn new things, choice is not easy, any field is vast and it takes life time just to be good at one thing, imagine trying to be good at multiple things, it wont work for many people.
Other day I had to explore Continue reading
Quick personal vpn – wireguard with aws
I have written about wire-guard previously about how easy it is to set-up a personal vpn
What is the issue: I have never explained the use-case clearly in that post, let me try to re-attempt the write-up again
you see, on a personal basis I need to access few websites which are hosted in India, issue with these websites is that they dont allow any traffic external to the country
There are many browser based proxies out there which can do this task just fine and also paid services, my problem is that when you are exchanging user/password information and financial transactions over these proxies you don’t know how exactly all this data getting exchanged and transmitted
Few Tips from my experience before i get into the post :
-> you need to change ubuntu instances ipv4 forwarding so that it will forward packets through the instance also nats it
root@ip-172-31-34-66:~# cat /etc/sysctl.conf | egrep -i ip_forward net.ipv4.ip_forward=1 root@ip-172-31-34-66:~#
-> Make sure you dont start routing everything first, take public DNS’s and test the system with allowed_ips in the configuration file else you will loose internet access and if not done correctly you will Continue reading
Progress on Image classification and Home-assistant and Dyson integration
Contunuing from the previous post, I was determined to learn some sort of image classification in my free time, I went with Tensorflow/keras as the language or ml language as they have a lot of tutorials around it.
Below youtube series from tensor-flow team will help you get started if you are interested in this
The other part that was a sort of problem for a long time is integration of Dyson Fan with any sort of automation platform like Alexa or python api, Dyson does have an app but apparently they have some issues in UK/Ireland integration, after some research I found that home-assistant supports integration and after a lot of documentation and trials, it looks like home-assistant has now support for it.
Here is the below file that you need to use in the configuration.yaml snippet.
dyson:
username: yourdysonemailaccount
password: yourpassword
language: GB
devices:
- device_id: xxx-xx-xxxxxx
device_ip: a.b.c.d
This has now control for any sort of automation
-Rakesh
Aws Deeplens – meet the devil dog – part 1
Now on first look she is so adorable, don’t be fooled by looks this dog is responsible for destruction of usb cables, foot wear , headphones, trash can openings and garden destruction.
Idea inspired from : https://youtu.be/ALKz1eKj4n0
Aws deeplens – https://aws.amazon.com/deeplens/
So here is the idea, over the course of next few days to months, I will start capturing many constructive and destructive images of my dog and start training a AI model which will give us a reasonable idea on what exactly she is up to when we leave her alone and alert us accordingly.
Am not a ML developer let alone be an expert, but aws makes it easy to train and deploy models and you don’t have to know much to get started. I developed another ML model with aws rekognition which identifies unwanted plants and shrubs in a raised bed.
I got set up this deeplens and deployed a model project in no less than 5 minutes and am being honest. Here is a sample model which identifies cat vs dog , this model comes is among example models.
It has a Mqtt topic which you can subscribe as well
-Rakesh
Improving the moisture model – the final phase
Back from this post http://r2079.com/2020/03/17/telnyx-api-p-sms-and-aws-iot-saves-my-plants-every-single-time/, I did see that using Telnyx and Aws MQTT did indicate the moisture. All good, so why drag this topic ?
Links reference :
https://www.espressif.com/en/products/socs/esp32 – microcontroller used in the project
https://vruzend.com/ – lithium ion 18650 batteries
https://micropython.org/ – micropython
http://telnyx.com/ – Telecom provider
https://aws.amazon.com/ – Cloud provider
Well, there are multiple aspects to the design itself
- The system always required usb external power – batteries well dint last long
- The system always required to be internal or inside home – reason it wasnt weather resistant
- Costly [ Mqtt Push and and if plant dried out, it would make Telnyx API send me a lot of messages which in turn is a cost ]
- I never wanted a 24×7 system, I wanted something which comes online once per day and then sort of sleeps for rest of the day
- Wanted to use micropython, python programming language is something I always found easy on beginners like me, where as C and CPP are difficult in my opinion.
- I wanted something small and less or Continue reading
New hosting Space on AWS
My blog was at https://r2079.wordpress.com and its now moved to https://r2079.com. Why this change?
First and Foremost – Thrill and Challenge
Secondly – Customization and Cost
Don’t get me wrong, I dint migrate because I wanted to get into web development, its not the case and Am not even at intermediate Level there!
Why – This is a custom domain. This is hosted with Route53 Amazon, WordPress is build on AWS custom instance. The Reasons are very simple
- I Wanted to include short flash videos in my old blog, word press apparently dint allow it
- Secondly, I wanted to take frequent backups – was only possible with a huge yearly cost for customization
- Paying for 1 Route53 domain, opened a lot of DNS options, I cancelled my Dynsubscription which was costing 54 Euros per year, while AWS would get most of it witht 12 USD.
- Most Importantly, I don’t have to pay anything just to block Ads!
So, This is where it is, I will try to maintain the website now and see how this goes, Till now Infrastructure was maintained and patched by WordPress , from now probably i have to take care of it.
Wireguard Server and QR Code scan in the mobile app – It’s that simple to set-up a VPN
Fancy a VPN build in under 10 minutes? , there are many vendors outside who offer mobile App and connectivity all through the world, most of the times ofcourse they under perform. Be it for beating Apps which impose Geographical restrictions etc.
What is wireguard ?
https://www.wireguard.com/ – you can read all about it
Why do you need it ?
Simple and easy to build your own VPN service plus Pay as you Go by turning off the cloud instance and Wire-guard has some cutting edge encryption at the software level, which makes it performs better even in cloud instances.
Do i need to Install anything?
All you need is to run a docker image.
https://hub.docker.com/r/linuxserver/wireguard – and you will have all install instructions
Ok i have installed whats Next?
Get your Mobile App, and scan the QR code generated by the system.
Next ?
Nothing, download the app and you are good to by scanning this and you are on your way to your own VPN
How do i verify my traffic stats ?
Log into docker and execute wg, all stats will be readily available
-Rakesh
BPF Compiler Collection – BCC in short
Network Tracing sometimes is really important, although most of the times tcpdump utility is quite handy there are other tools that can make life much easier.
while am no expert in eBPF and scripts, i do know how to use bcc-tools in some scenarios.
https://github.com/iovisor/bcc/blob/master/INSTALL.md#ubuntu—binary
Alright What is the scenario:
Let’s start with something small, you want to measure TCP connection latency.
Other interesting options, you can capture lifetime, stats and most importantly TCP-Retransmissions
Ebpf filters are safer and more powerful to implement, give this a consideration during any Linux troubleshooting scenarios.
-Rakesh
Tizona – Cli for Cloud Music
This is not a Network technology related post
https://tizonia.org/ – Opensource Project
In their own words ‘The most powerful cloud music player for the Linux terminal, with support for Spotify (Premium), Google Play Music (free and paid tiers), SoundCloud, YouTube, TuneIn radios, Plex servers and Chromecast devices.’
What this has to do with this Post:
I have just implemented in an old Raspberry Pi which I was about to throw away. Secondly, it’s a passive project from at least 4 years to play cloud music on Linux CLI.
Why this obsession?
Ever think about music play all day but a very very low volume, am that kind of person. I don’t really listen to it very particularly all the times, but some music lingering in low volumes [Am speaking about extremely low volumes] will always help.
Why not use your LAPTOP ?
The very point of spreading it accross the home and not having to deal with GUI interfaces. Its much better for me not to have control as well, I just skip the tracks and will never be able to listen to new tracks.
Secondly, I dont want to spend a lot of money on crazy hardware with Continue reading
Ostinato – Drone Mode – Traffic Generator
I have started this below article detailing a small subset of what Ostinato is capable of doing, in short, is a traffic generator with great options to modify and dictate the type of traffic that you want to generate and can do it at decent speeds
What is the Issue: The issue is that Ostinato in default mode is only supported in GUI mode, which is good for many cases but some times especially if you want to send some decent traffic to stress test your MPLS LSP a PC working from home scenario over VPN is not an ideal bet.
Explain more: Well, imagine you have a Cloud provider Instance, most of the instances are CLI based while Ostinato is GUI based.
So How to Proceed: After seeing what Tool is capable of we purchased the full suite which has Python Api support Plugin as well. So it operates something like this
How does it look:
so you have start the drone first, before doing anything else, i was stuck in this phase for more than 2 hours just because i was dumb not to read all the instructions
You can build the packet and save Continue reading
Telnyx api/sms saves my plants! Every single time.
When its a holiday, I do some house-hold farming, mostly into Hydroponics which is based on water and associated nutrients and does not require soil as a medium.
Let me show an example.
As you can see, Plant drinks up water, and the one which you are seeing is lemon plant from its seed stage, Monitoring water level is extremely important and for me there are many more in home so cant be keeping track of everything.
So sensor gets the data, Raspberry Pi talks to AWS IOT securely since Microcontrollers are still a pain to handle SSL and Rpi makes it easy and after processing MQTT message AWS IOT will process based on the Rules, in our case if the treshold is below 500 then plant is drying up.
Why Not Inbuilt SES/SMS with AWS ?
SES works great but I check my emails only once in two days and there is no way to alert based on the email, SMS I wanted to used multiple number pool (thinking to expand) and Cost with Telnyx is Really Amazing to do any Communication services
I have tried other vendors and results are not great either my carrier has some integration problems Continue reading
Ostinato – Packet Generator – Test your broadcast and mitigation strategies
I used ostinato long before when it was in early stages probably, at that point I never had any real need to use a packet generator as the place I was working was already equipped with IXIA and Spirant best in Class Packet generators.
https://ostinato.org/ – is the link
Whats the use case – Well I wanted to test LSP loadbalancing and also specific scenario of BUM traffic and how well it can be contained lets say in Switching level QFX and also VPLS level
Its doing a good job, though it started as open source, author now sells it with some minimal fee to keep up with the development costs.
You could also use scapy and i wrote a small program to do this , problem is scapy sorts of waits for some of the responses and i have not yet figured out what needs to be solved there
Testing Ostinato on QFX and its really good so far. Storm control got in and stopped transmitting any packets through the interface.
At the end, this is a nice to have tool for any sort of packet crafting and limited scale testing for proof of concept scenarios.
-Rakesh
Lora – Test and results – awesome
What is LORA – Best reference – https://www.thethingsnetwork.org
I will anyways attempt to define this as a newbie who just got into this technology
“with the help of a specific Receiver and Transmitter and using Radio waves you can send small bits of information to long ranges without the need of repeaters or wireless IP networking gear. “
Why is this useful – I have many use cases simple one being measuring soil moisture for plants and measuring water level in an overhead tank.
I currently have a very expensive setup of a Raspberry Pi zero wireless, a Repeater to repeat my Base Ip Network and a solar panel to power the power-hungry sensor as Raspberry pi does not know how to go into Deepsleep and wakeup.
Lora Makes it simple and even in reasonable living space, wireless routers do not perform well to cover the range up till the back yard garden.
What do you need – Listed in the below article and its nicely written
Most of the Sample Code – https://randomnerdtutorials.com/ttgo-lora32-sx1276-arduino-ide/
I have tested its range till 300m without any issues at all, that’s like 4 times what my wireless router can do!
Dyndns and Openvpn – Remote Management
I have visited my home and was doing some hobby IT setup with Raspberry Pi’s, the problem is that i had problems many times accessing my home PC is another Remote Location due to many reasons, lets say crappy ISP. I contacted my ISP and they said I need to take a static IP and also pay for opening up two non standard ports. Its like you pay to get tortured and then additional headache of Port forwarding.
To add more to the pain, the IP that i get from my upstream provider is a Private IP, wow I havent seen that for a while. Anyways, to get around this I was thinking about using OPENVPN as a solution along with Dyndns.
Now, setup is very simple
Clint-pc (Location 1) ———-AWS(OPENVPN)————Client-pc (Location 2)
Why AWS -> Accessible and Cost
Problem is changing IP, I dont have any business requirement or criticality to buy a Elastic IP , but whole point will be lost if my clients wont know what to access, worse I will never have access to location-2 if am in location-1 to change IP Addresses
I have mapped OPENVPN with dyndns script.
https://help.dyn.com/ddclient/
This really solved Continue reading
Editing Files in a Docker Container
This is the quick and easy way I learned to edit some files within a Docker container. Professional DevOps engineers might be doing it in a different way, this is the network engineers way of doing things 
-Rakesh
Pomodoro Timer with Esp8266 ,micropython and slack – a small weekend project
Hi All,
Pomodoro technique is really effective, more than the technique it’s more or less like a good stress buster to me.
https://en.wikipedia.org/wiki/Pomodoro_Technique
There are few problems with the timer management
1. It’s impractical for me to manually add 25-minute timer every time
2. I can do a small script but again my computer should always be on
3. Use some app or use a timer with sound (both of them are really disturbing for myself and also colleagues around me)
What did I use
- With anything involving DIY/IoT, there are two important aspects – It should be small and portable and secondly, cost should below.
Components Used :
- ESP8266
- 0.96Inch 128×64 OLED display (Pictures below)
- Micropython ( I could have gone with C++ but honestly I don’t know the language, hence I had to go through a lot of pain to make it work in microphone)
- Slack for daytime notifications about timer stages
Challenges :
- How do I make sure I don’t look at the clock every 25 minutes? Sound is not an option, so I used a slack webhook to notify me
- Code was big and hence there were memory allocation issues in Continue reading
Grafana and Influx – Infrastructure Engineers Language
If you want to understand what Infra engineer speaks and use a tool provided by them you need to have some exposure to the tool itself, you don’t have to be an expert.
Monitoring systems that I see nowadays are mostly centric around Prometheus while the Database used for storing any time-series events is InfluxDB. How do you actually map beautifully, its via Grafana
Grafana – https://grafana.com/
Influxdb – https://www.influxdata.com/
Prometheus – https://prometheus.io/
Now the problem here is that many tools are programmed on a daily basis, from a Network Engineer point of view I understood a few things.
Not everything you need to know the end to end like an Expert for that tool and some choices is purely based on Cost than anything else.
How would you really understand this? I set up a BME680 sensor in my home and will precisely use Grafana and Influx to map the recordings
Tools Used
https://learn.adafruit.com/adafruit-bme680-humidity-temperature-barometic-pressure-voc-gas
Raspberry Pi 3
Docker Images – Grafana and Influx
Sample Influx Script – https://github.com/yukthr/auts/blob/master/random_programs/influx-test.py
And finally Beautiful Grafana
All of this is open source and are not hard after the invent of Docker. Give Continue reading