Continuing – Bird Project and Extensions
I will keep this post very short, In order to fuel and make the model sustain, https://r2079.wordpress.com/2022/03/18/capturing-bird-photos/, I have thought about including a solar panel and powering the system with a Lithium Ion battery.
I have installed the system and it’s been working fine, I understood I was very bad at Soldering.
- TP4056 -> This takes energy from solar panel and feeds to Li Battery.
- Solar Panel -> I have a small panel which fits on the hosting box
- Lithium Ion Batteries and charger (4x2500mAH)
I hope this runs continuously using solar power.
-Rakesh
How to enable Recurring Cloudwatch Alarms?
If you do not deal with AWS/CloudWatch you don’t have to read this post.
What: The issue was simple, we had a cloud watch alarm for Lambda Function invocation, now the way I wanted was to send us recurring email notifications if the alarm was not addressed, apparently this is not a cloud-watch native feature and there is a work-around for this.
Short Story: Implementing this will have a new step function which will start alarming based on an alert-timer, this won’t by default apply to all the alarms that you configure, you need to specifically tag it with a keyword, more of those options detailed in the article, so based on the timer you set, Cloud-watch say send SNS notification or any action of your choice to get implemented.
Why Article if you have a Link that explains ?: To start with not everything that I encountered was straightforward, the install process requires you to have a docker environment, and a proper node install and then a CDK install, I never did that and it did waste some time so I wanted to document and also this might help anyone to implement the same.
Spoilers:
Capturing Bird Photos
Continuing from the previous post which I have set up to install a bird feeder, I have mentioned that I would install a camera based on the https://mynaturewatch.net/daylight-camera-instructions project and I did install it, this took some really good photos and would like to share some of them.
Few points:
- The battery that I used is a 10000mAh power-bank and it lasted for 2-3 days
- The box that I used is a baby food box and I nailed them to the boundary wooden poles from inside.
- This Bird Feeder is not the one I 3d Printed, this is an old bird feeder that someone gave it to me
Glad to see these pics, many more to come with Idea to install and make it sustain through solar power
-Rakesh
A Bird feeder from re-usable plastics – we have to start something small to protect other beings.
Disclaimer: I speak about reducing plastic and still I use a 3d printer to print a very specific fixture, I have weighed in options and plastic that I end up printing will be out there hopefully feeding birds or can safely be re-cycled until I come up with something innovative, as of now this is the least I could come upon re-using old bottles, helping birds feed and using some plastic to print a fixture. If I have to buy a commercial one, that is again some plastic and I won’t end up re-using it as well, so 3d printing this small fixture for me outweighs the other currently viable solution and this is something I can give out to neighbours and they can easily associate with.
Am proud to say that I have started to take some responsibility towards the environment and other living beings, that to say it’s my first step and now I appreciate even more what environmentalists and other people do to protect other beings, earth and fellow human beings which we don’t have any time to notice and appreciate them.
Once I started doing the below things which are very trivial I feel good and Continue reading
Aws Advanced Networking — Enhanced Networking — 3— Intel 82599 VF
The first two posts covered SRIOV/ ENA settings and use-cases, the next one in the series is about using Intel 82599 Virtual Functions adapter.
Post 1: https://r2079.wordpress.com/2021/12/28/enhanced-networking-1-sriov-aws/
Post 2: https://r2079.wordpress.com/2022/01/08/enhanced-networking-2-verifying-ena/
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sriov-networking.html
Instance Types: Select from the following supported instance types: C3, C4, D2, I2, M4 (excluding m4.16xlarge), and R3.
How can we verify:
At instance Level:
aws ec2 describe-instance-attribute --instance-id i-xx --attribute sriovNetSupport
[cloudshell-user@ip-10-0-119-152 ~]$ aws ec2 describe-instance-attribute --instance-id i-xx --attribute sriovNetSupport
{
"InstanceId": "i-xx",
"SriovNetSupport": {
"Value": "simple" -> Simple indicates its enabled, if not enabled its empty
}
}At an AMI Level
aws ec2 describe-images --image-id ami-07d8796a2b0f8d29c --query "Images[].EnaSupport"
[cloudshell-user@ip-10-0-119-152 ~]$ aws ec2 describe-images --image-id ami-07d8796a2b0f8d29c --query "Images[].SriovNetSupport"
[
"simple"
]At an Interface Level
ubuntu@ip-172-31-25-23:~$ ethtool -i ens3
driver: ixgbevf
version: 4.1.0-k
firmware-version:
expansion-rom-version:
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: yes
ubuntu@ip-172-31-25-23:~$Latest Ubuntu HVM and Amazon Linux AMI have drivers for Enhanced Networking, IXGBEVF module and required modules for sriovNetSupport.
There is also the best practices Github guide for ENA Linux best practices and operating system optimisation.
https://github.com/amzn/amzn-drivers/blob/master/kernel/linux/ena/ENA_Linux_Best_Practices.rst
Enhanced Networking – 2 – Verifying ENA
Previous Post – https://r2079.wordpress.com/2021/12/28/enhanced-networking-1-sriov-aws/
As Discussed in Part-1 SRIOV (Enhanced Networking on EC2) can be enabled in two ways, the first in the series is by far the simplest one, Enabling it with using ENA (Elastic Network Adapter).
Great, would that work for any instance – The answer is NO!, below are the specifications, to make summarize any Instance other than C4, D2, M4 instances smaller than m4.16xlarge, or T2 from current generation Instances.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html
How do I Check:
The Latest Ubuntu / Amazon Linux AMI include the module required for Enhanced networking with ENA installed and enabled for support, if you happen to use the old AMI’s the procedure listed in the above webpage will help
Testing:
I spin up a T3.large instance and below is how it looks like
You also have the option to verify it in a Cloud shell
How do you know if AMI supports it?
Finally on the interface itself
The next post will be similar but would cover an Intel specific Network Adapter.
-Rakesh
Tello Drone — Initial Review and Python Programming
What is this about ?
A small starter programmable drone that is meant for mostly indoor and has 80 grams of weight with a flight time of 13 minutes.
Some Highlights
https://www.ryzerobotics.com/tello
– Small Drone with under 100 grams weight
– Suitable for kids and anyone who is starting out to get into drones and programmable ones
– Two sites (Tello and tello.edu) offers various addons to support learning and make it more customised for learning
– 13 minutes of Flight time
– 100m Flight distance
– 720p HD Transmission
– 2 Antennas
– you can also have VR headset compatibility
– In collab with DJI and Intel
– Operation via various Apps (Paid and Free ones), Programming Languages ( we are interested in this)
Fancy Features
– Throw and Go — you can just toss Tello into the air
– 8d Flips (needs battery more than 50%)
– Bounce mode (flies up and down from your hand)
Things that I didn’t like :
-First and foremost, there is no way this connects to your home Wifi, Drone goes into an AP Broadcast mode (meaning this starts broadcasting its own AP and we have to connect to it)
This Continue reading
Enhanced Networking – 1. SRIOV – AWS
In preparation for AWS Advanced Networking Speciality Track, I have come across Enhanced Networking and application in various scenarios. I am going to cover this in a few different parts in the blog posts.
Enhanced Networking
Enhanced Networking – boils down to speeds (100Gbps vs 10Gbps) , from the picture below you can use enable Enhanced Networking either by choosing ENA stands for Elastic Network Adapter (which is supported by most of current gen instances) or by using Intel 82599 Virtual Function Interface which supports speeds upto 10Gbps with specific Instances.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html
I have heard about SR-IOV but never did a deep dive into what exactly it does, to be honest. While going through Enhanced Networking documentation on AWS documentation, I have seen mention of SR-IOV at several places.
What is SR-IOV ?
More on SR-IOV is written in this blog-post, but in short SR-IOV which stands for Single Root I/O Virtualisation provides device virtualisation by using virtual-functions/DMA(Direct memory access)/virtual function devices (VFD’s) and lowering CPU interrupts.
So, the first question is if this enabled by default ? No. you have to manually enable it, per Amazon website https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#current-gen-instances , all current generation instances Continue reading
Image/Data set collection – ML Hobby Post 2
Why is this required?
- While Deep Learning is in many different categories ( like Vision, Text (NLP), Audio, Recommendation system), My interest is always in Vision or anything which involves images, I somehow to find it closer to embed into a hobby than other aspects
- Any Part of Image-based Learning, involves a set of Images that are needed to train the model on what is our intention to recognize parameters, for example, consider the below image set called CIFAR-10.
- https://www.cs.toronto.edu/~kriz/cifar.html — This is the URL, this has a predefined collection of 10 different categories of Images that can be used for training the classifier of an image if the Image is among any of the 10 categories.
- Ordinarily, Let us say you wanted to categorize a Dog in a given picture or you had a scenario where you had months collected time-lapse photos and wanted to categorize Dog and filter out the images which involved Dog in it, you don’t have to collect so many images to train, test and build the model, Data is readily available
- Other Popular datasets — https://docs.fast.ai/data.external.html has lot of pre-collected Datasets at our disposal and https://www. Continue reading
ML Models – Random Notes 1
As always been interested to explore technology, Machine Learning has been in my mind for quite some time now. The pre-requisites to know or get into ML have always been associated with a Ph.D. Degree, High-end mathematics, and obviously a ton of understanding around software engineering. As the platform and technology evolved ease of learning machine learning is becoming more and more approachable for most people.
I tried to understand ML and implemented a few models before using different frameworks. I have started with Tensorflow in my free time and then took some internal training and each training had each frame work, initially, it was Apache MXNet, later it was Pytorch, my head was both in the right place and not in the right place I should say, I prefer widely a practical approach to things and was trying to deploy few models from everyday scenarios that am around, I focussed mostly on getting the model deployment with the platform/framework while I have pushed the technicalities for later part of the study which usually works for me, this time as this is not a full-time profession nor I have any intentions to get into this as a full-time ML Continue reading
Having a Cloud Router – Always handy!
Note:
This can be set up for free of cost in a sense of having 100$ cloud credit and Trial license for vMX and is for testing for a limited amount of time for and setting a router in the cloud, In the long run, even this testing would cost some money.
This will guide you on how to set up a prod system properly on AWS – https://www.juniper.net/documentation/us/en/software/vmx/vmx-aws/topics/concept/vmx-aws-overview.html
I Wish
Often I wish that I have some sort of online cloud router (either Cisco or Juniper) to look around some knobs and also to do some rapid testing or rehash of some of the concepts. I prefer mostly Juniper for their awesome logical-systems concept and it’s easy to build a good number of 14 routers that can also run MPLS.
Solution
With Docker and Kubernetes, many of the NFV functions are even easier, I came across implementations of Juniper cRPD Route Reflectors in production environments and its all based on docker-containers, and their orchestration and vMX and other product lines are very well engineered at this point in time and no longer they are at starting stage.
What is that Continue reading
Vodfaone NB-IOT, Fipy and AWS
Timeline : 2-4 months on a occasional weekends
Having a sensor to sense things is one story, Having an on-field sensor is another story in itself. While powering up the sensor and weather proofing is not the intention of this blog post but sure they are the other aspects that one needs to address before moving to higher layer communications.
success only once out of 3 times of sowing
Issue – Have a sensor on Field, make sure it communicates itself to AWS IOT end platform. This is not that simple, that too without using high cost end products which abstracts everything from end-user. To put it in simple terms, you have a field or a farm which is quite away from your home and without a power resource, what would you do ?
Lora is one the most effective option that works for us, while I did setup a Gateway and end-node Lora Device, this post aims at something more simple without involving Lora, I will cover Continue reading
AWS DeepLens and Telegram saving bird’s food and plants from my Dog!
Weekend project after a while!. Not Long ago, I did mention that my cute Dog is responsible for destructing most of the wall and paper, when enquired pro’s said its part of puppy growing up and it still needs time.Along with Dog, I personally would love to feed and enjoy petting other species to be honest out of which birds and plants are major, there are earthworms and sea life but Dog is not a threat to them based on the placement.
Links
Git: https://github.com/yukthr/auts/tree/master/aws_deeplens
Credits: Some of the code block is picked up from : https://aws.amazon.com/blogs/machine-learning/building-a-smart-garage-door-opener-with-aws-deeplens-and-amazon-rekognition/
Tripod stand: https://www.amazon.co.uk/gp/product/B08FGSV9CZ/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8&psc=1
AWS DeepLens
https://aws.amazon.com/deeplens/ – Is ML/AI enabled camera which can be used at edge inferencing. Slipping down to my usual choice of words, this piece of equipment has an onboard camera , a Gpu , a Cpu and ubuntu host OS. Easily integrates with AWS and runs a local Lambda for inferencing, hence decisions are made on the device instead of reaching out to AWS to do the logic.
What is the problem!
There is a small garden at the rear of the Continue reading
A Near 2 year Part Time Project Done – A sustaining Model!
Ever since I got interested in plants getting some sort of metrics has been a part time obsession.
Iteration 1 – No wireless and no outdoor model with always on usb power.
Iteration 2 – Learnt about ESP8266 microcontroller and deep sleep feature
Iteration 3 – Saving battery through deep sleep and battery power instead of usb mains, Adding ESP32 Microcontroller.
Iteration 4 – Study about Lithium Ion batteries
Iteration 5 – Making model wireless and usb free power, running on batteries
Iteration 6 – Containerising the entire software and integration with AWS and Telnyx
Iteration 7 – Making the model sustaining on itself through solar power and making it weather resistant
This completes an End to End IOT Model with a micro controller , a moisture sensor and two lithium ION batteries which get charged based on a small solar panel. Am going to extend this to LoRa Wan and will try to achieve ultra low power long distance.
The idea is that there is an allotment 6 kms from the place I live and I will see if AWS and LoRa Wan Supports me for protocol needs.
Docker containers associated with this project
DeepLearning – Intro Fun
I am enjoying insights into deep learning and computer vision and how machine learning is applied on images. While Deep Learning and Nueral networks are subjects for experts in field, I wanted simply to be an end user to do some experiments on my dog and nothing more. I have to mention that I had internal resources to get training on image augmentation techniques, DLL Nueral network frameworks pretrained to be used readily but am still trying to find some way into what this technology is all about and how to implement in my AWS Deeplens.
https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf
This is a good book to get introduced into pytorch and deep learning terminology. There is a very interesting examples of doing image manipulation of a horse to zebra, and how you can transfer that learning to anything.
Below is how I used on my Dog and results, I had fun starting this and hope you will have fun too, you dont need any great programming or math to just access some of the Neural networks and do some analysis
Normal Image
I wrote this post to let anyone interested to know that I dint had to write any complicated coding Continue reading
Keep up with Tech ? How Far ?
One of the hardest things to do is to keep up with tech, now thats some thing every once in a while we hear from our colleagues and we might say it to ourselves as well. There are two things which needs some discussion here
- New certifications in Networking scope
- New paths like programming and cloud services
After working for a while, am at a point where I have been exposed to so many technologies within networking, there is Service provider, datacenter, enterprise and other aspects like programming and cloud services.
The choice to take both paths is not efficient if I give it a good thought, but on the other hand its always good to know other aspects of the job as well, am not trying to be diplomatic but rather speaking from experience here.
One path is to be an expert in certain aspect of the field, the other side is to invest time to learn new things, choice is not easy, any field is vast and it takes life time just to be good at one thing, imagine trying to be good at multiple things, it wont work for many people.
Other day I had to explore Continue reading
Quick personal vpn – wireguard with aws
I have written about wire-guard previously about how easy it is to set-up a personal vpn
What is the issue: I have never explained the use-case clearly in that post, let me try to re-attempt the write-up again
you see, on a personal basis I need to access few websites which are hosted in India, issue with these websites is that they dont allow any traffic external to the country
There are many browser based proxies out there which can do this task just fine and also paid services, my problem is that when you are exchanging user/password information and financial transactions over these proxies you don’t know how exactly all this data getting exchanged and transmitted
Few Tips from my experience before i get into the post :
-> you need to change ubuntu instances ipv4 forwarding so that it will forward packets through the instance also nats it
root@ip-172-31-34-66:~# cat /etc/sysctl.conf | egrep -i ip_forward net.ipv4.ip_forward=1 root@ip-172-31-34-66:~#
-> Make sure you dont start routing everything first, take public DNS’s and test the system with allowed_ips in the configuration file else you will loose internet access and if not done correctly you will Continue reading
Progress on Image classification and Home-assistant and Dyson integration
Contunuing from the previous post, I was determined to learn some sort of image classification in my free time, I went with Tensorflow/keras as the language or ml language as they have a lot of tutorials around it.
Below youtube series from tensor-flow team will help you get started if you are interested in this
The other part that was a sort of problem for a long time is integration of Dyson Fan with any sort of automation platform like Alexa or python api, Dyson does have an app but apparently they have some issues in UK/Ireland integration, after some research I found that home-assistant supports integration and after a lot of documentation and trials, it looks like home-assistant has now support for it.
Here is the below file that you need to use in the configuration.yaml snippet.
dyson:
username: yourdysonemailaccount
password: yourpassword
language: GB
devices:
- device_id: xxx-xx-xxxxxx
device_ip: a.b.c.d
This has now control for any sort of automation
-Rakesh
Aws Deeplens – meet the devil dog – part 1
Now on first look she is so adorable, don’t be fooled by looks this dog is responsible for destruction of usb cables, foot wear , headphones, trash can openings and garden destruction.
Idea inspired from : https://youtu.be/ALKz1eKj4n0
Aws deeplens – https://aws.amazon.com/deeplens/
So here is the idea, over the course of next few days to months, I will start capturing many constructive and destructive images of my dog and start training a AI model which will give us a reasonable idea on what exactly she is up to when we leave her alone and alert us accordingly.
Am not a ML developer let alone be an expert, but aws makes it easy to train and deploy models and you don’t have to know much to get started. I developed another ML model with aws rekognition which identifies unwanted plants and shrubs in a raised bed.
I got set up this deeplens and deployed a model project in no less than 5 minutes and am being honest. Here is a sample model which identifies cat vs dog , this model comes is among example models.
It has a Mqtt topic which you can subscribe as well
-Rakesh
Improving the moisture model – the final phase
Back from this post http://r2079.com/2020/03/17/telnyx-api-p-sms-and-aws-iot-saves-my-plants-every-single-time/, I did see that using Telnyx and Aws MQTT did indicate the moisture. All good, so why drag this topic ?
Links reference :
https://www.espressif.com/en/products/socs/esp32 – microcontroller used in the project
https://vruzend.com/ – lithium ion 18650 batteries
https://micropython.org/ – micropython
http://telnyx.com/ – Telecom provider
https://aws.amazon.com/ – Cloud provider
Well, there are multiple aspects to the design itself
- The system always required usb external power – batteries well dint last long
- The system always required to be internal or inside home – reason it wasnt weather resistant
- Costly [ Mqtt Push and and if plant dried out, it would make Telnyx API send me a lot of messages which in turn is a cost ]
- I never wanted a 24×7 system, I wanted something which comes online once per day and then sort of sleeps for rest of the day
- Wanted to use micropython, python programming language is something I always found easy on beginners like me, where as C and CPP are difficult in my opinion.
- I wanted something small and less or Continue reading