Rakesh M

Author Archives: Rakesh M

Energy Independence, at least for gadgets and electronics; what’s the point of growing spinach and Mint?

This post is a bit different from my usual networking content. I won’t be discussing power grid inefficiencies or pollution. After learning some principles from permaculture, an ethical design science focused on life, it’s more of a personal reflection. I’m concerned about our society’s reliance on non-producing, ever-consuming and our mindset to depend on external producers. Even simple things that can be eliminated or produced without much effort have become needlessly complex, creating unnecessary consumption, erasing the human spark to make or say no to mindless consumerism, and creating vast dump yards. We used to be producers but have become ultimate consumers of life’s simple things.

TLDR: It’s not about affordability or money; I think everyone here can afford spinach or charge their devices happily, paying a service provider; the question is, why are we dependent on external grids and super-stores for simple things in life that were free at one point of time in our society, over consumerism, isolated and end up creating our own silo kingdoms?

Why did it make sense to me?

At a very high level, we all know that the climatic changes and food production, in general, have affected the topsoil so much that Continue reading

A While, Career update and Ends with various Junos load commands.

It has been quite some time since my last blog post. The past few months have been busy, leaving little time to write. I am happy to share that I have started working as a site-reliability developer at Oracle. While it has only been a short while, I am enjoying the work.

Reflecting on the past year, I am happy to say it has been productive overall. I have had the opportunity to collaborate with various networking and software teams, which has taught me a lot about high-scale traffic patterns and migrations. Working in an operations role has been beneficial since it requires constant fire-fighting and documentation for improvements. This has made me more aware of traffic patterns and ways to improve reliability.

What can be improved?

However, I could benefit from more study on the software architecture and distributed system design parameters from a business perspective. I want to write more this year; the first half was better, but lacked in the second half, where most of the writing was private.

From a personal standpoint, 2024 needs more travel absolutely and some improvements in garden automation for summer 2024

Other updates

Finally, I have been working on some JNCIA-Devops Continue reading

AWS Advanced Networking Speciality 1.3:Considerations for encryption and authentication with load balancers (for example, TLStermination, TLS passthrough)

< MEDIUM :https://towardsaws.com/aws-advanced-networking-speciality-1-3-considerations-402e0d057dfb >

List of blogs on AWS Advanced Networking Speciality Exam — https://medium.com/@raaki-88/list/aws-advanced-network-speciality-24009c3d8474

High-Level points that the article covers — Exam topics

Data Protection in ELB

AWS Shared-Responsibility Model defines how data protection applies in ELBs. It boils down to AWS protecting global infrastructure while the service consumer is more responsible for preserving the content and control over the hosted content.

Few important suggestions for accessing/Securing

  • MFA for accounts
  • TLS 1.2 or TLS 1.3 for AWS resource communication
  • Logging with AWS CloudTrail
  • Amazon Macie — Discovering and securing sensitive data in S3
  • FIP140–2 — Fips Endpoint

Encryption

Encryption at rest: Server-side encryption for S3 (SSE-S3) is used for ELB access logs. ELB automatically encrypts each log file before storing it in the S3 bucket and decrypts the access log files when you access them. Each log file is encrypted with a unique key, which is encrypted with a master key that is regularly rotated.

Encryption in Transit:

HTTPS/TLS traffic can be terminated at the ELB. ELB can encrypt and decrypt the traffic instead of additional EC2 instances or current EC2 backend instances doing this TLS termination. Using ACM (AWS Certificate Continue reading

AWS Advanced Networking Speciality 1.3: AWS Load Balancer Controller for Kubernetes clusters

< MEDIUM:https://raaki-88.medium.com/aws-advanced-networking-speciality-1-3-aws-load-balancer-controller-for-kubernetes-clusters-d491149b99c9 >

List of blogs on AWS Advanced Networking Speciality Exam — https://medium.com/@raaki-88/list/aws-advanced-network-speciality-24009c3d8474

Various Aspects of this post

Before understanding LoadBalancer Service, it’s worth understanding a few things about NodePort service.

NodePort Service :

NodePort service opens a port on each node. External agents can connect directly to the node on the NodePort. If not mentioned, a randomly chosen service is picked up for NodePort. LoadBalancing here is managed by front-end service, which listens to a port and load balances across the Pods, which responds to service requests.

LoadBalancer Service:

Like NodePort Service, the LoadBalancer service extends the functionality by adding a load balancer in front of all the nodes. Kubernetes requests ELB and registers all the nodes. It’s worth noting that Load Balancer will not detect where the pods are running. Worker nodes are added as backend instances in the load balancer. The classic-load balancer is the default LB the service chooses and can be changed to NLB(Network Load Balancer). CLB routes the requests to Front-end, then to internal service ports Continue reading

AWS Advanced Networking Speciality 1.3 : Configuration options for load balancer target groups (for example, TCP, GENEVE, IP compared with instance)

Advanced Network Speciality Exam — Blogs

<MEDIUM : https://towardsaws.com/aws-advanced-networking-speciality-1-3-5484de6c8da >

A Target group routes requests to one or more registered targets. They can be EC2 Instances, IP addresses, Kubernetes Cluster, Lambda Functions etc. Target groups are specified when you create a listener rule. You can also define various health checks and associate them with each target-groups.

Typical load-balancer components

What is Geneve, and what is the context with ELB: Generic Network Virtualisation Encapsulation

In the context of Gateway Load Balancer, a flow can be associated with either 5-Tuple or 3-Tuple.A flow can be associated with either a 5-tuple or 3-tuple flow in load balancers.

A 5-tuple flow includes the source IP address, destination IP address, source port, destination port, and protocol number. This is used for TCP, UDP, and SCTP protocols.

A 3-tuple flow includes the source IP address, destination IP address, and protocol number. This is used for ICMP and ICMPv6 protocols.

Gateway Load balancers and their registered virtual appliances use GENEVE protocol to exchange application traffic on port 6081

References :

https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html

https://datatracker.ietf.org/doc/html/rfc8926 Continue reading

Speciality 1.3 : Integrations of load balancers and other AWS services (for example, Global Accelerator, CloudFront, AWS WAF, Route 53, Amazon Elastic Kubernetes Service [Amazon EKS], AWS Certificate Manager [ACM])

< MEDIUM : https://raaki-88.medium.com/aws-advanced-networking-speciality-1-3-deedc0217ea6 >

Global Accelerator — A service that provides static ip addresses with your accelerator. These IP addresses are Anycast from the AWS edge network, meaning the global accelerator diverts your application’s traffic to the nearest region to the client.

Two types of Global Accelerators — Standard Accelerators and Custom Routing accelerators.

Standard Accelerators uses aws global network to route traffic to the optimal regional endpoint based on health, client location and policies that the user configures, increasing availability and decreasing latency to the end users. Standard-accelerator endpoints can be Network Load balancers, Application load balancers from load balancing context. Custom routing accelerators do not support load balancer endpoints as of today.

When using accelerators and Load-balancers, update DNS records so that application traffic uses accelerator end-point, redirecting the traffic to load-balancer endpoints.

CloudFront and AWS-WAF with ELB:

When using an application load balancer in ELB, cloud-front meant to cache the objects can reduce the load on ALBs and improve performance. CF can also protect ALB and internal services from DDOS attacks, as with AWS WAF. But for this to succeed, administrators Continue reading

AWS Advanced Networking Speciality 1.3 AWS Advanced Networking: Connectivity patterns that apply to load balancing based on the use case

< MEDIUM: https://raaki-88.medium.com/aws-advanced-networking-speciality-1-3-23eb011b74df >

Exam Topic — https://d1.awsstatic.com/training-and-certification/docs-advnetworking-spec/AWS-Certified-Advanced-Networking-Specialty_Exam-Guide.pdf

Previous posts :

https://towardsaws.com/aws-advanced-networking-task-statement-1-3-c457fa0ed893

https://raaki-88.medium.com/aws-advanced-networking-speciality-1-3-3ffe2a43e2f3

Internal ELB — An internal Load balancer is not exposed to the internet and is deployed in a private subnet. A DNS record gets created, which will have a private-IP address of the load-balancer. It’s worth noting to know DNS records will be publicly resolvable. The main intention is to distribute traffic to EC2 instances. Across availability zones, provided all of them have access to VPCs.

External ELB — Also called an Internet-Facing Load Balancer and deployed in the Public subnet. Similar to Internal ELB, this can also be used to distribute and balance traffic across two availability zones.

Example Architecture Reference — https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/deploy-an-amazon-api-gateway-api-on-an-internal-website-using-private-endpoints-and-an-application-load-balancer.html?did=pg_card&trk=pg_card

– Rakesh

AWS Advanced Networking Speciality 1.3 AWS Advanced Networking: Different types of load balancers and how they meet requirements for network design, high availability, and security

< MEDIUM: https://raaki-88.medium.com/aws-advanced-networking-speciality-1-3-3ffe2a43e2f3 >

https://medium.com/towards-aws/aws-advanced-networking-task-statement-1-3-c457fa0ed893 — Has intro details for the speciality exam topic

Different types of Load-Balancers

  • CLB — Classic Load Balancer
  • ALB — Application Load Balancer
  • NLB — Network Load Balancer
  • GLB — Gateway Load Balancer

High Availability Aspect: ELB (Can be any load-balancing) can distribute traffic across various different targets, including EC2-Instances, Containers, and IP addresses in either a single AZ or multiple AZs within a region.

Health Checks: An additional health check can be included to ensure that the end hos serving the application is healthy. This is typically done through HTTP status codes, with a return value 200 indicating a healthy host. If discrepancies are found during the health check, the ELB can gracefully drain traffic and divert it to another host in the target group. If auto-scaling is set up, it can also auto-scale as needed.

Network Design: Depending on the type of traffic and Application traffic pattern, the load and burst-rate choice of load-balancer will differ.

Various Features — High-Availability, High-Throughput, Health-Checks, Sticky-Sessions, Operational-Monitoring and Logging, Delete-Protection.

TLS Termination — You can also have integrated certificate management and SSL decryption which offloads end-host CPU load and acts as a central Continue reading

AWS Advanced Networking — Task Statement 1.3: How load balancing works at Layer 3, layer 4, and Layer 7 of the OSI model

< MEDIUM: https://towardsaws.com/aws-advanced-networking-task-statement-1-3-c457fa0ed893 >

Ref: https://aws.amazon.com/elasticloadbalancing/

Why Elastic Load Balancer?

  • To Distribute incoming application traffic across EC2 instances / Containers / IP Addresses and these entities, generally, can be called targets and grouped as Target-Groups.
  • Another advantage is performing health checks, such as packet-loss or high-latency, and can be integrated with Auto-scaling, hence Elastic.
  • Depending on the ELB type and its operational requirement, this can further be subdivided into CLB, ALB, GLB, and NLB.

CLB — Classic Load Balancer

– AWS Recommends ALB today instead of CLB
– Intended for EC2 instances which are built in EC2-Classic Network
– Layer 4 or Layer 7 Load Balancing
– Provides SSL Offloading and IPv6 support for Classic Networks

ALB — Application Load Balancer

  • Works at the 7th Layer of the OSI Model.
  • Supports applications that run in Containers
  • Supports Content-based Routing
  • HTTP/HTTPS, Mobile Apps, Containers in EC2, and Microservices benefit greatly from ALB

Why Elastic Load Balancer?

  • To Distribute incoming application traffic across EC2 instances / Containers / IP Addresses and these entities, generally, can be called targets and grouped as Target-Groups.
  • Another advantage is performing health checks, such as packet-loss or high-latency, and can be integrated with Auto-scaling, Continue reading

Set up your Own wireguard-based Cloud VPN server without installing software on your laptop — GL-A1300 Slate Plus Wireless Router

< MEDIUM: https://aws.plainenglish.io/set-up-your-own-wireguard-based-cloud-vpn-server-without-installing-software-on-your-laptop-1bdc41b0b55 >

Note: This requires the purchase of a wireless router which is capable of running a Wireguard package in this case it’s Slate-Plus GL-A1300 and I do not have any affiliate or ads program with them, I simply liked it for its effectiveness and low cost.

The Need :

For one reason many of us want a VPN server which does decent encryption but won’t charge us a lot of money, in some cases, it can be done free of cost and in others for not want us to install a variety of software which messes up with internal client routing and also against some of the IT-Policies, even if it’s a browser-based plugin.

The Choice :

Wireguard: https://www.wireguard.com/ — VPN Software, Software-based encryption, extremely fast and light-weight.

GL-A1300 Slate-Plus — Wireless Router with support for Wireguard which is not a feature in many of the current market routers, had OpenWrt as the installed software.

Features

The GL-A1300 Slate Plus wireless VPN encrypted travel router comes packed with features that will make your life easier while travelling. Here are just a few of the most important:

AWS Rekognition — A simple classifier without having to build a lot of in-house or Cloud Environments — Sample Bird-Project

< MEDIUM : https://aws.plainenglish.io/aws-rekognition-a-simple-classifier-without-having-to-build-a-lot-of-in-house-or-cloud-a43fc6e9e8a >

A few weeks ago, I set up a bird feeder and used it to capture bird images, the classifier itself was not that accurate but was doing a decent job, what I have realised is that not every time we end up with highly accurate on-board edge classification especially while learning how to implement them.

So after a few weeks, there were a lot of images some of them sure enough had birds while some of them were taken in Pitch Dark and am not even sure what made the classifier figure out a birdie in the snapshot from the camera.

Now for me in order to make an Image, I have to rely on a re-classifier doing the job for me, initially, I thought I will write a lambda-based classifier as a learning experiment, but then I thought it was a one-time process every 6 months or so, so I went ahead with a managed service option and in this case, its AWS Rekognition, which is quite amazing.

https://aws.amazon.com/rekognition/

Code Snippet used for Classification

import os
import concurrent.futures
import boto3

def detect_birds(image_path):
# Configure AWS credentials and region
session Continue reading

Setting up your own Cloud-GPU Server, Jupyter and Anaconda — Easy and complete walkthrough

< MEDIUM: https://medium.com/@raaki-88/setting-up-your-own-cloud-gpu-server-jupyter-and-anaconda-easy-and-complete-walkthrough-2b3db94b6bf6 >

Note: One of the important tips for lab environments is to set an auto-shutdown timer, below is one such setting in GCP

I have been working on a few hosted environments which include AWS Sagemaker Notebook instances, Google Cloud Colab, Gradient (Paperspace) etc and all of them are really good and needed monthly subscriptions, I decided to have my own GPU server instance which can be personalized and I get charged on a granular basis.

Installing it is not easy, first, you need to find a cloud-computing instance which has GPU support enabled, AWS and GCP are straightforward in this section as the selection is really easy.

Let’s break this into 3 stages

  1. Selecting a GPU server-based instance for ML practice.
  2. Installing Jupyter Server — Pain-Point Making it accessible from the internet.
  3. Installing Package managers like Anaconda — Pain-Point having Kernel of conda reflect in Jupyter lab.

Stage-1

For a change, I will be using GCP in this case from my usual choice of AWS here.

Choose GPU alongside the Instance

Generic Guidelines — https://cloud.google.com/deep-learning-vm/docs/cloud-marketplace

rakesh@instance-1:~$ sudo apt install jupyter-notebook

# Step1: generate the file by typing this line in console

jupyter notebook  Continue reading

Nvidia Jetson Nano — Initial thoughts, impressions, AI Specialist Certification.

< Medium: https://raaki-88.medium.com/nvidia-jetson-nano-initial-thoughts-impressions-ai-specialist-certification-2b9af95e1bba >

While browsing through various ways to get my AI-enabled bird camera setup, I came across Nvidia Jetson Nano, there are varied versions of this product and availability are limited, I Am in Europe and ordered the Nano Developer kit from Amazon US, shipping was fast with a good amount of inbound tax as well.

https://developer.nvidia.com/embedded/jetson-nano-developer-kit — is the one I have purchased while both new and old versions are available.

Unboxing Video :

Unboxing Video — Nvidia Jetson Nano

Initial Impressions and disadvantages:

  • I am surprised that this does not have WIFI and only works on an Ethernet adapter, so I ended up purchasing a Wifi-dongle which operated out of the box, I recommend a TP-Link adapter but anything will work and here is the link — https://www.amazon.co.uk/dp/B07LGMD97Z?psc=1&ref=ppx_yo2ov_dt_b_product_details
  • One of the factors is that Nvidia on its website claims that they support Raspberry Pi cameras which I have many and the reality is that they won’t end up supporting any of the latest cameras which are based on the IMX7* series, they can support the IMX219 series if you are lucky but anything other than that is a Continue reading

Image Search with Bing — ML/AI Fast.ai and AWS Sage Maker

< MEDIUM: https://aws.plainenglish.io/image-search-with-bing-ml-ai-fast-ai-and-aws-sage-maker-61fae1647c >

If you have heard about awesome AI course that fast.ai offers which is free of charge then you should definitely checkout. https://course.fast.ai/ is where you will find out all the details.

Course takes a very hands on approach and anyone can write and bring up their ML model in under 2 hours of the course. As a part of any image classification one of the basic requirements is to have number of images which is often referred to as Dataset and this Dataset is split into Test/Train/Validation for the model training.

For some of the easier examples, you can rely on search engine to give you those images for you, previously fast.ai used Microsoft’s bing.com for image search but later they replaced it with DDG (DuckDuckGo). while DDG is really nice I had throttle issues and some of the packages were outdated and hard to read.

So, I have re-written the same image-search Python function which uses Microsoft’s bing.com search engine

What are Pre-req:

  • Azure Cloud account
  • API Keys for the service

Procedure to generate keys

  • Go to Azure, create a Resource
  • Search for “Bing Search”
  • Select “Bing Continue reading

Automating Green-House Photos through Event-Bridge Pipes and Lambda

< MEDIUM: https://medium.com/towards-aws/automating-green-house-photos-through-event-bridge-pipes-and-lambda-434461b89f55 >

Image sent to Telegram

I have a small greenhouse which was in the pipeline for over 2 years and I finally decided to build it. Whoever is in gardening will agree that anything grows better in the greenhouse at least it appears to be so.

Now, the initial impression is all good but I have plans to learn and explore both the plant sides of things and also some using some part of image analysis for a predictive action, for all that to happen I need a camera and a picture to start with.

Hardware —

  1. Raspberry Pi — I have an old one at home, you can technically have any shape or size as long as it fits your need, My recommendation — is Raspberry Pi Zero

What are the other simplest alternatives:

  • I could have written a Python script which directly could have sent the image to Telegram storing the image locally or uploading it to S3

The reason I choose to go with Event-bridge Pipe is to put this more into practice and from there on connect more Lambda and step-functions for future expansion of the project.

Architecture Diagram for sending Images Continue reading

Home Automation — Finally Roller Curtains and Nightmares

< MEDIUM: https://raaki-88.medium.com/home-automation-finally-roller-curtains-and-nightmares-b8ef1fc473d9 >

Am a fan and enthusiast of home automation, tried various things in the past and now settled with few things which I would like to share.

  1. Light Automation is the first and most popular thing to do — Initially I started wiring with PIR sensor available in Amazon, later upgraded the entire light system to Philips Hue and along with Philips Motion sensor, so far so good. Its expensive than my initial solution but had to choose that because of wired vs wireless situation in home
  2. Smart-plugs are another common set of devices — while I have both combinations of commercially available one and also I personally flashed few of Sonoff smart switches with Tasmota firmware

. . .

Lets get to the Curtain Rollers — So for these here is the catch, I have a remote for these and thats about it, nothing more nothing less, My ideas were mostly around having network connectivity and manipulating them.

  • First and foremost thought that these are blue-tooth based and am wrong
  • I went to DFROBOT and bought IR-Transmitter and IR-Receiver, nope they dint work
  • I went Amazon and bought some other IR-transmitter and Receiver — Again, wasted a Continue reading

Enabling Nested-Virtualisation on Google cloud platform Instance

< MEDIUM: https://raaki-88.medium.com/enabling-nested-virtualisation-on-google-cloud-platform-instance-7f80f3120834 >

Important Excerpt from the below page.

https://cloud.google.com/compute/docs/instances/nested-virtualization/overview

You must run Linux-based OSes that can run QEMU; you can’t use Windows Server images.

You can’t use E2, N2D, or N1 with attached GPUs, and A2 machine types.

You must use Intel Haswell or later processors; AMD processors are not supported. If the default processor for a zone is Sandy Bridge or Ivy Bridge, change the minimum CPU selection for the VMs in that zone to Intel Haswell or later. For information about the processors supported in each zone, see Available regions and zones.

Though there are many use cases, I will speak from a networking standpoint. Let us say you need to do some sort of lab based on popular vMX Juniper or Cisco or any other vendor, if you have a bare metal instance, you have the ability to access the virtualised CPU cores and allocate them to the Qemu which will be the underlying emulator.

Issue

Almost by default most of the cloud providers will disable access to VT-x because of various reasons and some instances are not capable of supporting this by default. So either choose a custom instance with Continue reading

Buffer overflow — Linux Process — Stack Creation and Inspection

< MEDIUM: https://raaki-88.medium.com/buffer-overflow-linux-process-stack-creation-and-i-d6f28b0239dc >

Process and what happens during process creation have been discussed in this post previously — https://medium.com/@raaki-88/linux-process-what-happens-under-the-hood-49e8bcf6173c

Now, let’s understand what is buffer overflow:

A buffer overflow is a type of software vulnerability that occurs when a program tries to store more data in a buffer (a temporary storage area) than it can hold. This can cause the program to overwrite adjacent memory locations, potentially leading to the execution of malicious code or the crashing of the program. Buffer overflow attacks are a common method used by hackers to gain unauthorized access to a system.

Generally, C and C++ languages are more vulnerable to Buffer Overflow while programming languages like Python and Go have implementations which protect stack.

I have written the program in Python but had to use underlying C functionality to achieve similarly.

#!/usr/bin/python3
import ctypes
import pdb
buffer = ctypes.create_string_buffer(8)
ctypes.memmove(buffer, b"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",1000)
print('end of the program')

This is a very simple implementation where we created a buffer which can hold 8 bytes of memory, next we will create a new object which moves from one block of the memory to another but with a newer size, which will Continue reading

Organise Efficiently with Zapier — Dropbox / S3 / Sheets— Integration to organise scanned documents and important attachments

< MEDIUM: https://medium.com/@raaki-88/organise-efficiently-with-zapier-dropbox-s3-sheets-integration-to-organise-scanned-4f47d51f4a54 >

One biggest problem with my google drive is that it’s flooded with a lot of documents, images and everything which seems really important during that instant of time with names which are almost impossible to search later.

I tried various Google APIs and Python programs with Oauth2.0 and its integration is, not something easy and needs tinkering for the OAuth consent page.

I wanted something easier, a workflow when I scan documents in the scanner-pro app on IPAD/iPhone and upload them to storage it should then be organised with certain rules which can be easily searchable and also listable. What I mean by listable is that I need some sort of Google Sheet integration which can just enter the filename and date once it’s uploaded to S3.

When there is an excel sheet even if the search is available it gives me so much pleasure to fire up pandas and analyse or search for it, just makes me happy

Note: I am a Paid user of Zapier and using S3 is a premium app, Am not an advertiser for Zapier in any way, I found the service useful

Moving on, here is the workflow

Continue reading

Basic Step-Functions Input and Output and Lambda— Passing Data From one to another

< MEDIUM: https://medium.com/aws-in-plain-english/basic-step-functions-input-and-output-and-lambda-passing-data-from-one-to-another-b433666f6216 >

With so much focus on serverless in Re-Invent 2022 and the advantages of Step Functions, I have started to transform some of my code from Lambda to Step Functions.

Step-Function was hard until I figured out how data values can be mapped for input and how data can be passed and transformed between Lambda functions. I have made a small attempt for someone who is starting in step functions for understanding the various steps involved.

Basically, Step Functions can be used to construct business logic and Lambda can be used to transform the data instead of transporting with Lambda-Invokes from Lambda Functions.

Let’s take the following example

I have step_function_1 which has the requirement to invoke another lambda if my_var is 1 else do not do anything.

This is a simple if-else logic followed by the lambda-invoke function

Now, the power of step-functions will come into play to write these conditional and also pass data from Lambda to Other making it super scalable for editing in future and all of the code will seem very logical and pictorial, best part is this can be designed instead of learning Amazon’s State Language.

let’s try to do Continue reading

1 2 3 6