vmWare vRealize Automation (with embedded vRO) – Full Example of Custom Resources for Executing Ansible Playbooks from Blueprints

This articles exists because I spend a lot of time in the last year inside vmware vRealize Automation (vRA) that includes an embedded vRealize Orchestrator (vRO) on a undisclosed project. And the main take-away is that the documentation in certain aspects that are on the edge of new features of vRA/vRO are, to put it mildly “documented in a very chaotic way”. Additionally, what worked in vRA 8.0 – 8.2 suddenly stopped working in 8.3 and 8.4 when upgraded. So this product is not yet as stable as I would like and for this article it means that it will probably be completely outdated in a year or so. Nevertheless I wanted to put together a simple example that is working end-to-end and providing practical jump of point for building more.

And I have decided that bridging vRA/vRO with traditional Ansible playbook is a good idea for an example. Because it combines the vRAs great Infrastructure As A Code (IaaS) capabilities of using blueprints with a more infrastructure focused automation of Ansible. We will be building an abstract object on a blueprint (these types of objects are called “custom resource” because you are defining what they Continue reading

Network Topology Visualization #3 – Exploring other D3 visualization options for a DataCenter

Foreword

First a disclaimer #1, in contrast to the previous articles, here I will NOT describe the collection of data that will be visualized because these datasets were manufactured manually for the sake of example. So this time I will just say what was the target behind a given visualization example and give you a link to a live demo of it. You can examine the code and source JSON files yourself using the usual browser developer tools and should be fairly easy to adopt. 

Secondly a disclaimer #2, the examples here were developed as part of a part-time job with company called Langner GmbH. (visit Langner.com), as a form consultancy exploring options of enhancing their OT-BASE software with some javascript/D3 based visuals. This engagement ended in early 2020 (so more than a year ago now) and they were kind enough to give me permission to publish some examples here. With data properly anonymized. 

So let’s get to the examples. There is a total of 4 examples and I will NOT explain them one by one in detail here much (e.g their code). However it should be fairly easy for you to just Continue reading

[mini-update] Network Topology Visualization #2 – Using SNMP as data source and enhanced visuals

After a longer break, I managed to get back to my small visualization experiment from last time, and improve it a little. Firstly, the NETCONF interface was switched for a more standard SNMP one, even if that is not as cutting edge anymore. 

So without keeping you waiting, here is firs the visualization example, as you can see I was playing here a little and got inspired by the Mischief-makers map from Harry Potter. Of course if you do not like the visuals just re-do the CSS to get rid of it. 

Also, this time I made the code public using the much more common github.com here. 

Usage/Tutorial

Once you download this project from github, you will find several files in the directory, your starting point are the two configuration files called:

• config.ini  – holds SNMP configuration like community string and a list of IPs for each network device to be SNMP queried
• pyconfig.py – holds low level configuration on how to visualize the data recieved and how to order the devices detected into layers

Lets start with the config.ini, which is much smaller. Here is an example.

[DEFAULT]
SnmpVersion = 2c
SnmpCommunityString  Continue reading

Autopilot for Elite Dangerous using OpenCV and thoughts on CV enabled bots in visual-to-keyboard loop

Ok, first let’s get one thing clear. This is an educational project for me learning some computer vision algorithms (I want to make an insect identification system to protect a bee hive, – but that is for much later article) and the game Elite Dangerous provided only interesting guinea pig here to test some principles. This was never intended as a game cheat/bot or anything like that, although in the last chapter I will give my thoughts on AI becoming a thing playing games using undetectable external “human loops” (e.g. looking at monitor and pushing keyboard) that no anti-cheat will ever catch, but that is way beyond my motivation as I personally like Elite as it is and definitely do not want to destroy its internal mechanics by creating a farming/trading bot this way. That is also the reason why code of my experiments is not disclosed. If I do not figure out some clever way how to share this without some script-kiddies turning this experiment into a cheat I never will share this code. If you are here looking for a game cheat, you will not get it. If you are here to learn how to program Continue reading

Network Topology Visualization – Example of Using LLDP Neighborships, NETCONF and little Python/Javascript

Ok, this is a continuation of two streams of articles here, first my recent NETCONF tutorial here, and secondly my very old project (back then in Java) of visualization of network topologies using SNMP information called “HelloRoute”. So this is a resurrection of a very old ideas, just using newer methods and tools. But first a foreword on visualization.

Foreword – Visualization use in Network Infrastructure by Author’s experience

Well, as far as I would say, automated network visualization or documentation never really took of as primary source of documentation, everywhere I look we still maintain manually created maps with version control, trying to keep them up-to-date in change process and etc… , the reason why this is so is the context that human author can give the map, for example office networks mapped by purpose or parts of buildings, or by legal organizations. Have a look on the picture below, this is a difference between human and automated maps in most generic network modeling tools.

Human vs computer generated network diagrams

Now to not completely kill the point of you finishing this tutorial, I BELIEVE THE PROBLEM IS THAT VISUALIZATION TOOLS ON MARKET ARE MOSTLY GENERIC PRODUCTS, Continue reading

HP Networking/Comware NETCONF interface quick tutorial (using python’s ncclient and pyhpecw7)

So let’s learn about NETCONF, but first a bit of history and perspective. Everyone in networking business at least once heard about SNMP (Simple Network Management Protocol), which is the goto protocol for monitoring your network devices, and wondered how cool it would be if you could not only monitor your network with it, but actively configure it (sort of like “SDN wannabe”). But for that purpose the SNMP  was not really useful, it supported some write operations but they were so generic and incomplete that it was not really feasible. That is where NETCONF came around 2011 as a standard (it was here before but its RFC 6241 was ratified then) and changed the game in favor of configuring any device, while not restricting vendors from declaring their own NETCONF data structures to fit their features, but lets first check the protocol first before diving into the data structures.

NETCONF is a RCP (remote procedure call) based protocol, using XML formating as payload and YAML language as data modeling (the part that explains to you what XML to send to configure something).

LAB TOPOLOGY

Ok, lets get to the point, in our excercise I will be focused on the Continue reading

[minipost] Protecting SSH on Mikrotik with 3-strike SSH ban using only firewall rules

After working with Mikrotik / RouterBoard routers for a long time, I recently needed to replace an aging old wifi router at my parents and the recent brand of very cheap Mikrotik WIFI integrated routers (RB941-2nD-TC shown on left) that you can get under 20,-EUR was a great deal with an added bonus that I want to manage all this remotely and not visit physically every time there is a wifi problem.  So following my previous post on how to put a little script into Mirkotik to email you it’s public address whenever it changes (a mandatory to manage parent’s home router using dynamic public IP from ISP) I was also concerned about publicly opened SSH port and wanted at least basic protection on it. Most of you are probably using already some great tool such as fail2ban on linux, that scans log files and if it notices three bad logins to SSH from an IP, it will put the IP into a blocking filter on the local linux iptables firewall so it can no longer harass your system. Well I needed something similar on my home Mikrotik router/firewall, but without impacting its performance or doing a lot Continue reading

Example of private VLAN isolation across Virtual and Physical servers using ESX/dvSwitch and HP Networking Comware switches

The target was simple, we have an internal cloud datacenter at work that provides users and customers both virtual machines and physical machines. Each machine has to network interface cards (NICs), one is in control of the user/customer using SDN layer, the secondary NIC is for our support to monitor and help troubleshoot these machines when needed. This second NIC will be our target today. In the past we used per-user or per-customer firewall separations that was configuration intensive nightmare, but was reliable. However since we learned private VLANs are now supported by vmWares Distributed vSwitch (dvSwitch), we immediately tried to make it cooperate with private VLANs on physical switches. And since it worked like a charm, let me share with you a quick lab example. But theory first!

Theory of separating management rail between different customers with and without private VLANs

Solving management separation with a lot of subnets and firewalls

Fortunately Private VLANs arrived for most major vendors and promissed the ability to have one giant subnet and separate every host from each other on L2 using some basic principle of declaring ports as either promiscuous (can talk to any other port type), community (can talk to ports Continue reading

HPE’s DCN / Nuage SDN – Part 3 – REST API introduction

REST-full API, the new trendy bandwagon of cloud automation and SDN due to its simplicity and universal compatibility. TI is essentially a http transfer protocol with JSON payload. And if I confused you here, please stay calm, we are still doing networking here, we will just try to use a different control mechanism than good old console or visual GUI for the sake of automation later. Also here on this site we already had one exposure to REST API use with the HP’s SDN controller both using raw curl and automation of this controller using perl web application as tutorials.

Index of article series:

• HPE’s DCN / Nuage SDN – Part 1 – Introduction and LAB Installation Tutorial
• HPE’s DCN / Nuage SDN – Part 2 – First Steps Creating Virtual/Overlay Customer Network
• HPE’s DCN / Nuage SDN – Part 3 – REST API introduction

Introduction to part #3 about REST API

I was thinking for a long time how to structure this article, and at the end I decided. That instead of showing too much boring REST API calls, I am going to show only a few elemental REST API calls of Nuage, but from different points of view. We Continue reading

[minipost] Capturing bidirectional traffic of virtual machine (VMs) on vmWare ESX 6.x

Here I was getting a trouble with communication between an ESX virtual machine and the nearby switch (Nuage/DCN controller VM talking with a VTEP switch if someone is interested) and because that switch was direct destination of the control plane packets (OVSDB over TCP) I was not having much success creating a mirroring interface on the switch. So I learned how to capture a specific virtual machine traffic directly on the ESXi host’s SSH console and to not forget that, I will document this here.

Step 1 – enable SSH to the ESX host

In most cases this is not running by default, so go to the ESXi server direct terminal or iLo and via “F2″enter system customization and enter troubleshooting section:

ESX host troubleshooting options location

Right behind this menu should be “SSH Enable” option, and you simply hit that with enter

ALTERNATIVE: If you have vCenter deployed, you can use its GUI to enable SSH on a specific host like this:

Locate ESX Host in vCenter and open its security profile

Inside security profiles enable SSH server

Step 2 – locating switchport ID

My virtual machine was called “DCN4.0R5_VSC1” and was simply connected to logical vswitch0 in Continue reading

HPE’s DCN / Nuage SDN – Part 2 – First Steps Creating Virtual/Overlay Customer Network

In the previous part 1, we have installed basic HPE DCN system on a group of ESXi hosts. But we didn’t actually done anything inside it, so lets fix this by creating a first “HelloWorld” customer that we will call “NetworkGeekStuff” and deploy some virtual machines to this virtual network. In this part we are going to fix that and we will create a very basic virtual customer, a username/password for that customers administrator and create a small 3 tier ( database / internal  / dmz) network using HPE DCN’s overlay virtual network. And at the very end, we are going to connect to this network a few virtual machines.

Index of article series:

• HPE’s DCN / Nuage SDN – Part 1 – Introduction and LAB Installation Tutorial
• HPE’s DCN / Nuage SDN – Part 2 – First Steps Creating Virtual/Overlay Customer Network

Starting LAB state

We will start exactly where we ended on previous part 1, but to double-check, I am going to show the main views of my vCenter and VSD environment to show how “empty” it is after a pure install that we did so far. So starting with this, below is my view on vCenter boxes, with Continue reading

HPE’s DCN / Nuage SDN – Part 1 – Introduction and LAB Installation Tutorial

Nuage Networks is a spinoff from Alcatel-Lucent (now under Nokia as they acquisition Alcatel recently) and also a name of software defined network (SDN) overlay solution for datacenters and direct competitor for a bit more widely known vmWare’s NSX. However Alcatel/Nokia are not the only backers here, Hewlett Packard Enterprise (HPE) also got a vested interest in this technology and jumped on the partnership with Alcatel/Nokia generating their own spin-off called “HPE’s Distributed Cloud Networking” or HPE DCN for short. In this article I am going to quickly summarize what it should be capable of doing, and then I am going do a run down of how to install this system in a home lab environment, while trying to minimize the requirements for a home install of this beast sacrificing a few redundancies that you would normally have in a DC setup.

I am marking this article as “Part 1” because this installation will only provide the most very basic overlay solution and there will be follow-ups that will go into different aspects later (e.g. redundancy, cooperation with vmWare ESX high availability or L3 VTEPs). For now we are going to setup a simple lab, trying to minimize the Continue reading

Tutorial for small Hadoop cloud cluster LAB using virtual machines and compiling/running first “Hello World” Map-Reduce example project

I had Hadoop experience now for more than a year, thanks to a great series of Cloud Computing courses on Coursera.org, now after ~6 months of running via several cloud systems, I finally have time to put down some of my more practical notes in a form of an article here. I will not go much into theory, my target here would be to help someone construct his first small Hadoop cluster at home and show some of my amateur “HelloWorld” code that will count all words in all works of W. Shakespeare using the MapReduce. This should leave with with both a small cluster and a working compilation project using Maven to expand on your own later …

What I have used for my cluster is a home PC with 32G of RAM to run everything inside using vmWare Workstation. But this guide is applicable even if you run this usingVirtualBox, physical machines, or using virtual machines on some Internet cloud (e.g. AWS/Azure). The point will simply be 4 independent OS linux boxes that are together one a shared LAN to communicate between each other.

Lab Topology

For this one there is not much to say Continue reading

Ping on you! – Beta 2.0 is online

Nicer, more stable, AJAX/JSON enabled pingonyou.com and also preparations in background to introduce the 24/7 Monitoring feature.

So here we are. First a bit of short history, if you have seen the first version of the pingonyou.com, you probably had a sense of 90’s view of the page just by looking at it. Everything was static HTML and all updates were actually refreshing the whole page (including every new line in the ping functionality needed a whole site refresh!). This was also including the fact that the design and color selection was really very basic. If you want to have a look on the old version, simply go to previous articles on this blog (where pictures are) for a comparison or here is a picture of beta 1.0. But lets now focus on the new version which is available from today!

Ping on You – Beta 2.0

The first thing is the color-scheme, after trying out several dozens of combinations, the blue variant was the best overall that provide both good console readability and combined with the fact that the functions that it provides needed to be categorized much better. First everyone that comes Continue reading

[minipost]Quick LAB/config example for IPv6 BGP between HP Networking Comware v5 andCisco

A small lab showing basic configuration of BGP between Cisco and HP (Comware v5). This is just something small we deploying recently, there is nothing grand here, only a minor configuration example to follow later when needed.

NOTE on HP Comware v5  vs newer Comware v7, I understand I am using older version of the operating system on HP devices, the point is that this article is using one of my real work projects where Comware v5 was used without possibility to upgrade. However ALL Ipv6 functions that we needed were provided already on this older Comware, and when I checked, Comware v7 variant of this LAB is only changing commands syntax (actually quite easy to convert from v5 to v7 only following the “?”), therefore this article will remain in Comware v5 and I believe many readers will take the principles and will have no problem to upgrade to Comware v7 on their own.

Lab Topology:

This is a simple topology that is trying to simulate a typical L3 Edge / Distribution / Access with several HP 5800 layer3 switches and Cisco 3750 is simulating a typical WAN provider with dual-homing access. Of course all with limits Continue reading

Ping on You! – small weekend web/CGI project

For best article visual quality, open Ping on You! – small weekend web/CGI project directly at NetworkGeekStuff.

After a few previous web programming endeavors (both here on networkgeekstuff.com with SDN programming and in real life job) I had a taste to develop “something” and since I am a network geek, I wondered what I have needed in the past as network tools that would ideally be placed on the internet. So I came to the very basic list of things like ping, nslookup or putty as tool accessible only via a browser.

So without more delay, I give you first beta release of Ping on You! – pingonyou.com the list of current features is:

• ping
• nslookup
• wget (for http response checks)
• whois / ripe database

Here is a quick view how it looks when you try to ping google’s public DNS service on 8.8.8.8, since in the background a linux like ping tool is used, you have the same output:

Additionally, here are some features that I plan to do when I get more time, however no release date right now because there are much more complex than the above ones:

• TCP port openings checks Continue reading

[minipost] Windows partition editing with diskpart

For best article visual quality, open [minipost] Windows partition editing with diskpart directly at NetworkGeekStuff.

This will really be a micro-post as I only want to document this for my benefit. This is a way how to change partition table for disks, or USB sticks. In my example, I had a linux live boot USB stick that I needed to quickly convert into a usable storage USB stick with NTFS under windows and of course the visual GUI tools under My Computer -> Manage -> Disk Management was not having full visibility on all partitions that the linux created on this USB stick, so this is how to actually do partitioning on windows.

So without more delay, this is an example how to clean the USB stick partition table and reformat it for windows use:

Checkpoint Firewall CLI tool “dbedit” and quick lab examples

For best article visual quality, open Checkpoint Firewall CLI tool “dbedit” and quick lab examples directly at NetworkGeekStuff.

In this article, I am going to give you a quick guide how to run a single checkpoint FW as virtual machine quickly on your notebook and then super-quick introduction to configuring such checkpoint firewall via CLI instead of the much more typical SmartDashboard. This articles is very focused on what I personally needed to do lab for in work and is in no way a comprehensive guide to the “dbedit” tool from Checkpoint or any firewall automation.

Background

We are using Checkpoint firewalls in our customer networks at work and are heavily using SmartDashboard and other GUI based tools to manage these firewalls in a large datacenter environments (rulebase of 10k+ firewall rules!) because that is simply our internal standard. However recently there came a push to try to automate a certain aspects of configuring these firewalls because several customer wanted to achieve shorter lead-times at least on few aspects of firewall configurations.

And since Checkpoint FWs do not support any real API for managing policies with it, it came down to CLI tools like dbedit, which we will explore Continue reading

Soviet Mi-24V Hind E, 1/72 scale

For best article visual quality, open Soviet Mi-24V Hind E, 1/72 scale directly at NetworkGeekStuff.

The Mil Mi-24 (Russian: Миль Ми-24; NATO reporting name: Hind) is a large helicopter gunship and attack helicopter and low-capacity troop transport with room for eight passengers. It is produced by Mil Moscow Helicopter Plant and has been operated since 1972 by the Soviet Air Force and its successors, along with more than 30 other nations.

In NATO circles, the export versions, Mi-25 and Mi-35, are denoted with a letter suffix as “Hind D” and “Hind E” respectively. Soviet pilots called the Mi-24 the “flying tank”, or летающий танк (letayushchiy tank). More common unofficial nicknames were “Crocodile” (Крокодил or Krokodil) due to the helicopter’s camouflage scheme and “Drinking Glass” (Стакан or Stakan) because of the flat glass plates which surround the cockpit of the Mi-24.

In my small interpretation, I have used the Revell kid that was released in 2014 targeting the Czech Air Force variant, but it also has the option of doing Soviet markings, so I decided to go with this much more generally known view as I have a certain nostalgia on this thing appearing in the old Operation Flashpoint (2001) by Continue reading

Tutorial for creating first external SDN application for HP SDN VAN controller – Part 3/3: “Node Cutter” SDN application in perl with web interface

For best article visual quality, open Tutorial for creating first external SDN application for HP SDN VAN controller – Part 3/3: “Node Cutter” SDN application in perl with web interface directly at NetworkGeekStuff.

In this tutorial series, I will show you by example, how to build your first external REST API based SDN application for HP SDN VAN controller, with web interface for the user control. Target will be to learn how to use REST API, curl and perl scripting to generate some basic and useful code to view and also manipulate network traffic.

This article is part of “Tutorial for creating first external SDN application for HP SDN VAN controller” series consisting these articles:

• Part 1/3 – LAB creation and REST API introduction
• Part 2/3 – Influencing Flows via cURL commands
• Part 3/3 – “Node Cutter” perl application with web interface

In this Part 3/3, I will show you my first example SDN application that I have written just to demonstrate the combination of REST API interface with perl and the ability to have a web based interface for the user to this interface. Please note that this application is purely demonstrative on the REST API interface and Continue reading