Test-driving OSPF on RouterOS

Kernel Programming 101 – Creating your own Linux Kernel Module

Have you ever wanted to write your own kernel module? This multi-part blog post will talk about creating your first kernel module, using the proc file system, gathering and updating statistics and will cover topics such as your kernel as a menu items to menuconfig, setting defaults in the kernel config, registering network callbacks for specific types of packets taking Cisco CDP as an example and some tips-tricks in writing and debugging kernel code.  A lot of my own learning has been through blogs and experiments. One source that is really great and does summarize a lot of what my blog will talk about and more is TLDP (The Linux Documentation Project).

Today programming involving systems  is about providing the flexibility and pace for software development by providing user space APIs that interact with the linux kernel through system calls. This approach also provides the abstraction needed to carve out the complexity in direct kernel programming. So before we dive into kernel programming-101, lets answer this question - Why Kernel Programming? 

When there's need to perform operations without  cpu cycles wastage and/or reduce user space copy overheads, kernel programming suits the bill. Taking networking and linux networking stack as Continue reading

ContainerWorld2016 conference in review – Part II (Container Orchestration)

© Arun Sriraman

Picking up from Part I of ContainerWorld2016 conference in review, this post describes the various orchestration mechanisms available to manage containers. There are more than just the tree mechanisms out there to setup & manage containers but these; Kubernetes, Mesos and Docker Swarm are gaining traction and popularity. The Open Container Night Meetup (SVDevOps meetup group) although wasn't part of the conference tracks and hosted after day 1 sessions at the same venue was very informative. As part of this meetup's lightening talks, Adrian Otto provided a good in-depth comparison of the various container orchestration tools along with the roadmap for Openstack Magnum project and Carina by Rackspace.

When talking about containers and orchestration, as Adrian put it, one can think of two methods or paradigms of interacting with any system - imperative where you have complete control on how you want the system to work  along with configuration knobs accessible to you at every step and declarative where you describe the outcome and the system automates everything for you making it simple and easy but giving you less flexibility and configurability. The degree of configuration control and ease of use are two important factors that Continue reading

Setting up bonding on Ubuntu 14.04

This one was a twister to me. Linux bonding is fairly simple and there are numerous blogs out there explaining the steps to get two interfaces on a linux machine bonded. There's tons of information and step-by-step guides describing what needs to be done on an Ubuntu based system for bond configuration. I was surprised when I failed to get it working after multiple attempts. I finally ended up mixing steps from more than two different blog sources to get the setup configured correctly and more importantly persist it over reboots.

Before diving deep with specifics and yet another step-by-step guide to setting up bonds on an Ubuntu 12.02 or 14.04 machine, lets see what linux bonds are and why bonding in the first place. If you were a kid in the 90's or before you'd know what a NIC is. These days with PC's going almost extinct in the home and personal computing space, it is quite hard for users to be acquainted with network cards housing one or more NICs (Network Interface Controller). This era is all about wireless - you still do have a wireless card in your laptop or notebooks or tablets but that's Continue reading

ContainerWorld2016 conference in review – Part I

© ContainerWorld (Informa) - Taken from container world photo gallery

One would think that attending a multi-day conference is sedentary, just meeting new people and lending your ear to speakers and watching presentations. On the contrary it isn't so. Its extremely tiring and by the end of the day you are completely drained out. In this sense,  ContainerWorld2016  that took place last week (17th, 18th Feb) proved no different than other conferences and I ended quite exhausted at the end of the conference. Although exhausting, it was informative and it turned out to be more of a vendor neutral & community oriented conference compared to many others. Talk subjects varied from customer production deployment of containers to pain points of adopting the cloud native model. This multi-part post tries to summarize take-aways and interesting discussions that took place over the two days.

The conference was well received with participation from multiple vendors and customers such as RedHat, Canonical, Docker, Google, Yelp, CapitalOne, Paypal, eBay, Netflix, Veritas and Nordstrom to name a few. Right from day one keynote all the way to closing remarks on day two, customers and vendors alike reasoned the various advantages Continue reading

Want to sell your brand to a developer? Laptop stickers could be yet another way!

© Arun Sriraman

A sign of pride and fun for some but chaos for others; laptop stickers as I recently figured out is yet another way of marketing and branding. I'm sure a lot of people have wondered why a company gives away goodies in conferences and exhibitions. It's for direct+indirect marketing. When you see people walking around wearing a company printed T-Shirt or a cap, that company reaches people it never spoke to directly. It can be thought about as "flooding" in networking terms.

If its known that stickers are a marketing technique, why would someone want to mutilate their laptop with them?

Good question indeed.  I have seen people fret at the sight of my laptop and come at me with the expression of - "Oh my god! What has he done to the poor laptop".. The picture you see above is my work laptop. I like stickers and I put them on my laptop because I agree with a product or the company that makes a product and would want to share this with others (i.m.o. it also makes it look cool).

Laptop stickers though a smaller marketing channel, are nevertheless effective. Continue reading

New to Openstack and worried about networking pre-setup? Try VXLAN or GRE first!

As an avid developer/coder I tend to try out various languages. I do have my favorites but when I embark on a new tool or language the first experience definitely leaves behind a mark. Failing at  getting something to work in the very first try is fine as long as you don't end up hating the technology. I guess I've iterated this over and over again on other posts but networking is complex and hard. Getting it right does require some (not really, it's actually a lot :-) ) amount of experimentation. With technologies changing every so often it's imperative to at least give them a try if not jump the train and adopt them.

Openstack being an opensource project as massive as the linux kernel it does have a ton of features. The neutron component that facilitates networking is quite a handful for beginners and it gets even messier when you will need to configure neutron to work with the network architecture that is already present in your datacenter. Getting a production ready neutron setup with say VLAN backed networks is a handful. After working with Neutron for some time now, i've realized that it isn't as bad Continue reading

Picasa is finally going lights off! What’s next..

Google Inc. yesterday, finally announced axing of the Picasa Desktop app (no more updates to it) and "read only" access to Picasa Web Albums starting 1st May 2016. Although quite brutal, this hasn't been unexpected. I am however one of the many people who loves Picasa Web Albums and even Google+ photos.

A Lifescape Inc. technology after being acquired by Google was released as Picasa in 2002. As many other Google services Picasa has finally become yet another service to be axed out to promote its newer counterpart in photo sharing & organizing - Google Photos. Picasa brought in online photo backup and sharing space a new perspective. Many others have followed suite (Flickr, 500px..). The picasa desktop application was very handy in quickly exporting and importing pictures to and from the web. Organizing pictures was a breeze. Picasa came as both a paid and a free service.

Photo sharing functionality has been moved to different apps/services within Google Inc. ever since. I remember a similar move couple of years ago when I received a mail stating that picasa web albums were being migrated to Google+, Google's social networking platform. Google+ has its own story and Continue reading

Looking forward to ContainerWorld2016 next week!

Container technology has been there for years but is now picking steam thanks to Docker. Container packaging, image registry hosting for containers has directly affected both use and adoption of this technology. "Cloud Native" applications are being designed and written with the direct intent of running them on containers. Today I see containers in the same "boat" as virtualization years ago. Virtualization started slowly in the late 90's and has sprawled massively moving into every datacenter out there. It brought about a massive change in the way hardware resources were consumed making it highly cost effective. Virtualization was technology only used by linux pro's till it was made easy by companies to consume. The story with containers seem to be the same.

A linux container is a virtualization environment (VE) created using kernel support. It provides process isolation and resource isolation/allocation through cgroups (control groups). Docker is a tool that wraps programs to run as isolated processes in its own filesystems. There are various posts out there explaining the differences between docker and a linux container. One big advantage that containers give is the flexibility to move between various hosts quickly and easily.We've now moved Continue reading

A glance into host routes: Tenant networking & routing using Neutron (Openstack)

Software Defined Networks (SDN) and solutions have been making a lot of noise for a few years now. Rather slowly the networking industry has begun to notice this and affect change. Not only this but SDN has also become a pioneer - a big brother - a guide persona to other complementary technologies. We now have Software Defined - Storage, Data Center, Infrastructure and so on. It's Software Defined "everything" and Software Defined "anything". Software is slowly invading the big hardware only players and the sole reason being ease of customization and lower both; capex and opex. Networking in particular is very volatile and extremely configurable.

The neutron project of openstack is also fairly customizable bringing complexity with it. I recently ran into a requirement of having isolated networks talk to each other as well as some specific networks though isolated having access to the outside world (be it outside the cloud or the company WAN). This is what's giving rise to this particular blog post. I will lay out the premise of the discussion and then explain the solution. For networking experts out there, this might seem fairly obvious. I would suggest you stop right here and jump over Continue reading

Moving from ifconfig to ip

Have you started using CentOS7.2? If you are a networking ninja and were all mojo about "ifconfig and route" commands, you are going to be in for a not-so-pleasant surprise in the later versions of Linux. Release after release the community talked about moving to "ip" commands and removing  net-tools package that houses commands "ifconfig and route" from minimalistic installs and going with the iproute package. And now finally with CentOS7.2 these packages aren't there anymore. It is however still possible to get these programs by installing the "net-tools" package, which I do quite often but I also suggest trying to use the "ip" suite that comes with iproute package instead. Not only does it give extended functionality but it comes-with and so you would not have to install new packages.

I will try to describe the 1:1 mapping of the most common commands I use to just figure out what a linux box is configured with. There are tons of articles & cheat sheets out there which can be used too.

ifconfig commands - See & set interface properties

Data Center Networking – Openstack Neutron networking terms demystified!

Unikernel Systems joins Docker

Today, Unikernel Systems announced that it is joining Docker. Please read the main notice posted at unikernel.com and the Docker blog post at http://blog.docker.com/2016/01/unikernel/.

An important part of the work of Unikernel Systems is to support the growing unikernel movement, which includes contributing to a number of open source projects as well as supporting this community website, unikernel.org. The Unikernel Systems team will continue this work.

Through further interaction with the Docker project and community, the open source unikernel community will receive increased visibility and contributions which will accelerate the adoption of unikernels.

Links:

• Unikernel Systems announcement
• Docker blog post
• Docker Press Release

Discuss this on devel.unikernel.org

Creating VLAN interfaces in Linux

 lsmod | grep 8021q  

Running VLAN, VXLAN and GRE together using Neutron & Openstack

There are numerous blogs out there with step by step instructions to setup OVS networking for openstack neutron to run various types of networks. I came up with this post to best explain how everything works in the Neutron land and what a typical openstack deployment looks like. If you are looking for a more step-by-step procedure to set things up this post is a good place to start i.e. understanding the networking concepts and design behind running all the commands listed on other blogs.

A Neutron openstack deployment or any cloud environment these days typically consist of a Network Controller (SDN controller) and a cluster of compute hosts or servers. The type of networking you choose to connect all of these together is entirely upto you and I've seen various ways by which this can be done. There is no correct or right way of doing this. It's what works for you but if you don't really have anything already setup and are planning on starting afresh you could probably use this as a template to design your networking.

What you see above are three servers - one network node and two compute nodes.
First we'll go through Continue reading

Gotchas for using a different subnet for a VM than that of the host in Openstack

It is definitely possible to have a completely different subnet for a VM than that of the host machine running libvirt and KVM using linux bridging. This is done by using NAT technique. The reason I decided to put this down in my post is to just have it on record for me to refer in the future. Just keep in mind that I have created the instances through nova & openstack.

As always networking doesn't always work as designed or planned to and there's no fun if you don't see packet drops and unknown network issues breaking communication. After experimenting extensively and carefully jotting down the changes that was needed to be done, here are the list of gotchas' I've come up with:

• Libvirt or other network filters do not block packets (Skip this step if you aren't using nova networks and Openstack)

You can check to see what the network filter is programmed to do. To do this first find the instance ID for your instance and then find the libvirt-network filter rule for the same. You can edit the rule to set the subnet that you want to allow.

          Find instances Continue reading

Quick tips and reference on tcpdump tool

Linux bridges are powerful virtual switches that come with the networking stack of core Linux. Bridging is loaded as part of the bridge kernel module. Linux bridges amongst others have made it possible to network virtual machines and containers on a KVM based linux node.

Very so often I have had to go in and figure out where the packets either egressing the VM or ingress traffic to the VM magically vanishes and tcpdump has been one of the most valuable tools for me. Debugging in the networking world pretty much has a standard algorithm to it: Start from the source and check for packets along the route to the destination at every hop. Once you figure out where the packet disappears, where the black hole is, it is half the problem solved.

To do exactly this we have a variety of debugging tools ranging from Wireshark that captures packets to tcpdump and sniffers. All of these have one thing in common - they are user processes that hook onto specific kernel parameters to capture the packets. For example, although you do not explicitly set an interface in promiscuous mode this interface is moved to promiscuous mode when the packet Continue reading

Running devstack older than Kilo release

 #git tag -l  --> Lists the tags present in the repository.  
 #git checkout tags/ -b    --> Checkout code from a tag.   

IncludeOS is now free and open source!

Alfred Bratterud is Assistant Professor and PhD scholar at Oslo and Akershus University College of Applied Science where he is currently working full time leading the development of IncludeOS at the NetSys research group.

We’ve finally lifted the lid on IncludeOS, just in time for the IEEE CloudCom paper presentation recently. A preprint of the paper is available from our repo. However, we’ve done quite a lot of work since the paper was written, so here’s an update on what IncludeOS is now, and what you can expect in the near future.

It’s a bit like a JVM, but for x86 C++

A Java Virtual Machine is a portable language runtime environment. Java is portable across hardware architectures and operating systems because it uses a common instruction set. Once you’ve started a Java program, you can’t log into it (unless your program itself provides the facilities), and you can’t boot up any other programs inside it.

IncludeOS is like a safe language runtime for C++ programs, compiled into the x86 instruction set. This has the obvious advantage of removing one layer of abstraction, compared to Java: with hardware virtualization the code will execute directly on the CPU. Like with Continue reading

Interface naming in Linux – Choose the name you want for your interfaces using udev

Have you tried the recent CentOS7.X flavor or the latest of the Redhat versions? If you have then you would have noticed the change in nomenclature of network interfaces. While the traditional approach was to use "eth" shortened from "Ethernet" as a precursor word followed by a sequence of numbers starting at 0 to name network interfaces in a system and now from the v197 scheme we have the udev rules choose names automatically for interfaces using naming schemes dependent on either the firmware/BIOS indexes for on board NICs or slot numbers for add-on nics or the mac of the nic or the physical/geo location.

Although this intuitively sounds complicated it makes life much more easier and reliable. The older scheme worked in a way that could make naming unpredictable. When a nic interface driver gets initialized udev allocates the next available number to that nic and if a host has more than one nic card (either on board or external-extended) there is a possibility of the driver load order to change thus changing the name for the NICs. A power user could add rules to udev scripts to fix a name for a particular mac address in order Continue reading