New to Openstack and worried about networking pre-setup? Try VXLAN or GRE first!
As an avid developer/coder I tend to try out various languages. I do have my favorites but when I embark on a new tool or language the first experience definitely leaves behind a mark. Failing at getting something to work in the very first try is fine as long as you don't end up hating the technology. I guess I've iterated this over and over again on other posts but networking is complex and hard. Getting it right does require some (not really, it's actually a lot :-) ) amount of experimentation. With technologies changing every so often it's imperative to at least give them a try if not jump the train and adopt them.
Openstack being an opensource project as massive as the linux kernel it does have a ton of features. The neutron component that facilitates networking is quite a handful for beginners and it gets even messier when you will need to configure neutron to work with the network architecture that is already present in your datacenter. Getting a production ready neutron setup with say VLAN backed networks is a handful. After working with Neutron for some time now, i've realized that it isn't as bad Continue reading
Picasa is finally going lights off! What’s next..
Google Inc. yesterday, finally announced axing of the Picasa Desktop app (no more updates to it) and "read only" access to Picasa Web Albums starting 1st May 2016. Although quite brutal, this hasn't been unexpected. I am however one of the many people who loves Picasa Web Albums and even Google+ photos.
A Lifescape Inc. technology after being acquired by Google was released as Picasa in 2002. As many other Google services Picasa has finally become yet another service to be axed out to promote its newer counterpart in photo sharing & organizing - Google Photos. Picasa brought in online photo backup and sharing space a new perspective. Many others have followed suite (Flickr, 500px..). The picasa desktop application was very handy in quickly exporting and importing pictures to and from the web. Organizing pictures was a breeze. Picasa came as both a paid and a free service.
Photo sharing functionality has been moved to different apps/services within Google Inc. ever since. I remember a similar move couple of years ago when I received a mail stating that picasa web albums were being migrated to Google+, Google's social networking platform. Google+ has its own story and Continue reading
Looking forward to ContainerWorld2016 next week!
Container technology has been there for years but is now picking steam thanks to Docker. Container packaging, image registry hosting for containers has directly affected both use and adoption of this technology. "Cloud Native" applications are being designed and written with the direct intent of running them on containers. Today I see containers in the same "boat" as virtualization years ago. Virtualization started slowly in the late 90's and has sprawled massively moving into every datacenter out there. It brought about a massive change in the way hardware resources were consumed making it highly cost effective. Virtualization was technology only used by linux pro's till it was made easy by companies to consume. The story with containers seem to be the same.
A linux container is a virtualization environment (VE) created using kernel support. It provides process isolation and resource isolation/allocation through cgroups (control groups). Docker is a tool that wraps programs to run as isolated processes in its own filesystems. There are various posts out there explaining the differences between docker and a linux container. One big advantage that containers give is the flexibility to move between various hosts quickly and easily.We've now moved Continue reading
A glance into host routes: Tenant networking & routing using Neutron (Openstack)
Software Defined Networks (SDN) and solutions have been making a lot of noise for a few years now. Rather slowly the networking industry has begun to notice this and affect change. Not only this but SDN has also become a pioneer - a big brother - a guide persona to other complementary technologies. We now have Software Defined - Storage, Data Center, Infrastructure and so on. It's Software Defined "everything" and Software Defined "anything". Software is slowly invading the big hardware only players and the sole reason being ease of customization and lower both; capex and opex. Networking in particular is very volatile and extremely configurable.
The neutron project of openstack is also fairly customizable bringing complexity with it. I recently ran into a requirement of having isolated networks talk to each other as well as some specific networks though isolated having access to the outside world (be it outside the cloud or the company WAN). This is what's giving rise to this particular blog post. I will lay out the premise of the discussion and then explain the solution. For networking experts out there, this might seem fairly obvious. I would suggest you stop right here and jump over Continue reading
Moving from ifconfig to ip
I will try to describe the 1:1 mapping of the most common commands I use to just figure out what a linux box is configured with. There are tons of articles & cheat sheets out there which can be used too.
ifconfig commands - See & set interface properties
Data Center Networking – Openstack Neutron networking terms demystified!
Coming from a networking background I am used to the various TLAs (ironically: Three Letter Acronyms) and terms that sometime mean the same but completely different otherwise. Networking in the data-center is no different. People have now slowly moved away from a leaf-spine topology where an advanced pricey piece of equipment sits in the spine and relatively cheaper leaf devices connect the hosts and servers. We now have an old architecture that was used during telephony in a new form with new terms. Terms such as "Clos topology" where the the leaf-spine architecture is replaced by layers or levels of similar cheaper hardware allowing flexibility in its configuration forming a sort of mesh making it both scalable and reliable; and special protocols such as DCTCP (Data Center TCP) being interspersed in technical papers and company technology presentations. An outlook of these technologies is big enough to be a completely separate read and so I will not venture into it here.
Google's datacenter - Clos Topology research & implementation
Facebooks' datacenter architecture - 5 stage clos topology
With all these changes in the data center we now also see administrators isolating a complete data center/s or carving them Continue reading
Google's datacenter - Clos Topology research & implementation
Facebooks' datacenter architecture - 5 stage clos topology
With all these changes in the data center we now also see administrators isolating a complete data center/s or carving them Continue reading
Unikernel Systems joins Docker
Today, Unikernel Systems announced that it is joining Docker. Please read the main notice posted at unikernel.com and the Docker blog post at http://blog.docker.com/2016/01/unikernel/.
An important part of the work of Unikernel Systems is to support the growing unikernel movement, which includes contributing to a number of open source projects as well as supporting this community website, unikernel.org. The Unikernel Systems team will continue this work.
Through further interaction with the Docker project and community, the open source unikernel community will receive increased visibility and contributions which will accelerate the adoption of unikernels.
Links:
Discuss this on devel.unikernel.org
Creating VLAN interfaces in Linux
Communicating over multiple VLAN's is possible by using VLAN sub interfaces in linux. A vlan interface can be created in linux which shows up as a network interface device. Each of these interfaces are used as you would use a normal linux interface - assign an IP to it, attach it to a bridge, add routing tables entries and more. One use case is when you need a VM to act as an L2 gateway having one leg on one VLAN and another leg on the other.
First thing you will need is to load the 8021q linux kernel module that is responsible for VLAN tagging/untagging. See RFC.
Check if you have VLAN module. You can check the ouput of lsmod and figure out if 8021q is loaded or I simply like to do this:
You should see 8021q and some other lines in the output.
Add module to linux. Note that you'll need to automate this. You can add it in systemctl Continue reading
As always this can be achieved in multiple ways: using the vconfig command, adding a new interface network-script file (CentOS/Redhat) or by using the "ip" command. I will describe all three methods here:
First thing you will need is to load the 8021q linux kernel module that is responsible for VLAN tagging/untagging. See RFC.
Check if you have VLAN module. You can check the ouput of lsmod and figure out if 8021q is loaded or I simply like to do this:
lsmod | grep 8021q
You should see 8021q and some other lines in the output.
Add module to linux. Note that you'll need to automate this. You can add it in systemctl Continue reading
Running VLAN, VXLAN and GRE together using Neutron & Openstack
There are numerous blogs out there with step by step instructions to setup OVS networking for openstack neutron to run various types of networks. I came up with this post to best explain how everything works in the Neutron land and what a typical openstack deployment looks like. If you are looking for a more step-by-step procedure to set things up this post is a good place to start i.e. understanding the networking concepts and design behind running all the commands listed on other blogs.
A Neutron openstack deployment or any cloud environment these days typically consist of a Network Controller (SDN controller) and a cluster of compute hosts or servers. The type of networking you choose to connect all of these together is entirely upto you and I've seen various ways by which this can be done. There is no correct or right way of doing this. It's what works for you but if you don't really have anything already setup and are planning on starting afresh you could probably use this as a template to design your networking.
What you see above are three servers - one network node and two compute nodes.
First we'll go through Continue reading
A Neutron openstack deployment or any cloud environment these days typically consist of a Network Controller (SDN controller) and a cluster of compute hosts or servers. The type of networking you choose to connect all of these together is entirely upto you and I've seen various ways by which this can be done. There is no correct or right way of doing this. It's what works for you but if you don't really have anything already setup and are planning on starting afresh you could probably use this as a template to design your networking.
What you see above are three servers - one network node and two compute nodes.
First we'll go through Continue reading
Gotchas for using a different subnet for a VM than that of the host in Openstack
It is definitely possible to have a completely different subnet for a VM than that of the host machine running libvirt and KVM using linux bridging. This is done by using NAT technique. The reason I decided to put this down in my post is to just have it on record for me to refer in the future. Just keep in mind that I have created the instances through nova & openstack.
As always networking doesn't always work as designed or planned to and there's no fun if you don't see packet drops and unknown network issues breaking communication. After experimenting extensively and carefully jotting down the changes that was needed to be done, here are the list of gotchas' I've come up with:
- Libvirt or other network filters do not block packets (Skip this step if you aren't using nova networks and Openstack)
You can check to see what the network filter is programmed to do. To do this first find the instance ID for your instance and then find the libvirt-network filter rule for the same. You can edit the rule to set the subnet that you want to allow.Find instances Continue reading
Quick tips and reference on tcpdump tool
Linux bridges are powerful virtual switches that come with the networking stack of core Linux. Bridging is loaded as part of the bridge kernel module. Linux bridges amongst others have made it possible to network virtual machines and containers on a KVM based linux node.
Very so often I have had to go in and figure out where the packets either egressing the VM or ingress traffic to the VM magically vanishes and tcpdump has been one of the most valuable tools for me. Debugging in the networking world pretty much has a standard algorithm to it: Start from the source and check for packets along the route to the destination at every hop. Once you figure out where the packet disappears, where the black hole is, it is half the problem solved.
To do exactly this we have a variety of debugging tools ranging from Wireshark that captures packets to tcpdump and sniffers. All of these have one thing in common - they are user processes that hook onto specific kernel parameters to capture the packets. For example, although you do not explicitly set an interface in promiscuous mode this interface is moved to promiscuous mode when the packet Continue reading
Running devstack older than Kilo release
| The famous network topology diagram as seen in Juno - Openstack (My preference over the one in Kilo/Liberty) |
Juno and older code is now represented as tags and may or may not be supported by the individual project teams. However thanks to subversioning and git, you can check out code of juno or older releases by using tags now.
#git tag -l --> Lists the tags present in the repository.
#git checkout tags/ -b --> Checkout code from a tag.
All stable older releases have now been named following the convention "
IncludeOS is now free and open source!
Alfred Bratterud is Assistant Professor and PhD scholar at Oslo and Akershus University College of Applied Science where he is currently working full time leading the development of IncludeOS at the NetSys research group.
We’ve finally lifted the lid on IncludeOS, just in time for the IEEE CloudCom paper presentation recently. A preprint of the paper is available from our repo. However, we’ve done quite a lot of work since the paper was written, so here’s an update on what IncludeOS is now, and what you can expect in the near future.
It’s a bit like a JVM, but for x86 C++
A Java Virtual Machine is a portable language runtime environment. Java is portable across hardware architectures and operating systems because it uses a common instruction set. Once you’ve started a Java program, you can’t log into it (unless your program itself provides the facilities), and you can’t boot up any other programs inside it.
IncludeOS is like a safe language runtime for C++ programs, compiled into the x86 instruction set. This has the obvious advantage of removing one layer of abstraction, compared to Java: with hardware virtualization the code will execute directly on the CPU. Like with Continue reading
Interface naming in Linux – Choose the name you want for your interfaces using udev
Have you tried the recent CentOS7.X flavor or the latest of the Redhat versions? If you have then you would have noticed the change in nomenclature of network interfaces. While the traditional approach was to use "eth" shortened from "Ethernet" as a precursor word followed by a sequence of numbers starting at 0 to name network interfaces in a system and now from the v197 scheme we have the udev rules choose names automatically for interfaces using naming schemes dependent on either the firmware/BIOS indexes for on board NICs or slot numbers for add-on nics or the mac of the nic or the physical/geo location.
Although this intuitively sounds complicated it makes life much more easier and reliable. The older scheme worked in a way that could make naming unpredictable. When a nic interface driver gets initialized udev allocates the next available number to that nic and if a host has more than one nic card (either on board or external-extended) there is a possibility of the driver load order to change thus changing the name for the NICs. A power user could add rules to udev scripts to fix a name for a particular mac address in order Continue reading
Understanding VLAN terminology – Access Trunk Native Tagged Untagged
Ever so often I myself have searched around for the exact difference between all of the different terms associated with VLAN's (Virtual Local Area Networks) and I'm sure a lot of people out there do the same. There's so much out there explaining these terms but it gets confusing when everything is virtualized. How does networking work in a virtualized host where the operating system networking stack provide virtual network functions such as a virtual switch?
I will start out by stating that networking works the same way it is expected to work in the physical world with wires and hardware boxes that switch or route traffic. To understand what these terms mean you need to first understand how VLANs work. VLAN segregation can happen on a IP range basis, on a port to port basis on the switch, based on protocol, based on mac addresses or just based on some other higher layer abstractions. The most common implementation is however port based and I will go ahead by explaining these terms with respect to this premise. Also some of these terms are specifically defined by Cisco switches and have then on gone to become popular in network design vocabulary Continue reading
I will start out by stating that networking works the same way it is expected to work in the physical world with wires and hardware boxes that switch or route traffic. To understand what these terms mean you need to first understand how VLANs work. VLAN segregation can happen on a IP range basis, on a port to port basis on the switch, based on protocol, based on mac addresses or just based on some other higher layer abstractions. The most common implementation is however port based and I will go ahead by explaining these terms with respect to this premise. Also some of these terms are specifically defined by Cisco switches and have then on gone to become popular in network design vocabulary Continue reading
Contain Your Unikernels!
After DockerCon EU in Barcelona several people asked me: “Is this for real?”. Yes it is, and today we are releasing the code for the entire “Unikernels, meet Docker!” demo on GitHub.
To get started, clone the DockerConEU2015-demo repository and follow the instructions in README.md. You will need a Linux host with Docker and KVM installed.
Apart from the MySQL, Nginx and PHP with Nibbleblog unikernels shown in the demo, the repository also contains some simpler examples to get you started that we did not have time to show live in the short time-slot. There’s also an in-progress MirageOS/KVM port, so stay tuned for a future post on that.
Presented as a ‘cool hack’ in the closing session of the conference, this demo is just a taste of what is possible. Next, I’m going to work with the wider unikernel and Docker developer community on a production quality version of this demo. The goal is to make unikernel technology easily accessible to as many developers as possible!
Personally, I would like to thank Amir Chaudhry, Justin Cormack, Anil Madhavapeddy, Richard Mortier, Mindy Preston and Jeremy Yallop for helping me put the demo Continue reading
Slack has transformed the way teams communicate & they now plan to go IPO!
Having kept my pensive empty for quite some time now its time for a kick start. And what better way to start than to talk about Slack - A communication tool that has revolutionized intra-team, inter-team interactions and day to day messaging.
After the dot com burst and the difficult times of 2009 technology industry is now at a point where innovation is ever more flourishing with new companies announcing their birth and the more mature ones going through multiple rounds of funding, acquisitions and IPOs. Slack is one such company that has gone from a small startup to a multi-billion dollar venture and now in the process of going IPO (TechCrunch). The customer first approach with product re-engineering from customer feedback has yet again proved to be one of the most important ingredients to being successful along with laser sharp focus areas. Slack's growth trajectory is clearly exponential, one that I'm sure any founder would love to have in their annual revenue reports.
Being an engineer in a startup, I am lucky enough to be using this communication tool daily. Having used other paid tools like Cisco Jabber to the free ones like Google Hangouts, Skype and Continue reading
Unikernels, meet Docker!
Today, unikernels took to the stage at DockerCon EU in Barcelona!
As part of the Cool Hacks session in the closing keynote, Anil Madhavapeddy (MirageOS project lead), showed how unikernels can be treated as any other container. He first used Docker to build a unikernel microservice and then followed up by deploying a real web application with database, webserver and PHP code all running as distinct unikernel microservices built using Rump Kernels. Docker managed the unikernels just like Linux containers but without needing to deploy a traditional operating system!
This kind of integration helps put unikernels into the hands of developers everywhere and combines the familiar tooling and real-world workflows of the container ecosystem with the improved security, efficiency and specialisation of unikernels. We’ll finish off this post with details of how you can get involved — but first, before we go into Anil’s demonstration in more detail, some background about why unikernels matter, and why it makes sense to use Docker this way.
Why Unikernels?
As companies have moved to using the cloud, there’s been a growing trend towards single-purpose machine images, but it’s clear that there is significant room for improvement. At present, every VM has to Continue reading
A community site for Unikernels
Word about unikernels is spreading and more people are trying to learn about this new approach to programming the cloud. This community site aims to collate information about the various projects and provide a focal point for early adopters to understand more about the technology and become involved in the projects themselves.
Image Credit: Blake Thomson from Noun Project