At the end of the day, any announcement that is NON networking focused NEEDS a networking focus.  I just attended the UCS Grand Slam launch where there were a few announcements for UCS including the new UCS mini, capacity optimized UCS, and the UCS modular chassis (M series).  The latter was of the most interest to me.

The new M series de-couples CPU and memory from other peripherals on board including networking and storage.  CPU and memory exist on a replaceable / upgradeable cartridge and can be upgraded without touching the network or SSDs on board.  This smells and feels like the disaggregation happening in the Open Compute Project (OCP), but of course, the Cisco solution can be managed with all existing and new UCS systems via UCS Manager.  This could definitely be the start of a new computing paradigm for the Enterprise. 

Platform Overview

Each UCS M series chassis has 8 cartridges in a 2 RU form factor.  Each cartridge has two (2) quad core CPUs and 64GB memory, thus each server has a single quad core CPU and 32GB memory.  There are also 4 SSDs and dual 1400W power supplies per Continue reading

With some downtime this weekend, I was able to watch a few documentaries on NetFlix.  There were a few great ones on Mark Zuckerberg, Warren Buffet, Mark Cuban, and Steve Jobs.  Many of them came from the Bloomberg Game Changers series, but for Steve Jobs, I watched the Steve Jobs: The Lost Interview that was filmed in 1995, lost for almost two decades and then was released in 2012.  I highly recommend all of them, but for this post, I want to highlight something Jobs said nearly 20 years ago.

What does this have to do with Networking?

There have been numerous articles over the past few months, some by me, that either advocate that network engineers should learn how to program, that there is no need for them to learn how to program, or they should simply learn to think like programmers.

Now check out what Steve Jobs said in the following video back in 1995:

It’s an interesting time in networking, isn’t it?  I can probably quote myself saying that for as long as I’ve been blogging and about a year before that.  Supposedly 2015 is the year of POCs, bakeoffs, and seeing which startups continue to get funding, and which ones slowly dissolve.  As we start to see who the winners and losers may be, I thought it would be good to highlight the last 7 years and where the major focuses areas have been and see what could be next.

Hello OpenFlow!

By now, many of us know who Martin Casado is and what he’s done.  His PhD work, Ethane, at Stanford with Nick McKeown and team led to the pre 1.0 work of OpenFlow.  For the first several years of the network (r)evolution, it was all about OpenFlow.  By 2009, the phrase Software Defined Networking had emerged and referred to OpenFlow enabled architectures.  It was easy to understand.

• As the industry chatter increased on OpenFlow architectures, hardware commoditization, and the de-coupling of the control plane and data plane, Casado had already started Nicira with McKeown and Shenker.
• When limitations were seen on what Continue reading

I had a conversation recently with someone who has more of a sysadmin background.  We started talking about the intersection of DevOps and networking and while his environment wasn’t large, there was one pain point he talked about – he doesn't have access to the network switches to ensure they are configured properly for “his” servers and to ensure packets aren't being dropped, etc. when there are issues with the application, server, or network.  And by the way, he really doesn't want access to the data center switches, because after all, many fear logging into network devices that are in production.  

Could DevOps and network automation help here?

In fact, the answer is yes.  The goal is to get the right data into the right hands as quick as possible.  An automation platform can be used to query the switch to get the exact data the admin needs.  For those that have help desks supporting large campus networks, the same philosophy can be used there as well.   Help desk, junior admins, and cross-functional team members can now get what they need in just a few seconds.

In order to test this out, I’ve Continue reading

There was a recent blog by Mark Burgess, founder and creator of CFEngine. It is a must read (on his personal blog).  He really makes you think where we are as an industry, question if we are on the right path, and quite frankly calling out certain technologies as pity attempts compared to what is needed. Regardless of all that, we cannot forget one key point, the industry is in fact moving forward right now. 

As I read his post, I remembered a conversation that I saw on twitter not too long between John Willis, Lori MacVittie, and Joe Onisick. It was more or less on the intersection of SDN and DevOps. 

The article by Burgess and the Twitter conversation really got me thinking. 

When you combine these interactions, even at the highest levels, you have to wonder, what is the right approach from both a vendor (product) and user standpoint? Are we on the right path? Do we have it all wrong? Must we fail first before getting it right?

SDN Will Simplify

SDN controllers will totally simplify things, but even then, the APIs they expose are arguably too low level for the average consumer of Continue reading

Over the past few months, I’ve been posting on using Ansible for network automation.  Changing things up a bit, this post will cover using Ansible for server automation and I’ll share a few Ansible playbooks that I’ve built and have been using to bootstrap servers and prep them for various applications such as OpenStack and NSX deployments.  

Step 1 - Playbook 1
Creating password-less root account

Since Ansible uses SSH by default for connecting to the servers, you will realize the first thing that needs to be done is to copy the public key of where you will execute playbooks from onto the “new” server.  To do this, I use a playbook that is called server_one_time_run.yml.  You will notice that in this playbook, and only in this playbook, I have remote_user set to jedelman and sudo set to yes.

I’ve been testing against bare-metal and virtual machine installs using an Ubuntu ISO image.  During the OS install process, “jedelman” is the account that was created on all hosts and virtual machines. 

This playbook runs and copies over the public key in the root directory.  We are essentially creating a password-less login for Continue reading

Over the past few years, I’ve had the opportunity to work with best and the brightest in the industry.  The reach started with my co-workers, partners, and vendors, but gradually expanded due to the likes of maintaining a blog and occasionally being on Twitter.  In a recent exchange with someone who gave me a massive pivot and jump start in my career almost 10 years ago, it reminded me of a presentation this same person gave back then.  

One of the key themes of this presentation was “Intrinsic Motivation.”  This was the first time I had ever heard the phrase – the speaker talked about one’s inner desire and self-motivation as the reason for wanting, learning, and doing.  It’s a feeling that is hard to describe, but I remember thinking during the presentation, “Hmm, I think I may have that.”  It’s not something many of us talk about, but those that have it can often see it or sense it in others.  On the surface, it could be called a passion.  It could be a hobby you love or when work starts to overlap as your hobby.  Maybe it’s just OCD coupled Continue reading

In my previous post about Docker, I focused on an introduction to networking with Docker.  That post had a fair amount of traction mainly due to it being #dockercon the week it was published, and seemingly, people had an interest in learning more about it.  Following the post, there were a few folks (@hartley and others) that pointed me to some great links about more advanced concepts in Docker and a site that validated what I was speculating with leveraging overlay tunnels as means for connectivity between nodes running Docker.

After reading through a few posts and the links about libswarm and libchan, it was on my mental to-do list to learn more about them and test more advanced designs.  In the meantime, the friendly folks at Digital Ocean let me know about a local meetup where James Turnbull, VP of Services at Docker, was presenting.  The meetup was tonight.  It was a short presentation, but for me, it was eye opening (it may be because I’ve been *mostly* focused on networking).  So, you won’t find a deep dive here like the last post, but rather, some general thoughts on Docker motivated from Continue reading

There has been a ton of information out there on Docker over the last week.  Because the impact on networking is often overlooked for new technologies, I figured I’d get a head start to understand the basics of Docker Networking.  This post documents the steps I took to test docker analyzing the network constructs that are automatically configured during container creation.

First, I installed Docker using instructions for Ubuntu 12.04 (LTS) 64-bit.

Post install, but before a container was created, here is the output of my Ubuntu machine.  Two interfaces: eth3 (192.168.1.134) and lo (127.0.0.1).  This Ubuntu machine is running in virtual box and eth3 is bridged onto my home network of 192.168.1.0/24.

Creating my first Docker container. This took about a minute (maybe less) to download and start.  Pretty impressive.  Notice the last line in the screen shot below.  It takes you right into the container shown at ‘root@c7ad293f989:/#’ 

In a new bash prompt because the existing shell is now used for the container, check out an ‘ifconfig.’  Notice the two new additions: docker0 and veth068f.  docker0 is a Linux bridge and veth068f Continue reading

Automating the configuration, provisioning, and management of particular workflows for cloud gets a lot of attention these days.  While automation makes perfect sense for deploying workloads faster, there are also other areas where automation can be leveraged to improve the overall operational efficiency of the IT Ops team. 

One of these areas is automating the validation of configuration changes.   This could mean validating changes deployed via the CLI for existing networks or validating changes made by SDN controllers for those new shiny physical AND virtual networks.  It doesn’t matter.  Connectivity tests can also be automated.  Much more can be automated than configuration and policy for data centers.

I remember doing annual power shutdowns of the data center and IDFs where I worked years ago.  I remember doing OS upgrades on critical network devices.  I also remember the chaos and the amount of people that needed to be on a bridge validating “everything looked okay” when the devices came back online.  Was everything always okay?  Hardly, but it wasn’t until business started on the following Monday, those one off issues were uncovered and fixed.  If there were tools verifying routing tables, Continue reading

If you read this site often, you already know I’ve been doing quite a bit of work with Ansible specifically as it pertains to networking.  While I will be showing another video very soon in a follow up post, I wanted to take a step back and cover a few things before doing so.  The focus here is less about the technology and more my general mindset around automation PLATFORMS, code, open source, and why I do it.  Just something I’d like to share because I’m occasionally asked questions around these topics.

Culture 

It’s not about the tool, in my case Ansible, it’s about the process, methodology, and ideas that go into thinking differently.  For me personally, Ansible just had a lower barrier for entry, but I’ve grown to quite like it.  Actually, I liked it from the get-go.   In order to effectively embrace platforms like Ansible, there needs to be a change in culture first.  It was amazing to see the emphasis on culture during the recent DevOps days in Pittsburgh.  I tuned in and out during the live stream and it seemed every time I had a chance to watch, it Continue reading

When networks are deployed in a box by box model, network admins know exactly what, where, and how something is being configured.  In highly dynamic environments, this may not be the case.  This is why it’s crucial to understand what is really going on behind the scenes.  In OpenStack, there are several components that together are comprised to make OpenStack Networking (aka Neutron).  These include the Neutron server, dhcp agent, metadata agent, L3 agent, and then the agents that would reside in the infrastructure to be programmed (on either physical and/or virtual switches).  For example, in Open vSwitch deployments, there would be a Neutron OVS agent on each host/server.  And this could vary based on which particular vendor plugin is being used too!

In this post, I’m going to mainly focus on the Neutron Layer 3 agent because I had a hard time grasping this one at first.  It turns out that it’s not so bad after all.

When I first started reading about Neutron, I saw many references that there was only one (1) layer 3 agent supported in a given deployment.  That just didn’t seem to make sense because that Continue reading

[Special and huge thanks to Scott Lowe for answering an endless amount of questions I had while writing this post and testing with NSX/OVS over the last few days. Thanks to Deepesh as well who I bounced OVS questions off of when I needed to give Scott a break. ]

In Open vSwitch 101, I described the three main components that make up Open vSwitch (OVS) from an architectural standpoint, namely ovs-vswitchd, ovsdb-server, and the fast path kernel module.  If you start to work with OVS, the first thing you realize is that it takes quite a bit more knowledge to really understand it.  This post will focus on some design principles and options when running OVS on a hypervisor like KVM in conjunction with a network virtualization solution.

To make this a little more practical, we will use a scenario which consists of a KVM host with 3 physical NICs – 1 x 1G and 2 x 10G.  The 1G interface will be used for management and the 2 x 10G interfaces will be used for actual transport of VM traffic. 

This example will also assume the use of an overlay network virtualization solution Continue reading

There is so much discussion on if network engineers need to be programmers that I was almost getting pissed off last week.  It was an odd and funny feeling.  Anyway, I've written in the past here and here about the use of Ansible for networking.  In this post and video, the goal is to show why network engineers don’t need to be "hardcore programmers."

Below is a short demo of using Ansible to automate basic network configuration tasks on Cisco routers.  As more Ansible modules come out (or any other tool that does the job), we will quickly realize, the network engineer doesn’t need to be a “hardcore programmer,” but rather understand the tool and articulate the requirements such that new modules can easily be added by others.  If the tools we’ll have in the future are “platforms” that can be customized by the consumer/customer meaning the vendor isn’t required to add more functionality, then that’ll pretty awesome for both the network engineer and the business.  It’s a win win.

FYI- the modules being demo’d are leveraging Cisco’s onePK as the API to connect to and make changes to the routers.

Hope you enjoy. Continue reading

In the last post, I talked about how Ansible could be used for various forms of network automation.  In the comments, Michael asked if Ansible could also be used for network test automation and verification.  Since I’m just starting to explore Ansible, I figured why not try it out.  The short answer is, it’s possible.  Let’s take a look at an example proving this out.

I just built a playbook that verifies a router, or group of routers, has reachability to a defined list of destinations.  Maybe you are deploying a new site? Maybe a new host?  Is the route being sent correctly to all sites?  Now, a playbook like this can be used to ensure every site (from the router) has reachability to the new site, subnet, or host.  You define the destination.  Cool, right?

Update: Before reading anymore, it's recommended to read the first post on Ansible for Networking.

Let’s do a quick walk through.

Here is the playbook.

As you can see, there is not a lot to it.  This playbook that we are looking at, called ‘pinger’, is using a module called ‘auto_pinger’.  Continue reading

[This article is the outcome of some great conversations and exchanges I’ve had recently with Jeremy Schulman (@nwkautomaniac) around automation and Devops in the world of networking.  Thank you to Jeremy for those late tweaks before getting this posted!  Thanks to Kirk Byers (@kirkbyers) as well - he was also gracious enough to respond to clarify a few things and assisted with this post indirectly.]

There have been numerous articles written that describe the what and the why of Devops.  Reading through a few of these, you find references to CAMS --- you’ll read how “Devops is about CAMS.”  CAMS stands for Culture, Automation, Measurement, and Sharing.  Imagine working in an environment where automation is embraced?  We know most networks are not leveraging nearly any type of automation.  While we usually talk about engineers (of all types) not embracing automation, is the harsh reality most organizations are from having the right culture to embrace automation? 

What if there was a way to start testing network automation without taking a risk in a production environment?  And to top it off, the network automation was being done using some of the same tools that Continue reading

You can’t listen to an interview or podcast, an industry panel, or read a Q&A about the future of networking that doesn't involve skill sets.  The biggest question of them all – what skills should network engineers focus on so they don’t become irrelevant? If you really want to know what skills make sense, why ask, when you can do an easy search to see what skills companies are looking for these days in a variety of roles.  Combine SDN with DevOps into your search criteria and the results may surprise you.  They sure surprised me.  

I don’t usually peruse the job boards, but I’m on LinkedIn daily, so I figured I’d put their job site to the test and see what skills are in demand by searching for specific products and tools.  This is the best way to search, right?  I did numerous searches using the following keywords: SDN, Puppet, Chef, Ansible, Python, Cisco, network, and CCIE.

Here are the results:

• There were only five (5) jobs available when searching for SDN, CCIE, and Python.  Reminder, I only searched LinkedIn.
• The first was for a vendor as a “SDN Cloud Architect.”  Continue reading

It’s been two weeks since I attended my 3rd consecutive Open Networking Summit (ONS) and I’m glad to say, I finally found some time to get some notes and thoughts on paper about the conference.  Here are some on SDN at Google and Microsoft, and how they compare and contrast to industry incumbents’ solutions, but also how programmable NFV can be game changing in the Enterprise.  I also include thoughts on how Embrane and Big Switch play into this.

Enter Andromeda

Google talked about their [home grown] network virtualization solution.  It leverages a custom SDN controller called Andromeda that controls physical switches, virtual switches, programmable NFV devices, and also ties into the storage platforms deployed.  Google talked about how they have showed industry leadership with technologies such as GFS, MapReduce, B4 WAN, etc.  If I extrapolate, they expect Andromeda to do for data center networking what GFS is doing for distributed scale out storage.  Who will be the Nutanix of networking?

Google, like few others, still define SDN as the separation of the control plane and data plane.  Google states, “logically centralized/ hierarchical control plane with peer to peer data plane beats full Continue reading

Over the past few weeks, I’ve written about the idea behind a common programmable abstraction layer.  Previous articles are here and here.  It’s worth stating that something like a CPAL can be used with or without SDN controllers and with or without cloud management platforms.  As can be seen from the previous write ups and the video/demo below, today its primary focus is data extraction and data visibility.  It can use device APIs or controller APIs.  It’s about accessing the data you need quicker.  It’s that simple.  No more jumping from device to device and having to manage text and excel files.  

Edit 3/15/2014:
Github repo for CPAL

If there is a controller in the environment, you can still view data around particular physical and virtual switches in the environments by creating the right modules.  Same can be said if there was a CMP/CMS deployed.  While a CPAL can easily make changes to the network, it’s about taking small steps that can have a larger impact on how we use new APIs on network devices and controllers.  And if we don’t strive for a common framework now, we will end Continue reading

Two of the three companies promoting white box, now more commonly known as bare metal, switching are Cumulus and Big Switch Networks.  There has been coverage on each of these companies, but the question always arises, “does Cumulus support OpenFlow?”  I had the chance to talk to JR Rivers, Cumulus CEO, at the last Open Networking User Group (ONUG) during a Tech Field Day video and heard the answer from him then, but hadn’t seen anything documented publicly. 

There was a SDN Meetup at Stanford last week where JR gave his take on SDN and a great overview on Cumulus, which happens to be on Vimeo.  More importantly, he touches upon the question regarding OpenFlow support in the Cumulus Linux software stack during the video.

Coming directly from JR, his response (around the 58 minute mark):