SDN Deployments/Worries Rise Among Service Providers

SDN Deployments/Worries Rise Among Service Providers

by Steve Harriman, VP of Marketing - November 18, 2014

For the second consecutive year, Packet Design surveyed attendees of the SDN/MPLS International Conference in Washington, D.C. about SDN adoption, business drivers, and concerns. The answers – mostly from service providers – were very interesting compared to the 2013 survey. Overall, use of SDN in production networks is up, but so are concerns about everything from industry standards to management skills to tools. Here’s a rundown of the results (also summarized in our announcement). 

Production SDNs Increase 

More than half of this year’s survey respondents (53 percent) said they have some production SDN deployed, compared to only 19 percent of survey respondents in 2013. Of those, about 42 percent have up to 25 percent of their networks SDN-enabled. Only 11 percent indicated having between 26 and 75 percent of their networks as production SDN. 

Business Agility, New Services Up as Top Business Drivers 

The number of survey respondents who consider “agility to respond faster to business demands” as their number one driver jumped by 150 percent (65 percent in 2014 vs. only 26 percent in 2013). Nearly Continue reading

MAC Address Aggregation and Translation as an Alternative to L2 Overlays

Not so long ago, if you wanted to build a data center network, it was perfectly feasible to place your layer three edge on the top-of-rack switches and address each rack as its own subnet. You could leverage ECMP for simple load-sharing across uplinks to the aggregation layer. This made for an extremely efficient, easily managed data center network.

Then, server virtualization took off. Which was great, except now we had this requirement that a virtual machine might need to move from one rack to another. With our L3 edge resting at the top of the rack, this meant we'd need to re-address each VM as it was moved (which is apparently a big problem on the application side). So, now we have two options: We can either retract the L3 edge up a layer and have a giant L2 network spanning dozens of racks, or we could build a layer two overlay on top of our existing layer three infrastructure.

Most people opt for some form of the L2 overlay approach, because no one wants to maintain a flat L2 network with dozens or hundreds of thousands of end hosts, right? But why is that?

Continue reading · 1 Continue reading

Introduction to Using Cisco NX-API

I’ve posted a few times in the past about Cisco’s NX-API and realized I hadn’t provided any guidance on how to get started using the API itself. In this post, I share two videos that are meant to serve as a quick start to those who don’t have a development background and are looking to test NX-API.

The first video looks at the NX-API sandbox and how you map the data represented in the sandbox back into objects that you can use while working in Python.
The second video shows where to get the modules that I use in the first video, namely xmltodict and

Note: the device module that I use is primarily used with XML data being returned from the device. The easiest thing for those who want to test is to follow the steps outlined in the videos although there are mechanisms to switch to JSON. This device module does not support json-rpc (as that is still fairly new in NX-API).

And, don’t forget, you’ll need to connect to your Nexus 3K/9K via the management interface to work with NX-API.


Introduction Part 1

Introduction Part 2

Other NX-API links to check out:

Cisco Nexus Continue reading

L2 DCI failure remediation with proxy ARP

I had the pleasure of taking an interstate road trip with Ethan Banks last weekend. Naturally, we talked shop a bit.

Somewhere in Connecticut we were speculating about the possibility of using host routes and proxy ARP to restore connectivity between members of a subnet when an L2 DCI fails.

Would a router produce proxy ARP replies for destinations which are members of a directly connected network?

I labbed it up to find out.
Don't do it!
The routers run different HSRP groups. One per data center.

Hosts are configured to use the local HSRP address as their default gateway. There is no FHRP localization other than the configuration on the hosts. This is not a scheme that facilitates VM mobility between sites.

Each router uses an IP SLA monitor to check the L2 interconnect. This is accomplished by pinging the other site's HSRP address. The IP SLA monitor drives a tracking object (track 1). The tracking object drives a second tracking object (track 2) with inverted logic: when the routers can't ping each other track 2 transitions to "up":
 ! IP SLA monitor on R1
ip sla 1
icmp-echo source-interface FastEthernet0/1
ip Continue reading

Prepare Your Site for Traffic Spikes this Holiday Season

The holiday season is approaching, and everyone is thinking about gifts for their friends and family. As people increasingly shop online, this means huge spikes in traffic for web sites---especially ecommerce sites. We want you to get the most out of this year’s surge in web traffic, so we’ve created a list of tips to help you prepare your site to ensure your visitors have a reliable and fast experience.

Make sure your site can handle traffic spikes:

1) Contact your hosting provider to understand the limits of your hosting plan

Even though CloudFlare offsets most of the load to your website via caching and request filtering, a certain amount of traffic will still pass through to your host. Knowing the limits of your plan can help prevent a bottleneck from your hosting plan.

2) Reduce the number of unwanted requests to your infrastructure

CloudFlare allows you to block IP address individually or IPs from entire regions. If you don’t want or need traffic from certain IPs or regions, you can block them using your Threat Control panel. This is useful for sites who know where their visitors usually come from.

For example, if you run an ecommerce site with Continue reading

Complexity vs Security

Many of the ‘security’ measures in our networks add complexity. That may be an acceptable tradeoff, if we make a meaningful difference to security. But often it feels like we just add complexity for no real benefit.

Here’s some examples of what I’m talking about:

  • Multiple Firewall Layers: Many networks use multiple layers of firewalls. If you have a strong policy that says all traffic must go via a server within a DMZ, this makes sense. But often we end up with the same connections going through multiple firewalls. We end up configuring the same rules in multiple places. No security benefit, but increased chance of making mistakes, and added troubleshooting complexity.
  • Chained proxies: It’s pretty common to use a proxy server, to enforce HR and security controls on what users browse. But some organisations have chained proxies, where an internal proxy server connects to an upstream proxy server to get Internet access. The upstream proxy doesn’t add anything from a policy or control perspective. It’s just another point to configure and troubleshoot.
  • NAT/Routing: First let me be clear: NAT is not complete security in itself, but it can form a valid part of your overall network security policy. That Continue reading

State of the State for Open vSwitch

This week, VMware will be hosting the Open vSwitch 2014 Fall Conference, with more than 200 attendees and nearly two dozen talks on a variety of subjects from a key participants.  The full schedule is available here, and we’ll be doing a wrap up of some of the takeaways from the conference a bit later.

For the uninitiated, Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license.  It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, 802.1ag).  In addition, it is designed to support distribution across multiple physical servers similar to VMware’s vDS or Cisco’s Nexus 1000V. See full feature list here

For more information on OVS, I encourage you to check out the OVS website.

In the mean time, take a read about latest Open vSwitch developments in this post on Network Heresy by OVS core contributors Justin Pettit, Ben Pfaff, and Ethan Jackson.

Accelerating Open vSwitch to “Ludicrous Speed”


But I’ve got an ‘Excellent Signal’!!?

Ever so often I find myself troubleshooting some type of wireless related issue, and while wireless issue’s vary from Slow performance Clients can’t connect Poor voice performance Or even random disconnects, the list is endless. However one of the common things I hear during the troubleshooting process is without a doubt along the lines of: […]

The Degree or the Certification: You are Not a Widget

One of the things that bothers me the most about the Internet of Things (IOT) is how blithely we slip from talking about objects as things to people as things. Among all the things I do not want to be, a “thing,” attached to the “Internet of Things,” is not one of them. What does this have to do with the question of whether you should get a degree or a certification? Simply this: You shouldn’t treat yourself as a widget, either.

Let me explain.

I can’t count the number of times I’ve heard people say, “You should get a certification because it provides more bang for the buck.” In fact, in one rather amusing line of reasoning on the subject, Peter Thiel (who started the Thiel Foundation to encourage smart young people to quit college and take up a career instead), said in a recent interview:

Educational institutions are far too often interested in churning out graduates (i.e., getting their money) without imparting the ability to think rather than just work the system.

To paraphrase, you should opt out of college because colleges are just in the game to make money off you, and you’ll make Continue reading

ESXi whitebox server

I usually have access to an ESX box at work where I can run multiple VMs and virtual routers for labbing and testing. I’ve also wanted one at home. It’s nice to be able to quickly spin up VMs when needed without always running them through my laptop. While virtual routers don’t need lots of […]

Just Published: Overlay Virtual Networks in Software Defined Data Centers

Overlay virtual networks are one of my favorite topics – it seems I wrote over a hundred blog posts describing various aspects of this emerging (or is it reinvented) technology since Cisco launched VXLAN in 2011.

During the summer of 2014 I organized my blog posts on overlay networks and SDDC into a digital book. I want to make this information as useful and as widely distributed as possible – for a limited time you can download the PDF free of charge.

Learn more about the book

Networking’s UX victims

Our perception of nirvana is impacted mightily by current conditions. For people who live in third world countries, for example, merely having running water or reliable electricity can be a life-altering boon. Meanwhile, those of us who are more accustomed to the creature comforts of life consider slow internet or a poorly seasoned meal worthy of public scorn (even if we add the hashtag #firstworldproblems).

So how is the current state of networking impacting its user base?

A new normal

Perhaps the most insidious effect of poor conditions is that prolonged exposure can actually cause us to reset our baseline for normal. When we are subjected to extended periods of great or even long periods of suck, we adjust our expectations.

In networking, this means that our current normal has been forged through diligent neglect of actual user experience for decades. It’s not so much purposeful behavior by the incumbent networking players so much as placing focus elsewhere. For at least the last few decades, the future of networking has always been defined by the next protocol or knob. That is to say that the focus for product development has always been about bolstering device capability.

With the focus Continue reading

BGPSEC: Basic Operation

I’m going to take a little break from my other two series to inject a short series on BGPSEC. I’ll return to HTIRW and RFCs you need to know shortly. BGPSEC is a set of standards currently under consideration in the IETF to secure BGP beyond the origin AS – in other words, to secure […]

Author information

Russ White

Russ White
Principal Engineer at Ericsson

Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. He recently published The Art of Network Architecture, is currently working on a new book in the area of network complexity with Addison Wesley, a book on innovation from Continue reading

Custom Python Packages With PyPi For Networking Peoples


Quite frequently I write and build code to control and harvest data from network infrastructure. Whether it’s writing modules for a stack automation tool like Ansible or writing modules for run to completion scripts, code is often spawned to meet requirements. Sometimes this code is more than a little complex with many dependencies. In addition, sometimes code is written to extend an existing module, like the Python NetConf library NCClient. This happened recently for example when the NCClient library failed to meet expectations.

Two options exist for complicated projects from an arbitrary point of view:

  1. Include all of the dependencies in a directory and provide everything including the kitchen sink to allow your code to function as per design.
  2. Create an installable which takes care of all dependencies.

With Python, option 2. is an interesting one that I’ve never previously thought about doing.

Information for this post and my initial requirement came from these two sources which were most helpful:

PyPi Introduction

How many times have you installed a module using something like

sudo pip install 'blah'
without really wondering what the repository is that hosts the code for you to just automagically Continue reading

War Stories: Cursed VLANs

I’ve written before about switch ports being permanently disabled. This time it’s something new to me: VLANs that refuse to forward frames.

A Simple Network

The network was pretty straightforward. A pair of firewalls connecting through a pair of switches to a pair of routers:

Cursed VLAN

Sub-interfaces were used on the routers and firewalls, with trunks to the switches. VLAN 100 was used for, and VLAN 200 was used for The switches were configured to pass VLANs 100 & 200.

All was working as expected. All devices could see each other on all VLANs.

Until it stopped

We received reports that we’d lost reachability to Router A’s VLAN 200 sub-interface. After doing some investigation, we could see that Firewall-A could no longer see Router A’s MAC address on G0.200. But everything else was fine – the VLAN 100 interface worked perfectly. So we knew it couldn’t be a physical interface issue.

Hmmm. What’s going on? First instinct: check the switch port configuration. Has anything changed? Nope. VLAN 200 still there, configured as expected. The router & firewall were still tagging frames with VLAN 200. But they couldn’t see each other, and the Continue reading

Secret Sunday – Greg Ferro

Greg FerroJust over a month ago I accepted Etherealmind’s “30 Blogs in 30 Days Challenge”, and this Friday I ‘m pleased to say that I completed the challenge without missing a day. It seems appropriate then that I should use today’s Secret Sunday to give a shout out to the man behind the mission, Greg Ferro (aka Etherealmind).

In some ways it feels like cheating to include Greg because he’s unlikely to be a new discovery for anybody reading my blog, but credit is due where it’s due; you can’t argue that Greg does not give back to the networking community. His Etherealmind website has a huge following and a wealth of content written over a number of years. He’s a co-host of the PacketPushers podcasts, generating hundreds of podcasts dedicated to the networking industry. He has written three books that he has published on LeanPub and has a fourth on the way, co-authored with the venerable Brent Salisbury.

Greg is never short of an opinion, and I suspect that one of the reasons for his large following is that he isn’t afraid to go out there and let you know what he thinks, even if “you” are a vendor, and what Continue reading

Ops Work vs Project Work

There’s a constant tension between delivering new services, and running the existing services well. How do you figure out how to prioritise work between Operations tasks and Project work? Skewing too far either way leads to problems. Maybe the answer is in how we structure Operations tasks?


  • Operations work: Dealing with outages, trouble tickets, support requests, etc. System monitoring – reviewing data for capacity planning, and identifying new areas to monitor. Automated repetitive tasks. Patches, upgrades, minor changes to existing services. Accountants would call this work OpEx.
  • Project work: Design, test and deployment of new services. Major upgrades or enhancements to existing services. This is usually classified as CapEx. For some businesses, this work is customer-billable.

What happens when you’re imbalanced?

  • Too much Project work: If you’re flat out deploying new systems (and dealing with the fallout), it’s easy to let Operations work slip. Maybe you don’t get around to automating that log rotation script, or paying attention to the slope of that consumption graph. It’s OK for a while too…things seem to be trucking along. But then you start having outages due to simple things like logs filling directories, or you hit a capacity limit, and there’s a 6-week Continue reading