The Care and Feeding of a High Maintenance Network

The Care and Feeding of a High Maintenance Network

A network is an organic creation. The minute it’s born, when all new core and edge connections are made and routing is turned up, things begin to change. Many changes are self-driven due to unexpected interactions: Equal Cost Paths (ECMPs), Asymmetric Paths, etc. Other changes are due to the random nature of the Internet and are readily noticeable at the peering points into the newborn network.

Some people think that once the switch is turned on things will just work as designed. I’ve found that is rarely the case. Networks need care and feeding. Tools to check on the processing capacity, resource consumption, and well being of the network and its individual elements are required.

For the monitoring aspect of this “care and feeding,” simple SNMP tools may be used. They are perfectly adequate for tracking and graphing CPU rates, available memory and throughput for connections between network elements. However, when it comes to understanding the network’s routing and traffic patterns, using SNMP-based tools is rarely the best method.

Today’s dynamic IP networks require visibility into what’s happening Continue reading

Root Cause Analysis – It’s Not Perfect

Automated Root Cause Analysis promises a lot. High-end network monitoring systems promise that they can automatically isolate network problems, and only tell you about the thing that needs fixing. This sounds very enticing. Who wants a flood of alarms, when we could get just one alarm, telling us what we need to fix? But it’s not perfect, and you do need to pay attention to it.

Consider this contrived network:

What happens if the upstream link from the router fails?

From the perspective of the NMS, all systems at that site are unreachable. A simple NMS that is unaware of topology will create 4 alarms – one for each of the router, the switches and the server. A smarter NMS will recognise that it only needs one alarm, for the router WAN link being unreachable (and therefore the whole site is offline). It will know that the switches and server are unreachable, but those alarms will be suppressed by the key incident.

This all sounds like a good idea. Why wouldn’t you want that?

But what if the NMS view of the network is incomplete? What might happen then?

Consider the same network as above, but this time a new WAN router has been Continue reading

On choosing VMware NSX or Cisco ACI

Are you stuck in the middle of a battle to choose VMware NSX or Cisco ACI? In this post I’ll attempt to bring some clarity and strategic guidance in first choosing the right path, then propose how the two technologies can co-exist. I’ll start with the message below from a reader asking for my opinion on the matter:

Hi Brad,

I’m involved in a new Data Center networking project where Cisco is proposing the Cisco ACI solution. I am starting to dig-in to the technology, but my immediate “gut reaction” is to use Cisco for a standard Clos-type Leaf and Spine switch network and use NSX for providing Layer 3 to Layer 7 services.

I am interested in hearing your opinion about Cisco ACI versus VMware NSX, since you have worked for both companies. If you have time, it would be great to share your views on this subject.

As you can imagine, this is a highly political discussion and our network team are Cisco-centric and resisting my ideas. We are a VMware/Cisco shop and I want the best fit for our SDDC strategy.

For the sake of discussion, lets assume that your IT organization wants to optimize for better Continue reading

On choosing VMware NSX or Cisco ACI

Are you stuck in the middle of a battle to choose VMware NSX or Cisco ACI? In this post I’ll attempt to bring some clarity and strategic guidance in first choosing the right path, then propose how the two technologies can co-exist. I’ll start with the message below from a reader asking for my opinion on the matter:

Hi Brad,

I’m involved in a new Data Center networking project where Cisco is proposing the Cisco ACI solution. I am starting to dig-in to the technology, but my immediate “gut reaction” is to use Cisco for a standard Clos-type Leaf and Spine switch network and use NSX for providing Layer 3 to Layer 7 services.

I am interested in hearing your opinion about Cisco ACI versus VMware NSX, since you have worked for both companies. If you have time, it would be great to share your views on this subject.

As you can imagine, this is a highly political discussion and our network team are Cisco-centric and resisting my ideas. We are a VMware/Cisco shop and I want the best fit for our SDDC strategy.

For the sake of discussion, lets assume that your IT organization wants to optimize for better Continue reading

On choosing VMware NSX or Cisco ACI

Are you stuck in the middle of a battle to choose VMware NSX or Cisco ACI? In this post I’ll attempt to bring some clarity and strategic guidance in first choosing the right path, then propose how the two technologies can co-exist. I’ll start with the message below from a reader asking for my opinion on the matter:

Hi Brad,

I’m involved in a new Data Center networking project where Cisco is proposing the Cisco ACI solution. I am starting to dig-in to the technology, but my immediate “gut reaction” is to use Cisco for a standard Clos-type Leaf and Spine switch network and use NSX for providing Layer 3 to Layer 7 services.

I am interested in hearing your opinion about Cisco ACI versus VMware NSX, since you have worked for both companies. If you have time, it would be great to share your views on this subject.

As you can imagine, this is a highly political discussion and our network team are Cisco-centric and resisting my ideas. We are a VMware/Cisco shop and I want the best fit for our SDDC strategy.

For the sake of discussion, lets assume that your IT organization wants to optimize for better Continue reading

On choosing VMware NSX or Cisco ACI

Learning NSX, Part 18: Routing Without Network Address Translation

This is part 18 of the Learning NSX blog series, in which I talk about using layer 3 (L3) routing with VMware NSX but without network address translation (NAT). This post describes a configuration that offers yet another connectivity option for OpenStack cloud administrators and operators.

In part 6, I showed you how to add a gateway appliance to your NSX installation. Part 9 leveraged the gateway appliances to create a L3 gateway service, which—as I explained in part 15—provides the functionality for logical routers in OpenStack. (Logical routing was covered in part 14.) Part 16 expanded the routing configuration to support multiple external networks. This post expands the options again by showing you how to do logical routing without using network address translation (NAT). Of course, it would probably be helpful to read the entire series; links to all posts can be found on the Learning NVP/NSX page.

As I mentioned, so far you’ve seen three different external connectivity options:

• Routing (layer 3 connectivity) to a single external network
• Routing (layer 3 connectivity) to multiple external networks using VLANs
• Bridging (layer 2 connectivity) between a logical network and a physical broadcast domain

Both of the routed Continue reading

Cloudflare – An Awesome IPv6 Move – Thank you!

Recently Cloudflare made a pretty cool move, and made their IPv6 services available to all of their customers – even the free ones, like me! So first things first, huge kudos to Cloudflare for offering this up; it has offered … Continue reading →

If you liked this post, please do click through to the source at Cloudflare – An Awesome IPv6 Move – Thank you! and give me a share/like. Thank you!

SDN Job Report – Methods

Ready for that first #SDN job? How many jobs are available out there? Are there jobs with an SDN focus, or simply jobs that include SDN as just one small part of the job? Are SDN jobs really beginning to emerge?

Well, I too am curious about this kind of question. So a while back, I started tracking mentions of SDN on a couple of job sites: Dice.com and Monster.com. For today’s post, I’ll set the stage for how we’ve gathered the data. In the following posts, I’ll show the numbers for the 3^rd quarter 2014.

My Own Ponderings of Searching for “SDN” Jobs

A while back, I was wondering about the job market for networking jobs. We’re in the middle of a time when networking jobs may undergo a lot of change. Our individual preparation for our careers needs to consider both the existing world (where most of today’s jobs still are) and this possible future world with lots of SDN jobs. And I thought, “Wouldn’t it be great if someone would post articles on occasion about emerging SDN-related jobs, and compare that to existing networking jobs.”

And then I wondered if I could do Continue reading

SDN and legacy companies: laggards or pragmatists?

There was an interesting Twitter thread over the weekend initiated by Ethan Banks (@ecbanks). He commented that there was too much technique churn in SDN and NetOps (the networking equivalent of DevOps). His point was that in the face of all the change in how to do things, it left users in an impossible spot. How can up pick up a new technology if the frameworks around how to use it are consistently changing?

His conclusion was that we cannot herd these cats. But what is really going on?

No consensus on operating models

The most basic truth here is that there is no real consensus on operating models around any of the new technology. While there are rough agreements on a few architectural principles (and even there, far more is in the air than well grounded), there is really not a lot of best practices to which companies can pin their operations.

Sure, it might be obvious to people that SDN is here to stay. But what exactly does that mean? And which SDN do I evaluate, purchase, and eventually deploy? Do I go with OpenFlow because ONF has convinced me that openness is the primary tenet? Do I Continue reading

The Degree, or the Certification: First Thoughts

Having just come off doing a presentation on “being a great engineer,” I can tell you what the number one question people asked was: Should I get a degree, or a certification? In fact, several people were irritated that Denise and I were even talking about anything else, because it’s the only question that counts.

Let me counter that thought. If you’re asking whether you should get a degree or a certification, you’re asking the wrong question.

It’s not that I don’t have anything invested in certifications. I hold a CCIE (2635), CCDE (2007:001), and CCAr. I’ve written questions for the CCIE. I was on the original SME team that invented the CCDE and CCAr certifications. I’ve taught certification classes, written certification books, and generally been involved in the certification world for a long time.

It’s not that I don’t have anything invested in college, either. I have one four year degree, two Master’s degrees, and I’m currently working like crazy to gain acceptance into an PhD program (Philosophy, in Apologetics and Culture, if you’re curious). I’ve taught as an adjunct in the NC State MS program, and I’m on Capella University’s advisory council. I teach on a regular basis Continue reading

Simple Bash Ping Script

Been looking for something like this for a while:

http://etherealmind.com/tech-notes-ping-sweep-ip-subnet/

for i in `seq 1 255`; do ping -c 1 192.168.1.$i | tr \n ‘ ‘ | awk ‘/1 received/ {print $2}'; done

I usually use nMAP for this, but in some instances you may not have access to it. For those times, Bash should work really well.

Android Bugs

I have a little Samsung Galaxy S2 which is a perfectly adequate phone. I use the Google Apps on it to keep some semblance of order in my life. However, I run into some occasional bugs which require a bit of work to solve. So I’m putting in the fixes here so they’re easily available.

1. Weather widget displays an incorrect location. Documentation date 03.11.2014
This is fixed by going to Settings->Application Manager and performing a forced stop on the TouchWiz Home application.

2. Lockscreen displaying an incorrect location. Documentation date 03.11.2014
First I thought this was related to the first problem. However, this can be fixed by going to Applications->Security and changing the location detection to refresh automatically. It looks like adding a new city in the weather widget updates the location in the lockscreen, but removing it doesn’t force an update to the lockscreen.

More bugs will be documented here as they’re found.

Learning NSX, Part 18: Routing Without Network Address Translation

This is part 18 of the Learning NSX blog series, in which I talk about using layer 3 (L3) routing with VMware NSX but without network address translation (NAT). This post describes a configuration that offers yet another connectivity option for OpenStack cloud administrators and operators.

In part 6, I showed you how to add a gateway appliance to your NSX installation. Part 9 leveraged the gateway appliances to create a L3 gateway service, which—as I explained in part 15—provides the functionality for logical routers in OpenStack. (Logical routing was covered in part 14.) Part 16 expanded the routing configuration to support multiple external networks. This post expands the options again by showing you how to do logical routing without using network address translation (NAT). Of course, it would probably be helpful to read the entire series; links to all posts can be found on the Learning NVP/NSX page.

As I mentioned, so far you’ve seen three different external connectivity options:

• Routing (layer 3 connectivity) to a single external network

• Routing (layer 3 connectivity) to multiple external networks using VLANs

• Bridging (layer 2 connectivity) between a logical network and a physical broadcast domain

Both of the routed Continue reading

Response: The Inner Ring

This is some of Russ White's finest writing on the career development using a lecture from CS Lewis as a baseline

The post Response: The Inner Ring appeared first on EtherealMind.

HTIRW: Provider Peering and Revenue Streams (Part 2)

This is a continuation from last week’s post on provider peering streams. Second Example: Customer to Noncustomer Assume traffic is coming in from A and is destined to M. How can AS64501 maximize revenue stream in this situation? There is only place to make money (the [A,C] link), and there is one place where its […]

Author information

Russ White

Principal Engineer at Ericsson

Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. He recently published The Art of Network Architecture, is currently working on a new book in the area of network complexity with Addison Wesley, a book on innovation from Continue reading

Use a Disaster Recovery Project to Build Your New Cloud

It doesn’t make sense to build a new data center network to support legacy bare-metal server infrastructure. You’ll have to use relatively expensive 1G/10G ports to be able to connect the current and future servers, and once the server and virtualization engineers wake up and do hardware refresh you’ll end up with way too many ports (oh, and you do know that transceivers could cost more than the switching hardware, right?).

BYOD: Just another money-grab?

BYOD policies sound alluring. No more forced use of a crappy old corporate laptop – “hey look, we’ll let you choose whatever you want!” But I think it is a way to shift the cost burden over to employees. It will be done slowly, over several years, and we’ll welcome it. But it will lead to employees carrying more costs. I guess we should be careful with what we wish for.

In my teens I spent many years working in the produce & butchery departments at a local supermarket. When I started out, the contracts still had the last vestiges of union-dominated times. So we got paid allowances for laundry, extra allowances if we’d passed some school exams, higher rates for overtime, meal allowances, etc. During the years I was there, these were eroded. Each year they gave us pay rises that were nominally higher than inflation, and yet another allowance was ‘incorporated’ into my wages. Sometimes allowances would remain for older employees. When I left, I was being paid significantly more than new employees, in part because I still had several extra allowances.

I think we’ll see the same thing with BYOD programs. I think it will go like this:

1. Announce BYOD Continue reading

Helpful Concepts for the Fresh New Geek

Someone recently asked me to be a professional mentor, an occurrence that becomes more surreal the longer I consider it in its implications and entirety.  So far the recipient of my educational transgressions appears content, but the experience has reminded me of several ranty moments I’ve had over the years regarding what new network geeks […]

Author information

Keith Tokash

Keith Tokash, CCIE (R&S) #21236, began his career in 1999, and has spent the last decade running around large content and small ISP networks. He spends his spare time with his newborn son, on the mat at the local Jiu-Jitsu gym, and trying to keep his fat yap shut.

The post Helpful Concepts for the Fresh New Geek appeared first on Packet Pushers Podcast and was written by Keith Tokash.

Positioning an IT Conversation

About a  week ago, I took my wife’s van to the shop. The main issue was it was making a popping noise in the front end. I only observed the noise when steering sharply and the vehicle was in motion. Typically this occurred when parking. Although I was nearly certain this was an issue with a CV joint, I only told the mechanic about the symptoms we had observed.

The reason I didn’t lead the conversation to the CV joint is that I wanted the mechanic to look at the problem objectively. I knew he was the expert and I wanted him to solve the problem instead of replacing a part. In order to shift the responsibility, I needed the mechanic to diagnose the problem and create a plan of action.

Positioning IT Conversations to Solve Problems

At this point in my career, I have worked in various areas of technology. Over the years, I’ve had customers that tell me exactly what they think they need. In some cases, they’re correct. However, there are times that their solution does not fully solve the problem they are observing. On the other hand, some customers take a smarter approach and explain the problem they are trying to solve.

When Continue reading