Decoding LACP Port State

It’s frustrating when the output to a show command gives exactly the information needed, but in a format which is unintelligible. So it is with the Partner Port State field in the NXOS show lacp neighbor interface command which reports the partner port state as a hexadecimal value. To help with LACP troubleshooting, here’s a quick breakdown of the port states reported on by LACP, and how they might be seen in Junos OS and NXOS.

LACP Port State

The LACP port state (also known as the actor state) field is a single byte, each bit of which is a flag indicating a particular status. In this table, mux (i.e. a multiplexer) refers to the logical unit which aggregates the links into a single logical transmitter/receiver.

The meaning of each bit is as follows:

Pre-Provisioning Your FEXen For Fun and Profit

In this post, I’ll discuss how to protect your income by using the FEX pre-provisioning capability of NXOS. I discovered the hard way that not pre-provisioning your FEX can have catastrophic side effects. What better story to post on Friday the 13th?

FEXy Time

Attaching a FEX to a Nexus switch is relatively simple; a few commands on each of the two switches the FEX connects to and it’s up and running. It’s also possible to pre-provision the FEX modules in the configuration. The documentation doesn’t make it entirely clear why this would be desirable, beyond the rather cryptic:

In some Virtual Port Channel (vPC) topologies, pre-provisioning is required for the configuration synchronization feature. Pre-provisioning allows you to synchronize the configuration for an interface that is online with one peer but offline with another peer.

Got that? In other words, pre-provisioning makes it possible to configure a FEX module that isn’t there yet, or that is powered down, or is only connected to one side of a VPC pair for some inexplicable reason. Maybe I’ve ordered some
(plural of FEX) and want to configure the ports ahead of time? Whatever the rationale for doing so, I’ve never previously needed pre-provisioning Continue reading

My Lexicon: Fexen

Fexen (noun, pl.; pronounced Fex-uhn)

Usage

“Do we have any copper FEXen on those switches?”

Explanation

Fexen is the plural of FEX (the Cisco Nexus Fabric Extender modules). Oh, I know, “FEXes” is just as easy to say, but somehow FEXen seems to work better. Try and use this word in conversation today and see how it feels.

“We have about 20 FEXen distributed around the data center.”

I think you’ll like it.

If you liked this post, please do click through to the source at My Lexicon: Fexen and give me a share/like. Thank you!

My Lexicon: Nexii

Nexii (noun, pl.; pronounced nex-eye)

Usage

“I have built a leaf/spine fabric using Nexii.”

Explanation

Nexii is the plural of Nexus, obviously. To talk about “Cisco Nexuses” is ugly. Referring to “Cisco Nexus switches” is syllabically inefficient. Nexii is the perfect blend between inappropriate Latin noun pluralization and verbal optimization.

“We need to upgrade the software on our Nexii.”

You’ll thank me later.

If you liked this post, please do click through to the source at My Lexicon: Nexii and give me a share/like. Thank you!

Task List Tracker for the Mac (DIY Version)

As a Mac user, how do you keep track of the tasks you need to complete? I find myself swamped in things that need doing and every day more things get added to my list. The problem is, in the past I’ve relied too much on my memory to keep track of what I need to do, and I’m sadly aware that there are more things on my task list than I can keep track of, and all too frequently I get into work and think “What was I going to do this morning? I’m sure there was something high priority, but…”

It should be easy, you’d think, to maintain a list of tasks, assign some kind of priority, and have that list readily accessible while using my computer. I suspect there’s an app (indeed, that there are many apps) for that, but while I have tried a few, somehow I’ve not managed to integrate them into my daily workflow. I spoke to a colleague about this, and he said that he keeps a text file on his Desktop listing all his open tasks, and he updates it as needed. If it works for him, maybe it would work for Continue reading

Making a Clickable HTML Network Diagram using OmniGraffle

As a Mac user, I have to give my diagramming love to OmniGraffle and I try not to envy the Visio users too much. I maintain that Graffle diagrams subjectively look nicer than Visio, but in terms of features, Visio wins the day. Despite that, sometimes poor old Graffle does so something helpful and in this case, it’s being able to export a diagram as an image with an HTML image map.

The Plan For A Web-Based Network Diagram

My plan was to create a web-based network diagram for my home network where I could click on any device on the diagram and be connected to it using the appropriate protocol handler (e.g. SSH or HTTPS). This hypothetical page would not serve as a diagram of the network, but might also provide useful information for my long-suffering, geek wife, who tells me with despair in her eyes that she has no idea what the network looks like any more after I’ve messed around with it so much. She has a point. After considering making something in HTML, I realized that OmniGraffle would do the hard work for me, and it would be much easier to update later, too.

For Continue reading

CYA! Cover Your Assets (By Securing Them) (Thwack)

Still using local accounts for device access? Don’t know what a Term Process is? You need to CYA!

On the Solarwinds Thwack Geek Speak blog I looked at a variety of security (and related) features which should be configured on all devices. Please do take a trip to Thwack and check out my post, “CYA! Cover Your Assets (By Securing Them)“.

Please see my Disclosures page for more information about my role as a Solarwinds Ambassador.

If you liked this post, please do click through to the source at CYA! Cover Your Assets (By Securing Them) (Thwack) and give me a share/like. Thank you!

Handling A10 PCAP Files Using Automator in MacOS

I’m not a big user of Apple’s Automator tool, but sometimes it’s very useful. For example, A10 Networks load balancers make it pretty easy for administrators to capture packets without having to remember the syntax and appropriate command flags for a tcpdump command in the shell. Downloading the .pcap file is pretty easy too (especially using the web interface), but what gets downloaded is not just a single file; instead, it’s a gzip file containing a tar file which in turn contains (for the hardware I use) seventeen packet capture files. In this post I’ll explain what these files are, why it’s annoying, and how I work around this in MacOS.

Sixteen Candles

If you’re wondering how one packet capture turned into sixteen PCAP files, that’s perfectly reasonable and the answer is simple in its own way. The hardware I use has sixteen CPU cores, fifteen of which are used by default to process traffic, and inbound flows are spread across those cores. Thus when taking a packet capture, the system actually requests each core to dump the flows matching the filter specification. Each core effectively has awareness of both the client and server sides of any connection, so both Continue reading

New Coder: Real World Code Development

It’s one thing to learn the syntax for a programming language, but it’s another to have the ability to think through a problem and break it down into a logical set of tasks which the code can execute.

On the Solarwinds Thwack Geek Speak blog I worked through a real world automation process to see what the steps might be, and how it can sometimes be possible, and even advantageous, to reuse code or hand off a task to another tool. Please do take a trip to Thwack and check out my post, “New Coder: Real World Code Development“.

Please see my Disclosures page for more information about my role as a Solarwinds Ambassador.

If you liked this post, please do click through to the source at New Coder: Real World Code Development and give me a share/like. Thank you!

New Coder: Which language is the right one?

There are so many programming languages to choose from; as a new network or infrastructure coder, what are the best options to consider, and why? What are the differences between all these languages anyway?

On the Solarwinds Thwack Geek Speak blog I explain what interpreted and compiled languages are, what strong and weak typing means, I evaluate some common languages and make some recomendations. Please do take a trip to Thwack and check out my post, “New Coder: Which Language Is The Right One?“.

Please see my Disclosures page for more information about my role as a Solarwinds Ambassador.

If you liked this post, please do click through to the source at New Coder: Which language is the right one? and give me a share/like. Thank you!

So You Want To Code? It’s Only Logical! (Thwack)

How should somebody new to coding get started learning a language, writing code, and maybe even automating something? Where should they begin?

That’s the question I asked on the Solarwinds Thwack Geek Speak blog. In my post I look at what programming really is, and whether it’s going to be something that comes naturally, or will require a very conscious effort. Please do take a trip to Thwack and check out my post, “So you want to code? It’s only logical!“.

Please see my Disclosures page for more information about my role as a Solarwinds Ambassador.

If you liked this post, please do click through to the source at So You Want To Code? It’s Only Logical! (Thwack) and give me a share/like. Thank you!

Dell EMC Ethernet Switching Update

I’m at the Dell EMC World 2017 conference in Las Vegas this week, and I’ve been enjoying catching up on what the network group has been up to. In my previous experience, the legacy Dell Networking products have unfortunately been seen as those things that get thrown in when you buy a rack of servers. In other words, they lacked credibility or worse, the rack would come with another vendor’s switches in them, reinforcing the idea that Dell’s own products weren’t up to the job.

It’s my belief though, that two things in recent years have dramatically changed that perspective. The first is Dell EMC’s OS10, a modular network operating system which by all accounts is actually pretty capable. Previous OS incarnations were of varying quality, as has been the case with many vendor-branded switches, and with the release of OS10, Dell Networks (as it was at the time) put a stake in the ground and showed that they wanted things to be different.

The second element is disaggregation. Dell identified the opportunity to use what was becoming ubiquitous merchant silicon like the Broadcom Trident II chipset to be able to play at the exact same level as everybody Continue reading

No-Hassle Hardware Replacement with DCNM

Continuing my look at Cisco’s Data Center Network Manager (DCNM) software, I had to swap out a faulty spine switch recently, and got a chance to find out whether I could use DCNM to make the RMA process a little less painful than it would normally be.

Using DCNM for RMA

If I had a dream, albeit a rather sad one, it would be to be able to swap out a hardware component in my network quickly and efficiently, and not have to mess around.

DCNM has all the elements to make this feasible. Given a device serial number, I’ve already confirmed that DCNM can automatically deploy the correct firmware version and base configuration when deploying a new fabric, so there’s no reason it couldn’t be used to do the same thing for a replacement switch, but this time deploying the complete, production configuration in one step. Happily I already have the configuration for the spine switch I’m replacing, because DCNM takes frequent backups. I also have the appropriate firmware loaded to DCNM’s software repository because I used it to build the fabric in the first place. So how do I approach the RMA?

Telling DCNM About The New Hardware

Once the replacement switch was received Continue reading

DCNM ISSU: Disaster or Triumph? Let’s Find Out.

If you’ve recovered from the shock of hearing me say something positive about Cisco’s Data Center Network Manager (DCNM) product, then you’ll want to hold on tightly to your underbritches as I tell you that I just used DCNM ISSU to perform disruptive software upgrades on eight of the switches in my ethernet fabric, and–spoiler alert–it was actually a fairly pleasant experience!

DCNM ISSU

DCNM offers management of ISSU (In-Service Software Upgrade). ISSU usually implies some kind of hitless (to the revenue ports) upgrade, historically made possible on the Nexus 7000, for example, by having dual supervisor modules and using Non-Stop Forwarding (NSF) to keep the forwarding plane intact while the supervisors failover. With the single-CPU layer-2 Nexus 5600 switches, however, the data plane can be told to continue forwarding frames while the control plane reboots with new code, allowing for an upgrade to take place without interruption.

Disruptive Upgrades

Unfortunately it’s not always possible to perform a non-disruptive upgrade. The code version I was installing included a fix to the linecard BIOS, so the linecards had to be reloaded as well as the main CPU. In other words, the switch has to be rebooted after the upgrade, and there’s Continue reading

TIP: How To Do MD5 and SHA1 File Checksum Validation

It’s always a good idea to calculate an MD5 or SHA1 file checksum to validate file integrity after download or transfer, especially when dealing with firmware binaries. While most modern systems are smart enough to validate images before attempting an installation, not all are so wise, and I’m sure I’m not the only one to have seen a device bricked (or stuck in ROMMON or a similar bootloader or equivalent) after a bad image was uploaded.

Here’s a quick reference guide to creating file checksums on Macos (OSX), Windows and Linux.

File Checksum Validation

There are various ways to check md5/sha1 checksums depending on your preferred platform. Vendors tend to publish the MD5 or SHA1 checksums (or both) for downloadable files, so it’s silly not to do checksum validation and confirm that the file has downloaded completely and uncorrupted. I try to validate after each time I transfer a file so that I don’t waste time sending a corrupted file on to the next hop. For example:

• Download image file from Cisco’s website
• — View the checksum and compare
• SCP the file to a target jump server
• — View the checksum and compare
• SCP the file to the end device

Continue reading

When IOS XR Licenses Don’t Activate, What Then?

I came across a small but irritating issue with ASR / IOS XR licensing today, and since I found a way to fix it, I’m sharing my results.

Licensing IOS XR on the ASR9k

I have an ASR9006 with two A9K-MOD160-TR linecards on which I need to run VRFs, so I purchased two of the A9K-IVRF-LIC linecard-based VRF licenses. I got the PAK keys from my reseller, and went to Cisco’s licensing portal to fulfill both of them following the usual process with the PID and S/N information taken from admin show license udi. I downloaded the license file and transferred it to an accessible jump server, then from the regular privileged exec mode (rather than the admin exec mode), I used sftp to transfer the file to the router.

Why not use the admin exec to transfer the licenses?

Simple: to transfer the license file within the admin exec means using tftp or ftp:

RP/0/RSP0/CPU0:asr9006-1(admin)#copy ?
  /recurse        Recursively list subdirectories encountered
  WORD            Copy from file
  bootflash:      Copy from bootflash: file system
  disk0:          Copy from disk0: file system
  disk0a:         Copy from disk0a: file system
  disk1:          Copy from disk1: file system
  disk1a:         Copy from disk1a: file system
  disk2:          Copy from disk2: file system
 Continue reading

USB Consoling Myself With Opengear’s ACM7004-5

Have you ever tried using the USB Console port on your network hardware? Me neither, and that’s mainly because the instructions typically begin with Download and install the USB console driver for your operating system, at which point I exhale deeply and get out my USB serial adapter instead. I think Opengear must have heard me sighing because the ACM7004-5 Remote Site Gateway device they’ve sent me to look at has four USB console ports built in.

Opengear ACM7004-5

The compact ACM7004-5 packs more power than its diminutive stature might at first suggest. Taking a look at the back panel reveals a densely-packed set of ports offering a total of four switched GigabitEthernet ports, four serial console ports (RJ-45) and four USB ports:

As with the other small Opengear devices I’ve tested, this model comes with a single rackmount bracket so it can be attached within a rack with relative ease. It’s only about five inches wide, so it’s not too hard to find a free space to locate it. If you aren’t blessed with a rack, there are small rubber feet that can be stuck on the underside. The power port is interesting; I find myself shouting for Continue reading

How Does NetBeez Rate For Troubleshooting?

Continuing from my previous NetBeez post, I’d like to share some more detail on the charting and reporting capabilities of the product, and my experience using NetBeez to troubleshoot some real network issues.

Incidentally, as advertising slogans go, this one is surprisingly effective; I was surprised at how many people do actually approach and say “Ok go on then, tell me about your beez?”

Hands On Operations

I have been able to spend some time digging around the interface in anger, as it were, and seeing whether the NetBeez tools might raise an alert that otherwise wasn’t caught by other systems. To that end, I have one happy story, but also a number of things I found I wanted to be able to do, but couldn’t. These are things I might not have thought about had I not actually been using them for real, rather than just with test data.

Charts

The actual charts are quite nicely put together, although getting there can be a little cumbersome unless linked directly from an alert or something. For example, here is the top of the list of Resources within one of my Target test sets:

If I click on the PING Continue reading

Microservices Gone Wild – Tech Dive Part 4

In this last post of my four-part series on microservices, I’ll look at some of the positive aspects of microservices, and how much simpler they can potentially make things once you overcome the up-front effort required to make them work.

Scalability

When a monolithic app needs to scale, how can that be achieved? Well, for example:

• More RAM (if the app is memory-bound)
• More or Faster CPUs (if the app is CPU-bound)
• More instances of the app (front with a load balancer)

These are all effective ways to scale the application. What if one function within the application could really use a performance boost, even though the others are working just fine? Using a load balancer to distribute work requests can mean that scaling up the ability for a single module to process concurrent requests can be as simple as spinning up a few more containers and sharing the load:

There’s some effort required to allow the main program to issue concurrent calls, but the benefits can be worthwhile. Plus, of course, each of our microservices may be called by other programs, or may call each other as necessary, so there may be more than just one source of activity. Continue reading

Microservices Gone Wild – Tech Dive Part 2

In this post, I’ll outline the program I’ll be using to demonstrate how microservices work. It’s written in go but it’s pretty straightforward. At the end of the series of posts I will upload all of these examples to github as well, in case anybody wants to poke at them.

The Program – Squariply

For demonstration purposes, I’ll be discussing a very simple program that is currently implemented in a monolithic fashion. I’ve called it squariply for reasons that will momentarily become obvious.

Purpose

Squariply accepts two integers on the command line, calculates the product (i.e. multiplies the two numbers), then squares the resulting number before printing the final result out. Mathematically speaking, if the integers provided on the command line are a and b, the output will be equivalent to (a * b) ^ 2.

Monolithic Code

My extremely amateur go code looks like this:

package main

import (
    "fmt"
    "os"
    "strconv"
)

func main() {
    str_a := os.Args[1]
    str_b := os.Args[2]

    int_a, _ := strconv.Atoi(str_a)
    int_b, _ := strconv.Atoi(str_b)

    multiplyResult := int_a * int_b
    squareResult := multiplyResult * multiplyResult

    fmt.Printf("Result is %d\n", squareResult)
}

For the purposes of clarity, Continue reading