Author Archives: networkingnerd
Author Archives: networkingnerd
My son was complaining to me the other day that he missed on question on a multiple choice quiz in his class and he got a low B grade instead of getting a perfect score. When I asked him why he was frustrated he told me, “Because it was easy and I missed it. But I think the question was wrong.” As usual, I pressed him further to explain his reasoning and found out that the question was indeed ambiguous but the answer choices were pretty obviously wrong all over. He asked me why someone would write a test like that. Which is how he got a big lesson on writing test questions.
When you write a multiple choice test question for any reputable exam you are supposed to pick “wrong” answers, known as distractors, that ensure that the candidate doesn’t have a better than 25% chance of guessing the correct answer. You’ve probably seen this before because you took some kind of simple quiz that had answers that were completely wrong to the point of being easy to pick out. Those quizzes are usually designed to be passed with the minimum amount of effort.
I wanted to wait to put out a hot take on the Facebook issues from earlier this week because failures of this magnitude always have details that come out well after the actual excitement is done. A company like Facebook isn’t going to do the kind of in-depth post-mortem that we might like to see but the amount of information coming out from other areas does point to some interesting circumstances causing this situation.
Let me start off the whole thing by reiterating something important: Your network looks absolutely nothing like Facebook. The scale of what goes on there is unimaginable to the normal person. The average person has no conception of what one billion looks like. Likewise, the scale of the networking that goes on at Facebook is beyond the ken of most networking professionals. I’m not saying this to make your network feel inferior. More that I’m trying to help you understand that your network operations resemble those at Facebook in the same way that a model airplane resembles a space shuttle. They’re alike on the surface only.
Facebook has unique challenges that they have to face in their own way. Network automation there isn’t a bonus. It’s Continue reading
Have you tried to order networking gear recently? You’re probably cursing because the lead times on most everything are getting long. It’s not uncommon to see lead times on wireless access points or switch gear reaching 180 days or more. Reports from the Internet say that some people are still waiting to get things they ordered this spring. The prospect of rapid delivery of equipment is fading like the summer sun.
Why are we here? What happened? And can we do anything about it?
The pandemic has obviously had the biggest impact for a number of reasons. When a fabrication facility shuts down it doesn’t just ramp back up. Even when all the workers are healthy and the city where it is located is open for business it takes weeks to bring everything back online to full capacity. Just like any manufacturing facility you can’t just snap your fingers and get back to churning out the widgets.
The pandemic has also strained supply chains around the world. Even if the fabs had stayed open this entire time you’d be looking at a shortage of materials to make the equipment. Global supply chains were running extremely lean Continue reading
I know we talk about the subject of private 5G a lot in the industry but there are more players coming out every day looking to add their voice to the growing supporters of these solutions. And despite the fact that we tend to see 5G and Wi-Fi technologies as ships in the night this discussion isn’t going to go away any time soon. In part it’s because decision makers aren’t quite savvy enough to distinguish between the bands, thinking all wireless communications are pretty much the same.
I think we’re not going to see much overlap between these two technologies. But the reasons why aren’t quite what you might think.
Working from anywhere other than the traditional office is here to stay. Every major Silicon Valley company has looked at the cost benefit analysis and decided to let workers do their thing from where they live. How can I tell it’s permanent? Because they’re reducing salaries for those that choose to stay away from the Bay Area. That carrot is pretty enticing and for the companies to say that it’s not on the table for remote work going forward means they have no incentive to make people Continue reading
This week I tweeted something from a discussion we had during Networking Field Day that summed up my feelings about the state of documentation of application programming interfaces (APIs):
I laughed a bit as I wrote it because I’ve worked in department stores like Walmart in the past and I know the reasons why they tend to move things around. Comparing that to the way that APIs are documented is an interesting exercise in how people think about things like new capabilities and notification of changes.
In case you weren’t aware, everything in your average department store is carefully planned out. The things placed in the main aisles are decided on weeks in advance due to high traffic. The items placed at the ends of the aisles, or endcaps, are placed there to highlight high margin items or things that are popular enough to be sought out by customers. The makeup of the rest of the store is determined by a lot of metrics.
There are a Continue reading
It’s been a hectic week and I realized that I haven’t had a chance to share some of the latest stuff that I’ve been working on outside of Tech Field Day. I’ve been a guest on a couple of recent podcasts that I loved.
I was happy to be a guest on Episode 57 of the Art of Network Engineering podcast. AJ Murray invited me to take part with all the amazing co-hosts. We talked about some fun stuff including my CCIE study attempts, my journey through technology, and my role at Tech Field Day and how it came to be that I went from being a network engineer to an event lead.
The interplay between the hosts and I during the discussion was great. I felt like we probably could have gone another hour if we really wanted to. You should definitely take a listen and learn how I kept getting my butt kicked by the CCIE open-ended questions or what it’s like to be a technical person on a non-technical briefing.
I wanted to take minute to talk about a story I’ve been following that’s had some new developments this week. You may have seen an article talking about a backdoor in Juniper equipment that caused some issues. The issue at hand is complicated at the linked article does a good job of explaining some of the nuance. Here’s the short version:
Earlier this week there was a great tweet from my friends over at Juniper Networks about mistakes we’ve made in networking:
It got some interactions with the community, which is always nice, but it got me to thinking about how we solve problems and learn from our mistakes. I feel that we’ve reached a point where we’re learning from the things we’ve screwed up but we’re not passing it along like we used to.
Part of the reason why I started my blog was to capture ideas that had been floating in my head for a while. Troubleshooting steps or perhaps even ideas that I wanted to make sure I didn’t forget down the line. All of it was important to capture for the sake of posterity. After all, if you didn’t write it down did it even happen?
Along the way I found that the posts that got significant traction on my site were the ones that involved Continue reading
I’ve spent the better part of the last month fighting a transient issue with my home ISP. I thought I had it figure out after a hardware failure at the connection point but it crept back up after I got back from my Philmont trip. I spent a lot of energy upgrading my home equipment firmware and charting the seemingly random timing of the issue. I also called the technical support line and carefully explained what I was seeing and what had been done to work on the problem already.
The responses usually ranged from confused reactions to attempts to reset my cable modem, which never worked. It took several phone calls and lots of repeated explanations before I finally got a different answer from a technician. It turns out there was a known issue with the modem hardware! It’s something they’ve been working on for a few weeks and they’re not entirely sure what the ultimate fix is going to be. So for now I’m going to have to endure the daily resets. But at least I know I’m not going crazy!
Known issues are a way of life in technology. If you’ve worked with any Continue reading
I was saddened to learn last week that one of my former coworkers passed away unexpectedly. Duane Mersman started at the same time I did at United Systems and we both spent most of our time in the engineering area working on projects. We worked together on so many things that I honestly couldn’t keep count of them if I tried. He’s going to be missed by so many people.
Duane was, in many ways, my polar opposite at work. I was the hard-charging young buck that wanted to learn everything there was to know about stuff in about a week and just get my hands dirty trying to break it and learn from my mistakes. If you needed someone to install a phone system next week with zero formal training or learn how iSCSI was supposed to operate based on notes sketched on the back of a cocktail napkin I was your nerd. That meant we could often get things running quickly. It also meant I spent a lot of time trying to figure out why things weren’t working. I left quite a few forehead-shaped dents in data center walls.
Duane was not any of those Continue reading
I spent the past two weeks enjoying the scenic views at the Philmont Scout Ranch with my son and some of his fellow Scouts BSA troop mates. It was very much the kind of vacation that involved a lot of hiking, mountain climbing, and even some inclement weather. We all completely enjoyed ourselves and I learned a lot about hanging bear bags and taking care of blisters. I also learned a lot about leadership by watching the boys in the crew interact with each other.
Leadership styles are nothing new to the people that read my blog. I’ve talked about them at length in the past. One thing I noticed when I was on the trek was how different leadership styles can clash and create friction among teenagers. As adults we tend to gloss over delivery and just accept that people are the way they are. When you’re fourteen or fifteen you haven’t quite taken that lesson to heart yet. That means more pushing against styles that don’t work for you.
We have all worked for or with someone that has a very authoritarian style in the past. The kind of people that say, “Do this right now” Continue reading
In this week’s episode of the Gestalt IT Rundown, I jumped on my soapbox a bit regarding the latest Pegasus exploit. If you’re not familiar with Pegasus you should catch up with the latest news.
Pegasus is a toolkit designed by NSO Group from Israel. It’s designed for counterterrorism investigations. It’s essentially a piece of malware that can be dropped on a mobile phone through a series of unpatched exploits that allows you to create records of text messages, photos, and phone calls and send them to a location for analysis. On the surface it sounds like a tool that could be used to covertly gather intelligence on someone of interest and ensure that they’re known to law enforcement agencies so they can be stopped in the event of some kind of criminal activity.
If that’s where Pegasus stopped, I’d probably not care one way or the other. A tool used by law enforcement to figure out how to stop things that are tough to defend against. But because you’re reading this post you know that’s not where it stopped. Pegasus wasn’t merely a tool developed by intelligence agencies for targeted use. If I had to Continue reading
There was a tweet making the rounds this last week that gave me pause. Max Clark said that we should embrace single points of failure in the network. His post was an impassioned plea to networking rock stars out there to drop redundancy out of their networks and instead embrace these Single Points of Failure (SPoF). The main points Mr. Clark made boil down to a couple of major statements:
I’m sure more networking pros out there are practically bristling at these suggestions. Others may read through the original tweet and think this was a tongue-in-cheek post. Let’s look at the argument logically and understand why this has some merit but is ultimately flawed.
I’m going to tackle the second point first. The idea that you can use cheaper gear and have cold standby equipment just sitting on the shelf is one that I’ve heard of many Continue reading
I’ve worked for a Value Added Reseller (VAR) in the past and it was a good run of my career before I started working at Tech Field Day. The market was already changing eight years ago when I got out of the game. With the advent of the pandemic that’s especially true today. Quite a few of my friends say they’re feeling the pressure from their VAR employer to stretch beyond what they’re accustomed to doing or outright being treated in such a way as to be forced out or leaving on their own. They tell me they can’t quite understand why that’s happening. After some thought on the matter I think I know. Because you represent debt they need to retire.
We don’t start our careers knowing everything we need to know to make it. The industry spends a lot of time talking about careers and skill paths and getting your legs under you. Networking people need to learn Cisco or Juniper or whatever configuration language makes the most sense for them. Wireless people need to learn how to do site surveys and configure access points. Server people need to learn operating systems and hypervisors. We start Continue reading
We’re halfway through 2021 and it’s been going better than last year. Technology seems to be rebounding and we’re seeing companies trying to find ways to get employees to come back into the office. Of course, that is being met head on by the desire to not go back at all and continue to do the job from home that has been done over the past year. Something is going to have to give and I don’t know what that might be.
It’s not a secret that it’s hard to get stuff done. Procrastination is practically a super power for me. I’ve tried so many methods and systems to keep myself on track over the years that I should probably start a review site. Sadly, the battle of my executive function being on constant vacation and the inability to get organized saps a lot of my ability to execute. It’s gotten to the point where I’ve finally realized that I need to start tricking my brain into getting things done.
Any reputable researcher will tell you that dealing with neurodivergent behaviors like ADHD is all about understanding the reasons why you do the things you do. I know what needs to be done. I just don’t want to do it. Worse yet, anything that I can do to avoid working on something is going to capture my attention because I’d rather be doing something unproductive as opposed to something I don’t like. This can manifest itself in strange ways like preferring to do the dishes instead of writing a blog post or mowing the yard instead of practicing a presentation.
It’s taken me a while but I’ve finally come up Continue reading
I read a piece on LifeHacker yesterday that made me shake my head a bit. I’m sure the title SMART Goals Are Overrated was designed to get people to click on it, so from that perspective it succeeded. Wading into the discourse there was an outline of how SMART goals were originally designed for managers to give tasks to employees and how SMART doesn’t fit every goal you might want to set, especially personal aspirational ones. Since I have a lot of experience with using SMART goals both for myself and for others I wanted to give some perspective on why SMART may not be the best way to go for everything but you’re a fool if you don’t at least use it as a measuring tool.
As a recap, SMART is an acronym for the five key things you need to apply to your goal:
I was notified this week that I’m eligible for the 10-year CCIE plaque. Which means that it’s been a decade since I walked out of Cisco’s Building C in San Jose with a new number and a different outlook on my networking career. The cliche is that “so many things have changed” since that day and it’s absolutely accurate because the only constant in life is change.
I think the first thing that makes me think about the passage of time since my certification is the fact that the lab where I took the exam no longer exists. Building C was sold to the company that owns and operates the San Francisco 49ers stadium just down Tasman drive from the old letter buildings. Those real estate locations were much more valuable to the NFL than to Cisco. I can’t even really go and visit my old stomping grounds any more because the buildings were gutted, renovated, and offered to other operations that aren’t from Cisco.
Now, you don’t even go to San Jose or RTP for the lab. Three years ago the labs in the US moved to Richardson, TX. The central aspect of the location Continue reading
By now you’ve seen the news that longtime CEO of Aruba Keerti Melkote is retiring. He’s decided that his 20-year journey has come to a conclusion and he is stepping down into an advisory role until the end of the HPE fiscal year on October 31, 2021. Leaving along with him are CTO Partha Narasimhan and Chief Architect Pradeep Iyer. It’s a big shift in the way that things will be done going forward for Aruba. There are already plenty of hot takes out there about how this is going to be good or bad for Aruba and for HPE depending on which source you want to read. Because I just couldn’t resist I’m going to take a stab at it too.
Keerti is a great person. He’s smart and capable and has always surrounded himself with good people as well. The HPE acquisition honestly couldn’t have gone any better for him and his team. The term “reverse acquisition” gets used a lot and I think this is one of the few positive examples of it. Aruba became the networking division of HPE. They rebuilt the husk that was HP’s campus networking division and expanded it Continue reading
Imagine you’re deep into a massive issue. You’ve been troubleshooting for hours trying to figure out why something isn’t working. You’ve pulled in resources to help and you’re on the line with the TAC to try and get a resolution. You know this has to be related to something recent because you just got notified about it yesterday. You’re working through logs and configuration setting trying to gain insights into what went wrong. That’s when the TAC engineer hits you with with an armor-piecing question:
When did this start happening?
Now you’re sunk. When did you first start seeing it? Was it happening before and no one noticed? Did a tree fall in the forest and no one was around to hear the sound? What is the meaning of life now?
It’s not too hard to imagine the above scenario because we’ve found ourselves in it more times than we can count. We’ve started working on a problem and traced it back to a root cause only to find out that the actual inciting incident goes back even further than that. Maybe the symptoms just took a while to show up. Perhaps someone unknowingly “fixed” the issue with a Continue reading