Problem Replication or Why Do We Need to Break It Again?

There was a tweet the other day that posited that we don’t “need” to replicate problems to solve them. Ultimately the reason for the tweet was that a helpdesk refused to troubleshoot the problem until they could replicate the issue and the tweeter thought that wasn’t right. It made me start thinking about why troubleshooters are so bent on trying to make something happen again before we actually start trying to fix an issue.

The Definition of Insanity

Everyone by now has heard that the definition of insanity is doing the same thing over and over again and expecting a different result. While funny and a bit oversimplified the reality of troubleshooting is that you are trying to make it do something different with the same inputs. Because if you can make it do the same thing over and over again you’re closer to the root cause of the issue.

Root cause is the key to problem solving. If you don’t fix what’s actually wrong you are only dealing with symptoms and not issues. However, you can’t know what’s actually wrong until you can make it happen more than once. That’s because you have to narrow the actual issue down Continue reading

Making It Work in 2023

We’re back to the first of the year once again. January 1, 2023 is a Sunday which feels somewhat subdued. That stands in contrast to the rest of the year that felt like a rollercoaster always one heartbeat away from careening out of control. As is the tradition, I’ll look at the things I wanted to spend more time working on in 2022:

• More Analytical Content: I have to honestly give myself a no on this one, at least from a technical perspective. I did spend some time making analytical content for my Tomversations series. However, the real difference in analytical content came from my posts about leadership and more “soft skill” focused ideas. I’ve gotten more comments about those posts than anything in 2022 and I couldn’t be more proud.
• Saying No to More Things: This is the part where I would insert an animated GIF of someone laughing manically. While I did make strides in telling people that I have way too much going on to take care of one extra thing the reality is that I took on more things that I probably should have. That’s something that I definitely do need to change but the real Continue reading

Testing Your Weakest Links as a Chain

You may have heard in the news this week that there was a big issue with Southwest Airlines this holiday season. The issues are myriad and this is going to make for some great case studies for students in the future. However, one thing I wanted to touch on briefly in this whole debacle was the issue of a cascade failure.

The short version is that a weather disruption in the flight schedule became a much bigger problem when the process for rescheduling the flight crews was overwhelmed. Turns out that even after the big computer system upgrades and all the IT work that has gone into putting together a modern airfare booking system that one process was still very manual. The air crew rescheduling department was relatively small in nature and couldn’t keep up with the demands placed on it by the disruptions. It got to the point where Southwest had to reduce their number of flights in order to get the system back to normal.

Worst Case Scenario

I’m not an expert at airline scheduling but I have spent a lot of time planning for disaster recovery. One of the things that we focus on more than anything Continue reading

The Power of Complaining Properly

Recently I’ve started listening to a new podcast all about the brain and behaviors called Hidden Brain. It’s got a lot great content and you should totally check it out. One of the latest episodes deals with complaining and how it can make us less productive and more likely to repeat patterns or shut people out.

Complaining is as old as language. I’m sure as soon as the first person to create communications around spoken words was able to teach another person one of the first things they did was complain about the weather or something they hated. Our mind is built to express itself about things we don’t like, such as bad drivers or silly behaviors at work.

The episode explores the ways that our brain can trap us in cycles of complaining simply for the sake of complaining. It also discusses how we should try to spend more time trying to be productive in how we address complaints. I’ve experienced this a lot in IT as well as in my career after being directly involved in IT and there’s a lot of merit in changing the way we complain about things.

Airing Grievances

Complaining without a suggested solution Continue reading

The Power of Continuing Education on Certifications

I’m about six months away from recertifying my CCIE and even though I could just go Emeritus now I’m working on completing some continuing education at the end of the year to push it out another three years. I am once again very thankful that Cisco has this as an option instead of taking a test over and over again as the only option to renew my certifications.

As I embark on another journey to keep myself current in the networking community, I realize that the flexibility that education credits offer is more important that just passing a test or learning a new skill. Employers should also be thrilled that knowledge workers have the ability to work on other skills and be recognized for them. Because there are two different paths that this can lead to.

To Be The Best

One of the things that most professionals recognize with continuing education is that you can leverage your skills to race through things. If you’re already an expert at something like BGP or spanning tree why not take courses to improve the depth of your knowledge? This is part of the reason why there are a number of double CCIEs that Continue reading

ChatGPT and Creating For Yourself

I’m sure you’ve been inundated by posts about ChatGPT over the past couple of weeks. If you managed to avoid it the short version is that there is a new model from OpenAI that can write articles, create poetry, and basically answer your homework. Lots of people are testing it out for things as mundane as writing Amazon reviews or creating configurations for routers.

It’s not a universal hit though. Stack Overflow banned ChatGPT code answers because they’re almost always wrong. My own limited tests show that it can create a lot of words from a prompt that seem to sound correct but feel hollow. Many others have accused the algorithm of scraping content from others on the Internet and sampling it into answers to make it sound accurate but not the best answer to the question.

Are we ready for AI to do our writing for us? Is the era of the novelist or technical writer finished? Should we just hang up our keyboards and call it a day?

Byte-Sized Content

When I was deciding what I wanted to do with my life after college I took the GMAT to see if I could get into grad school for Continue reading

DPUs Could Change The Network Forever

You wouldn’t think that AWS re:Invent would be a big week for networking, would you? Most of the announcements are focused on everything related to the data center but teasing out the networking specific pieces isn’t as easy. That’s why I found mention of a new-ish protocol in an unrelated article to be fascinating.

In this Register piece about CPUs there’s a mention of the Nitro DPU. More importantly there’s also a reference to something that Amazon has apparently been working on for the last couple of years. It turns out that the world’s largest online bookstore and data center company is looking to get rid of TCP.

Rebuilding Transport

The new protocol was developed in 2020. Referred to as Scalable Reliable Datagram (SRD), it was build to solve specific challenges Amazon was seeing related to performance in their cloud. Amazon decided that TCP had bigger issues for them that they needed to address.

The first was that dropped packets required retransmission. In an environment like the Internet that makes sense. You want to get the data you lost. However, when TCP was developed fifty years ago the amount of data that was lost in transit was tiny compared to Continue reading

Time to Talk

It’s a holiday week here in the US so most people are working lighter days or just taking the whole week off. They’re looking forward to spending time with family and friends. Perhaps they’re already plotting their best strategy for shopping during Black Friday and snagging a new TV or watch. Whatever the case may be there’s lots things going on all over.

One thing that I feel needs to happen is conversation. Not just the kind of idle conversation that we make when we don’t know what to talk about. I also don’t mean the kinds of deep conversations that we need to prepare ourselves to have. I’m talking about the ones where we learn. The ones we have with friends and family where we pick up tidbits of stories and preserve them for the future.

It sounds rather morbid but these conversations aren’t going to be available forever. Our older loved ones are getting older every year. Time marches on and we never know when that time I going to run out. I have several friends that have lost loved ones this year and still others that have realized the time is growing shorter. Mortality is something that Continue reading

Play To Your Team Strengths

This past weekend I went to a training course for an event that I’m participating in next year. One of the quotes that came up during the course was about picking the team that will help you during the event. The quote sounded something like this:

Get the right people on the right bus in the right seats and figure out where you want to go.

Sounds simple, right? Right people, right bus, right seats. Not everyone is going to be a good fit for your team and even if they are they may not be in the right position to do their best work. But how do you know what they’re good at?

Not-So-Well Rounded

Last night, I listened to this excellent Art of Network Engineering episode. The guest was a friend of mine in the industry, Mike Bushong (@MBushong). He’s a very talented person and he knows how to lead people. He’s one of the people that would love to work for given the opportunity. He’s also very astute and he has learned a lot of lessons about enabling people on a team.

One of the things he discussed in the episode was about people’s strengths. Continue reading

Continuity is Not Recovery

It was a long weekend for me but it wasn’t quite as long as it could have been. The school district my son attends is in the middle of a ransomware attack. I got an email from them on Friday afternoon telling us to make sure that any district-owned assets are powered off until further notice to keep our home networks from being compromised. That’s pretty sound advice so we did it immediately.

I know that the folks working on the problem spent the whole weekend trying to clean it up and make sure there isn’t any chance of getting reinfected. However, I also wondered how that would impact school this week. The growing amount of coursework that happens online or is delivered via computer is large enough that going from that to a full stop of no devices is probably jarring. That got me to thinking once more about the difference between continuity and recovery

Keeping The Lights On

We talk about disaster recovery a lot. Backups of any kind are designed to get back what was lost. Whether it’s a natural disaster or a security incident you want to be able to recover things back to the way Continue reading

Wi-Fi 6E Growing Pains For Apple

You may have seen that the new iPad Pro has Wi-Fi 6E support. That caused a lot of my wireless friends to jump out and order one, as I expected. As I previously mentioned, 2023 is going to be a big year for Wi-Fi 6E. I was wrong about the 6E radio on the new iPhone but given the direction that Apple is going with the iPad Pro and probably the MacBook as well we’re in for a lot of fun. Why? Because Apple is changing their stance on how to configure 6GHz networks.

An SSID By Any Other Name

If you’ve ever set up wireless networks before you know there are some different suggestions about how to configure the SSIDs with multiple bands. One school of thought says that you need to combine both 2.4GHz and 5GHz in the same SSID and let the device figure out which one is the best to use. This is the way that I have mine set up at home.

However, if you do a quick Google search you’ll find a lot of other wisdom that suggests creating two different SSIDs that only work on a single band. The thought process Continue reading

Cleaning Out The Cruft

I spent the weekend doing something I really should have done a long time ago. I went through my piles of technology that I was going to get around to using one day and finally got rid of anything I didn’t recognize. Old access points, old networking gear, and even older widgets that went to devices that I don’t even remember owning.

Do you have one of these piles? Boxes? Corners of your office or cave? The odds are good there’s a pile of stuff that you keep thinking you’re eventually going to get around to doing something with some day. Except some day hasn’t come yet. So maybe it’s time to get rid of that pile. Trust me you’re going to feel better for getting rid of that stuff.

What to do with it? It needs to be properly recycled so don’t just toss it in the trash can. Anything with electric circuits needs to be properly disposed of so look for an electronics recycling facility. Yes, there are stories that electronics recycling isn’t all it’s cracked up to be but it’s better than polluting with e-waste everywhere.

Consider donating the devices to a trade school or other maker Continue reading

Hedgehog – The Network OS Distro?

You’ve probably seen by now that there’s a new entrant into the market for network operating systems. Hedgehog came out of stealth mode this week to fanfare from the networking community. If you read through the website you might question why I labeled them as a network operating system. While they aren’t technically the OS I think it’s more important to look at them as an OS distribution.

Cacophony of Choice

Hedgehog starts from a very simple premise. Cloud networking is where we’re all headed. Whether or not you’re running entirely on-premises, fully in the public cloud, or in some kind of super-multi-hybrid cloud offering you’re all chasing the same thing. You want a stable system that acts as a force multiplier for your operations teams to reduce deployment times for users to get their builds done. It’s been said before but the idea of cloud is to get IT out of the way of the business.

Streamlining processes means automating a lot of the things that were formerly done by people. That means building repeatable and consistent tools to make that happen. If anyone has ever worked on AWS or Google Cloud you have lots of access to that Continue reading

Why Do We Accept Bad Wireless Clients?

We recorded a fun roundtable discussion last week during Mobility Field Day that talked about the challenges that wireless architects face in their daily lives. It’s about an hour but it’s packed with great discussions about hard things we deal with:

One of the surprises for me is that all the conversations came back to how terrible wireless clients can be. The discussion kept coming back to how hard it is to find quality clients and how we adjust our expectations for the bad ones.

Driven to Madness

Did you know that 70% of Windows crashes are caused by third-party drivers? That’s Microsoft’s own research saying it. That doesn’t mean that Windows is any better or more stable with their OS design compared to Linux or MacOS. However, I’ve fiddled with drivers on Linux and I can tell you how horrible that experience can be¹. Windows is quite tolerant of hardware that wouldn’t work anywhere else. As long as the manufacturer provides a driver you’re going to get something that works most of the time.

Apply that logic to a wireless networking card. You can buy just about anything and install it on your system and it will mostly Continue reading

Monday Mobility Quick Thoughts

I’m getting ready for Mobility Field Day 8 later this week and there’s been a lot of effort making sure we’re ready to go. That means I’ve spent lots of time thinking about event planning instead of writing. So I wanted to share some quick thoughts with you ahead of this week as well as WLPC Europe next week.

• I remain convinced than half of the objections that are raised by oversight organizations when it comes to adopting new technology come from the fact they got caught flat-footed and weren’t ready for it to be popular. Whether it’s the Wi-Fi 6E safety issue or the report earlier this year from the FAA about 5G and airports it just seems like organizations spend less time doing actual investigation and more time writing press releases about how they are ready to figure it all out yet.
• I also remain cautiously optimistic that the new Apple devices rumored to be coming out later this year, namely the iPad Pro and MacBook Pro with M2 chips, will have Wi-Fi 6E support. Yes, the iPhone didn’t. It’s also a smaller device with less room to add new hardware. The iPad and MacBook have historically gotten Continue reading

Intelligence and Wisdom

I spent the last week at the Philmont Leadership Challenge in beautiful Cimarron, NM. I had the chance to learn a bit more about servant leadership and work on my outdoor skills a little. I also had some time to reflect on an interesting question posed to me by one of the members of my crew.

He asked me, “You seem wise. How did you get so wise?” This caught me flat-flooted for a moment because I’d never really considered myself to be a very wise person. Experienced perhaps but not wise like Yoda or Gandalf. So I answered him as I thought more about it.

Intelligence is knowing what to do. Wisdom is knowing what not to do.

The more I thought about that quote the more I realized the importance of the distinction.

Basic Botany

There’s another saying that people tweeted back at me when I shared the above quote. It’s used in the context of describing Intelligence and Wisdom for Dungeons and Dragons roleplaying:

Intelligence is knowing that a tomato is a fruit. Wisdom is not putting tomatoes in a fruit salad.

It’s silly and funny but it gets right to the point and is a Continue reading

Redundancy Is Not Resiliency

Most people carry a spare tire in their car. It’s there in case you get a flat and need to change the tire before you can be on your way again. In my old VAR job I drove a lot away from home and to the middle of nowhere so I didn’t want to rely on roadside assistance. Instead I just grabbed the extra tire out of the back if I needed it and went on my way. However, the process wasn’t entirely hitless. Even the pit crew for a racing team needs time to change tires. I could probably get it done in 20 minutes with appropriate cursing but those were 20 minutes that I wasn’t doing anything else beyond fixing a tire.

Spare tires are redundant. You have an extra thing to replace something that isn’t working. IT operations teams are familiar with redundant systems. Maybe you have a cold spare on the shelf for a switch that might go down. You might have a cold or warm data center location for a disaster. You could even have redundant devices in your enterprise to help you get back in to your equipment if something causes it to go Continue reading

Brand Protection

I woke up at 5am this morning to order a new iPhone. I did this because I wanted the new camera upgrades along with some other nice-to-haves. Why did I get an iPhone and not a new Samsung? Why didn’t I look at any of the other phones on the market? It’s because I am a loyal Apple customer at this point. Does that mean I think the iPhone is perfect? Far from it! But I will choose it in spite of the flaws because I know it has room to be better.

That whole story is repeated time and again in technology. People find themselves drawn to particular companies or brands. They pick a new phone or computer or car based on their familiarity with the way they work or the design choices that are made. But does that mean they have to be loyal to that company no matter what?

Agree to Disagree

One of the things that I feel is absolutely paramount to being a trusted advisor in the technology space is the ability to be critical of a product or brand. If you look at a lot of the ambassador or influencer program agreements you’ll see Continue reading

When Were You Last a Beginner?

In a couple of weeks I’m taking the opportunity to broaden my leadership horizons by attending the BSA leadership course known as Philmont Leadership Challenge. It’s a course that builds on a lot of the things that I’ve been learning and teaching for the past five years. It’s designed to be a sort of capstone for servant leadership and learning how to inspire others. I’m excited to be a part of it in large part because I get to participate for a change.

Being a member of the staff for my local council Wood Badge courses has given me a great opportunity to learn the material inside and out. I love being able to teach and see others grow into leaders. It’s also inspired me to share some of those lessons here to help others in the IT community that might not have the chance to attend a course like that. However the past 3 years have also shown me the value of being a beginner at something from time to time.

Square One

Everyone is new at something. No one is born knowing every piece of information they’ll need to know for their entire lives. We learn language and Continue reading

Certification Comfort Food

I’m a big fan of comfort food. Maybe more than I should be. The idea of something simple and tasty just hits the right spot a lot of time, especially when I’m stressed or don’t have time to do something more involved. I know I really need to be better about cooking but you can’t beat a quick meal that uses something simple and gets the job done, right?

Now, before you ask yourself what I’m on about this week, I want you to think about that analogy in terms of certifications and learning. When we’re starting out in the industry or we’re learning a new skill we have to pick up basic ideas. The more advanced or radical the technology the more we need the kinds of explanations that make the concepts simple to understand. We need the equivalent of learning comfort food. Simple, digestible, and easy to prepare.

Climbing the Ladder

As our skills improve we have the choice to continue on and develop our capabilities to greater depths. Perhaps we want to learn everything there is to know about BGP and policies. We could even parlay that networking knowledge into new adjacencies that build on our skill Continue reading