Murphy the Chaos Manager

I had the opportunity to sit in on a great briefing from Gremlin the other day about chaos engineering. Ken Nalbone (@KenNalbone) has a great review of their software and approach to things here. The more time I spent thinking about chaos engineering and IT, the more I realized that it has more in common with Murphy’s Law that we realize.

Anything That Can Go Wrong

If there’s more than one way to do a job and one of those ways will end in disaster, then somebody will do it that way. – Edward Murphy

 

Anything that can go wrong will go wrong. – Major John Paul Stapp

We live by the adage of Murphy’s Law in IT. Anything that can go wrong will go wrong. And usually it goes wrong at the worst possible time. Database query functions will go wrong when you need them the most. And usually at the height of something like Amazon Prime Day. Data center outages only seem to happen at 4 am on a Sunday during a holiday.

But why do things go wrong like this? Is it because the universe just has it out for IT people? Are we Continue reading

Why Is The CCIE Lab Moving?

Cisco confirmed big CCIE rumor this week that the RTP lab was going to be moved to Richardson, TX.

The language Cisco used is pretty neutral. San Jose and RTP are being shut down as full time lab locations and everyone is moving to Richardson. We knew about this thanks to the detective work of Jeff Fry, who managed to figure this out over a week ago. Now that we know what is happening, why is it coming to pass?

They Don’t Build Them Like They Used To

Real estate is expensive. Anyone that’s ever bought a house will tell you that. Now, imagine that on a commercial scale. Many companies will get the minimum amount of building that they need to get by. Sometimes they’re bursting at the seams before they upgrade to a new facility.

Other companies are big about having lots of area. These are the companies that Continue reading

Presenting To The D-Suite

Do you present to an audience? Odds are good that most of us have had to do it more than once in our life or career. Some of us do it rather often. And there’s no shortage of advice out there about how to present to an audience. A lot of it is aimed at people that are trying to speak to a general audience. Still more of it is designed as a primer on how to speak to executives, often from a sales pitch perspective. But, how do you present to the people that get stuff done? Instead of honing your skills for the C-Suite, let’s look at what it takes to present to the D-Suite.

1. No Problemo

If you’ve listened to a presentation aimed at execs any time recently, such as on Shark Tank or Dragon’s Den, you know all about The Problem. It’s a required part of every introduction. You need to present a huge problem that needs to be solved. You need to discuss why this problem is so important. Once you’ve got every head nodding, that’s when you jump in with your solution. You highlight why you are the only person that can do Continue reading

Clear Skys for IBM and Red Hat

There was a lot of buzz this week when IBM announced they were acquiring Red Hat. A lot has been discussed about this in the past five days, including some coverage that I recorded with the Gestalt IT team on Monday. What I wanted to discuss quickly here is the aspirations that IBM now has for the cloud. Or, more appropriately, what they aren’t going to be doing.

Build You Own Cloud

It’s funny how many cloud providers started springing from the earth as soon as AWS started turning a profit. Microsoft and Google seem to be doing a good job of challenging for the crown. But the next tier down is littered with people trying to make a go of it. VMware with vCloud Air before they sold it. Oracle. Digital Ocean. IBM. And that doesn’t count the number of companies offering a specific function, like storage, and are calling themselves a cloud service provider.

IBM was well positioned to be a contender in the cloud service provider (CSP) market. Except they started the race with a huge disadvantage. IBM was a company that was focused on selling solutions to their customers. Just like Oracle, IBM’s primary customer was Continue reading

How High Can The CCIE Go?

Congratulations to Michael Wong, CCIE #60064! And yes, you’re reading that right. Cisco has certified 30,000 new CCIEs in the last nine years. The next big milestone for CCIE nerds will be 65,536, otherwise known as CCIE 0x10000. How did we get here? And what does this really mean for everyone in the networking industry?

A Short Disclaimer

Before we get started here, a short disclaimer. I am currently on the Cisco CCIE Advisory Board for 2018 and 2019. My opinions here do not reflect those of Cisco, only me. No insider information has been used in the crafting of this post. Any sources are freely available or represent my own opinions.

Ticket To Ride

Why the push for a certified workforce? It really does make sense when you look at it in perspective. More trained people means more people that know how to implement your system properly. More people implementing your systems means more people that will pick that solution over others when they’re offered. And that means more sales. And hopefully also less support time spent by your organization based on the trained people doing the job right in the first place.

You can’t fault people for wanting Continue reading

What Makes a Security Company?

When you think of a “security” company, what comes to mind? Is it a software house making leaps in technology to save us from DDoS attacks or malicious actors? Maybe it’s a company that makes firewalls or intrusion detection systems that stand guard to keep the bad people out of places they aren’t supposed to be. Or maybe it’s something else entirely.

Tradition Since Twenty Minutes Ago

What comes to mind when you think of a traditional security company? What kinds of technology do they make? Maybe it’s a firewall. Maybe it’s an anti-virus program. Or maybe it’s something else that you’ve never thought of.

Is a lock company like Schlage a security company? Perhaps they aren’t a “traditional” IT security company but you can guarantee that you’ve seen their products protecting data centers and IDF closets. What about a Halon system manufacturer? They may not be a first thought for security, but you can believe that a fire in your data center is going cause security issues. Also, I remember that I learned more about Halon and wet/dry pipe fire sprinkler systems from my CISSP study than anywhere else.

The problem with classifying security companies as “traditional” or “non-traditional” Continue reading

Security Is Bananas

I think we’ve reached peak bombshell report discussion at this point. It all started this time around with the big news from Bloomberg that China implanted spy chips into SuperMicro boards in the assembly phase. Then came the denials from Amazon and Apple and event SuperMicro. Then started the armchair quarterbacking from everyone, including TechCrunch. From bad sources to lack of technical details all the way up to the crazy conspiracy theories that someone at Bloomberg was trying to goose their quarterly bonus with a short sale or that the Chinese planted the story to cover up future hacking incidents, I think we’ve covered the entire gamut of everything that the SuperMicro story could and couldn’t be.

So what more could there be to say about this? Well, nothing about SuperMicro specifically. But there’s a lot to say about the fact that we were both oblivious and completely unsurprised about an attack on the supply chain of a manufacturer. While the story moved the stock markets pretty effectively for a few days, none of the security people I’ve talked to were shocked by the idea of someone with the power of a nation state inserting themselves into the supply chain Continue reading

The Why of Security

Security is a field of questions. We find ourselves asking
all kinds of them all the time. Who is trying to get into my network? What are
they using? How can I stop them? But I feel that the most important question is
the one we ask the least. And the answer to that question provides the
motivation to really fix problems as well as conserving the effort necessary to
do so.

The Why’s Old Sage

If you’re someone with kids, imagine a conversation like
this one for a moment:

Your child runs into the kitchen with a lit torch in their hands and asks “Hey, where do we keep the gasoline?”

Now, some of you are probably laughing. And some of you are
probably imagining all kinds of crazy going on here. But I’m sure that most of
you probably started asking a lot of questions like:

• – Why does my child have a lit torch in the house?
• – Why do they want to know where the gasoline is?
• – Why do they want to put these two things together?
• – Why am I not stopping this right now?

Usually, the rest of the Five Ws follow Continue reading

Outing Your Outages

How are you supposed to handle outages? What happens when everything around you goes upside down in an instant? How much communication is “too much”? Or “not enough”? And is all of this written down now instead of being figured out when the world is on fire?

Team Players

You might have noticed this week that Webex Teams spent most of the week down. Hard. Well, you might have noticed if you used Microsoft Teams, Slack, or any other messaging service that wasn’t offline. Webex Teams went offline about 8:00pm EDT Monday night. At first, most people just thought it was a momentary outage and things would be back up. However, as the hours wore on and Cisco started updating the incident page with more info it soon became apparent that Teams was not coming back soon. In fact, it took until Thursday for most of the functions to be restored from whatever knocked them offline.

What happened? Well, most companies don’t like to admit what exactly went wrong. For every CloudFlare or provider that has full disclosures on their site of outages, there are many more companies that will eventually release a statement with the least amount of technical Continue reading

Writing Is Hard

Writing isn’t the easiest thing in the world to do. There are a lot of times that people sit down to pour out their thoughts onto virtual paper and nothing happens. Or they spend hours and hours researching a topic only to put something together that falls apart because of assumptions about a key point that aren’t true.

The world is becoming more and more enamored with other forms of media. We like listening to podcasts instead of reading. We prefer short videos instead of long articles. Visual aids beat a wall of text any day. Even though each of these content types has a script it still feels better having a conversation. Informal chat beats formal prose every day.

Written Wringers

I got into blogging because my typing fingers are way more eloquent than the thoughts running through my brain. I had tons of ideas that I needed to put down on paper and the best way to do that was to build a simple blog and get to it. It’s been eight years of posting and I still feel like I have a ton to say. But it’s not easy to make the words flow all the time.

Continue reading

A Matter of Perspective

Have you ever taken the opportunity to think about something from a completely different perspective? Or seen someone experience something you have seen through new eyes? It’s not easy for sure. But it is a very enlightening experience that can help you understand why people sometimes see things entirely differently even when presented with the same information.

Overcast Networking

The first time I saw this in action was with Aviatrix Systems. I first got to see them at Cisco Live 2018. They did a 1-hour presentation about their solution and gave everyone an overview of what it could do. For the networking people in the room it was pretty straightforward. Aviatrix did a lot of the things that networking should do. It was just in the cloud instead of in a data center. It’s not that Aviatrix wasn’t impressive. It’s the networking people have a very clear idea of what a networking platform should do.

Fast forward two months to Cloud Field Day 4. Aviatrix presents again, only this time to a group of cloud professionals. The message was a little more refined from their first presentation. They included some different topics to appeal more to a cloud audience, such Continue reading

A Review of Ubiquiti Wireless

About six months ago, I got fed up with my Meraki MR34 APs. They ran just fine, but they needed attention. They needed licenses. They needed me to pay for a dashboard I used rarely but yet had to keep up yearly. And that dashboard had most of the “advanced” features hidden away under lock and key. I was beyond frustrated. I happen to be at the Wireless LAN Professionals Conference (WLPC) and ran into Darrell DeRosia (@Darrell_DeRosia) about my plight. His response was pretty simple:

“Dude, you should check out Ubiquiti.”

Now, my understanding of Ubiquiti up to that point was practically nothing. I knew they sold into the SMB side of the market. They weren’t “enterprise grade” like Cisco or Aruba or even Meraki. I didn’t even know the specs on their APs. After a conversation with Darrell and some of the fine folks at Ubiquiti, I replaced my MR34s with a UniFI AP-AC-HD and an AP-AC-InWall-Pro. I also installed one of their UniFi Security Gateways to upgrade my existing Linksys connection device.

You may recall my issue with redundancy and my cable modem battery when I tried to install the UniFi Security Gateway for Continue reading

The Long And Winding Network Road

How do you see your network? Odds are good it looks like a big collection of devices and protocols that you use to connect everything. It doesn’t matter what those devices are. They’re just another source of packets that you have to deal with. Sometimes those devices are more needy than others. Maybe it’s a phone server that needs QoS. Or a storage device that needs a dedicated transport to guarantee that nothing is lost.

But what does the network look like to those developers?

Work Is A Highway

When is the last time you thought about how the network looks to people? Here’s a thought exercise for you:

Think about a highway. Think about all the engineering that goes into building a highway. How many companies are involved in building it. How many resources are required. Now, think of that every time you want to go to the store.

It’s a bit overwhelming. There are dozens, if not hundreds, of companies that are dedicated to building highways and other surface streets. Perhaps they are architects or construction crews or even just maintenance workers. But all of them have a function. All for the sake of letting us drive on Continue reading

Fixing My Twitter

It’s no surprise that Twitter’s developers are messing around with the platform. Again. This time, it’s the implementation of changes announced back in May. Twitter is finally cutting off access to their API that third party clients have been using for the past few years. They’re forcing these clients to use their new API structure for things like notifications and removing support for streaming. This new API structure also has a hefty price tag. For 250 users it’s almost $3,000/month.

You can imagine the feedback that Twitter has gotten. Users of popular programs like Tweetbot and Twitterific were forced to degrade client functionality thanks to the implementation of these changes. Twitter power users have been voicing their opinions with the hashtag #BreakingMyTwitter. I’m among the people that are frustrated that Twitter is chasing the dollar instead of the users.

Breaking The Bank

Twitter is beholden to a harsh mistress. Wall Street doesn’t care about user interface or API accessibility. They care about money. They care are results and profit. And if you aren’t turning a profit you’re a loser that people will abandon. So Twitter has to make money somehow. And how is Twitter supposed to make money in today’s Continue reading

The Cargo Cult of Google Tools

You should definitely watch this amazing video from Ben Sigelman of LightStep that was recorded at Cloud Field Day 4. The good stuff comes right up front.

In less than five minutes, he takes apart crazy notions that we have in the world today. I like the observation that you can’t build a system more than three or four orders of magnitude. Yes, you really shouldn’t be using Hadoop for simple things. And Machine Learning is not a magic wand that fixes every problem.

However, my favorite thing was the quick mention of how emulating Google for the sake of using their tools for every solution is folly. Ben should know, because he is an ex-Googler. I think I can sum up this entire discussion in less than a minute of his talk here:

Google’s solutions were built for scale that basically doesn’t exist outside of a maybe a handful of companies with a trillion dollar valuation. It’s foolish to assume that their solutions are better. They’re just more scalable. But they are actually very feature-poor. There’s a tradeoff there. We should not be imitating what Google did without thinking about why they did it. Sometimes the “whys” will apply Continue reading

Are We Seeing SD-WAN Washing?

You may have seen a tweet from me last week referencing a news story that Fortinet was now in the SD-WAN market:

It came as a shock to me because Fortinet wasn’t even on my radar as an SD-WAN vendor. I knew they were doing brisk business in the firewall and security space, but SD-WAN? What does it really mean?

SD Boxes

Fortinet’s claim to be a player in the SD-WAN space brings the number of vendors doing SD-WAN to well over 50. That’s a lot of players. But how did the come out of left field to land a deal rumored to be over a million dollars for a space that they weren’t even really playing in six months ago?

Fortinet makes edge firewalls. They make decent edge firewalls. When I used to work for a VAR we used them quite a bit. We even used their smaller units as remote appliances to allow us to connect to remote networks and do managed maintenance services. At no time during that whole engagement Continue reading

Cisco and the Two-Factor Two-Step

In case you missed the news, Cisco announced yesterday that they are buying Duo Security. This is a great move on Cisco’s part. They need to beef up their security portfolio to compete against not only Palo Alto Networks but also against all the up-and-coming startups that are trying to solve problems that are largely being ignored by large enterprise security vendors. But how does an authentication vendor help Cisco?

Who Are You?

The world relies on passwords to run. Banks, email, and even your mobile device has some kind of passcode. We memorize them, write them down, or sometimes just use a password manager (like 1Password) to keep them safe. But passwords can be guessed. Trivial passwords are especially vulnerable. And when you factor in things like rainbow tables, it gets even scarier.

The most secure systems require you to have some additional form of authentication. You may have heard this termed as Two Factor Authentication (2FA). 2FA makes sure that no one is just going to be able to guess your password. The most commonly accepted forms of multi-factor authentication are:

• Something You Know – Password, PIN, etc
• Something You Have – Credit Card, Auth token, Continue reading

It’s About Time and Project Management

I stumbled across a Reddit thread today from /u/Magician_Hiker that posed a question I’ve always found fascinating. When we work on projects, it always seems like there is a disconnect between the project management team and the engineering team doing the work. The statement posted at the top of this thread is as follows:

Project Managers only plan for when things go right.

Engineers always plan for when things go wrong.

How did we get here? And can anything be done about it?

Projecting Management

I’ve had a turn or two at project management. I got my Project+ many years back, and even more years before that I had to learn all about project management in college. The science behind project management is storied and deep. The idea of having someone assigned to keep things running on task and making sure all the little details get taken care of is a huge boon as the size of projects grow.

As an engineer, can you imagine trying to juggle three different installations across 5 different sites that all need to be coordinated together? Can you think about the effort needed to make sure that everything works together and is done on Continue reading

Friday Musings on Network Analytics

I’ve been at Networking Field Day this week, and as always the conversations have been great and focused around a variety of networking topics. One that keeps jumping out at me is network analytics. There’s a few things that have come up that were especially interesting to me:

• Don’t ask yourself if networking monitoring is not worth your time. Odds are good you’re already monitoring stuff in your network and you don’t even realize it. Many networking vendors enable basic analytics for troubleshooting purposes. You need to figure out how to build that into a bigger part of your workflows.
• Remember that analytics can integrate with platforms you’re already using. If you’re using ServiceNow you can integrate everything into it. No better way to learn how analytics can help you than to setup some kind of ticket generation for down networks. And, if that automation causes you to get overloaded with link flaps you’ll have even more motivation to figure out why your provider can’t keep things running.
• Don’t discount open source tools. The world has come a long way since MRTG and Cacti. In fact, a lot of the flagship analytics platforms are built with open source tools as Continue reading

Independence, Impartiality, and Perspective

In case you haven’t noticed recently, there are a lot of people that have been going to work for vendors and manufacturers of computer equipment. Microsoft has scored more than a few of them, along with Cohesity, Rubrik, and many others. This is something that I see frequently from my position at Tech Field Day. We typically hear the rumblings of a person looking to move on to a different position early on because we talk to a great number of companies. We also hear about it because it represents a big shift for those who are potential delegates for our events. Because going to a vendor means loss of their independence. But what does that really mean?

Undeclaring Independence

When people go to work for a manufacturer of a computing product, the necessarily lose their independence. But that’s not the only case where that happens. You can also not be truly independent if you work for reseller. If your company specializes in Cisco and EMC, are you truly independent when discussion Juniper and NetApp? If you make your money by selling one group of products you’re going to be unconsciously biased toward them. If you’ve been burned or had Continue reading