Adventures in Upgrading Netbox

I’ve been using Netbox for a while now, and, frankly, I can’t live without it. If you’ve never heard of it, it’s a Source of Truth for your network automation tasks started by Jeremy Stretch. I use it to document my networks (hardware inventory, subnets, physical connections, etc.), which provides my automation tasks a place to pull and push all sorts of information like management IPs, rack locations, power connections, network drops…the list goes on. In better words, your automation tools can ask Netbox what the state of your network is, and send it an update if that tool discovers something different. There are plenty of better places to discuss the benefits of a Souce of Truth, so just do the Googles for it.

My production instance is running Netbox 2.7.6, which is very old. The latest version of Netbox as of today is 3.3.7, so that should tell you how far behind we are. I’ve had mine running for over two years, and, in the meantime, the world has moved forward. If I update the server it’s running on (Ubuntu 20.04), then Netbox breaks. Yes, it’s so far behind Continue reading

BGP Configuration on FortiOS

I’ve never done a post on Forti-anything, but I’m really appreciating the products Fortinet is putting out lately. They’re transitioning from “run your SMB off of our stuff” to “actually, we’re pretty good for larger companies”, so their GUI lacks features to keep the SMB from blowing stuff up, The advanced features are there in the CLI, and I wanted to use it to show that difference between the GUI and the real config.

Let’s review some of the basic configuration elements of BGP first. You need an autonomous system (AS) number and a router ID for your side. You also need the AS number of the remote system. You need the IP address on their side (usually the interface facing you). That looks something like this. We’re going to be ‘Fortigate 1’ for this exercise.

With just this information, we can turn up a BGP neighbor that does absolutely nothing. To actually send some routes, you need to tell BGP what to send. We’ll keep this simple and add just connected networks. Adding to the diagram, we get this.

Now we have something of value (though choosing BGP over OSPF or RIP for this little scenario is pretty horrible). Continue reading

Modular Network OS with Nokia SR Linux

I was lucky enough to have been invited to attend Network Field Day 29 this past September in San Jose, CA. This event brings independent thought leaders together with a number of IT product vendors to share information and opinions. We saw presentations from a pretty full range of vendors — from the chips to observability. It was a great event and worth a few hours to check out the videos. Thanks to Gestalt IT for getting me involved.

Nokia was among the list of high-end companies we saw. No, they don’t make phones any more (though they do market their name to products), but they are still in the full-power, throw-packets-as-fast-as-you-can markets for hyperscalers and such. If you’re old like I am, you might remember Nokia as the hardware that Checkpoint ran on for a while. My brain has done its best to filter memories of those devices, but, luckily, the Nokia team is doing some much better things these days.

SR Linux was one of the focuses and the big hitter for me. This is a modernization of the SR OS that was introduced 20 years or so ago, and gets us into a “world of streaming telemetry. Continue reading

Nyansa Voyance at NFD18

Disclaimer : I was lucky enough to have been invited to attend Network Field Day 18 this past July in Silicon Valley. This event brings independent thought leaders to a number of IT product vendors to share information and opinions. I was not paid to attend any of these presentations, but Tech Field Day did provide travel, room, and meals for the event. There is no expectation of providing any blog content, and any posts that come from the event are from my own interest. I’m writing about Nyansa strictly from demonstrations of the product.  I’ve not installed it on my own network and have no experience running it.

Anyway,…on with the show!

Nyansa (pronounced nee-ahn’-sa) is focused on user expereince on the access network. Their product, Voyance, analyzes data from a list of sources to provide a view into what client machines are seeing. This is more than just logs from the machine itself. We’re talkin about taking behaviors on the wireless, access network, WAN, and Internet, and correlating those data points to predict user experience issues and recommend actions to remediate those problems. As we discussed in the presentation, there are products that do each of Continue reading

Automating My World

I’ve told this story 984828934 time in the past year, but bear with me.  We got a new director-type last year, and he has challenged all of us to do things differently.  As in everything.  Anything that we’re doing today should be done differently by next year.  This isn’t saying that we’re doing things wrong.  This is just a challenge mix things up, integrate new tools, and get rid of the noise.  Our group has responded big-time, and we’re now doing most of our day-to-day tasks with a tool of some kind.  A couple weeks ago, I realized that I did a whole day’s work without logging directly into any gear — everything was through a tool.  It was a proud moment for me and the group.

To kick off this new adventure, we’re starting with writing all our own stuff in-house; we’re obviously not talking about a full, commercial orchestration deployment here.  We’ve talking about taking care of the menial tasks that we are way too expensive to be doing.  Simple tasks.  Common tasks.  Repeatable tasks.  All game.  What’s the MAC address of that host? Continue reading

Cisco Live 2018 – Yes, I Went Too

It’s been a very busy month or so. June is always like that, it seems. There’s ARRL Field Day, which is always the last rainy weekend in June. This year, Cisco Live was in June, and that typically includes Tech Field Day activities. Right before that, we had the whole family in town for a family reunion. There was all sorts of stuff going on. Now that most of that has blown over, I’ve collected my thoughts and wanted to talk about Cisco Live this year.

Those who are of any importance in the networking world (LOL!) converged on Orlando this to attend the conference. Orlando brings back all sorts of memories — from Taverna Opa to Sizzler to LISP explained with plates — and we’re all familiar with the Orange County Convention Center. It’s a great facility with enough room to handle the largest of gatherings. I don’t think I saw the attendance numbers, but I would guess there were 30,000 attendees at Cisco Live this year. A typical crowd for the event, and the venue was more than adequate.

This year, I went on the Imagine Pass instead of the full conference pass. This pass included Continue reading

An Update for my Adoring Fans

I feel like a teenage girl with a fashion blog who hasn’t posted in 6 months and comes back with “I know I haven’t posted in a while…”  Sigh.  It’s been right at a year since I actually published a post, so I figured I would give everyone an update.

I’ve had some personal things going on lately, and those have taken all of my energy.  We’ve made it through those rough times, so my energy is coming back.  I’m feeling better every day, and I hope I can get back to producing some content.  And, let me tell you…I’ve got some stuff to talk about.

*insert star wipe here*

We got a new director-level dude at the office, and he’s really mixing things up for us.  His philosophy includes changing the way we do everything that we do.  Like literally everything.  He ran a report for me on my ticket queue and showed me that 60% of my ticket count was on stupid stuff that’s below my pay grade.  His advice : Make somebody else do it.  So I did.  I taught myself some more Python (not hard since Continue reading

Cisco Live US 2017 – Saturday Adventure

For the last couple years, on the Saturday before Cisco Live US kicks off, we like to go and do something in the host city.  Nothing big.  Nothing fancy.  Just something we aren’t going to be able to do once the conference gets going.  In San Diego, we went to the zoo.  Last year, we went to the National Atomic Testing Museum.  This year, we’re going to the National Museum of Organized Crime and Law Enforcement…aka, the Mob Museum.

It’s Vegas.  This placed used to be full of mobsters doing some unsavory things.  I like to think it’s a bit more legit these days, but it’ll be fun to see how the mob and the law danced around back in the day.  It’s in the old historic post office and courthouse a couple block off Fremont Street and is complete with full courtroom and the wall where the Saint Valentine’s Day Massacre occurred. I should be a good time assuming we make it past the police lineup.

All are invited.  If you want to join us, drop me a message on Twitter.  We’ll probably meet for lunch then head over there in the early afternoon.  Or head over there for lunch. Continue reading

Cisco Live US 2017 – The Plan So Far

Put it on your calendar.  Cisco Live US is June 25 – 29, 2017, in Las Vegas.  This is the largest conference I go to every year, and it’s the highlight of my professional year.  I’ve been going for a few years now and enjoy it for the content and camaraderie.  What are we doing this year?

We’ll fly in on Friday again and do something.  No idea what, but I imagine we’ll throw out an invitation for dinner to the public and meet somewhere.  If you’re going to be in town, let me know, and we’ll meet up.

The Saturday Adventure was going to be ham radio related since that’s ARRL Field Day.  I reached out to the Las Vegas ham club, and they told me that the clubs out there all go to the top of a mountain to operate.  The problem : that mountain is 44.8 miles away from Mandalay.  That’s one helluvan Uber ride, so that’s out.  I looked at some other epic sites like the Grand Canyon and Hoover Dam, but, based on past participation, the time requirements for those don’t make the cut for the group.  We probably need to meet Continue reading

Cisco Clock Issue – This Is Really Bad

Check out this advisory from Cisco that came out a couple days ago.  You need to read it and act on it immediately!  I’ll summarize for you : Thanks to a faulty clock signal component, certain Cisco devices will stop functioning after about 18 months and become really expensive bricks!  Reading through it, you’ll see phrases like “we expect product failures” and “is not recoverable.”  Seriously, what the hell? This really warms the heart.

The fault affects a couple Meraki devices, the Nexus 9504, and some models of the ISR 4000s – the ISR4331, ISR4321, and ISR4351.  The 4000s are part of Cisco’s flagship branch routers, and I know several people (including myself!) who have some of the affected units deployed in production.  Some unnamed people on Twitter tell me that they have 50 and even 120 of these guys deployed in the field.  That’s a lot of faulty clocks.

The fix is to open a TAC case and get a new device.  Cisco is using the word “platform” when talking about replacement, meaning that they’ll send you a naked device.  If you have cards or memory upgrades or a Continue reading

ASIC Programmability from Barefoot Networks

Full disclosure : I was lucky to be among a group of networking influencers invited to Silicon Valley to visit some networking companies and see what they were offering to the market.  I was flown out and given accommodations at the expense of Gestalt IT – the company that organized the event.  I was given some swag by each company, but I was never paid to write a positive review on the product.  Heck, I’m not even expected to write at all.

Think about the fastest switch in your network and why it’s so fast.  Traditionally, it’s because the manufacturer has developed a very efficient ASIC that does switching very well (give me some leeway here and forget about routing, encapsulation, etc.), but it really can’t do anything else.  Want a new switching feature?  Well, your switch can’t do that if the ASIC doesn’t support it.  No big deal – the manufacturer just needs to make a new ASIC that supports it, right?  This sounds simple, but, generally, this is a many-years process and requires a hardware update on your end.  This is not a good solution in a world where new features and technologies Continue reading

QoS? Really?

I wrote this post during Cisco Live and said “I’ll just give it a once-over tonight and publish it.”  That was something like 6 weeks ago now. What a loser I am.

Yes, really. QoS has actually gotten some attention this year. After how many years of living in the dark and being feared by junior and senior engineers alike, we’re seeing some really cool technologies coming out for it.

I was honored to be invited to Tech Field Day Extra this morning while I’m at Cisco Live.  If you don’t know about TFD, you’re missing out.  A group of influencers gather in a room and get very deep and very technical presentations from vendors.  Today, Cisco came and talked about a couple of topics including branch security and QoS.  Obviously, the QoS was the big hitter for me.

Tim Szigeti (@tim_szigeti) kicked off the QoS conversation by talking about some of the recent advancements in QoS in both hardware and software. In hardware, he discussed the programmability of the new ASICs that Cisco is using in their switches and routers.  These ASICs are dumb out of the box, but they are very willing to learn.  Want it Continue reading

Cisco Live 2016 – Everything Is Coming Together

It seems that Cisco Live is about the only thing I blog about in the last…well, few years.  At least I’m still writing, even if it is twice a year.  :)

Here’s a summary about Cisco Live for those who live in a dark hole.  It’s July 10 – 14, 2016, in Las Vegas.  If you do anything with Cisco, you should go.  If you do anything with technology that isn’t Cisco, you should go.  Bring your significant other.  There’s plenty to do for everyone.  Anyway, on to the details for this year’s show.

The Mandalay Bay’s South Convention Center will be our campgroun this year.  This is the same place from back in 2011, but with quite a few updates to the layout.  I can’t remember the numbers from the last time, but I imagine that number will be close to double this year.  I have all confidence that the convention center can handle us, though.  They did such a wonderful job last time.  I hope the food is as good as I remember; it’s been pretty poor going the last few years in Orlando, San Continue reading

Cisco Live – The Complaints

You should know by now that I always find something to complain about.  Is that a bad thing?  Probably.  Does it help improve things?  Absolutely!

Again, I love going to Cisco Live every year.  Without question, it’s my favorite event of the year.  It’s a great event with great people and great things to do.  With that said, let’s look at what could have been a bit better this year.

• Seating – There was a terrible lack of seating around the convention center this year.  I spent a good part of the time standing around when I wasn’t in a session, which is not good for a lazy, fat guy like me.  We’re talking standing for 5 days here.  In the past, there have been plenty of places to sit throughout the event, so I don’t really know what happened here.
• Logistics at the CAE – This happens every year.  You’re told to go in one gate at the CAE, but no one listens or your information is wrong; you wind up standing in line behind 18849298 people who were all behind you at one point.  Then you have to get your bag searched.  Then they take half your swag Continue reading

Cisco Live 2015 – Helping Others

Another year, another Cisco Live.  Boy, was it a good one.  San Diego is a great city, and convention center there is plenty big to take care of all 25k attendees.  On top of that, the city itself is equipped to handle groups of 40 roaming the streets looking for food and entertainment.

This year’s event had the usual stuff that everyone talks about – breakout session, keynotes, exams, etc. – but Cisco stepped outside of technology this year by helping others.

• Stop Hunger Now Volunteer Space – Cisco partnered with Stop Hunger Now and set up a space where attendees could assemble meal kits to fight hunger worldwide.  I believe the final number was 100,656 kits assembled, which is a fantastic number.  I’m ashamed to say, though, that I missed this due to scheduling issues, but I’m proud of Cisco and the attendees for participating.
• Mike Rowe Works Foundation – The keynote speaker this year was Mike Rowe, whom most of us know from his TV show Dirty Jobs.  His foundation helps to support skilled labor movements in order to keep all jobs meaningful.  As part of his appearance, Cisco gave his foundation a nice check for $20,000 and promised another Continue reading

FEMA and Your Business Continuity Plan

I passed the ROUTE exam a few days/weeks/months/something ago and decided to pursue certifications of another sort for a while. The wife and I are trying our best to help the community through our ham radio training, so I decided to go down that path a bit further. One thing I was interested in doing is to do EmComm during declared emergencies. That meant I had to take two FEMA courses online to be allowed in the EOC. I thought they would be terribly boring, but I found them to be quite familiar.

The first course was on the Incident Command System (ICS). The main idea is that, in the event of an emergency of any kind or size, an Incident Commander (IC) is assigned to be responsible for the recovery effort.  This mean analysis of the incident, generating an action plan (a very key component), and execution of said plan. If the IC can complete the action plan by himself, then off he goes. If he or she needs some additional resources like people or equipment, then he or she is empowered to draft that help from any entity that’s involved.

Another one of the big points of ICS is Continue reading

Summary Post – Methods to Manipulate OSPF Costs

There are three ways to manipulate the interface cost in OSPF.  One is very direct, one changes the presentation of the interface, and the other changes the calculations for every interface.

Set the cost of the interface directly – Just give it the number you want.  Easy.  This is the number OSPF will use in the SPF calculations without doing any math on the interface.

R1(config-if)#ip ospf cost 8482

Set the bandwidth of the interface – The formula that OSPF uses to calculate interface cost is pretty easy to remember – (reference bandwidth) / (interface bandwidth).  Changing the interface bandwidth will obviously change the result of the calculation.  The same caveat for EIGRP route manipulation holds true here; if you change the bandwidth of the interface, you may affect other things like QoS…or EIGRP, now that I mention it.

R1#sh ip ospf inter brief
Interface    PID   Area            IP Address/Mask    Cost  State Nbrs F/C
Fa0/0        1     0               192.0.2.1/24       10    DR    0/0
R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#int f0/0
R1(config-if)#bandwidth 10
R1(config-if)#do show ip ospf interf brief
Interface    PID   Area            IP Address/Mask    Cost  State Nbrs F/C
Fa0/0        1     0               192.0.2.1/24        Continue reading

Summary Post – OSPF Network Statement Order and Matching

When you configure OSPF network statements, IOS orders them most-specific to least-specific then does a top-to-bottom match of the interfaces. It doesn’t matter which order you put them in, the configuration will always be ordered with the longest prefix matches first.  Lab time!

I have router R1 with these interfaces.

R1#sh ip int brief
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            10.0.0.1        YES manual up                    up
FastEthernet0/1            unassigned      YES unset  administratively down down
Loopback100                10.0.101.1      YES manual up                    up
Loopback200                10.2.101.1      YES manual up                    up

Let’s add the OSPF configuration where 10.0.0.0/8 is in area 2 then check what OSPF thinks is happening.

R1(config)#router ospf 1
R1(config-router)#network 10.0.0.0 0.255.255.255 area 2
...
R1#show ip ospf interface brief
Interface    PID   Area            IP Address/Mask    Cost  State Nbrs F/C
Lo100        1     2               10.0.101.1/24      1     LOOP  0/0
Lo200        1     2               10.2.101.1/24      1     LOOP  0/0
Fa0/0        1     2               10.0.0.1/24        10    WAIT  0/0

All the interfaces are in area 2 as expected. Now let’s add 10.0.0.0/16 into area 1 to see what happens.

R1(config-router)#network 10.0.0.0  Continue reading

Advertising a Default Route Into EIGRP

Let’s get an IPv4 default route into EIGRP.  There are a few methods to do it.  I hate most of them, though.  I think it will be obvious which one I like.

Here’s the lab I have set up to test everything.  I want R4 to generate the default in each case.

Default Network – Candidate default.  I don’t think I’ve ever used that all my years in networking, but here’s how to use it in EIGRP for a default route.  You basically say “If you don’t know where to send a packet, send it to where network X lives.”  We’re going to set the 192.168.1.0/24 as the default network, so, in our case X = 192.168.1.0. R4 will tag that route as a default candidate when it advertises it to the rest of the network.  The config is easy but requires a stateful (yes, stateful) network to beconfigured as the default.

R4 config:
R4(config)#ip default-network 192.168.1.0
!
R1 routes:
R1#sh ip route
...
     4.0.0.0/24 is subnetted, 1 subnets
D       4.4.4.0 [90/435200] via 192.0.2.3, 00:08:33, FastEthernet0/0
                [90/435200] via 192.0.2.2,  Continue reading

EIGRP and OSPF – Are We Connected?

For both OSPF and EIGRP routers to become neighbors, their interface’s primary IP address must be on the same subnet. That statement is true. There is a difference in the definition of “same subnet”, though.

In OSPF, both routers have to be configured to be on the same subnet with the same mask or else they won’t neighbor up.  When an hello packet is sent, the subnet mask is sent embedded in there.  The router does a quick look to be sure the subnets are defined the same way on both ends.  If everything doesn’t match, they don’t neighbor. Here’s a Wireshark screenshot to show you the OSPF hello.  Note: See edit below.

In EIGRP,the subnet mask isn’t sent in the hello packet, so that doesn’t come into play.  Each router does a subnet calculation on the source address of the potential suitor, and, if that guy falls within the connected network, the peering magic happens.  Here’s another Wireshark shot for you to enjoy.

Send any Wireshark certification vouchers questions my way.

Edit:  I did some further research on Julius’s comment about point-to-point links in OSPF.  It is absolutely true that point-to-point links do indeed ignore the subnet Continue reading