Fixing high CPU use on Cisco 7600/6500
Recently some time ago (this blog post has also been lying in draft for a while)
someone came to me with a problem they had with a Cisco 7600.
It felt sluggish and "show proc cpu" showed that the weak CPU was very loaded.
This is how I fixed it.
"show proc cpu history" showed that the CPU use had been high for quite a while, and too far back to check against any config changes. The CPU use of the router was not being logged outside of what this command can show.
"show proc cpu sorted" showed that almost all the CPU time was spent in interrupt mode. This is shown after the slash in the first row of the output. 15% in this example:
Interrupt mode CPU time is (a bit simplified and restricted to the topic at hand) used when the router has to react to some user traffic. Now why would the 7600 use the Continue readingRouter# show proc cpu sorted CPU utilization for five seconds: 18%/15%; one minute: 31%; five minutes: 42% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 198 124625752 909637916 137 0.87% 0.94% 0.94% 0 IP Input [...]
Healthy Paranoia Show 18: Illusion, Lies and Neuroscience with Alex Stone
Ladies and gentleman, prepare to be mystified and amazed by another episode of Healthy Paranoia. Where even the unicorns are nerdy and the evil bit is always set on your packets. Just in time for Halloween, get ready for some tricks and definitely treats, because we’re going to discuss the intersection of magic, social engineering […]
Author information
The post Healthy Paranoia Show 18: Illusion, Lies and Neuroscience with Alex Stone appeared first on Packet Pushers Podcast and was written by Mrs. Y.
Next-hop resolution and point-to-point
I had this blog post lying around as a draft for a long time. I didn't think it was was "meaty" enough yet, but since I'm no longer a network consultant I don't think it'll become any meatier. So here it goes.
Here I will describe the process of L3-to-L2 mapping, or next-hop resolution and how it works with point-to-point circuits like PPP, ATM and Frame relay. It's the process of finding out what to actually do with a packet once the relevant routing table entry has been identified.
It's deceptively simpler than on a LAN segment, but since people generally learn Ethernet before they learn point-to-point nowadays I'm writing it anyway.
When a packet is to be sent to an address on the same subnet a L3-to-L2 mapping is done to look up the L2 destination address (if any) to apply.
The packet is then encapsulated in a L2 frame and sent out the interface.
On a normal Ethernet LAN segment ARP is used to look up L3-to-L2, and the frame will then have that (L2) MAC address as its destination. The frame will then be received by (and only by) the intended destination.
In a point-to-point interface there Continue reading
Quiz #20 – NAT between Two Partner Companies
Your company has a border router (R2) that is connected to two partner companies: Partner-DB (R1) providing database services and Partner-APP (R3) that provides different application services to your web servers in DMZ (200.200.200.0/24). You are requested to configure NAT according to some requirements.
I’m a Networking Guy, and I’m Here To Talk About Programming
I’m fortunate enough to work and be connected with some stellar networking professionals. I mean it - they’re rock stars. In my quest to surround myself with smart folks like this - in an attempt to at the very least learn by osmosis - I’ve clearly succeeded. I haven’t been in the industry for that long - but I’ve chosen networking (among other things) to be what I want to focus on professionally, and these are the best people to learn it from.I’m a Networking Guy, and I’m Here To Talk About Programming
I’m fortunate enough to work and be connected with some stellar networking professionals. I mean it - they’re rock stars. In my quest to surround myself with smart folks like this - in an attempt to at the very least learn by osmosis - I’ve clearly succeeded. I haven’t been in the industry for that long - but I’ve chosen networking (among other things) to be what I want to focus on professionally, and these are the best people to learn it from.Forwarding UDP broadcast traffic mechanisms
How does the internet work - We know what is networking
We will speak here about some basics about Forwarding UDP broadcast traffic. If you were wondering what Forwarding UDP broadcast traffic actually is I will try to explain it here in few words. If you have more that one broadcast domains in your local network, let’s say that you have three VLANs. In normal networking theory it’s normal […]
SearchSDN Article: Packet Design Pivots Route Analytics Toward SDN
This item has no description. Follow link to view item.Spanning Tree Exercise and Revisiting Root Guard
This was actually spurned from a comment I received on another one of my blog posts that you can find here. Seeing that comment, I white boarded it and realized that I may have been completely wrong in regards to how Root Guard could “break a network”.
Let’s say we have the following topology:
- Core 1 is the root for VLAN 10 with a configured priority of 4096, and is the secondary root for VLAN 20 with a configured priority of 8192. We alternate this with Core 2 in order to load balance VLAN traffic.
- Access 3 and 4 are left in default configuration regarding spanning tree.
- Two workstations are present – one in VLAN 10, and another in VLAN 20. Their default gateways are SVIs that are on the Core switches.
- For simplicity, switch MAC addresses are the number contained in their names. Example: Access 4’s MAC address is “4”.
- All link costs are the same.
- All links between switches are trunks transporting all VLANs.
Let’s work through the spanning tree topologies.
Core 1 – Root bridge for VLAN 10. All ports designated.
Core 2 – Port 1 will be a root port Continue reading
With SDN, Do We Still Need CCIEs?
"With SDN, we don't need CCIEs anymore. Anyone can run the network with a simple click-and-drag GUI." Really."SDN makes the knowledge of traditional networking is not relevant anymore. We need more people who can write code instead." Wow.
"SDN with Openflow removes all the current routing protocols. So why wasting your time to study CCIE?" Speechless.
Let's start with definition.
According to Wikipedia, SDN is "...an approach to building computer networks that separates and abstracts elements of these systems..." There are two important keywords there: separate, and abstraction. Separate means decouple Control Plane and Data/Forwarding Plane function. If in 'traditional networking' both Contol and Data functions are contained within a single device, SDN makes the separation so the Control plane can be moved to a device or system that is located at the central of the network. More intelligent control function that can see the whole network end-to-end.
And the Control plane can be customized, manipulated, re-programmed and so on, regardless the state of the Data plane. This is the first level of the abstraction.
Why is abstraction important? Because we want to separate the complexity. Think about building multiple layers that separate Continue reading
The real Slim Shady
Historically when an application team needed compute and storage resources they would kick off a workflow that pulled in several teams to design, procure and deploy the required infrastructure (compute, storage & network). The whole process generally took a few months from request to delivery of the infrastructure.
The reason for this onerous approach was really that application groups generally dictated their choice of compute technology. Since most applications scaled vertically, the systems and storage scaled likewise. When the application needed more horsepower, it was addressed with bigger more powerful computers and faster storage technology. The hardware for the request was then staged followed by a less-than-optimal migration to the new hardware.
The subtlety that gets lost regarding server virtualization is that a virtualization cluster is based on [near] identical hardware. The first machines that were virtualized where the ones who’s computer and storage requirements could be met by the hardware that the cluster was based on. These tended to be the applications that were not vertically scaled. The business-critical vertically scaled applications continued to demand special treatment, driving the overall infrastructure deployment model used by the enterprise.
The subtlety that gets lost regarding server virtualization is that a virtualization cluster is based on [near] identical hardware. The first machines that were virtualized where the ones who’s computer and storage requirements could be met by the hardware that the cluster was based on. These tended to be the applications that were not vertically scaled. The business-critical vertically scaled applications continued to demand special treatment, driving the overall infrastructure deployment model used by the enterprise.
The data center Continue reading
Live digital threats
I wanted to bring these free tools to my blog to keep track of them, as most probably I will forget about, and to share with anybody interested in gathering information in the security areas. First tool was revealed to my in tweet from Greg Ferro: Blog post: Response: Digital Attack Map http://t.co/rEhfYAN77c pic.twitter.com/0O1L6kELsK — […]Network Function Abstraction Redux (Now with DevOps!)
I wrote a few days ago about how cool projects like OpenDaylight are abstracting network functions into consumable policies that non-network folks can use (and that’s a good thing!). I felt this quick follow-up was necessary. Providing the right tools to the application folks that allow network provisioning to occur as quickly as anything else that’s software-defined, such as servers, while keeping those tools light on the learning curve, is exactly what the apps folks have been wanting from the network for the last 10 years or so.Network Function Abstraction Redux (Now with DevOps!)
I wrote a few days ago about how cool projects like OpenDaylight are abstracting network functions into consumable policies that non-network folks can use (and that’s a good thing!). I felt this quick follow-up was necessary. Providing the right tools to the application folks that allow network provisioning to occur as quickly as anything else that’s software-defined, such as servers, while keeping those tools light on the learning curve, is exactly what the apps folks have been wanting from the network for the last 10 years or so.Cisco switches and smartport macros
Smartport macros are not more than some templates you can define on Cisco switches that will apply the same configuration on multiple ports. It’s not a subject that needs too many discussions, but it can be useful for your Cisco certification preparation or real life Cisco switch administration. Configuration is very simple and it goes […]Real world SDN: VMware NSX on Packet Pushers
If you like to listen to geeky networking podcasts on your way to the office, this is surely one not to miss. Scott Lowe and I appeared as guests on the Packet Pushers podcast, sponsored by VMware, and hosted by Ethan Banks and Greg Ferro to discuss the technical details of VMware NSX, and the […]Installing VMware tools on Cisco ACS
As of ACS v5.4 Cisco has finally included VMware tools for their ADE OS. Unfortunately, when you upgrade, they do not get installed automatically as the installation is triggered during the initial install. This post is for those of us that have upgraded to version 5.4 and didn’t choose to do a fresh install.
First of all, you need to get your hands on the Root Patch. This Root Patch allows you root shell access to the ADE OS, which is just a customized version of Redhat Linux. You can get this patch from TAC by asking them nicely, or telling them you need to install VMware tools on your ACS 5.4 install. I’m sure if you’re clever you can find a copy out in the wild as well. But your mileage may vary…
Installing the ACS Root Patch
This part is pretty simple. Using the ADE OS application installer, install the package using a predefined repository…
acs/eladmino# application install RootPatch-ACS-5-4.tar.gz ftp Save the current ADE-OS running configuration? (yes/no) [yes] ? Generating configuration... Saved the ADE-OS running configuration to startup successfully Initiating Application installation... Application successfully installed acs/eladmino#
Using the Root Patch
After the install, you Continue reading
Five Functional Facts about OTV
Following on from my previous “triple-F” article (Five Functional Facts about FabricPath), I thought I would apply the same concept to the topic of Overlay Transport Virtualization (OTV). This post will not describe much of the foundational concepts of OTV, but will dive right into how it actually functions in practice. A reasonable introduction to OTV can be found in my series on Data Center Interconnects.
So without any more preamble, here are five functional facts about OTV.
#1 – OTV Adds 42 Bytes of Overhead
OTV, being an encapsulation technology, adds additional headers to the encapsulated payload. Without rehashing too much of the basics, OTV extends a Layer 2 domain across a Layer 3 cloud. In order to preserve the Layer 2 semantics on either side of the cloud, OTV scoops up the entire Layer 2 packet on one side, transports it across the cloud in the middle, and puts it on the LAN in the other side. This preserves the entire Ethernet header including the original source/dest MAC, and even the CoS bits and VLAN tag.
So to begin with, we’re putting a (potentially) full-sized Ethernet frame – with headers – inside another Ethernet frame. That Continue reading
Show 165 – Running Code Is What Defines The Rules
Ethan Banks and Greg Ferro are joined on this week’s Packet Pushers podcast by Teren Bryson, Paul Stewart, and Michele Chubirka. This is a community show, meaning it’s just a bunch of engineers chatting about the industry and our experiences. No vendors looking over our shoulders at all. Here’s what we yammer on about. Topics […]
Author information
The post Show 165 – Running Code Is What Defines The Rules appeared first on Packet Pushers Podcast and was written by Ethan Banks.