Failure Analysis: An Interesting way to Break CAPWAP
I recently stumbled into what I think is a very interesting failure scenario with a Cisco Wireless solution. This was a traditional controller based solution that leveraged a CAPWAP data and control plane. The symptoms were fairly consistent and strange.
Symptoms:
- When issues are occurring, all uploads reduce to about 1.5Mb/s
- Installing a new AP seems to solve the issue
- Issue re-occurs in a few minutes
- Issues only occur for one specific site
- Wireless is configured consistently across 5 sites
- RF is not an issue
Topology:

When I got involved with this, a few people had reviewed the configuration and TAC had been involved for some time. While on-site, I took a look at RF and channel utilization (expecting to find it to be ugly since I knew it was heavily dependent on 2.4Ghz). My first order of business was to spin up a test AP in its own group and advertise a test SSID on a 5Ghz channel. Upon doing so, both iPerf and Speedtest were >50Mb/s. My initial thought was that the density needed to be increased and the radios tweaked to get more clients on 5Ghz. However, a few minutes into my testing–my upload also Continue reading