Nasty Multicast VSS bug on Catalyst 4500-X

I ran into an “exciting” bug yesterday. It was seen in a 4500-X VSS pair running 3.7.0 code. When there has been a switchover meaning that the secondary switch became active, there’s a risk that information is not properly synced between the switches. What we were seeing was that this VSS pair was “eating” the packets, essentially black holing them. Any multicast that came into the VSS pair would not be properly forwarded even though the Outgoing Interface List (OIL) had been properly built. Everything else looked normal, PIM neighbors were active, OILs were good (except no S,G), routing was there, RPF check was passing and so on.

It turns out that there is a bug called CSCus13479 which requires CCO login to view. The bug is active when Portchannels are used and PIM is run over them. To see if an interface is misbehaving, use the following command:

hrn3-4500x-vss-01#sh platfo hardware rxvlan-map-table vl 200 <<< Ingress port

Executing the command on VSS member switch role = VSS Active, id = 1


Vlan 200:
l2LookupId: 200
srcMissIgnored: 0
ipv4UnicastEn: 1
ipv4MulticastEn: 1 <<<<<
ipv6UnicastEn: 0
ipv6MulticastEn: 0
mplsUnicastEn: 0
mplsMulticastEn: 0
privateVlanMode: Normal
ipv4UcastRpfMode: None
ipv6UcastRpfMode: None
routingTableId: 1
rpSet: 0
flcIpLookupKeyType: IpForUcastAndMcast
flcOtherL3LookupKeyTypeIndex: 0
vlanFlcKeyCtrlTableIndex: 0
vlanFlcCtrl: 0


Executing the command on VSS member switch role = VSS Standby, id = 2


Vlan 200:
l2LookupId: 200
srcMissIgnored: 0
ipv4UnicastEn: 1
ipv4MulticastEn: 0 <<<<<
ipv6UnicastEn: 0
ipv6MulticastEn: 0
mplsUnicastEn: 0
mplsMulticastEn: 0
privateVlanMode: Normal
ipv4UcastRpfMode: None
ipv6UcastRpfMode: None
routingTableId: 1
rpSet: 0
flcIpLookupKeyType: IpForUcastAndMcast
flcOtherL3LookupKeyTypeIndex: 0
vlanFlcKeyCtrlTableIndex: 0
vlanFlcCtrl: 0

From the output you can see that "ipv4MulticastEn" is set to 1 on one switch and 0 to the other one. The state has not been properly synched or somehow misprogrammed which leads to this issue with black holing multicast. It was not an easy one to catch so I hope this post might help someone.

This also shows that there are always drawbacks to clustering so weigh the risk of running in systems in clusters and having bugs affecting both devices as opposed to running them stand alone. There's always a tradeoff between complexity, topologies and how a network can be designed depending on your choice.

Catalyst 3750 IPv6 ACL Limitations

I recently ran into some limitations of IPv6 ACLs on the Catalyst 3750 platform. I had developed an ACL to protect from receiving traffic from unwanted address ranges such as ::, ::1, ::FFFF:0:0/96. The first address is the unspecified address, the second one is the loopback address and the last one is IPv4 mapped traffic. The ACL also contained an entry to deny traffic with routing-type 0.

Note that no error is output when adding the entries in the ACL, only when applying the ACL to an interface.

% This ACL contains following unsupported entries.
% Remove those entries and try again.
    deny ipv6 any any routing-type 0 sequence 20
    deny ipv6 host ::1 any sequence 290
    deny ipv6 host :: any sequence 310
    deny ipv6 ::FFFF:0.0.0.0/96 any sequence 330
% This ACL can not be attached to the interface.
SW1(config-if)#
%PARSE_RC-4-PRC_NON_COMPLIANCE: `ipv6 traffic-filter v6-ACL-IN in'

From the configuration guide, the following limitations apply to the Catalyst 3750 platform.

What this means is that we can’t match on flowlabel, routing-header and undetermined transport upper layer protocol. We also need to match on networks ranging from /0 to /64 and host addresses that are belonging to global unicast Continue reading

Blog Migrated!

Hi!

I have decided to migrate my blog from wordpress.com to a private environment. The main reasons being that I felt that I had outgrown the normal wordpress.com site. I wanted to be able to install plugins and get more accustomed to running my own environment. These days it can’t hurt picking up some Linux skills.

The other reason is that I haven’t made a dime on the blog, in fact since I’ve had to pay hosting costs I’ve been losing money on the blog every year. By placing some ads I hope I can make enough for the hosting and anything extra would help me in getting things I need to generate more content.

The blog should now be reachable over both v4 and v6 and have SSL enabled.

Please bear with me if you find any things that are broken. I have migrated the content but I’m sure things will pop up. If they do, please notify me.

/Daniel

NAT saves the day!

Introduction NAT is bad, it breaks end to end connectivity. It’s misused as a security tool. Using NAT kills kittens. Yes yes, we all know that. That doesn’t mean that there aren’t valid use cases for NAT and when NAT can save the day. What was the problem? Imagine that you have a device that […]

Author information

ddib

Daniel Dib is a network engineer and CCIE #37149. He mainly works with enterprise networks and network design. You can find his original content at lostintransit.se and on Twitter @Danieldibswe

The post NAT saves the day! appeared first on Packet Pushers Podcast and was written by ddib.