Using BFD to Track WAN Status and Change HSRP Priority

It’s been five years since I started this blog! Time flies and a lot has happened since. Thanks for being along for the ride. What better way to celebrate than a blog post?

This post is going to be short and to the point.

Many of us run HSRP or VRRP. It is quite common to run it in a topology where you have dual routers and dual exits to the WAN and you don’t want to black hole your traffic.

One traditional way of achieving this is by tracking the interface that goes towards the WAN. There are a couple of drawbacks to this approach though:

• You may not get link down on failure (connecting to switch)
• You may experience an error that does not produce link down event

The next option is to use IP SLA that sends ICMP Echo towards the next-hop of the WAN or some destination further into the network. Ehanced Object Tracking (EOT) can then be used to create a track object that decrements the priority of the HSRP active router when the ICMP Echo probe fails. This works better but there are still some drawbacks to this approach:

• Frequency can’t be set to Continue reading

IPv6 Multicast

These are my notes for IPv6 multicast for the CCDE exam. Overview

• Prefix FF::/8 reserved for multicast
• Multicast Listener Discovery (MLD) replaces IGMP
  • MLD is part of ICMPv6
  • MLDv1 equivalent to IGMPv2
  • MLDv2 equivalent to IGMPv3
• ASM, SSM and Bidir supported
• PIM identified by IPv6 next header 103
• BSR and static RP supported
• No support for MSDP
  • Anycast supported through PIM, defined in RFC4610

• Any Source Multicast (ASM)
  • PIM-SM, PIM-BiDir
  • Default for generic multicast and unicast prefix-based multicast
  • Starts with FF3x::/12
• Source Specific Multicast (SSM)
  • PIM-SSM
  • FF3X::/32 is allocated for SSM by IANA
  • Currently prefix and plen is zero so FF3X::/96 is useable for SSM
• Embedded RP groups
  • PIM-SM, PIM-BIDir
  • Starts with FF70::/12

IPv6 Multicast Addressing

IPv6 multicast address format includes variable bits to define what type of address it is and what the scope is of the multicast group. The scope can be:

1 – Node

2 – Link

3 – Subnet

4 – Admin

5 – Site

8 – Organization

E – Global

The flags define if embedded RP is used, if the address is based on unicast and if the address is IANA assigned or not (temporary). The unicast based IPv6 multicast address allows an organization to Continue reading

Service Provider IPv6 Deployment

These are my study notes regarding IPv6 deployment in SP networks in preparation for the CCDE exam.

Drivers for implementing IPv6

• External drivers
  • SP customers that need access to IPv6 resources
  • SP customers that need to interconnect their IPv6 sites
  • SP customers that need to interface with their own customers over iPv6
• Internal drivers
  • Handle problems that may be hard to fix with IPv4 such as large number of devices (cell phones, IP cameras, sensors etc)
  • Public IPv4 address exhaustion
  • Private IPv4 address exhaustion
• Strategic drivers
  • Long term expansion plans and service offerings
  • Preparing for new services and gaining competitive advantage

Infrastructure

• SP Core Infrastructure
  • Native IPv4 core
  • L2TPv3 for VPNs
  • MPLS core
  • MPLS VPNs

My reflection is that most cores would be MPLS enabled, however there are projects such as Terastream in Deutsche Telekom where the entire core is IPv6 enabled and L2TPv3 is used in place of MPLS.

• IPv6 in Native IPv4 Environments
  • Tunnel v6 in v4
  • Native v6 with dedicated resources
  • Dual stack

The easiest way to get going with v6 was to tunnel it over v4. The next logical step was to enable v6 but on separate interfaces to not disturb the “real” traffic and to be Continue reading

Coming Updates to the CCIE Program

With everything going on in the industry, what is happening to the CCIE program?

I recently watched a webinar on coming updates to the CCIE program. I have also been talking to the CCIE and CCDE program managers which I am proud to call my friends. The certifications are a big part of Cisco’s business, people are afraid that certifications will lose value as Software Defined Networking (SDN) gains more traction in the industry. What is Cisco’s response to the ever changing landscape of networking?

We have already seen Cisco announce the CCNA cloud and CCNA industrial which shows that Cisco follows the market. Will we see a CCIE cloud or CCIE SDN? Doubtful… Why? Because SDN is not a track in itself, it will be part of all tracks… The CCIE DC will be refreshed to include topics like Application Centric Infrastructure (ACI) in the blueprint. When? It’s not official yet which means you have at least 6 months. My guess is that we will see an announcement before this year ends which would mean that the update is around a year away.

CCIE DC is the natural fit for SDN. What about the other tracks? Expect other tracks Continue reading

Design Considerations for North/South Flows in the Data Center

Traditional data centers have been built by using standard switches and running Spanning Tree (STP). STP blocks redundant links and builds a loop-free tree which is rooted at the STP root. This kind of topology wastes a lot of links which means that there is a decrease in bisectional bandwidth in the network. A traditional design may look like below where the blocking links have been marked with red color.

If we then remove the blocked links, the tree topology becomes very clear and you can see that there is only a single path between the servers. This wastes a lot of bandwidth and does not provide enough bisectional bandwidth. Bisectional bandwidth is the bandwidth that is available from the left half of the network to the right half of the network.

The traffic flow is highlighted below.

Technologies like FabricPath (FP) or TRILL can overcome these limitations by running ISIS and building loop-free topologies but not blocking any links. They can also take advantage of Equal Cost Multi Path (ECMP) paths to provide load sharing without doing any complex VLAN manipulations like with STP. A leaf and spine design is most commonly used to provide for a high amount Continue reading

Introduction to Storage Networking and Design

Introduction

Storage and storage protocols are not generally well known by network engineers. Networking and srorage have traditionally been two silos, Modern networks and data centers are looking to consolidate these two nettworks into one and to run them on a common transport such as Ethernet,

Hard Disks and Types of Storage

Hard disks can use different type of connectors and protocols.

• Advanced Technology Attachment (ATA)
• Serial ATA (SATA)
• Fibre Channel (FC)
• Small Computer System Interface (SCSI)
• Serial Attached SCSI (SAS)

ATA and SATA and SCSI are older standards, newer disks will typically use SATA or SAS where SATA is more geared towards the enterprise market. FC is used to attach to Storage Area Network (SAN)

Storage can either be file-level storage or block-level storage. File-level storage provides access to a file system through protocols such as Network File System (NFS) or Common Internet File System (CIFS). Block-level storage can be seen as raw storage that does not come with a file system. Block-level storage presents Logical Unit Number (LUN) to servers and the server may then format that raw storage with a file system. VmWare uses VmWare File System (VMFS) to format raw devices.

DAS, NAS and SAN

Storage can Continue reading

Next Generation Multicast – NG-MVPN

Introduction

Multicast is a great technology that although it provides great benefits, is seldomly deployed. It’s a lot like IPv6 in that regard. Service providers or enterprises that run MPLS and want to provide multicast services have not been able to use MPLS to provide multicast  Multicast has then typically been delivered by using Draft Rosen which is a mGRE technology to provide multicast. This post starts with a brief overview of Draft Rosen.

Draft Rosen

Draft Rosen uses GRE as an overlay protocol. That means that all multicast packets will be encapsulated inside GRE. A virtual LAN is emulated by having all PE routers in the VPN join a multicast group. This is known as the default Multicast Distribution Tree (MDT). The default MDT is used for PIM hello’s and other PIM signaling but also for data traffic. If the source sends a lot of traffic it is inefficient to use the default MDT and a data MDT can be created. The data MDT will only include PE’s that have receivers for the group in use.

Draft Rosen is fairly simple to deploy and works well but it has a few drawbacks. Let’s take a look at these:

• Overhead – Continue reading

My CLUS 2015 Schedule for San Diego

With roughly two months to go before Cisco Live starts, here is my preliminary schedule for San Diego.

I have two CCDE sessions booked to help me prepare for the CCDE exam. I have the written scheduled on wednesday and we’ll see how that goes.

I have a pretty strong focus on DC because I want to learn more in that area and that should also help me prepare for the CCDE.

I have the Routed Fast Convergence because it’s a good session and Denise Fishburne is an amazing instructor and person.

Are you going? Do you have any sessions in common? Please say hi if we meet in San Diego.

OSPF Design Considerations

Introduction

Open Shortest Path First (OSPF) is a link state protocol that has been around for a long time. It is geneally well understood, but design considerations often focus on the maximum number of routers in an area. What other design considerations are important for OSPF? What can we do to build a scalable network with OSPF as the Interior Gateway Protocol (IGP)?

Prefix Suppression

The main goal of any IGP is to be stable, converge quickly and to provide loop free connectivity. OSPF is a link state protocol and all routers within an area maintain an identical Link State Data Base (LSDB). How the LSDB is built it out of scope for this post but one relevant factor is that OSPF by default advertises stub links for all the OSPF enabled interfaces. This means that every router running OSPF installs these transit links into the routing table. In most networks these routes are not needed, only connectivity between loopbacks is needed because peering is setup between the loopbacks. What is the drawback of this default behavior?

• Larger LSDB
• SPF run time increased
• Growth of the routing table

To change this behavior, there is a feature called prefix suppression. When Continue reading

A Quick Look at Cisco FabricPath

Cisco FabricPath is a proprietary protocol that uses ISIS to populate a “routing table” that is used for layer 2 forwarding.

Whether we like or not, there is often a need for layer 2 in the Datacenter for the following reasons:

• Some applications or protocols require to be layer 2 adjacent
• It allows for virtual machine/workload mobility
• Systems administrators are more familiar with switching than routing

A traditional network with layer 2 and Spanning Tree (STP) has a lot of limitations that makes it less than optimal for a Datacenter:

• Local problems have a network-wide impact
• The tree topology provides limited bandwidth
• The tree topology also introduces suboptimal paths
• MAC address tables don’t scale

In the traditional network, because STP is running, a tree topology is built. This works better for for flows that are North to South, meaning that traffic passes from the Access layer, up to Distribution, to the Core and then down to Distribution and to the Access layer again. This puts a lot of strain on Core interconnects and is not well suited for East-West traffic which is the name for server to server traffic.

A traditional Datacenter design will look something like this:

If we Continue reading

CLUS Keynote Speaker – It’s a Dirty Job but Somebody’s Gotta Do It

Did you guess by the title who will be the celebrity keynote speaker for CLUS San Diego? It’s none other than Mike Rowe, also known as the dirtiest man on TV.

Mike is the man behind “Dirty Jobs” on the Discovery Channel. Little did he know when pitching the idea to Discovery that they would order 39 episodes of it. Mike traveled through 50 states and completed 300 different jobs going through swamps, sewers, oil derricks, lumberjack camps and what not.

Mike is also a narrator and can be heard in “American Chopper”, “American Hot Rod”, “Deadliest Catch”, “How the Universe Works” and other TV shows.

He is also a public speaker and often hired by Fortune 500 companies to tell their employees frightening stories of maggot farmers and sheep castrators.

Mike also believes in skilled trades and in working smart AND hard. He has written extensively on the country’s relationship with work and the skill gap.

I’m sure Mike’s speach will be very interesting…and maybe a bit gross…

The following two links take you to Cisco Live main page and the registration packages:

Cisco Live
Cisco Live registration packages

HSRP AWARE PIM

In environments that require redundancy towards clients, HSRP will normally be running. HSRP is a proven protocol and it works but how do we handle when we have clients that need multicast? What triggers multicast to converge when the Active Router (AR) goes down? The following topology is used:

One thing to notice here is that R3 is the PIM DR even though R2 is the HSRP AR. The network has been setup with OSPF, PIM and R1 is the RP. Both R2 and R3 will receive IGMP reports but only R3 will send PIM Join, due to it being the PIM DR. R3 builds the (*,G) towards the RP:

R3#sh ip mroute 239.0.0.1
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report, 
       Z - Multicast Tunnel, z - MDT-data group sender, 
        Continue reading

Network Design Webinar With Yours Truly at CLN

I’m hosting a network design webinar at the Cisco Learning Network on Feb 19th, 20.00 UTC+1.

As you may know, I am studying for the CCDE so I’m focusing on design right now but my other reason for hosting this is to remind people that with all the buzzwords around SDN and NfV going around, the networking fundamentals still hold true. TCP/IP is as important as ever, building a properly designed network is a must if you want to have overlays running on it. If you build a house and do a sloppy job with the foundation, what will happen? The same holds true in networking.

I will introduce the concepts of network design. What does a network designer do? What tools are used? What is CAPEX? What is OPEX? What certifications are available? What is important in network design? We will also look at a couple of design scenarios and reason about the impact of our choices. There is always a tradeoff!

If you are interested in network design or just want to tune in to yours truly, follow this link to CLN.

I hope to see you there!

Cisco Live in San Diego – Will You Make It?

“Make it” was one of the first singles released by the the band Aerosmith. Since then these guys have been rocking away for about 40 years. What does this have to do with Cisco Live? Aerosmith will be the band playing at the Customer Appreciation Event (CAE). A good time is pretty much guaranteed. Aerosmith knows how to entertain a crowd.

The CAE will take place at Petco Park, the home of the San Diego Padres. This photo shows the arena in the evening, looks quite spectacular to me.

Cisco Live is much more than just having fun though. If you want to make it in the IT industry, there is a lot to gain by going to Cisco Live. Here are some of my reason why I want to go:

• Stay on top of new technologies – Where is ACI going?
• Dip my toes into other technologies that I find interesting
• Gain deep level knowledge of platforms or features that will benefit me and my customers
• Go to sessions that will aid me on my certification path
• Connect with people!
• Learn a lot while having fun at the same time!
• Learn from the experience of others

When you are Continue reading

STP Notes for CCDE

These are my study notes for CCDE based on “CCIE Routing and Switching v5.0 Official Cert Guide, Volume 1, Fifth Edition” and “Designing Cisco Network Service Architectures (ARCH) Foundation Learning Guide: (CCDP ARCH 642-874), Third Edition“, “INE – Understanding MSTP” and “Spanning Tree Design Guidelines for Cisco NX-OS Software and Virtual PortChannels“. This post is not meant to cover STP and all its aspects, it’s a summary of key concepts and design aspects of running STP.

STP

STP was originally defined in IEEE 802.1D and improvements were defined in amendments to the standard. RSTP was defined in amendment 802.1w and MSTP was defined in 802.1s. The latest 802.1D-2004 standard does not include “legacy STP”, it covers RSTP. MSTP was integrated into 802.1Q-2005 and later revisions.

STP has two types of BPDUs: Configuration BPDUs and Topology Change Notification BPDUs. To handle topology change, there are two flags in the Configuration BPDU: Topology Change Acknowledgment flag and Topology Change flag.

MessageAge is an estimation of the age of BPDU since it was generated by root, root sends it with an age of 0 and other switches increment this Continue reading

Noction Intelligent Routing Platform (IRP) – What is it?

I was contacted by some people at Noction and asked if I was interested in writing about their platform, the Intelligent Routing Platform (IRP). Since it’s a product that uses Border Gateway Protocol (BGP), it peaked my interest. First let’s make the following things clear:

• I am not being paid to write this blog post
• My opinions can’t be bought
• I will only write about a product if it’s something that interests me

BGP is the glue of the Internet (with DNS) and what keeps everything running. BGP is a well designed and scalable protocol which has been around for a long time. It has grown from carrying a few hundred routes to half a million routes. However, there will always be use cases where BGP might not fit your business model.

In Noction’s white paper they define the following as the network’s major challenges:

• Meeting the customer’s demand for 100% uptime
• Facing the low latency requirement
• Achieving reliable data transmission
• Avoiding network congestion and blackouts
• Achieving consistency of throughput
• Keeping bandwidth usage below predefined commit levels
• Reducing the cost and time of network troubleshooting

The product is designed for multihomed networks running BGP. You can’t optimize network flows if Continue reading

Cisco Reveals New Products – The Time of Multigigabit is Here

Wireless networks are becoming faster and faster. With 802.11ac Wave 2, wireless networks will be capable of achieving speeds up to 6.8 Gbps. This creates challenges when connecting APs to switches which normally run Ethernet at 1GE or 10GE. To meet these evolving demands, Cisco has as of today revealed some new products.

Cisco is releasing a new compact switch supporting multigigabit technology, the Cisco Catalyst 3560-CX. The most compelling new features are support for multigigabit interfaces, more power available for PoE, support for 10GE on the uplinks and being able to be deployed as an Instant Access switch. It also support PoE pass through which can help save on long cable runs. The Catalyst 3560-CX supports two multigigabit interfaces.

This device is fanless, so it can be deployed in cubicles to decrease the need for a wiring closet. It also has the support for role based security. Cisco’s goal is to provide for a better working environment, which they call “Next Generation Workspace”.

If you are a technical person, you are probably wonder about the multigigabit ports. IEEE only has 1GE, 10GE and so on. Cisco started the NBASE-T Alliance with Aquantia, Freescale, and Xilinx. Other members Continue reading

QoS Design Notes for CCDE

Trying to get my CCDE studies going again. I’ve finished the End to End QoS Design book (relevant parts) and here are my notes on QoS design.

Basic QoS

Different applications require different treatment, the most important parameters are:

• Delay: The time it takes from the sending endpoint to reach the receiving endpoint
• Jitter: The variation in end to end delay between sequential packets
• Packet loss: The number of packets sent compared to the number of received as a percentage

Characteristics of voice traffic:

• Smooth
• Benign
• Drop sensitive
• Delay sensitive
• UDP priority

One-way requirements for voice:

• Latency ≤ 150 ms
• Jitter ≤ 30 ms
• Loss ≤ 1%
• Bandwidth (30-128Kbps)

Characteristics for video traffic:

• Bursty
• Greedy
• Drop sensitive
• Delay sensitive
• UDP priority

One-way requirements for video:

• Latency ≤ 200-400 ms
• Jitter ≤ 30-50 ms
• Loss ≤ 0.1-1%
• Bandwidth (384Kbps-20+ Mbps)

Characteristics for data traffic:

• Smooth/bursty
• Benign/greedy
• Drop insensitive
• Delay insensitive
• TCP retransmits

Quality of Service (QoS) – Managed unfairness, measured numerically in latency, jitter and packetloss

Quality of Experience (QoE) – End user perception of network performance, subjective and can’t be measured

Tools

Classification and marking tools: Session, or flows, are analyzed to determine what class the packets belong to Continue reading

Unique RD per PE in MPLS VPN for Load Sharing and Faster Convergence

This post describes how load sharing and faster convergence in MPLS VPNs is possible by using an unique RD per VRF per PE. It assumes you are already familiar with MPLS but here is a quick recap.

The Route Distinguisher (RD) is used in MPLS VPNs to create unique routes. With IPv4, an IP address is 32 bits long but several customers may and probably will use the same networks. If CustomerA uses 10.0.0.0/24 and CustomerX also uses 10.0.0.0/24, we must in some way make this route unique to transport it over MPBGP. The RD does exactly this by prepending a 64 bit value and together with the IPv4 address, creating a 96-bit VPNv4 prefix. This is all the RD does, it has nothing to do with the VPN in itself. It is common to create RD consisting of AS_number:VPN_identifier so that a VPN has the same RD on all PEs where it exists.

The Route Target (RT) is what defines the VPN, which routes are imported to the VPN and the topology of the VPN. These are extended communities that are tagged on to the BGP Update and transported over MPBGP.

MPLS uses Continue reading

Book Review – End-to-End QoS Network Design: Quality of Service for Rich-Media & Cloud Networks, Second Edition

As part of my CCDE studies, I needed a good resource on QoS. There have basically been two good books on QoS before, the first edition of End to End Qos Network Design and Qos-Enabled Networks: Tools and Foundations. The first edition of this book is good but very dated, it was released back in 2004. Qos-Enabled Networks is a great book but it’s written to not be vendor specific, so you will not get details on platforms or configuration snippets.

In my opinion, earlier books gave a good foundation to understand QoS concepts but there were too few design cases, they were lacking platform information and not enough examples to be able to act as a reference. Since the first edition of this book, a lot has happened, new products and new Places In the Network (PIN) such as Datacenter, Wireless and to some degree MPLS.

The book is written by Tim Szigeti, Christina Hattingh, Robert Barton and Kenneth Briley Jr. Tim is a long time CCIE, technical leader at Cisco. He is the QoS gury responsible for a lot of the Cisco Validated Designs (CVDs) and a frequent presenter at Cisco Live. Christina is a former Technical Marketing Continue reading