Vince Schuele

Author Archives: Vince Schuele

IP Infusion: EVPN-MPLS first look on GA 6.0

IP Infusion just released OcNOS version 6.0 and the release notes, as well as press release, show a focus on EVPN with an MPLS data plane. Don’t forget EVPN and VxLAN aren’t mutually exclusive, EVPN runs on and was originally designed for a MPLS data plane. I recently discussed this on a podcast EVPN doesn’t need VxLAN if you want to know more on that topic.

Lets take a look at basic EVPN-VPWS and EVPN-VPLS deployment. Since we’re looking at an MPLS data plane we will utilize ISIS-SR for MPLS. We’re utilizing ISIS-SR as it is increasingly replacing LDP and RSVP-TE for label distribution.

IGP and Label Distribution

First let’s look at the IGP setup and label distribution as everything else will be built on top of this. run int lo
interface lo
 ip address
 ip address secondary
 ipv6 address ::1/128
 ipv6 address 2001:db8::1/128
 prefix-sid index 101
 ip router isis UNDERLAY
 ipv6 router isis UNDERLAY

We have to set an index to create the node-sid for this device. In this case we use 101. run segment-routing
  Continue reading

Situational Awareness for Network Migrations

At IP Architechs we perform a lot of network migrations and it is no secret network migrations/ maintenance windows can be one of the most nerve-racking things for engineers, managers, and business leaders for a variety of reasons.

For the engineers the uncertainty might be caused by fear of failure, not being able to predict the outcome due to complexity, rushed on preparation to meet a deadline, or a litany of other reasons.

For managers and business leaders it might be more along the lines of; what happens if this goes wrong, how will this effect my bottom line, are there going to be 1000s of trouble tickets come 8/9am when everyone hits the office, and so on.

The Preparation

We’re going to look at this at the perspective of the engineer throughout. The prep work is probably one of the most important pieces of success. This is where you do many things including but not limited to:

  • building and testing the configuration to be implemented
  • making a rollback plan — this might be something as simple as move a cable and shut an interface or a multistep/multi-device plan
  • know the situation surrounding the window

Lets explore understanding the situation Continue reading

BGP Communities part 4: Active/Active datacenter

If you read part 2 of this series and came out wondering this is great but:

How do I connect to the internet?

Does this breakdown once I need to have connections?

What else do I have to do to manage state?

We’ll set out to answer these questions and show how it works. There are some dependancies such as your provider supporting customer BGP TE communities as laid out in part 3.

This seems to be the elusive grail in enterprise networking that everyone wants but is unsure of where to start. Hopefully, a few of those questions have been answered throughout this series but be sure to understand what you’re getting into and that your team can support it before and after you leave.

The overall topology

We’ve got data center 1 (DC1) and data center 2 (DC2). They each have a connection to an internal router in ASN 60500. A lot of networks I come across have dedicated routers coming out of the DC to terminate internet connections and support full tables. These router usually only pass a default internally. I don’t have the full tables but instead copy the topology and pass a default into the Continue reading

BGP communities part 3: Customer BGP Traffic Engineering communities

If you’ve ever been asked to prioritize one internet connection over another for any variety of reasons, cost, latency, SLA, etc… this is for you.

Often I hear the same tactics to solve this problem:

  • AS-PATH prepending
  • conditional advertisements
  • scripting
  • some other manual process

However, most carriers offer customer BGP TE communities that you can use to influence traffic within their AS, with one notable exception Hurricane Electric. If you’re not sure what a BGP community take a quick look at this post on them first.

Lets explore how to utilize these, where to find them, and how they might give more deterministic path selection than the options laid out above.

BGP Topology

Default behavior with no modification

First to get familiar with the topology and show reachability we’ll leave all settings as “defaults” with no modifications.

ISP-1-RTR-1#traceroute source
Type escape sequence to abort.
Tracing the route to
VRF info: (vrf in name/id, vrf out name/id)
  1 1 msec 1 msec 1 msec
  2 1 msec 0 msec 0 msec
  3 2 msec *  1  Continue reading

Networking Field Day: Service Provider – Arista VPLS

During the first networking field day: service provider one of the big topics was EVPN versus VPLS. Arista has put a lot of work into their EVPN deployment and this has give then a ton of success in the data center. However, a large portion of the provider space, especially last mile providers, rely on VPLS heavily. This naturally led to discussion about Arista VPLS support.

I’m pleased to see that there is now basic support in EOS as of EOS 4.27.2F and more on the roadmap. Hopefully, we’ll see the off ramp, RFC8560, from VPLS to EVPN which was a hot button topic throughout the week.

In the release notes for EOS 4.27.2F it calls our basic VPLS support. So I took a look. Reviewing the new 4.27.2F manual I found support for LDP PWs on RFC4447 which is virtual private wire support. This also appeared to be in EOS 4.26 but not earlier. I might have missed a different call out in the doc, if I did someone please point me in the right direction so I can update this post!

In the meantime lets review how this works:

mpls  Continue reading

App delivery for an improved pizza experience

It’s been a while since we started work on one of our newest projects.  We have been trying to solve a problem in app location.  It all came from the notion that Little Caesars know where my pizza is, so why can’t the network resolve where the app is?    We also thought it would be novel use of Anycast because the app can be anywhere. 

So, what problems specifically have we solved using this design?  Intent based gateways are a signaling mechanism allows the apps to be delivered along with the pizza.  As we can see app Buffalo Wings can reach both the intent based gateway and Fried Pickles using TI-LFA, which strips the fat bits before they reach the gateway.   Our unique caching solution using Tupperware, which are stacked in K8s, allows for the apps to be delivered in a bursty nexthop specific competitive manner.  This has proven to keep the apps warm within the physical layer.

In our example, the Delivery Center Interconnect,  we are doing an east to west Multi Pizza Layered Service that can drop the apps with full BTU into any of the regions.  The apps are Continue reading

BGP communities for traffic steering – part 2: State Management across Data Centers

This post has been a while in the making and follows up on an article about BGP communities that can be found here. Then we followed it up with some more discussion about FW design and place, or lack there of, on this podcast which inspired me to finish up “part 2”.

Anyone who has ever had to run active/active data centers and has come across this problem of how do I manage state?

You can ignore it and prepare yourself for a late night at the worst time.

Take everyone’s word that systems will never have to talk to the a system in a different security zone in the remote DC

Utilize communities and BGP policy to manage state; which we’ll focus on here

One of the biggest reasons we see for stretching a virtual routing and forwarding (vrf) is to move DC to DC flows of the same security zone below FWs. This reduces the load on the firewall and makes for easier rule management. However, it does introduce a state problem.

We’ll be using the smallest EVPN-multisite deployment you’ve ever seen with Nexus 9000v and Fortinet FWs.

Inter vrf intra data center

The first flow we’ll look Continue reading

Interop IP infusion and MikroTik: part 2 – VPLS

VPLS is a pretty common technology in ISPs to either sell layer 2 services or backhaul traffic to a centralized aggregation point to conserve IPv4 space; check out more on that here.

How can I take advantage of segment routing but still deliver the same services? We already looked at how to setup the label switched paths utilizing IP Infusion as a segment routing mapping server in this interop post. Now let’s see how we can deliver a VPLS service over this with mikrotik as a provider edge router.

Delivering a service with a L2VPN

After setting up the IGP and label distribution between the PEs we will start building the L2VPN.

Why a BGP signaled VPLS session instead of LDP signaled VPLS?

In the segment-routing domain there is no LDP running. However, some vendors support static pseudowires or other methods to bring up a targeted LDP session for VPLS. I did some basic testing here couldn’t easily identify the right combination of knobs to make this work. Don’t worry; I’ll come back to it.

BGP signaled VPLS is a standards based technology that both vendors support.

First thing we need to do after having loopback reachability is to build Continue reading

Interop LDP and Segment Routing with IP infusion and MikroTik


During networking field day service provider 1 there was a ton of talk about segment routing (SR) and ethernet virtual private networks (EVPN). One of the biggest questions was “how do we get there?” and while we won’t examine EVPN in this post (it’s coming in a future post don’t worry) we will look at how you can take advantage of SR while still having large portions of LDP in your network.

The team here at IP architechs works on a lot of MikroTik and whitebox gear so we’ll focus on a deployment using MikroTik and IP infusion.

MPLS and IGP setup

The first thing to accomplish is end to end reachability between the provider edge (PE) routers. MikroTik doesn’t support IS-IS so we will have to perform redistribution between the IS-IS segment and the OSPF segment as seen above.

MPLS only requires the /32s of the loopbacks for functionality so redistribution is limited to the /32 loopbacks of the PE routers.

ip prefix-list LDP-PE-LOOPBACKS
 seq 10 permit eq 32
ip prefix-list SR-PE-LOOPBACKS
 seq 10 permit eq 32
route-map REDIS-OSPF-TO-ISIS permit 10
 match ip address prefix-list LDP-PE-LOOPBACKS
route-map REDIS-ISIS-TO-OSPF permit  Continue reading

Migrating from fabricpath to EVPN/VxLAN


Do you have a 3 tier, switched, or vendor proprietary data center design?

Does it rely on spanning tree or proprietary solutions to eliminate spanning tree?

Not sure how to migrate to a new architecture without serious downtime?

If you answered yes to any of these questions then this post is for you. We’ll be looking at deploying an EVPN/VxLAN Data Center fabric and migrating a from a cisco fabricpath environment to the new design.

Although we will be focusing on a fabricpath migration many, if not all, of the principles apply to migrating a 3 tier architecture.

1. Building the new Data Center Fabric
2. Connecting the current fabricpath and new fabric
3. Migrating switched virtual interfaces
4. Migrating various types of physical devices

Building the new Data Center Fabric

The easiest part of designing and building the new fabric is the physical topology. This should be a symmetric topology to easily take advantage of equal cost multipath and add additional switches with ease. This is also known as a spine/leaf or clos topology. The basic idea is leafs connect to spines and spines connect to super spines. A leaf/spine should not connect to another switch of the Continue reading

Utilizing BGP Communities for traffic steering – part 1: Firewalls


I typically spend more time in the enterprise data center than most of our team members and this comes with its own unique set of problems. One discussion that seems to never fail to come up is “where do I put the Firewalls (FWs)?”. That is typically followed by I have a disaster recovery or backup site with FWs there as well. This inevitably leads to a state management problem. Let’s look at how we can utilize BGP to address this problem:

  • what is a BGP standard community
  • BGP best path selection process
  • how to utilize them to steer traffic

This is something most service providers deal with on a daily basis but can be new to an enterprise.

BGP Standard communities

A BGP community is a route attribute that, essentially provides extra information for someone to take action or glean information from the route such as where it came from (location, type, organizational role).

By definition, a community is a 32 bit number that can be included with a route and when utilizing the new community format is displayed as (0-65535):(0-65535). It is recommend to utilize the new community format versus the old community format which is Continue reading