What caused today’s Internet hiccup

Like others, you may have noticed some instability and general sluggishness on the Internet today.  In this post we’ll take a closer look at what happened, including some of the BGP details!

At around 8am UTC Internet users on different mailing lists, forums and twitter, reported slow connectivity and intermediate outages.  Examples can be found on the Outages mailing list company support site such as liquidweb and of course on Nanog.

How stable is the Internet
So how do we know if the Internet was really unstable today? One way to look at this is by looking at the outages visible in BGP over the last 12 months.  On average we see outages for about 6,033 unique prefixes per day, affecting on average 1470 unique Autonomous Systems. These numbers are global averages and it’s worth noting that certain networks or geographical areas are more stable than others.

Screen Shot 2014-08-12 at 9.01.06 PM

BGP stability and outages differ per country.

If we look at the number of detected outages by BGPmon today we see outage for 12,563 unique prefixes affecting 2,587unique Autonomous Systems. This is well above the daily average and indeed both the unique prefixes and the unique Autonomous Systems count are Continue reading

Rolling back to Maven 3.0.X on OSX (Homebrew)

The current version of Maven in Homebrew at the time of writing is 3.2.2

This is great... unless one of the plugins in your project doesn't support it and then you have to downgrade :(

Fortunately it's not too painful

:::bash
brew uninstall maven
brew tap homebrew/versions
brew install maven30

@dave-tucker

Cisco 881 or Cisco 881?

There are two versions of the Cisco 881 branch router:
  • Part numbers beginning with CISCO881, which have been end of lifed.
  • Part numbers beginning with C881, which are newly available.
There are a bunch of differences between these models, but it's hard to tell that a difference even exists, let alone what the differences are by looking at the available documentation. I just got my hands on a new C881 for the first time. Here's what I've noticed.

Physical Differences
New C881 on top, old CISCO881 (not wireless - don't believe the stickers) on bottom.
New C881 on top
Twin screw holes on the new C881...

...make the ACS-890-RM-19= work on the C881.
  • The USB port has been moved from one side to the other.
  • The "fake" screw hole on the side is now a threaded hole, which means that the C881 will accept the 891's rack mount hardware.
  • The Fa4 port has moved a bit.
  • The C881 is lead free, which seems to be what prompted all of these gyrations.
    Power Differences
    • We have a power switch!
    • There's no longer a dedicated PoE brick.
    • There's still a required internal PoE module, and it's got a different part number.
    Licensing Differences

    Tipping Point 2.0

    Nine months ago, I wrote about how advances in silicon designs and technologies were going to create a product set that will democratize the networking components in modern data centers. Specifically, that the Trident II family of products will perform the role that Xeon did on the compute side and provide the fulcrum on which open networking OSes would flip the industry.

    Today, I am happy to add to that story and talk a little about the secondary effects that that tipping point has set in motion. We had set about to make the networking space an open, agile and innovation-laden environment akin to the compute space, but we are finding a tremendous appetite for the story to be moved further forward and have networking and compute be treated as complete equals. The drive to manage compute and networking in a harmonious way, in the way that a bus and CPUs operate inside a single box, will have a familiar lynchpin – x86 processors.

    Let’s look at a little historical context around the progression of CPUs that sit inside data center switching systems. Traditionally, the CPU that sat inside a networking box operated its control plane. The calculus used to Continue reading

    Maybe MU-MIMO Matters

    Wireless

    As 802.11ac becomes more widely deployed in environments I find myself looking to the next wave and the promise it brings.  802.11ac Wave 1 for me really isn’t that groundbreaking.  It’s an incremental improvement on 802.11n.  Wave 1 really only serves to wake up the manufacturers to the fact that 5 GHz radios are needed on devices now.  The real interesting stuff comes in Wave 2.  Wider channels, more spatial streams, and a host of other improvements are on the way.  But the most important one for me is MU-MIMO.

    Me Mi Mo Mum

    Multi-user Multiple-Input Multiple-Output (MU-MIMO) is a huge upgrade over the MIMO specification in 802.11n.  MIMO allowed access points to multiplex signals on different channels into one data stream.  It accomplished this via Spatial Division Multiplexing (SDM).  This means that more antennas on an access point are a very good thing.  It increases the throughput above and beyond what could be accomplished with just a single antenna.  But it does have a drawback.

    Single-user MIMO can only talk to one client at a time.  All the work necessary to multiplex those data streams require the full attention of a single access point for Continue reading

    Thwarting BGP Route Hijacking with SDN as a Catalyst

    Thwarting BGP Route Hijacking with SDN as a Catalyst


    by Cengiz Alaettinoglu, CTO - August 12, 2014

    Following up on my last post about the security vulnerabilities in BGP, the IETF has taken two efforts to fix them. Back in 1995, the Routing Policy System Working Group was formed (I have chaired this working group, and many in the community, including folks from service providers and address registry operators, contributed). We have standardized a language called Routing Policy Specification Language (RPSL[ref]), and a security model (RP-SEC [ref]).

    Network operators, both service providers and enterprises, would register their authorized routes (by chain of trust starting from the Internet Assigned Numbers Authority), and the neighbor ASs they pass these routes to. Given the state of the art in 1994, the security credentials (authentication as well as authorization) would be checked at the time of registration. We then wrote a tool that read these validated policy specifications and generated router configurations that would be immune to these kinds of attacks. Unfortunately, RPSL adoption has been low (more on this later).

    IETF recently took another effort in its Secure Inter-Domain Routing Working Group (SIDR). The technology developed there can check the security credentials in-band Continue reading

    The Accumulated IGP Metric Attribute for BGP

    This is an interesting draft which can ensure better paths are chosen in certain corner cases. Before this draft, BGP was able to redistribute the IGP metric as a MED value into BGP. The issue with MED is that it’s very low on the BGP best path algorithm. Note that Cisco/Brocade consider weight as primary, […]

    VMworld 2014 Networking and Security Session Guide

    At last year’s show, we introduced you to VMware NSX, and presented a vision for how network virtualization will fundamentally change data center networking. We focused a lot on what NSX is, what it does, and why you should start planning to virtualize your network.

    This year, we’re still focused on the basics. We have a lot of content that will help those of you who are new to network virtualization and NSX start to establish a base. But of course, we have a whole year of selling NSX under our belt. And we want to share that experience with you in a VMworld program that will take you, and NSX, to the next level.

    Security and network micro-segmentation?  We’ve got it covered.  Customer deployment stories? You bet. Partners with real GA solutions, solving real-world problems? They are on the agenda.

    Take a pass through the list below, and then check out the schedule builder on VMworld.com to organize your week.

    We think the #NSXninjas will be out in full force at VMworld. Are you one?  We hope so!

    Monday August 25, 2014

    Networking Sessions

    NET1846

    Introduction to NSX

    11:00 – 12:00 PM

    NET1214

    NSX Continue reading

    Balanced Buffer Design for Mission-Critical Cloud Networks

    Leading customers and researchers in cloud and data center networking have been promoting the importance of understanding the impact of TCP/IP flow and congestion control, speed mismatch and adequate buffering for many decades. The problem space has not changed during this time, but the increase in the rates of speed by 100X and in storage capacity by 1000X have aggravated the problem of reliable performance under load for data intensive content and for storage applications, in particular. One Arista fan summed it up best by saying:

    “Basically the numbers have changed by order of magnitude, but the problem is the same!”

    Poor performance and inadequate buffering in a demanding network is a painful reminder that buffering, flow control, and congestion management must be properly designed. TCP/IP was not inherently built for rate-fairness, and packets are intentionally dropped (yes, only window fairness is possible). Yet the effect of these drops can be multiplicative given major speed mismatches of 10-100X inside the data center. In the past, QoS and rate metering were adequate. However, at multi-gigabit and terabit speeds and particularly as more storage moves from Fiber Channel (with buffer credits) to Ethernet, packet loss gets more acute.

    Benefits of Balanced Continue reading

    The Canadian Bitcoin Hijack

    A few days ago researchers at Dell SecureWorks published the details of an attacker repeatedly hijacking BGP prefixes for numerous large providers such as Amazon, OVH, Digital Ocean, LeaseWeb, Alibaba and more. The goal of the operation was to intercept data between Bitcoin miners and Bitcoin mining pools. They estimated that $83,000 was made with this attack in just four months. The original post has many of details which we won’t repeat here, instead will take a closer look at  the BGP details of this specific attack.

    Attack details
    Our friends at Dell SecureWorks decided not to name the network from which the hijacks originated. As a result we won’t name the exact Autonomous System either, instead we will suffice by saying that the originator of this hijack is a network operating in Eastern Canada.

    Initial experiment
    BGPmon detected the first hijack by this Canadian Autonomous System on October 8th 2013. For about 14 minutes a more specific /24 prefix for a Palestinian network was hijacked. Looking at geographical scope of the announcements and the probes that saw this route, we believe that in this case the route was only announced over the Toronto Internet Exchange.

    Bitcoin hijack
    On Feb Continue reading

    Let’s Connect at VMworld 2014

    I’ll be at VMworld in a couple of weeks. If you’re a vendor that would like to chat, please schedule me. I’d be happy to meet. If you’re a fellow IT engineer, I’d be happy to meet up as well. I’ll be hanging with folks from Tech Field Day, as well as Chris […]

    Downloads

    Here is a repository of Wi-Fi related documents and resources that WLAN administrators will find useful.

    If you have a Wi-Fi related document, tool, or resource that you would like included on this list, please contact me for inclusion! My contact info is listed on the right column of this website.

    Revolution Wi-Fi Downloads:

    Design+for+Capacity.png

    Designing WLANs for Capacity

    35 presentation slides, PDF format.

    This presentation covers an approach and methodology to integrating WLAN capacity planning into the WLAN design process to allow network engineers to effectively meet growing capacity demands by clients on wireless networks. It defines what capacity means for a WLAN, what factors determine capacity, provides an approach to designing for capacity, and where capacity planning should be integrated into the overall WLAN design process.

    Effective capacity planning is required for all WLANs, not just high-density environments.

    This information was presented at the WLAN Professionals Conference (2014).

    You can also

    watch the presentation video

    and download the

    capacity planning worksheets

    to help calculate capacity needs, which helps simplify the process and step the user through each step.

    Wi-Fi+SSID+Overhead+Calculator.png

    Wi-Fi SSID Overhead Calculator

    Excel Spreadsheet format.

    This tool allows WLAN administrators to assess the network performance impact that multiple SSIDs Continue reading

    SSID Overhead Calculator

    One of the most commonly cited best practices among Wi-Fi professionals is to the limit the number of SSIDs you have configured on your WLAN in order to reduce the amount of overhead on the network and to maintain high performance. But there is not a lot of public data out there to really drive home this point when explaining it to another engineer, management, or a customer. Simply telling someone that they shouldn't create more than 'X' number of SSIDs isn't very convincing.

    Therefore, I've created a visual tool to help you explain WHY too many SSIDs is a bad thing:

    The Wi-Fi SSID Overhead Calculator
    (Click Image to Download)
    Wi-Fi SSID Overhead Calculator

    This tool calculates the percentage of airtime used by 802.11 beacon frames based on the following variables:
    1. Beacon Data Rate - beacon frames are sent at the lowest Basic / Mandatory data rate configured on the WLAN. Beacons must be sent at a "legacy" data rate, meaning only 802.11a/b/g rates. Select the beacon data rate from the drop-down menu within the tool.
    2. Beacon Frame Size - beacon frames can vary in size based on the version of the 802.11 standard implemented (802. Continue reading

    Show 200 – State of the Pushers

    With 2.5 Million downloads over 4 years and more 250 shows, Greg and Ethan talk honestly and openly about the future of Packet Pushers, the increasing impact on our personal lives and the choices we face in the months ahead. What few people understand is that producing the Packet Pushers podcasts takes a lot of […]

    Author information

    Greg Ferro

    Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

    He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus.

    The post Show 200 – State of the Pushers appeared first on Packet Pushers Podcast and was written by Greg Ferro.

    Security policies on vSwitch/dvSwitch

    As described on previous posts both vSwitch and dvSwitch can enforce networking through three policies: Option Default on vSwitch dvSwitch PortGroup Promiscuous mode  Reject Reject MAC address changes  Accept Reject Forged transmits  Accept Reject Let’s describe what each policy can prevent and cannot. Promiscuous mode The promiscuous mode allows a VM to put a vNIC […]
    (Visited 195 times since 2013-06-04, 1 visits today)

    Segment Routing on IOS-XR

    Cisco has released some support for segment-routing on IOS-XR 5.2.0 so what better time to lab it up. I’ve got four IOS-XRv boxes running 5.2.0: RP/0/0/CPU0:XR1#sh ver | include XR Cisco IOS XR Software, Version 5.2.0[Default] Currently IS-IS is the only protocol with support in XR. There are drafts to get this working in both […]

    What is Metadata and Why Should I Care?

    August 2014 is proving yet again to be an amusing month in the Australian political scene, and in this case the source of the amusement was watching a number of Australian politicians fumble around the topic of digital surveillance and proposed legislation relating to data retention measures.

    VMware NSX Customer Story: Colt Decreases Data Center Networking Complexity

    Adoption of network virtualization and SDN technologies from VMware and Arista Networks simplifies cloud infrastructure and enables automation to reduce timescales of cloud and network service provisioning

    colt_logo_l_cmyk

    Offering the largest enterprise-class cloud footprint in Europe, Colt, an established leader in delivering integrated network, data center, voice and IT services, has implemented software-  defined networking [SDN] and network virtualization to simplify how its managed IT and cloud-based networking environment is deployed, managed and scaled throughout its data centers.

    Following an extensive review, Colt selected Arista to provide high speed 10 and 40 gigabit Ethernet cloud-centric switches as an underlay network fabric and VMware NSX™ network virtualization to deliver a fully decoupled software network overlay.

    SDN paves the way for automated cloud service delivery

    The shift to SDN will provide a flexible, scalable, efficient and cost effective way to support the delivery of Colt’s managed IT services, including cloud based services. This makes Colt one of the first service providers in Europe to adopt SDN in a production environment to remove  automate cloud service delivery.

    As a result of deploying a new network architecture based on Arista and VMware networking technologies, the time for Colt to add, change or modify services will Continue reading

    Using LISP for IPv6 tunnelling.

    In this post I would like to show how its possible to use a fairly new protocol, LISP, to interconnect IPv6 islands over an IPv4 backbone/core network.

    LISP stands for Locator ID Seperation Protocol. As the name suggest, its actually meant to decouple location from identity. This means it can be used for such cool things as mobility, being VM’s or a mobile data connection.

    However another aspect of using LISP involves its tunneling mechanism. This is what I will be using in my example to provide the IPv6 islands the ability to communicate over the IPv4-only backbone.

    There is alot of terminology involved with LISP, but i will only use some of them here for clarity. If you want to know more about LISP, a good place to start is http://lisp.cisco.com.

    The topology i will be using is a modified version of one presented in a Cisco Live presentation called “BRKRST-3046 – Advanced LISP – Whats in it for me?”. I encourage you to view this as well for more information.

    Here is the topology:

    LISP-IPv6-Topology

    Some background information about the setup. Both Site 1 and Site 2 are using EIGRP as the IGP. Both IPv4 and Continue reading