War Stories: Cursed VLANs

I’ve written before about switch ports being permanently disabled. This time it’s something new to me: VLANs that refuse to forward frames.

A Simple Network

The network was pretty straightforward. A pair of firewalls connecting through a pair of switches to a pair of routers:

Cursed VLAN

Sub-interfaces were used on the routers and firewalls, with trunks to the switches. VLAN 100 was used for 100.100.100.0/24, and VLAN 200 was used for 200.200.200.0/24. The switches were configured to pass VLANs 100 & 200.

All was working as expected. All devices could see each other on all VLANs.

Until it stopped

We received reports that we’d lost reachability to Router A’s VLAN 200 sub-interface. After doing some investigation, we could see that Firewall-A could no longer see Router A’s MAC address on G0.200. But everything else was fine – the VLAN 100 interface worked perfectly. So we knew it couldn’t be a physical interface issue.

Hmmm. What’s going on? First instinct: check the switch port configuration. Has anything changed? Nope. VLAN 200 still there, configured as expected. The router & firewall were still tagging frames with VLAN 200. But they couldn’t see each other, and the Continue reading

Secret Sunday – Greg Ferro

Greg FerroJust over a month ago I accepted Etherealmind’s “30 Blogs in 30 Days Challenge”, and this Friday I ‘m pleased to say that I completed the challenge without missing a day. It seems appropriate then that I should use today’s Secret Sunday to give a shout out to the man behind the mission, Greg Ferro (aka Etherealmind).

In some ways it feels like cheating to include Greg because he’s unlikely to be a new discovery for anybody reading my blog, but credit is due where it’s due; you can’t argue that Greg does not give back to the networking community. His Etherealmind website has a huge following and a wealth of content written over a number of years. He’s a co-host of the PacketPushers podcasts, generating hundreds of podcasts dedicated to the networking industry. He has written three books that he has published on LeanPub and has a fourth on the way, co-authored with the venerable Brent Salisbury.

Greg is never short of an opinion, and I suspect that one of the reasons for his large following is that he isn’t afraid to go out there and let you know what he thinks, even if “you” are a vendor, and what Continue reading

Ops Work vs Project Work

There’s a constant tension between delivering new services, and running the existing services well. How do you figure out how to prioritise work between Operations tasks and Project work? Skewing too far either way leads to problems. Maybe the answer is in how we structure Operations tasks?

Definitions

  • Operations work: Dealing with outages, trouble tickets, support requests, etc. System monitoring – reviewing data for capacity planning, and identifying new areas to monitor. Automated repetitive tasks. Patches, upgrades, minor changes to existing services. Accountants would call this work OpEx.
  • Project work: Design, test and deployment of new services. Major upgrades or enhancements to existing services. This is usually classified as CapEx. For some businesses, this work is customer-billable.

What happens when you’re imbalanced?

  • Too much Project work: If you’re flat out deploying new systems (and dealing with the fallout), it’s easy to let Operations work slip. Maybe you don’t get around to automating that log rotation script, or paying attention to the slope of that consumption graph. It’s OK for a while too…things seem to be trucking along. But then you start having outages due to simple things like logs filling directories, or you hit a capacity limit, and there’s a 6-week Continue reading

Network Break 21

IT Talent Shortage and Whiny CIOs, Podcasts Make Money, ACI vs NSX wobbles and Dell busts some moves at its conference.

Author information

Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus.

TwitterFacebookGoogle+LinkedIn

The post Network Break 21 appeared first on Packet Pushers Podcast and was written by Greg Ferro.

Facebook Altoona Network Diagram in 2-D

Facebook recently wrote about the network architecture they are using in their new Altoona data center facility.  If you haven't read through their article yet, it's definitely worth the read.

They have a few diagrams that outline the architecture.  One of them is in 3-D.  3-D diagrams are always more difficult for my brain to conceptualize (maybe it's just me), so I re-drew it in a more typical 2-D fashion.
There aren't details on quantity and speeds of ports for the spine and edge switches, so I represented them similar to how they have them in their diagram.  Assuming at least 144 ports of 40G in each spine switch it would allow the data center to be the home to over 100,000 servers.  For every downward facing port a spine switch has (across the plane), 2300 servers can be added.  That's the amount of servers per pod.  48 servers per rack * 48 racks per pod = 2300 servers.  Based on the switch types being used, this number could be more, but I'm using the characteristics of 48 x 10G ports for host facing ports + 4 x 40G ports for uplinks.

My diagram depicts 8 racks in the POD Continue reading

Infuriating Inconsistent Interfaces; F5 on the stand.

F5 Logo

Ok, it’s another f5 post and if you’re not using f5 you might think this is irrelevant to you. However, I beg you to read on because the issue I’m describing today has a relationship to SDN and network automation, and why they are such a pain to do in so many cases.

f5 SSL Profiles

The day began simply enough: news had broken about the “Poodle” SSLv3 vulnerability, and like the majority of network and server nerds we needed to disable or block SSLv3 as quickly as possible in order to remove that particular attack vector. My job was to look at the f5 load balancers, and to do so I realized that I needed to understand what SSL we had out there, and I’d also need to determine the exact change I would be making.

I wrote a couple of scripts to analyze our f5 configurations, and soon enough I had a spreadsheet showing all the SSL client profiles that were in use on each load balancer. It’s important, at this point, to understand how the f5 configures SSL profiles. Fundamentally, a custom profile inherits all of its settings from a “parent” profile, unless you specifically choose to Continue reading

Infuriating Inconsistent Interfaces; F5 on the stand.

Ok, it’s another f5 post and if you’re not using f5 you might think this is irrelevant to you. However, I beg you to read on because the issue I’m describing today has a relationship to SDN and network automation, … Continue reading →

If you liked this post, please do click through to the source at Infuriating Inconsistent Interfaces; F5 on the stand. and give me a share/like. Thank you!

Reinventing the wheel (or RFC 1925 sect 2.11)

Simon Wardley is another old-timer with low tolerance for people reinventing the broken wheels. I couldn’t resist sharing part of his blog post because it applies equally well to what we’re seeing in the SDN world:

No, I haven't read Gartner's recent research on this subject (I'm not a subscriber) and it seems weird to be reading "research" about stuff you've done in practice a decade ago (sounds familiar). Maybe they've found some magic juice? Experience however dictates that it'll be snake oil […]. I feel like the old car mechanic listening to the kid saying that his magic pill turns water into gas. I'm sure it doesn't ... maybe this time it will ... duh, suckered again.

Meanwhile the academics already talk about SDN 2.0.

Meeting Rules

Years ago a wise engineer gave me these rules for meetings:

  1. Never go into a meeting unless you know what the outcome will be.
  2. Plan to leave the meeting with less work than when you went in.

Stick to those rules, and you’ll do well.

OK, so maybe the second rule’s not so serious, but the first one has a grain of truth. You don’t need to know exactly what the decision should be, but you should be clear about what you want to get decided. If it’s particularly important, you should have already discussed it with the key attendees, and you should know what they’re thinking. You don’t want any surprises.

Too many meetings have no clear purpose, or they can only agree that ‘a decision needs to be made…pending further research.’ Avoid those sorts of meetings. Otherwise it ends up like…well….Every Meeting Ever:

Facebook Altoona Network Diagram in 2-D

Facebook recently wrote about the network architecture they are using in their new Altoona data center facility. If you haven’t read through their article yet, it’s definitely worth the read.

They have a few diagrams that outline the architecture. One of them is in 3-D. 3-D diagrams are always more difficult for my brain to conceptualize (maybe it’s just me), so I re-drew it in a more typical 2-D fashion.
There aren’t details on quantity and speeds of ports for the spine and edge switches, so I represented them similar to how they have them in their diagram. For every downward facing port a spine switch has (across the plane), 2300 servers can be added. That’s the amount of servers per pod. 48 servers per rack * 48 racks per pod = 2300 servers. Based on the switch types being used, this number could be more, but I’m using the characteristics of 48 x 10G ports for host facing ports + 4 x 40G ports for uplinks.

My diagram depicts 8 racks in the POD and 8 switches per plane. These numbers would be 48 in a fully built out network/diagram.

altoona

Feel free to comment and correct anything I may Continue reading

GNS3 – ASAv and XRv and IOU and XEv

I am able to run ASAv and XRv and IOU and XEv on my la;ptop, forming OSPF neighbor relationship between them.

I then pinged each loopback from the ASA and also pinged each loopback from IOU. This test shows:
  • One way broadcast and one way unicast are working - ARP
  • Unicast is working - ICMP
  • Multicast is working - OSPF

All thanks for GNS3 v1.1. Isn't it great?

Here is the topology:


And here is some show commands from the ASA:


GNS3 integration with Virtual box is very useful. Whatever you can run inside Virtualbox, you can connect to each other with endless possibilities.

My system76 laptop is running Ubuntu 14.04, 16GB, i7 and SSDs.

I was using the following resources:

  • gns3-gui
  • gns3-server
  • iouyap
  • dynamips (this is needed even if not using dynamips for IOS)
  • vboxwrapper
  • vpcs (optional, but very handy to test connectivity)
  • Virtualbox integration with GNS3 won't work without virtualbox SDK: http://forum.gns3.net/topic6145.html
  • Set permissions for dynamips and iouap:
    cd /usr/local/bin
    sudo setcap cap_dac_override,cap_net_admin,cap_net_raw+eip dynamips
    sudo setcap cap_net_raw,cap_net_admin+eip iouap
  • CSR1K install: http://herdingpackets.net/2014/02/06/using-the-cisco-csr1000v-in-gns3-with-virtualbox/
  • XRv install: http://www.noshutdown.ma/ios-xrv-step-step-install-gns3-integration/
    Notice the NIC type. It should be MT server.
    Notice Continue reading

    • CCIE Data Center Written Bootcamp :: December 15th – 19th

    We’re excited to announce an upcoming CCIE Data Center Written Bootcamp, beginning December 15th.

    It’s going to be delivered via our custom Online-HD-ILT™ training solution (high definition and interactive), Jason, our resident Data Center expert and CCIE DC instructor, will be delivering this specialized bootcamp. Not only is this course a must for CCIE Data Center written candidates  (Cisco exam ID 350-080), but it’s also perfect for engineers who are also preparing for their CCIE Data Center lab exam, as they will be able to easily transition straight into their lab studies. Also, if you’re looking to re-certify an existing CCIE, but would like a potential road map into another one, this bootcamp will provide you with the capabilities of doing so.

    This course lists at $999, but we’re providing a $500 coupon for individuals who purchase it now – through Monday, November 17th (Please use coupon code: DCW500BC upon checkout).

    Also, our annual (1-year and 2-year) iPeverything™ clients will be permitted to attend this course at absolutely no charge! Simply login, go to the schedule page within your Member’s Area, and register.

    Note: This course will begin at 9 AM EST, and will last 5 to 8 hours per Continue reading

    PlexxiPulse—A Big Week for Plexxi and a New Era in IT

    If you’ve been following Plexxi in the news and on social media, you will see that we announced our new CEO, Rich Napolitano, who comes to us from EMC to continue the company’s expansion and help lead Plexxi into a new era of IT. Rich has been a friend of Plexxi’s for a while; he sat on our Board of Directors before taking on his current role CEO. You can read Rich’s first blog post on why he joined Plexxi here. We also announced this week that Tim Lieto has been named senior vice president of sales and customer service to lead the company’s worldwide sales and channel effort. It sure is an exciting time for Plexxi, and we’re thrilled to have both Rich and Tim on board!

    In this week’s PlexxiTube of the week, Dan Backman explains how Plexxi’s Big Data fabric solution is managed.

    Cisco, Arista disaggregating?

    Jim Duffy wrote an interesting piece in Network World this week questioning whether Cisco and Arista should develop versions of their operating systems for bare metal hardware (similar to Cumulus). The industry shift that is taking place here is actually very simple. Research and development spend reflects where the value and Continue reading

    Cumulus and Ansible

    Simplify_the_Network_with_Ansible This past September Michael DeHaan took part in a joint Ansible and Cumulus Networks webinar on how Ansible and Cumulus can radically simplify the nature of modern IT management, and how well they can play together.

     

    Learn more here

    Watch a quick demo video of how to use Ansible to automate network switches running cumulus linux. 

    Cisco ISR: Enable Features, No Performance Hit?

    cisco-logo

    Last month I visited Interop NYC 2014 as a guest of Tech Field Day Extra! where our group was given a presentation about the new Cisco ISR routers by Matt Bolick, a Technical Marketing Engineer for Cisco.

    The Integrated Service Routers (ISRs) themselves seem pretty feature packed, covering four key areas:

    • Transport independence (DMVPN)
    • Intelligent Path Control (PfR v3)
    • Application Optimization (WAN optimization, ADC and WAAS)
    • Secure Connectivity (Scalable, strong encryption, IPS, web filtering, etc.)

    Rather than reinvent the wheel, Matt explained that the idea was to use existing protocols in a useful new way; in this case in particular to offer secure hybrid transport across MPLS and Internet for private cloud and DC access, probably ultimately moving to just Internet connectivity base on the shift Cisco has seen in how corporations see their branch offices (and specifically how much they want to reduce costs!).

    So far so cool, but I figure you can look up all the specifications and features for yourselves so I won’t bore you with much more of that here. There was something else that tickled me though.

    ISR Performance Figures

    The new routers have some interesting performance claims:

    Cisco ISR: Enable Features, No Performance Hit?

    Last month I visited Interop NYC 2014 as a guest of Tech Field Day Extra! where our group was given a presentation about the new Cisco ISR routers by Matt Bolick, a Technical Marketing Engineer for Cisco. The Integrated Service … Continue reading →

    If you liked this post, please do click through to the source at Cisco ISR: Enable Features, No Performance Hit? and give me a share/like. Thank you!

    4 Inevitable Questions When Joining a Monitoring Group, Pt. 1

    Leon Adato, Technical Product Marketing Manager with SolarWinds is our guest blogger today, with a sponsored post on the topic of alerting. The Four Questions For people who are interested in monitoring, there is a leap that you make when you go from watching systems that YOU care about, to monitoring systems that other people […]

    Author information

    Sponsored Blog Posts

    The Packet Pushers work with our vendors to present a limited number of sponsored blog posts to our community. This is one. If you're a vendor and think you have some blog content you'd like to sponsor, contact us via [email protected].

    The post 4 Inevitable Questions When Joining a Monitoring Group, Pt. 1 appeared first on Packet Pushers Podcast and was written by Sponsored Blog Posts.

    The Roost Stand Discount Code

    Original content from Roger's CCIE Blog Tracking the journey towards getting the ultimate Cisco Certification. The Routing & Switching Lab Exam
    Since my first post on the Roost Stand I have been overwhelmed with comments on my new portable laptop stand the biggest one is where did you buy it from? The great guys at Roost have given me a Roost Stand Discount Code that you can use at the checkout to get a 10% discount. If... [Read More]

    Other pages of interest:

    Post taken from CCIE Blog

    Original post The Roost Stand Discount Code

    1 … 3,728 3,729 3,730 3,731 3,732 … 3,894