War Stories: Gratuitous ARP and VRRP
Continuing our theme of ARP-related war stories, here’s another ARP/switching behaviour I’ve come across. This particular problem didn’t result in any outages, but the network wasn’t working as well as it should have, and started flooding frames unexpectedly. Here’s what was going on:
The Network
Breaking the network down to its simplest level, it looked like this:
The two routers were a VRRP pair. Router-A was 100.100.100 .11, Router-B was 100.100.100.12, and the virtual IP was 100.100.100.1. These acted as a default gateway for the client LAN. PCs connected to the client LAN got their network configuration from DHCP, and set their default gateway to 100.100.100.1. Using this, they were able to get access to resources behind the routers, such as Server-1 at 200.200.200.200. All worked well.
Obviously there was a lot more to the network than what I’ve shown here, but it’s not important.
The Issue
I said it was working well – so what was wrong? One day I was using Wireshark to diagnose a network issue between PC-A and Server-1. I ran Wireshark on PC-A, with a capture filter of “host 200.200.200.200″. The packet flow Continue reading
Taking the Bull by the Horns
Over the past few years, I’ve had the opportunity to work with best and the brightest in the industry. The reach started with my co-workers, partners, and vendors, but gradually expanded due to the likes of maintaining a blog and occasionally being on Twitter. In a recent exchange with someone who gave me a massive pivot and jump start in my career almost 10 years ago, it reminded me of a presentation this same person gave back then.
One of the key themes of this presentation was “Intrinsic Motivation.” This was the first time I had ever heard the phrase – the speaker talked about one’s inner desire and self-motivation as the reason for wanting, learning, and doing. It’s a feeling that is hard to describe, but I remember thinking during the presentation, “Hmm, I think I may have that.” It’s not something many of us talk about, but those that have it can often see it or sense it in others. On the surface, it could be called a passion. It could be a hobby you love or when work starts to overlap as your hobby. Maybe it’s just OCD coupled Continue reading
Podcast Spotlight — Software Gone Wild
Today’s podcast spotlight goes to Software Gone Wild. This is a newer podcast hosted by our friend Ivan Pepelnjak. The topics are focused on the growing pains the networking industry is experiencing and various forms of automation that are attempting to solve them. This includes various aspects of SDN, NFV and how others are using technology to deliver bigger/better/faster solutions.
Recent episodes include Network Automation @ Spotify and The F-Script with my good friend John Herbert.
Links
- Software Gone Wild
- Follow Ivan Pepelnjak (@ioshints) on twitter
Disclaimer: I have no affiliation with the Software Gone Wild podcast or any organization linked to, represented in or derived from content found in this article. This article represents my own opinions and may not be that of my employer.
The post Podcast Spotlight — Software Gone Wild appeared first on PacketU.
Refactoring Code: Fun With REST and JSON
Recently I’ve been writing some automation code for Atlassian’s JIRA project management / bug tracking product. Think of JIRA as a generic AGILE-aware tool that can provide everything from project tracking all the way down to task tracking, trouble ticketing … Continue reading
If you liked this post, please do click through to the source at Refactoring Code: Fun With REST and JSON and give me a share/like. Thank you!
What Happens When 20 Programs Poll The Network?
Packetpushers show 198 was a great episode about Network Automation. At one point, Greg asks:
“What happens when you’ve got 20 apps polling one device?”
Well, you might hit the same problem I did:
SECURITY-SSHD-6-INFO_GENERAL : Incoming SSH session rate limit exceeded
I have some Python scripts that poll performance and configuration data from a couple of ASR9Ks, and I was getting some gaps in my data. The scripts run on different polling cycles (some hourly, some every 15 minutes, etc). It wasn’t consistent, but now and then my script would fail to collect any data.
I dug into it, and found that I was hitting the default SSH rate limit of 60 per minute, calculated as 1 per second. Because I couldn’t control the exact scheduling of when my polls ran, I inserted a short random wait timer into some of them. That helped, and I had fewer failures, but it still wasn’t quite right.
So I used the command “ssh server rate-limit 120″ to allow 2 SSH connections per second. That has helped, and now I’m not getting any failures.
But it won’t be pretty if I do have 20 different apps all trying to poll at once.
(Yes, I know, I should Continue reading
Network Break 13
We have renamed the show to "The Network Break"Network Break 13
We have renamed the show to "The Network Break"
Author information
The post Network Break 13 appeared first on Packet Pushers Podcast and was written by Greg Ferro.
Network Break 13
We have renamed the show to "The Network Break"
The post Network Break 13 appeared first on Packet Pushers.
HP VSR1001 Virtual Services Router on GNS3
HP VSR is a Comware 7 router software application for a server which provides the same functionality as a physical router. Installed on either VMware or KVM virtual machine it offers routing, Firewall, IPSec, and MPLS VPN security services.
The tutorial gives you some ideas how to install HP VSR1000 (Virtual Service Router) running Comware 7 OS on Qemu disk and connects Qemu appliance to GNS3.
They are HP VSR1001, VSR1004 and VSR1008 models available for download. Differences between models are explained here. As the VSR1001 model has the lowest RAM requirements comparing to other models and we do not have to concern the forwarding performance, VSR 1001 demo ISO image is our choice. The demo is full featured, performance limited and requiring no license and with no expiration date.
HP VSR1001 Minimum Hardware Requirements
- CPU: 2.0 GHz
- Memory: 1 GB
- Disk space: 8 GB
- Network interfaces: 2 virtual NICs, E1000 and VirtIO virtual NICs are recommended, maximum 16 NICs supported
1. Download HP VSR1001 Virtual Services Router
Navigate to the Download page here
Picture 1 - HP VSR1001 Virtual Services Router Download Page
Click on the button >> on he right, beside the padlock icon. Either sign in with your HP Passport account or Continue reading
CCIE Collaboration Update – Racks and Videos
Update 1: CCIE Collaboration Racks are now available for rent. Sign in to your members account and click on Rack Rentals in the left navigation. Please note that these racks are currently in general beta release, meaning they are available to everyone, but if you happen to find a bug or an issue with the scheduler, rack control page, or rack equipment, please start a support case and kindly let us know about it so that we may remediate it quickly. We’ve had a number of closed beta testers and their tests have all gone very well. We should be out of general beta release within a few weeks. Full instructions on how to use our racks can be found in our new Collaboration Rack Rental Guide, and I will be releasing a few follow-up videos later today with links inside the guide, to further demo things and walk you through how to use these racks, including the use our new rack control panel.
Please note that while we do certainly still support L2VPN for connecting your phones directly, as well as SSL VPN for server access, we also provide support now for 100% VPN-less connectivity with only Continue reading
No turning back: Russia activates Crimean cable
| The Crimean peninsula depends critically on the Ukrainian mainland for infrastructure services: power, water, and Internet. That has begun to change in the last few days, as Crimean ISPs began receiving their first Internet services over the newly constructed Kerch Strait Cable, linking Crimea with the Russian mainland. The message: there is no turning back now in the process of infrastructure consolidation. |
|
| It’s a symbolic step that’s been months in the making. Following Russia’s annexation of Crimea from Ukraine in March, Prime Minister Dmitry Medvedev ordered the immediate construction of a new submarine cable across the Kerch Strait, one that would connect mainland Russia to the peninsula. |
At Medvedev’s direction, Russian state-owned telecommunications company Rostelecom quickly constructed a submarine cable across the Kerch Strait at a cost of 400-900 million rubles (11-25 million US dollars). On April 25th, Rostelecom announced that the cable was completed.
But laying a short cable through shallow littoral waters is simple work, compared to the process of convincing Crimea’s ISPs to accept Internet service — any Internet service — from a Russian carrier. April passed, and then May, and June. We knew that when the Continue reading
SolarWinds NPM 11 – Now Application Aware
I had a chance last week to speak with a couple of folks at Solarwinds about the release of their Network Performance Monitor (NPM) 11 product, which is being announced today. I don’t cover network management products too often, but Solarwinds … Continue reading
If you liked this post, please do click through to the source at SolarWinds NPM 11 – Now Application Aware and give me a share/like. Thank you!
Fun With Optics
I recently had a deployment where we needed to connect Cisco 6500s to Juniper MX960s.
There was a lot of confusion surrounding what fibre and optic modules needed to be used, so I’m documenting the initial state and the successful state here.
Initially, the Junipers had the following model numbers used for the optics:
XFP-10G-LR (identified using a “show chassis hardware”)
The 6500s:
10Gbase-SR
Interestingly, one of the links came up between the devices, while one did not. I tried swapping the cable (Single Mode) for Multimode for the non-working link, and that did not fix the issue. We then swapped the Cisco optic for an LR optic and used Single Mode fibre which did the trick.
Confusingly for someone from a Cisco background like me, the Juniper optic was labelled XFP-10G-L-OC192-SR1. I assumed the SR stood for short reach/range (it does) and tried to use Multimode fibre, as you would with Cisco SR optics. However, it turns out that the Juniper SR optic in this case used Single Mode fibre as its interface.
It is documented in this handy link here, which turned up with some Googling.
The confusion stems due to the reference to Short Reach differing between Continue reading
The New CCNP – Combining Exams
The new CCNP RS was just released. The last day to test with the old exams is
January 29, 2015.
What is usually seen is that people start to panic, they want to complete the
old exams before they are removed. There is no reason to panic though, you can
mix and match the old exams and the new exams. If you have taken the old
ROUTE and SWITCH, you can take the new TSHOOT and become a CCNP. If you have
the old SWITCH, you can take the new ROUTE and TSHOOT and become a CCNP.
All the valid combinations are available through a comparison tool from Cisco.
Which exams should you take? This depends on how far you are into your studies
and what your future plans are. If you plan to take the CCIE, the new ROUTE looks like
a good stepping stone to me. If you want to finish as quickly as possible, then take
the old exams. As mentioned above, if you don’t complete all three in time, you can take
one of the new ones to round off the CCNP.
Good luck to all the CCNP candidates out there!
DDoS mitigation with Cumulus Linux
 |
| Figure 1: Real-time SDN Analytics for DDoS mitigation |
This article builds on the test setup described in RESTful control of Cumulus Linux ACLs in order to implement the ONS 2014 SDN Idol winning distributed denial of service (DDoS) mitigation solution - Real-time SDN Analytics for DDoS mitigation.
The following sFlow-RT application implements basic DDoS mitigation functionality:
include('extras/json2.js');
// Define large flow as greater than 100Mbits/sec for 1 second or longer
var bytes_per_second = 100000000/8;
var duration_seconds = 1;
var id = 0;
var controls = {};
setFlow('udp_target',
{keys:'ipdestination,udpsourceport', value:'bytes',
filter:'direction=egress', t:duration_seconds}
);
setThreshold('attack',
{metric:'udp_target', value:bytes_per_second, byFlow:true, timeout:4,
filter:{ifspeed:[1000000000]}}
);
setEventHandler(function(evt) {
if(controls[evt.flowKey]) return;
var rulename = 'ddos' + id++;
var keys = evt.flowKey.split(',');
var acl = [
'[iptables]',
'# block UDP reflection attack',
'-A FORWARD --in-interface swp+ -d ' + keys[0]
+ ' -p udp --sport ' + keys[1] + ' -j DROP'
];
http('http://'+evt.agent+':8080/acl/'+rulename,
'put','application/json',JSON.stringify(acl));
controls[evt.flowKey] = {
agent:evt.agent,
dataSource:evt.dataSource,
rulename:rulename,
Continue reading
CCNP Exam update coming soon.
Looks like the Cisco Certification team has been busy lately, earlier this year the CCNP: Security track got an update and recently an update to the CCNP: Route/Switch was just announced. Before you get too worried if you are currently studying for the current exams, you have until January 2015 before the current exams get […]
ScienceLogic Global Network Manager
ScienceLogic 7.5 includes many enhancements and new features. One I’m interested in is “Global Manager” which can be used to massively scale out the ScienceLogic architecture. Here’s some more detail on why ScienceLogic introduced this feature, and what it does.
Problem: A Single Database
I’ve talked before about the ScienceLogic architecture, and noted that the Database can be a bottleneck:
You’ll notice that all the variations only ever have one “active” database at any one time. All the processing is done on this system, with the results replicated to the other databases. You can scale out your Collectors or User Interface by adding more servers – but you can’t scale out the core database. Right now you have to scale up the database – ie. allocate more RAM/CPU/IOPS. This gets around the performance bottlenecks, but comes at a cost.
In this diagram, we can see the database is at the heart of everything. We can have HA & DR options for it, but there is only ever one active DB:
Distributed Architecture – click for larger
We can have multiple web interfaces, but they all query the same database.
Solution: More Databases!
The new Global Manager option from Continue reading
27 – Bis – Path Optimisation with ASA cluster stretched across long distances – Part 2
How can we talk about security service extension across multiple locations without elaborating on path optimisation ? 
Path Optimization with ASA Cluster stretched across long Distances
In the previous post, 27 – Active/Active Firewall spanned across multiple sites – Part 1, we demonstrated the integration of ASA clustering in a DCI environment.
We discussed the need to maintain the active sessions stateful while the machines migrate to a new location. However, we see that, after the move, the original DC still receives new requests from outside, prior to sending them throughout the broadcast domain (via the extended layer 2), reaching the final destination endpoint in a distant location. This is the expected behavior and is due to the fact that the same IP broadcast domain is extended across all sites of concern. Hence the IP network (WAN) is natively not aware of the physical location of the end-node. The routing is the best path at the lowest cost via the most specific route. However, that behavior requires the requested workflow to “ping-pong” from site to site, adding pointless latency that may have some performance impact on applications distributed across long distances.
With the increasing demand for dynamic workload mobility Continue reading
The Pain of Licensing
Frequent readers of my blog and Twitter stream may have noticed that I have a special loathing in my heart for licensing. I’ve been subjected to some of the craziest runarounds because of licensing departments. I’ve had to yell over the phone to get something taken care of. I’ve had to produce paperwork so old it was yellowed at the edges. Why does this have to be so hard?
Licensing is a feature tracking mechanism. Manufacturers want to know what features you are using. It comes back to tracking research and development. A lot of time and effort goes into making the parts and pieces of a product. Many different departments put work into something before it goes out the door. Vendors need a way to track how popular a given feature might be to customers. This allows them to know where to allocate budgets for the development of said features.
Some things are considered essential. These core pieces are usually allocated to a team that gets the right funding no matter what. Or the features are so mature that there really isn’t much that can be done to drive additional revenue from them. When’s the Continue reading
A Customer Perspective: VMware NSX, Micro-Segmentation & Next-Generation Security
VMware NSX and Palo Alto Networks are transforming the data center by combining the 
fast provisioning of network and security services with next-generation security protection for East-West traffic. At VMworld, John Spiegel, Global IS Communications Manager for Columbia Sportswear will take the stage to discuss their architecture, their micro-segmentation use case and their experience. This is session SEC1977 taking place on Tuesday, Aug 26, 2:30-3:30 p.m.
Micro-segmentation is quickly emerging as one of the primary drivers for the adoption of NSX. Below, John shares Columbia’s security journey ahead of VMworld
+++++++++++++++++++++++++++++++++++++++
When I started at Columbia, we were about a $500 million company. Now we’re closing in on $2 billion and hoping to get to $3 billion rather quickly. So as you can imagine, our IT infrastructure has to scale with the business. In 2009, we embarked on a huge project to add a redundant data center for disaster recovery. As part of the project, we partnered with VMware and quickly created a nearly 100% virtualized datacenter. It was a huge success. But something was missing; a security solution that matched our virtualized data center. There just wasn’t a great way to insert security in order to Continue reading