The Network as a Complex Distributed System

Through http://blog.ipspace.net I landed on this article on acm.org discussing the complexity of distributed systems. Through some good examples, George Neville-Neil makes it clear that creating and scaling distributed systems is very complex and “any one that tells you it is easy is either drunk or lying, and possibly both”.

Networks are of course inherently distributed systems. Most everyone that has managed a good sized network before knows that like the example in the article, minor changes in traffic or connectivity can have huge implications on the overall performance of a network. In my time supporting some very large networks I have seen huge chain reactions of events based on what appear to be some minor issues.

Very few networks are extensively modeled before they are implemented. Manufacturers of machines, cars and many other things go through extensive modeling to understand the behaviors of what they created and their design choices. Using modeling they will look at all possible inputs and outputs, conditions, failure scenarios and anything else we can think of to see how their product behaves.

There are few if any true modeling tools for networks. We build networks with extensive distributed protocols to control connectivity Continue reading

Performance Tests and Out-of-Box Performance

Simonp made a perfectly valid point in a comment to my latest OVS blog post:

Obviously the page you're referring to is a quick-and-dirty benchmark. If you wanted the optimal numbers, you would have to tune quite a few parameters just like for hardware benchmarks (sysctl kernel parameters, Jumbo frames, ...).

While he’s absolutely right, this is not the performance data a typical user should be looking for.

Read more ...

InfluxDB and Grafana

Cluster performance metrics describes how to use sFlow-RT to calculate metrics and post them to Graphite. This article will describe how to use sFlow with the InfluxDB time series database and Grafana dashboard builder.

The diagram shows the measurement pipeline. Standard sFlow measurements from hosts, hypervisors, virtual machines, containers, load balancers, web servers and network switches stream to the sFlow-RT real-time analytics engine. Over 40 vendors implement the sFlow standard and compatible products are listed on sFlow.org. The open source Host sFlow agent exports standard sFlow metrics from hosts. For additional background, the Velocity conference talk provides an introduction to sFlow and case study from a large social networking site.
It is possible to simply convert the raw sFlow metrics into InfluxDB metrics. The sflow2graphite.pl script provides an example that can be modified to support InfluxDB's native format, or used unmodified with the InfluxDB Graphite input plugin. However, there are scaleability advantages to placing the sFlow-RT analytics engine in front of the time series database. For example, in large scale cloud environments the metrics for each member of a dynamic pool isn't necessarily worth trending since virtual machines are frequently added and removed. Instead, sFlow-RT tracks all the Continue reading

RPKI: BGP Security Hammpered by a Legal Agreement

Resource Public Key Infrastructure (RPKI) is a relatively new standard for establishing BGP route origination. I wrote a brief introductory article here. Apologies  for the self-promotion, but rather than rehash the basics here, I raise another issue that needs community attention: ARIN’s Relying Party Agreement (RPA: PDF link). Having said that, some basics are needed. […]

Author information

Andrew Gallo

Senior Information Systems Engineer

Andrew Gallo is a Washington, DC based Senior Information Systems Engineer
and Network Architect, responsible for design and implementation of the
enterprise network for a large university.

Areas of specialization include the University's wide area connections,
including a 150 kilometer DWDM ring, designing a multicampus routing
policy, and business continuity planning for two online datacenters.

Andrew started during the internet upswing of the mid to late 90s
installing and terminating fiber. As his career progressed, he has had
experience with technologies from FDDI to ATM, and all speeds of Ethernet,
including a recent deployment of several metro area 100Gbps circuits.

Focusing not only on data networks, Andrew has experience in traditional
TDM voice, VoIP, and real-time, unified collaboration technologies.

Areas of interest include optical transport, network virtualization and
software defined networking, and network science and graph Continue reading

Will I Be Blogging Again?

Dan wrote in with a question: Hey, I like your site as well… are you going to be doing any more posts? It just seems odd that your last blog post was the analyzation of the site and how it portends to your future work. anyway, good luck I know it's tacky to write a blog post about how you're not writing enough blog posts… but here goes.

Pioneers vs. Protectors in Cloud Networking Innovation

The innovation of hundreds of startup companies created the Internet, and the Internet has changed the world. Innovation continues to have a dramatic impact on networking in recent years. These new developments have changed the way applications, workloads and networks interact. Having been involved in this industry for more than three decades, I have witnessed and been part of these transformations from the 1980s to the 2015 era. Each phase of innovation has been characterized by new companies and entrants, as depicted below:

PHASES OF NETWORK INNOVATION
Epoch Vendors Network Technologies Trends
First
1980–1995		 AT&T, Sun, 3Com, NET, Proteon,
UB, BBN, DEC, IBM		 ARPANET, Circuits, Hubs,
SNA, Ethernet, Token
Ring, Routers		 Terminal-Mainframes and
Minis, Channel Attach
Second
1995–2010		 Cisco, Juniper, Nortel/Bay,
Alcatel, Lucent, Avaya		 Switching, Multiprotocol
Routing, LAN-WAN,
TCP/IP		 PC, Client-Server, Web,
North-South traffic
Third
2010–present		 Arista, VMware, Facebook, Microsoft,
Splunk, Red Hat, Palo Alto, Aruba,
many others		 The SDN Era of Open,
Programmable
Networking,
DevOps meets NetOps,
Universal Cloud Networks		 Mobile Virtual Workloads
and Workflows,
Big Data,
Hyperscale Web,
Virtual Machines /
Containers

Traits of a Pioneering Innovator vs. Protector

Dominant companies often fall by the wayside when they do not anticipate and react to clear market trends as Continue reading

Ansible on AWS: Free Best Practices Webinar on December 17th!

Dualspark

We'd like to invite you to a free webinar on December 17th featuring Ansible and our friends at DualSpark, an expert Amazon Web Services consulting partner.

Register for the Webinar Here

Ansible Automation on AWS: Best Practices by Battle-Hardened Experts

- Using Ansible to manage infrastructure in multi-tier deployments 
- Using CloudFormation and Ansible to manage configuration for more complicated scenarios 
- How Tower adds visibility to systems at runtime
 
Register Now
Presenters: 

Patrick McClory (DualSpark) is a software engineer and architect who fell into 'ops by accident.' Through years of experience in multi-platform and multi-layered systems, he's honed his craft and learned how to build  systems at scale that both leverage the best of breed software solutions and frameworks as well as the flexibility of highly configurable infrastructure before it was cool to call it infrastructure-as-code. Today, Patrick helps to run the boutique consulting firm DualSpark Partners which focuses on helping clients make a move to the cloud using cloud-native strategies from infrastructure management through to application design and development. Follow Patrick and DualSpark on Twitter.

Dave Johnson (Ansible) started his career at Red Hat prior to its IPO, ultimately building and leading Continue reading

BGP hijack incident by Syrian Telecommunications Establishment

The Syrian national Telecommunications Establishment (STE) has been in the news numerous times over the last few years, mostly because of the long lasting large scale Internet outages in Syria. This morning however we observed a new incident involving the two Autonomous systems for STE (AS29386 and AS29256). Starting at 08:33 UTC we detected  that hundreds of new prefixes were being announced by primarily AS 29386. The new BGP announcements by STE (AS29386) were for prefixes that are not owned or operates by the Syrian Telco and as a result triggered ‘hijack / origin AS’ alerts for numerous BGPmon users. The announcements lasted for a few minutes only and we saw paths changing back to the original origin AS at about 08:37 UTC.

RIPE stat has some great tools that visualize the event, this example shows what happened to the youtube prefix 208.117.232.0/24

Youtube prefix hijack

Propagation
STE buys upstream connectivity to the rest of the Internet via three providers, AS3491 (PCCW Global), AS3320 (Deutsche Telekom AG) and AS6762 (Telecom Italia Sparkle). The ‘bad’ BGP updates from this morning were only seen via Telecom Italia. This is either because STE only announced it to Telecom Italia, or because the other two providers filtered Continue reading

Vendor Whitebox Switches – Better Together?

ChocoPeanut

Whitebox switching has moved past the realm of original device manufacturers and has been taken up by traditional networking vendors. Andre Kindness (@AndreKindness) of Forrester recently posted that he fields several calls from his customers every day asking about a particular vendor’s approach to whitebox switching. But what do these vendor offerings look like? And can we predict how a given vendor will address the whitebox market?

Chocolate In My Peanut Butter

Dell was one of the first traditional networking vendors to announce a whitebox switch offering that decoupled the operating system from the switching hardware. Dell offered packages from Cumulus Linux and Big Switch Networks alongside their PowerConnect lineup. This makes sense when you consider that the operating system on the switch has never been the strong suit of Dell. The PowerConnect OS is not very popular with network engineers, being very dissimilar from more popular CLIs such as Cisco IOS and its look-alikes.  Their attempts to capitalize on the popularity of Force Ten OS (FTOS) and adapt it or use on PowerConnect switches has been difficult at best, due to the divide been hardware architecture of the two platforms.

What Dell is very good at is Continue reading

The Big YANG Theory

The Big YANG Theory

by Hariharan Ananthakrishnan, Distinguished Engineer - December 9, 2014

At this point in the evolution of the network, we think it is important to outline the history, pros, cons, and future of YANG. The data model in YANG helps in managing configuration for both traditional and software defined networks (even SDN needs some persistent state). Standardized YANG models will help in managing true multi-vendor networks. 

What Is YANG Exactly?
As I outlined in “The Current State of SDN Protocols,” YANG is a data modeling language used to model configuration and state data manipulated by the Network Configuration Protocol (NETCONF), NETCONF remote procedure calls and NETCONF notifications. YANG was developed by the NETMOD working group in the IETF and was published as RFC 6020. 

In the past few years, YANG gained a lot of traction with the open-source community. There are tools developed to validate YANG and transform YANG specification into other formats. Some tools can even generate JAVA code given a YANG specification. Router vendors noticed the traction and started contributing to model definitions, standardization and eventual support in their products. 

My Experience
I got involved with YANG when Continue reading

Operations Manager to OMi Migration Path

HP has finally announced a migration path for Operations Manager to OMi. It’s about time too. This looks like a good path. If you want to stick with HP Software for managing your services, you should be investigating it.

The writing’s been on the wall for a while. HP has stopped investment in Operations Manager. I asked last year if HP had abandoned Operations Manager. This year I noted that it was kicking, but only just. My advice was:

To customers using HP OM…start planning your migration away from it, if you haven’t already. To customers considering purchasing it: Don’t, unless you’re buying it as part of an overall BSM/OMi implementation, and the salesfolk have guaranteed you can change your licensing over at no cost in future.

Well, HP has finally announced the OM2OMi Evolution program. Key points:

  • License entitlement – OM servers can get equivalent licenses for OpsBridge Premium
  • Operations Agent 11 works with both OM and OMi, so you don’t have to do the Agent migration at the same time
  • Migration tools to assist with switching over

They do include this quote:

Well no one at HP is going to try to force you into replacing a product you love. Rest Continue reading

Johannesburg: CloudFlare’s 30th data center

Fire up the celebration braai, Jozi! CloudFlare is here, and it’s a big one. An important milestone (our 30th data center) calls for an equally important new location: Johannesburg, South Africa, our first data center in Africa.

For the local audience: Steek aan 'n braai ter viering, Jozi! CloudFlare is hier en dis 'n groot een. 'n Belangrike mylpaal (ons 30ste datasentrum), vra vir ewe belangrike en nuwe ligging: Johannesburg, Suid-Afrika, ons eerste datasentrum in Afrika.

Now serving Southern Africa

Prior to now nearly all CloudFlare traffic delivered to Africa was served from our London, Amsterdam and Hong Kong data centers with round trip latency of 200-350ms. Bandwidth in the region is notoriously expensive (it would make even the Australians blush) making it prohibitive to enter into the continent. That is, before now. Just a few months ago we were fortunate to enter into discussions with a number of partners in the region that share CloudFlare’s vision to help build a better Internet.

Our Johannesburg data center will not only make sites on CloudFlare more performant for Internet users in South Africa, but also for Internet users across all of Southern Africa (and beyond). From Botswana to Kenya, users Continue reading

phpipam v1.1 released :: security fixes, performance improvements, mod_rewrite optional

Dear all, I am happy to announce new version of phpipam IP address management – version 1.1.

This release fixed some bugs, provided some new features, but most importantly it was focused on:

  • Security fixes (SQL injection, XSS scripting, crypting DB passwords, brute-force attack prevention, …)
  • Performace improvements (Caching, reusing SQL connection),..
  • mod_rewrite no longer required, selectable URL structure under settings

Some screens:

installation vlans subnet

If you find phpIPAM useful for your company donations would be highly appreciated :)

You can demo it here: http://demo.phpipam.net/
You can download it on sourceforge site: phpipam-1.1.

Special thanks to all the people submitting bug reports, translators and feature testers!

Full changelog for this release is:

Enhancements:
----------------------------
+ Caching of SQL results to avoid multiple queries;
+ Reduced number of DB queries;
+ Added selected mail notifications to admins to be notified on IP/subnet change;
+ Added new subnetId index to ipaddresses table that significantly improves network loading;
+ Now using only 1 network connection towards MySQL server;
+ Updated pagination;
+ mod_rewrite no longer required, selectable URL structure under settings;
+ Added option not to display free ranges;
+ Added option to set maximum VLAN number;
+ Selectable custom fields to  Continue reading

Powerpoint Doesn’t Stink

“Presentations are just a waste of time.”

“Powerpoint stinks.”

“Can’t we do something other than another long, boring, presentation?”

“We should just ban Powerpoint.”

If I had a nickel for every time I’ve heard someone complain about Powerpoint, or presentations, I’d be rich enough to quit work and stop blogging. :-) Isn’t it about time we were honest with ourselves, though? Isn’t it about time we told the truth about this particular problem? Blaming Powerpoint for bad presentations is like blaming word processors for badly written books.

The problem isn’t Powerpoint. The problem is the person you see every morning looking at you in the mirror. The problem isn’t the tool, it’s that we stink at organizing and presenting our thoughts in any sort of reasonable way. So let’s talk about how to build a better presentation.

To begin: forget everything you’ve ever read in a book about making elevator pitches, making a presentation that impacts, with dash, flair, or whatever. There is a set of presentations that present as a story, with flair and dash, and there is another set that just doesn’t.

As an example, I was the Routing Protocols SGM for Cisco Live for Continue reading

Christmas Change Freeze – Good or Bad?

We’re approaching Christmas, and for many of us, that means we’re about to enter an extended change freeze. This means an extended period when we shouldn’t change anything, hoping to improve stability. ITIL Change Management tells us this is good. I’m not convinced.

The Christmas Change Freeze

Many businesses impose some form of change freeze across all production systems during the Christmas/New Years period. In theory, all network/compute/storage changes are deferred until January. In practice, high priority changes will still be made if you jump up and down enough. The rate of change should still be lower during this period though.

Some change freezes may only run from just before Christmas until early January. Other businesses will go into a change freeze for as long as five weeks. My experience is that Southern Hemisphere businesses have a longer change freeze than Northern Hemisphere ones. I assume this is because many staff take extended leave over the Austral summer.

Aside: In New Zealand, the term ‘Brown out’ is often used when referring to the Christmas Change Freeze. I have no idea why this term is used, as a ‘brownout’ normally refers to something quite different.

Why Have One?

There are differing opinions about the usefulness Continue reading

1 … 3,659 3,660 3,661 3,662 3,663 … 3,834