Russ

Author Archives: Russ

Reaction: Keith’s Law

Ethan pointed me to this post about complexity and incremental improvement in a slack message. There are some interesting things here, leading me in a number of different directions, that might be worth your reading time. The post begins with an explanation of what the author calls “Keith’s law”—

I am going to paraphrase the version he shared over lunch at the Facebook campus a few years ago and call it Keith’s Law: In a complex system, the cumulative effect of a large number of small optimizations is externally indistinguishable from a radical leap. If you want to do big things in a software-eaten world, it is absolutely crucial that you understand Keith’s Law. —Breaking Smart

The author attributes this to the property of emergence; given I don’t believe in blind emergence, I would attribute this effect to the combined intertwining of many intelligent actors producing an effect that at least many of them probably wanted (the improvement of the complex system), and each of them working in their own spheres to achieve that result without realizing the overall multiplier effect of their individual actions. If that was too long and complicated, perhaps this is shorter and better—

The law of Continue reading

Reaction: DevOps and Security

Over at TechBeacon, my friend Chris Romeo has an article up about DevOps and security. It’s interesting to me because this is actually an area I’d never thought about before, even though it makes sense. Given DevOps is essentially writing software to control infrastructure (like routers, compute, and storage), and software needs to be written in a way that is secure, then it should be obvious that DevOps software should be developed with good security principles gleaned from software development as part of the foundation.

And here we face a challenge, as Chris says—

There is no standard that defines security for DevOps, and the chances of a standard ever developing is small because different organizations are doing things their own way, and can’t even agree on a standard name. And while there is a standard for the secure development lifecycle (ISO/IEC 27034-1), few organizations are ever validated against it.

The key point in here is that every organization is doing things their own way. This isn’t wrong, of course, because every organization must have some “snowflakiness” to justify its existence, and that “snowflakiness” is often likely to show up, in a large way, in something like handling resources within Continue reading

I2RS and Remote Triggered Black Holes

In our last post, we looked at how I2RS is useful for managing elephant flows on a data center fabric. In this post, I want to cover a use case for I2RS that is outside the data center, along the network edge—remote triggered black holes (RTBH). Rather than looking directly at the I2RS use case, however, it’s better to begin by looking at the process for creating, and triggering, RTBH using “plain” BGP. Assume we have the small network illustrated below—

bgp-rtbh-01

In this network, we’d like to be able to trigger B and C to drop traffic sourced from 2001:db8:3e8:101::/64 inbound into our network (the cloudy part). To do this, we need a triggering router—we’ll use A—and some configuration on the two edge routers—B and C. We’ll assume B and C have up and running eBGP sessions to D and E, which are located in another AS. We’ll begin with the edge devices, as the configuration on these devices provides the setup for the trigger. On B and C, we must configure—

  • Unicast RPF; loose mode is okay. With loose RPF enabled, any route sourced from an address that is pointing to a null destination in the routing table will Continue reading