stretch
Author Archives: stretch
- Home → Author's archive:
stretch
Book Review: Model-Driven DevOps
Earlier this month at Cisco Live in Las Vegas, I attended a presentation by Steven Carter and Jason King titled Model-Driven DevOps: The Enterprise Automation Framework You've Been Looking For! They discussed the importance of robustly modeling your network data — a quality very dear to me, as you might expect — and having a clear strategy for automating changes, and shared an example automation architecture for reference. It was a refreshing take on introducing network engineers to DevOps methodologies, and the first time I've seen anyone clearly map software development practices to network automation.
Interested to dive deeper into the material than a 45-minute session would allow, I was pleased to learn that their talk derived from a book of the same name that they co-authored. Coupled with the lingering geeky high from Cisco Live, it made for an excellent read on the flight back home.
The book's content is broken into seven easily digestible chapters, each dedicated to a particular step along a path toward network automation:
- Chapter 1: A Lightbulb Goes Off
- Chapter 2: A Better Way
- Chapter 3: Consumable Infrastructure
- Chapter 4: Infrastructure as Code
- Chapter 5: Continuous Integration/Continuous Deployment
- Chapter 6: Implementation
- Chapter 7: Human Continue reading
Observations on Seven Years of Maintaining Open Source
June 27th marks the seventh anniversary of NetBox, a one-time hobby project which quickly took off and today largely consumes my life. What began as a proof-of-concept solution for the network engineering team at DigitalOcean is now perhaps the most widely deployed network source of truth in the world.
This feels like an opportune time to reflect on some lessons I've learned along the way, with the hope of offering mixed encouragement and caution to those considering a similar path. And as I've felt the urge to pick up blogging again, this post will also serve to share what I've been up to recently.
Most articles about open source in general are boring. Reading about licenses and software governance feels like a punishment. Keenly aware of this fact, I'll do my best to navigate around the theory and stick with observations that are of practical use to the aspiring open source maintainer.
Continue reading · No comments
Templating Device Configurations
One of the core functions of network automation is the ability to generate network device configurations from a template. This is a discrete, intentional process which unfortunately is often conflated with the totally separate act of applying a rendered configuration to a device. In this article we'll look at how to establish a template from existing configurations, define and organize variable data, and ultimately render a series of configurations automatically using a simple Python script.
What is a Template?
The term template describes any sort of mold or pattern from which new, identical objects can be created. For instance, a cookie cutter is a sort of template that can be used to create an arbitrary number of identically-shaped cookies from a sheet of dough. But in our case, we're inexplicably more interested in creating network device configuration files than baking cookies, and creating wholly identical copies of a file isn't terribly useful, since each network device typically has a handful of unique characteristics such as hostname, authentication credentials, IP addresses, and so on.
To address this need to define changing pieces of data within an otherwise unchanging document, we employ variables. A variable serves as a placeholder within the template, Continue reading
Real World APIs: Snagging a Global Entry Interview
As my new job will have me traveling a bit more often, I finally bit the bullet and signed up for Global Entry (which is similar to TSA PreCheck but works for international travel as well). A few days after submitting my application and payment, I was conditionally approved. The next step was to schedule an “interview,” which is essentially a 10-minute appointment where they ask a few questions and take biometrics. The interview must be done in person at one of relatively few CBP locations.
Here in Raleigh, North Carolina, my two closest locations are Richmond and Charlotte. Unfortunately, CBP’s scheduling portal indicated no availability for new appointments at either location. No additional context is provided, so I have no idea whether I should keep trying every few days, or attempt to schedule an appointment at a remote location to coincide with future travel.
My only hope at this point is that spots will eventually open up as other applicants cancel their appointments or CBP adds sufficient staff to meet demand. But that means manually logging into the portal, completing two-factor authentication, and checking both of my desired appointment locations each and every time.
Sounds like a great use Continue reading
Connection Restored
I was embarrassed to realize recently that it’s been well over two years since my last blog post. Life has a way of getting away from you, I suppose. But I’ve decided to try and reboot the blog, and hopefully get back to writing regularly. Let me kick things off my sharing what I’ve been up to recently.
Goodbye, DigitalOcean!
After nearly five years working at DigitalOcean, I made the difficult decision to part ways with the company. In my time there, I was fortunate to work with an amazing team, and witness the truly amazing evolution of a startup company from niche player to major cloud provider. Most of all, I’m thankful to DigitalOcean for the opportunity my role provided in extending from traditional network engineering into development and automation. I’ll miss working with my DO team, but I’m excited to see where the future will take them.
Three Months with Google Fiber
I'm one of the lucky few to benefit from Google Fiber's recent expansion into new regions (before they nixed the whole thing). I've had the service fire three months now and figured I should write up my experience with it thus far.
The Installation
Google Fiber announced that it would be expanding to the Raleigh-Durham metro area, known locally as "The Triangle", in January 2015. It's been a long game of hurry-up-and-wait since then, watching crews laying fiber all over town without hearing a peep from Google regarding availability. But in the fall of 2016, people were finally able to start signing up for service. Here's how my installation went.
September 3
Google Fiber registration opens! I sign up for service and pay a paltry $10 deposit, which gets credited toward my first bill. Over the next couple weeks, various utilities swing by to mark their lines in the ground. (Here's the color code for utility markings in the US, if you're curious.)
September 24
Google's contractor arrives on site to lay fiber from the curb to my house and to many of my neighbors' houses. Surprisingly, they cut my trench by hand, possibly due to the steep Continue reading
Three Months with Google Fiber
I'm one of the lucky few to benefit from Google Fiber's recent expansion into new regions (before they nixed the whole thing). I've had the service fire three months now and figured I should write up my experience with it thus far.
The Installation
Google Fiber announced that it would be expanding to the Raleigh-Durham metro area, known locally as "The Triangle", in January 2015. It's been a long game of hurry-up-and-wait since then, watching crews laying fiber all over town without hearing a peep from Google regarding availability. But in the fall of 2016, people were finally able to start signing up for service. Here's how my installation went.
September 3
Google Fiber registration opens! I sign up for service and pay a paltry $10 deposit, which gets credited toward my first bill. Over the next couple weeks, various utilities swing by to mark their lines in the ground. (Here's the color code for utility markings in the US, if you're curious.)
September 24
Google's contractor arrives on site to lay fiber from the curb to my house and to many of my neighbors' houses. Surprisingly, they cut my trench by hand, possibly due to the steep Continue reading
Legacy TLS cipher support in Firefox
After upgrading Firefox recently, I noticed that I could no longer access certain embedded devices via HTTPS. It seems that recent versions of Firefox and Chrome no longer support certain TLS ciphers due to recently discovered vulnerabilities. That's all well and good, except the error returned offers no recourse if you need to connect anyway.
Firefox returns the error SSL_ERROR_NO_CYPHER_OVERLAP with no option to temporarily allow connectivity. (Chrome reports a similar error named ERR_SSL_VERSION_OR_CIPHER_MISMATCH.) Presumably, this choice was made by the developers with the intention of forcing people to upgrade outdated devices. Unfortunately, in order to upgrade an out-of-date device, we typically must first be able to connect to it. I wasted a fair bit of time digging up a solution, so I figured I'd document the workaround here for when I inevitably run into this problem again a year from now and have forgotten what I did.
Legacy TLS cipher support in Firefox
After upgrading Firefox recently, I noticed that I could no longer access certain embedded devices via HTTPS. It seems that recent versions of Firefox and Chrome no longer support certain TLS ciphers due to recently discovered vulnerabilities. That's all well and good, except the error returned offers no recourse if you need to connect anyway.
Firefox returns the error SSL_ERROR_NO_CYPHER_OVERLAP with no option to temporarily allow connectivity. (Chrome reports a similar error named ERR_SSL_VERSION_OR_CIPHER_MISMATCH.) Presumably, this choice was made by the developers with the intention of forcing people to upgrade outdated devices. Unfortunately, in order to upgrade an out-of-date device, we typically must first be able to connect to it. I wasted a fair bit of time digging up a solution, so I figured I'd document the workaround here for when I inevitably run into this problem again a year from now and have forgotten what I did.
The Overlay Problem: Getting In and Out
I've been researching overlay network strategies recently. There are plenty of competing implementations available, employing various encapsulations and control plane designs. But every design I've encountered seems ultimately hampered by the same issue: scalability at the edge.
Why Build an Overlay?
Imagine a scenario where we've got 2,000 physical servers split across 50 racks. Each server functions as a hypervisor housing on average 100 virtual machines, resulting in a total of approximately 200,000 virtual hosts (~4,000 per rack).
In an ideal world, we could allocate a /20 of IPv4 space to each rack. The top-of-rack (ToR) L3 switches in each rack would advertise this /20 northbound toward the network core, resulting in a clean, efficient routing table in the core. This is, of course, how IP was intended to function.
Unfortunately, this approach isn't usually viable in the real world because we need to preserve the ability to move a virtual machine from one hypervisor to another (often residing in a different rack) without changing its assigned IP address. Establishing the L3 boundary at the ToR switch prevents us from doing this efficiently.
The Overlay Problem: Getting In and Out
I've been researching overlay network strategies recently. There are plenty of competing implementations available, employing various encapsulations and control plane designs. But every design I've encountered seems ultimately hampered by the same issue: scalability at the edge.
Why Build an Overlay?
Imagine a scenario where we've got 2,000 physical servers split across 50 racks. Each server functions as a hypervisor housing on average 100 virtual machines, resulting in a total of approximately 200,000 virtual hosts (~4,000 per rack).
In an ideal world, we could allocate a /20 of IPv4 space to each rack. The top-of-rack (ToR) L3 switches in each rack would advertise this /20 northbound toward the network core, resulting in a clean, efficient routing table in the core. This is, of course, how IP was intended to function.
Unfortunately, this approach isn't usually viable in the real world because we need to preserve the ability to move a virtual machine from one hypervisor to another (often residing in a different rack) without changing its assigned IP address. Establishing the L3 boundary at the ToR switch prevents us from doing this efficiently.
Taking the CCIE Lab in RTP
Cisco's campus in Research Triangle Park, North Carolina, is one of only two places in the United States where candidates can complete a CCIE lab exam (the other being in San Jose, California). People fly in from all over the eastern US and beyond to spend a day taking the exam. Lots of folks who've taken the exam have written up their experiences, but I haven't seen many talk at length about their time in RTP outside of Cisco's building 3.
I've lived just a few minutes away from the testing site for the past few years, and it occurred to me recently that visitors might benefit from some local knowledge.
Getting Here
Most people fly in via Raleigh-Durham International Airport (RDU). RDU is a medium-sized airport with two terminals. Most flights operate out of Terminal 2, except for Southwest Airlines, which is based in the newly-renovated Terminal 1.
As airports go, I'm a big fan of RDU. It's a very modern, clean, and well-organized facility. The interior of Terminal 2 is beautifully designed to resemble an early airplane wing and is flooded with natural light during the day. (It's also one of very few places where you can Continue reading
Taking the CCIE Lab in RTP
Cisco's campus in Research Triangle Park, North Carolina, is one of only two places in the United States where candidates can complete a CCIE lab exam (the other being in San Jose, California). People fly in from all over the eastern US and beyond to spend a day taking the exam. Lots of folks who've taken the exam have written up their experiences, but I haven't seen many talk at length about their time in RTP outside of Cisco's building 3.
I've lived just a few minutes away from the testing site for the past few years, and it occurred to me recently that visitors might benefit from some local knowledge.
Getting Here
Most people fly in via Raleigh-Durham International Airport (RDU). RDU is a medium-sized airport with two terminals. Most flights operate out of Terminal 2, except for Southwest Airlines, which is based in the newly-renovated Terminal 1.
As airports go, I'm a big fan of RDU. It's a very modern, clean, and well-organized facility. The interior of Terminal 2 is beautifully designed to resemble an early airplane wing and is flooded with natural light during the day. (It's also one of very few places where you can Continue reading
NetBox v1.1.0 Released
One year ago today, I made the first commit to a repository named "netbox" hosted internally at DigitalOcean. It was the first iteration of a tiny little app I scratched together using the Django Python framework to track IP prefix utilization. A year later, NetBox has grown into an extensive tool that we use to track IPs, racks, devices, connections, circuits, and even encrypted credentials. And I'm happy to say that it's now open source!
NetBox v1.1.0 Released
One year ago today, I made the first commit to a repository named "netbox" hosted internally at DigitalOcean. It was the first iteration of a tiny little app I scratched together using the Django Python framework to track IP prefix utilization. A year later, NetBox has grown into an extensive tool that we use to track IPs, racks, devices, connections, circuits, and even encrypted credentials. And I'm happy to say that it's now open source!
Announcing NetBox
Update: NetBox has been released!
Several years ago, I lamented the few options available for a provider-grade IPAM solution. Specifically, I explained why building a custom application would be undesirable:
Could I create a custom IPAM solution with everything we need? Sure! The problem is that I'm a network engineer, not a programmer (a natural division of labor which, it seems, is mostly to blame for the lack of robust IPAM solutions available). Even if I had the time to undertake such a project, I have little interest in providing long-term maintenance of it.
But I suppose time makes fools of us all.
Nearly one year ago, I started developing an IPAM application as part of my day job. Leveraging my experience with the Django Python framework, I had a working proof-of-concept in just a week. Over the next several months, the project grew more mature and began to take on additional roles: data center infrastructure management, circuit tracking, and credentials storage. Today, the tool functions as our "source of truth" for many aspects of our infrastructure. We call it NetBox.
Announcing NetBox
Update: NetBox has been released!
Several years ago, I lamented the few options available for a provider-grade IPAM solution. Specifically, I explained why building a custom application would be undesirable:
Could I create a custom IPAM solution with everything we need? Sure! The problem is that I'm a network engineer, not a programmer (a natural division of labor which, it seems, is mostly to blame for the lack of robust IPAM solutions available). Even if I had the time to undertake such a project, I have little interest in providing long-term maintenance of it.
But I suppose time makes fools of us all.
Nearly one year ago, I started developing an IPAM application as part of my day job. Leveraging my experience with the Django Python framework, I had a working proof-of-concept in just a week. Over the next several months, the project grew more mature and began to take on additional roles: data center infrastructure management, circuit tracking, and credentials storage. Today, the tool functions as our "source of truth" for many aspects of our infrastructure. We call it NetBox.
Don’t be Discouraged by Plagiarists
Recently, a friend pointed out that an individual had taken one of my cheat sheets, superimposed his own logo and URL on it, and published it as his own work. This is certainly not the first time I've been plagiarized, nor will it be the last, I suspect. I called out the individual on Twitter, and I'm very gratefully for the many people who helped me compel him to remove the illegitimate content. Eventually.
I wanted to write a quick post sharing my thoughts on this incident for the benefit of everyone who has expressed interest in starting their own blog or web site. I've heard plenty of people comment over the years to the effect of, "Why bother starting a blog if someone's just going to harvest the RSS feed and re-publish it on their own site to make a few bucks?" Indeed, this has always been a concern among producers of both free and paid content.
I wish I could tell you that plagiarism isn't that big a deal, or that it won't happen to you. But the truth is plagiarism is a huge problem in our industry (and across the Internet in general), and if Continue reading
Don’t be Discouraged by Plagiarists
Recently, a friend pointed out that an individual had taken one of my cheat sheets, superimposed his own logo and URL on it, and published it as his own work. This is certainly not the first time I've been plagiarized, nor will it be the last, I suspect. I called out the individual on Twitter, and I'm very gratefully for the many people who helped me compel him to remove the illegitimate content. Eventually.
I wanted to write a quick post sharing my thoughts on this incident for the benefit of everyone who has expressed interest in starting their own blog or web site. I've heard plenty of people comment over the years to the effect of, "Why bother starting a blog if someone's just going to harvest the RSS feed and re-publish it on their own site to make a few bucks?" Indeed, this has always been a concern among producers of both free and paid content.
I wish I could tell you that plagiarism isn't that big a deal, or that it won't happen to you. But the truth is plagiarism is a huge problem in our industry (and across the Internet in general), and if Continue reading
Can You Keep a Secret? (Part 2)
In part one, we saw how AES can be used to encrypt sensitive data so that it can be retrieved only by using an encryption key. The problem with this approach is that everyone who needs access to the data must have a copy of the key. If any one of these copies becomes compromised, the entire database must be re-encrypted using a new key, and the new key must be distributed securely to all parties involved. In this article, we'll see how symmetric encryption can be combined with asymmetric cryptography (namely RSA) to create a hybrid cryptosystem.
Let's begin by encrypting some data using AES as we did in part one. First we pad our plaintext's length to a multiple of 16 using null bytes, then generate a 256-bit encryption key and a 128-bit IV, and finally encrypt it with CFB-mode AES to generate a string of ciphertext.
>>> from Crypto.Cipher import AES >>> import os >>> plaintext = "Operation Neptune will launch on June 6th" >>> plaintext += (16 - len(plaintext) % 16) * chr(0) >>> encryption_key = os.urandom(32) >>> iv = os.urandom(16) >>> cipher = AES.new(encryption_key, AES.MODE_CFB, iv) >>> ciphertext = Continue reading