Writing a Custom IPAM Application
Four years ago, I lamented the lackluster selection of IPAM applications available for service providers. Unfortunately, it seems not much has changed lately. I was back to exploring IPAM offerings again recently, this time with the needs of a cloud hosting provider in mind. I demoed a few tools, but none of them seemed to fit the bill (or they did, but were laughably overpriced).
So, I decided to write my own. In my rantings a few years back, I had considered this option:
Could I create a custom IPAM solution with everything we need? Sure! The problem is that I'm a network engineer, not a programmer (a natural division of labor which, it seems, is mostly to blame for the lack of robust IPAM solutions available). Even if I had the time to undertake such a project, I have little interest in providing long-term maintenance of it.
My opinion has not changed, but I've come to realize that if I want a tool that fits my requirements, I will need to build it. And after surprisingly little time, I'm happy to report that I have now have a kick-ass IPAM tool that does exactly what I want it to.
What the CCIE does not Prove
I came across an article today about a 19-year-old who earned his CCIE. It reminded me of a Reddit post from a few weeks ago. Someone asked why, when evaluating a CCIE, hiring managers still demand a number of years of practical experience in the field.
I'm in a situation where I'm a CCNP with 3 years of experience. I want to get my CCIE but I keep being told left and right I don't have enough experience and I'll never get a CCIE job without 7 years of experience. Am I supposed to just laze around and wait until I get more experience? It just doesn't make sense.
This is a fairly common misunderstanding among people new to our field, and is largely the result of vendor marketing. People want so badly to believe that a certification proves their worth as an individual, when in reality its value is much more narrowly defined.
What the CCIE does not Prove
I came across an article today about a 19-year-old who earned his CCIE. It reminded me of a Reddit post from a few weeks ago. Someone asked why, when evaluating a CCIE, hiring managers still demand a number of years of practical experience in the field.
I'm in a situation where I'm a CCNP with 3 years of experience. I want to get my CCIE but I keep being told left and right I don't have enough experience and I'll never get a CCIE job without 7 years of experience. Am I supposed to just laze around and wait until I get more experience? It just doesn't make sense.
This is a fairly common misunderstanding among people new to our field, and is largely the result of vendor marketing. People want so badly to believe that a certification proves their worth as an individual, when in reality its value is much more narrowly defined.
What the CCIE does not Prove
I came across an article today about a 19-year-old who earned his CCIE. It reminded me of a Reddit post from a few weeks ago. Someone asked why, when evaluating a CCIE, hiring managers still demand a number of years of practical experience in the field.
I'm in a situation where I'm a CCNP with 3 years of experience. I want to get my CCIE but I keep being told left and right I don't have enough experience and I'll never get a CCIE job without 7 years of experience. Am I supposed to just laze around and wait until I get more experience? It just doesn't make sense.
This is a fairly common misunderstanding among people new to our field, and is largely the result of vendor marketing. People want so badly to believe that a certification proves their worth as an individual, when in reality its value is much more narrowly defined.
Traceroute and Not-so-Equal ECMP
I came across an odd little issue recently involving equal-cost multipath (ECMP) routing and traceroute. Traceroutes from within our network to destinations out on the Internet were following two different paths, with one path being one hop longer than the other. This resulted in mangled traceroute output, impeding our ability to troubleshoot.
The relevant network topology comprises a mesh of two edge routers and two core switches. Each edge router has a number of transit circuits to different providers, and advertises a default route via OSPF to the two core switches below. The core switches each load-balance traffic across both default routes to either edge routers.
Because each edge router has different providers, some destinations are routed out via edge1 and others via edge2, which means sometimes a packet will be routed to edge2 via edge1, or vice versa.
Routers typically employ a hash function using layer three and four information from each packet to pseudo-randomly distribute traffic across equal links. Typically, all packets belonging to a flow (e.g. all packets with the same source and destination IP and port numbers) follow the same path.
However, in this case traceroute packets were being split across two path of unequal Continue reading
Traceroute and Not-so-Equal ECMP
I came across an odd little issue recently involving equal-cost multipath (ECMP) routing and traceroute. Traceroutes from within our network to destinations out on the Internet were following two different paths, with one path being one hop longer than the other. This resulted in mangled traceroute output, impeding our ability to troubleshoot.
The relevant network topology comprises a mesh of two edge routers and two core switches. Each edge router has a number of transit circuits to different providers, and advertises a default route via OSPF to the two core switches below. The core switches each load-balance traffic across both default routes to either edge routers.
Because each edge router has different providers, some destinations are routed out via edge1 and others via edge2, which means sometimes a packet will be routed to edge2 via edge1, or vice versa.
Routers typically employ a hash function using layer three and four information from each packet to pseudo-randomly distribute traffic across equal links. Typically, all packets belonging to a flow (e.g. all packets with the same source and destination IP and port numbers) follow the same path.
However, in this case traceroute packets were being split across two path of unequal Continue reading
Traceroute and Not-so-Equal ECMP
I came across an odd little issue recently involving equal-cost multipath (ECMP) routing and traceroute. Traceroutes from within our network to destinations out on the Internet were following two different paths, with one path being one hop longer than the other. This resulted in mangled traceroute output, impeding our ability to troubleshoot.
The relevant network topology comprises a mesh of two edge routers and two core switches. Each edge router has a number of transit circuits to different providers, and advertises a default route via OSPF to the two core switches below. The core switches each load-balance traffic across both default routes to either edge routers.
Because each edge router has different providers, some destinations are routed out via edge1 and others via edge2, which means sometimes a packet will be routed to edge2 via edge1, or vice versa.
Routers typically employ a hash function using layer three and four information from each packet to pseudo-randomly distribute traffic across equal links. Typically, all packets belonging to a flow (e.g. all packets with the same source and destination IP and port numbers) follow the same path.
However, in this case traceroute packets were being split across two path of unequal Continue reading
Networking FAQ 4: Fundamentals
- At what OSI layer does protocol X operate?
- What's the difference between a router and a multilayer switch?
- What's the difference between forwarding and control planes?
- What's the difference between MTU and MSS?
- What's the difference between a VLAN interface and a BVI?
- How do tunnel interfaces work?
- What do NAT terms like "inside local" mean?
- Can I use the network and broadcast addresses in a NAT pool?
- Why do we need IP addresses? Can't we just use MAC addresses for everything?
- Does QoS provide more bandwidth?
Networking FAQ 4: Fundamentals
- At what OSI layer does protocol X operate?
- What's the difference between a router and a multilayer switch?
- What's the difference between forwarding and control planes?
- What's the difference between MTU and MSS?
- What's the difference between a VLAN interface and a BVI?
- How do tunnel interfaces work?
- What do NAT terms like "inside local" mean?
- Can I use the network and broadcast addresses in a NAT pool?
- Why do we need IP addresses? Can't we just use MAC addresses for everything?
- Does QoS provide more bandwidth?
Networking FAQ 4: Fundamentals
- At what OSI layer does protocol X operate?
- What's the difference between a router and a multilayer switch?
- What's the difference between forwarding and control planes?
- What's the difference between MTU and MSS?
- What's the difference between a VLAN interface and a BVI?
- How do tunnel interfaces work?
- What do NAT terms like "inside local" mean?
- Can I use the network and broadcast addresses in a NAT pool?
- Why do we need IP addresses? Can't we just use MAC addresses for everything?
- Does QoS provide more bandwidth?
Networking FAQ 3: Names and Addresses
- Where do IP addresses and domain names come from?
- Did we really run out of IPv4 addresses?
- Can I buy more IP addresses?
- Does IPv6 really provide a bazillion addresses?
- Why does IPv6 use hexadecimal addressing?
- What is IPAM?
- How do I create an IP addressing scheme?
- How does IPv6 subnetting work?
- What prefix length should I use on point-to-point links?
- How should I name devices on my network?
Continue reading · No comments
Networking FAQ 2: Certifications
- What are the most popular certifications in networking?
- How much is certification X worth?
- How should I study for certification exams?
- What's a "brain dump?"
- What is the exam experience like?
- I only just barely passed! Does it still count?
- My employer will pay for me to get a certification. Should I do it?
Continue reading · No comments
Networking FAQ 1: Breaking into the Field
This post is the first in a series I plan to publish over the next few months regarding frequently asked questions in networking. Each post will cover a different subject, roughly following the outline I shared last summer. I hope people find this useful!
- What kind of networking jobs are there?
- What are the different networking specialties?
- How do I get into networking?
- Do I need a college degree to be a networker?
- Do I need certifications?
- Do I need to know a programming language?
- What should I list on my resume?
- How much do networkers make?
- How do I find a job?
- Do you have any interview tips?
- What are the negative aspects of networking?
MAC Address Aggregation and Translation as an Alternative to L2 Overlays
Not so long ago, if you wanted to build a data center network, it was perfectly feasible to place your layer three edge on the top-of-rack switches and address each rack as its own subnet. You could leverage ECMP for simple load-sharing across uplinks to the aggregation layer. This made for an extremely efficient, easily managed data center network.
Then, server virtualization took off. Which was great, except now we had this requirement that a virtual machine might need to move from one rack to another. With our L3 edge resting at the top of the rack, this meant we'd need to re-address each VM as it was moved (which is apparently a big problem on the application side). So, now we have two options: We can either retract the L3 edge up a layer and have a giant L2 network spanning dozens of racks, or we could build a layer two overlay on top of our existing layer three infrastructure.
Most people opt for some form of the L2 overlay approach, because no one wants to maintain a flat L2 network with dozens or hundreds of thousands of end hosts, right? But why is that?
PDU-12C
"After all, what's the best part of Halloween?" Jimmy pleaded over the phone. He was trying yet again to convince Tom to skip work for the night and head over to the party he was throwing. Tom and Jimmy were good friends, but he already knew how the conversation was going to end.
"I dunno, the candy?" Tom played dumb.
"No, the eye candy! I'm telling you bro, you don't want to miss it. Rachel will be there." Jimmy sang the last bit tauntingly.
"I told you," Tom countered. "I've got work." It was around 6pm now, and he was just pulling into the parking lot outside the data center where he planned to spend the night recabling several racks of equipment. The scariest part of his Halloween would be picking through years' worth of undressed patch cabling.
"I don't get why you have to do that shit at night anyway. Why can't you do it during the day when you're stuck at work anyway?" Jimmy prodded.
Tom parked across from the building's entrance and turned off his car. Other than a couple vehicle belonging to the operations staff, the parking lot was deserted. He Continue reading
Cumulus Linux: First Impressions
Typically, when you buy a network router or switch, it comes bundled with some version of the manufacturer's operating system. Cisco routers come with IOS (or some derivative), Juniper routers come with Junos, and so on. But with the recent proliferation of merchant silicon, there seem to be fewer and fewer differences between competing devices under the hood. For instance, the Juniper QFX3500, the Cisco Nexus 3064, and the Arista 7050S are all powered by an off-the-shelf Broadcom chipset rather than custom ASICs developed in-house. Among such similar hardware platforms, the remaining differentiator is the software.
One company looking to benefit from this trend is Cumulus Networks. Cumulus does not produce or sell hardware, only a network operating system: Cumulus Linux. The Debian-based OS is built to run on whitebox hardware you can purchase from a number of partner Original Device Manufacturers (ODMs). (Their hardware compatability list includes a number of 10GE and 40GE switch models from different vendors.)
Cumulus Linux is, as the name implies, Linux. There is no "front end" CLI as on, for example, Arista platforms. Upon login you are presented with a Bash terminal and all the standard Linux utilities (plus a number of Continue reading
Preliminary Book Topics
As I announced earlier this summer, I'm working on writing a book targeted to people entering the field of computer networking. I've got a fair amount of content fleshed out already, but figured it might help to get some feedback on the tentative structure. The book is being written in a question-and-answer style, organized into chapters by subject.
Below is the preliminary table of contents. It's still very much a work in progress, but I'm curious what people think of this approach. Constructive criticism and suggestions for additional content are welcome!
Replacing an MPLS WAN with an Internet VPN Overlay
I received an email last week from a reader seeking advice on a fairly common predicament:
Our CIO has recently told us that he wants to get rid of MPLS because it is too costly and is leaning towards big internet lines running IPSEC VPNs to connect the whole of Africa.
As you can imagine, this has caused a huge debate between the networks team and management, we run high priority services such as Lync enterprise, SAP, video conferencing etc. and networks feel we need MPLS for guaranteed quality for these services but management feels the Internet is today stable enough to run just as good as MPLS.
What is your take on the MPLS vs Internet debate from a network engineer's point of view? And more so, would running those services over Internet work?
This is something I struggled with pretty frequently in a prior job working for a managed services provider. MPLS WANs are great because they provide flexible, private connectivity with guaranteed throughput. Most MPLS providers also allow you to choose from a menu of QoS schemes and classify your traffic so that real-time voice and video services are treated higher preference during periods of congestion.
Unfortunately, Continue reading
Beyond the Blog
I'm thinking about writing a book.
Obviously, there are a lot of networking books on the market today. Search for any mainstream certification on Amazon and you'll find titles from half a dozen publishers. The majority of these are oriented toward specific vendors (most commonly Cisco) and many parallel a given certification exam. These books are overall pretty great. Most of them.
There also exists a minority of books which cover topics outside of the vendor-driven mainstream, like Gary A. Donahue's Network Warrior published by O'Reilly, now in its second edition. I love this kind of independent title because its content isn't constrained to a particular mold. The author finds stuff he thinks is relevant and interesting, and he writes about it. This is the correct way to write a book.
But over the past few years it has become painfully evident to me that there are many areas of this field we simply don't talk about in print, at least not at the entry level where perhaps it would be most helpful. If you want a thirty-page lecture on subnetting or a terrible mnemonic for the OSI model, pick any CCNA book from the pile and you're good to Continue reading
PSA: Global IPv4 Routing Table Hits 500k Routes
Last week, the global IPv4 routing table has surpassed the 500 thousand route benchmark, according to the CIDR Report. The graph below shows its progression since the early nineties:
I last wrote about global IPv4 growth in August of 2009, when the table size was at a mere 300 thousand routes. While that benchmark was largely ceremonial, this one crosses a threshold which should may be of grave concern for many.
As has been pointed out on the NANOG mailing list, we are quickly approaching the hard forwarding plane capacity limits which exists on several very popular platforms, namely the Cisco 7600/6500 and RSP720/Sup720. The default TCAM partitioning scheme of these platforms allows for a maximum of 512 thousand IPv4 routes.
If you accept full Internet routes anywhere on your network, you'll want to verify the maximum table sizes for those platforms. On the 6500/7600 platform, the current partitioning scheme can be inspected with show mls cef maximum-routes:
Router# show mls cef maximum-routes FIB TCAM maximum routes : ======================= Current : --------- IPv4 + MPLS - 512k (default) IPv6 + IP Multicast - 256k (default)
The good news is that it's easy to repartition the default scheme (e. Continue reading