Category Archives for "Cumulus Networks Blog"

5 Network automation tips and tricks for NetOps

Despite what some people say, automation is not for the lazy. This opinion probably comes from the fact that the whole point of automation is to reduce repetitive tasks and make your life easier. Indeed automation can do just that, as well as giving you back hours each week for other tasks.

But getting your automation off the ground to begin with can be a challenge. It’s not as if you just decide, “Hey, we’re going to automate our network now!” and then you follow a foolproof, well-defined process to implement network automation across the board. You have to make many decisions that require long discussions, and necessitate ambitious and careful thinking about how you’re going to automate.

Just as with anything else in the IT world, there are no one-size-fits-all solutions, and no “best practices” that apply to every situation. But there are some common principles and crucial decision points that do apply to all automation endeavors.

In this post, I’ll give you five network automation tips and tricks to get clarity around your automation decisions and reduce any friction that may be inhibiting (further) adoption of network automation.

1. Choose whether you want flexibility or simplicity

Automating Continue reading

Cumulus content roundup: May

Hope you brought your networking acronyms dictionary with you – this month’s Cumulus content roundup is going full tech-geek and we’re NOT ashamed! We’re brushing up on EVPN, ECMP, DWDM and TGIF (okay, not the last one. But did that make you LOL?) See a term that makes you go WTF? Don’t worry — we’ve got webinars, videos, blog posts and more to help you differentiate between BGP and OMG.

From Cumulus Networks:

EVPN content hub: Deploying EVPN enables you to enhance your layer 3 data center with benefits such as multitenancy, scalability, ARP suppression and more. Don’t know where to begin? Browse this EVPN resources page to learn more about how you can incorporate EVPN into your Cumulus network.

Celebrating ECMP in Linux — part one: Equal Cost Multi-Path (ECMP) routes are a big component of all the super-trendy data center network designs that are en vogue right now. Read part one of this series about ECMP’s history, how it’s evolved and what Cumulus is doing to help.

Networking how-to video — What is Voyager?: Voyager is a Dense Wavelength Division Multiplexing (DWDM) platform Facebook brought to the Telecom Infra Project (TIP), bringing the first Continue reading

Solving challenges with Linux networking, programmable pipelines and tunnels

Exciting advances in modern data center networking

Many moons ago, Cumulus Networks set out to further the cause of open networking. The premise was simple: make networking operate like servers. To do that, we needed to develop an operating system platform, create a vibrant marketplace of compatible and compliant hardware and get a minimum set of features implemented in a robust way.

Today, these types of problems are largely behind us, and the problem set has moved in the right direction towards innovation and providing elegant solutions to the problems around scale, mobility and agility. Simply put, if “Linux is in the entire rack,” then it follows that the applications and services deployed via these racks should be able to move to any rack and be deployed for maximum overall efficiency.

The formula for this ephemeral agility then is based on two constructs.

  1. If the application can deploy anywhere, the policies governing the application’s ability to interact with the world need to be enforceable anywhere and on any rack in the entire data center.
  2. It should be possible to place an application on any rack and all the connectivity it needs should be available without needing any physical changes in Continue reading

How to manage access in a web-scale data center

One of the consistent questions that arises during the web-scale transition is the impact of managed access to networking infrastructure. How do we take traditional management techniques and adapt them to the new operational paradigm of web-scale networking, where automation drives the majority of changes and the infrastructure is treated as a holistic entity rather than node-by-node?

Local privileges

In the most basic way, we can migrate existing workflows to the new paradigm. Though inefficient, the old way of doing things still works with the new web-scale paradigm. The easiest way to do this is to restrict access to your switches using local privileges. In Linux, users are controlled using the adduser command, and the permissions for that user are controlled using the chmod commands.

A list of all users is stored in the /etc/passwd folder of Linux:

 [email protected]:~$ cat /etc/passwd
 list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
 gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
 systemd-timesync:x:100:103:systemd Time Synchronization,,,:/run/systemd:/bin/false
 systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/bin/false
 systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/bin/false
 systemd-bus-proxy:x:103:106:systemd Bus Proxy,,,:/run/systemd:/bin/false
 frr:x:104:109:Frr routing suite,,,:/var/run/frr/:/bin/false

Users can be added and deleted using the adduser and deluser commands:

 [email protected]:~$ sudo  Continue reading

NetQ + Kubernetes: bringing container visibility with the leading container orchestrator

Businesses today have to get applications to market faster than ever, but with the same or less budget. Because of this requirement, modern data centers are evolving to support a change in application delivery. In order to get applications to market faster and increase revenue, applications that were once built as one monolithic entity are becoming segmented and deployed separately, communicating amongst themselves. The pieces of applications, sometimes referred to as microservices, are often deployed as containers. This results in much faster deployment and a quicker update cycle. However, the network teams operating the infrastructure supporting the applications often have no visibility into how their networks are being utilized, and thus are making design, operations and troubleshooting decisions blindly. Now, Cumulus NetQ provides this visibility from container deployments all the way to the spine switches and beyond — accelerating operations and providing the crucial information to efficiently design and operate the networks running containers.

Understanding the challenges of container management

Traditionally, the new application design and deployment method using containers makes operating and managing the infrastructure to support them very challenging. The containers often have to talk with each other within or across data centers or to the outside world. An Continue reading

Announcing Cumulus Linux 3.6 and…early access to Voyager!

The reign of proprietary networking in optical and data center interconnect falls

In furthering our mission to bring S.O.U.L. to networking through Simple, Open, Untethered, Linux-based networking solutions, and just on the heels of our Cumulus NetQ 1.3 announcement around simplifying container networking and operations, today we continue the mission to advance web-scale networking in the digital age with our release of Cumulus Linux 3.6. Our focus has been to help organizations move towards a modern world of simplification, flexibility and scale — where complex applications reside on standardized infrastructure that is automated, repeatable and scalable. We see a world of agility built upon cloud principles; of converged administrative teams where sysadmins can manage the network and network admins can manage systems.

What’s new in Cumulus Linux 3.6

In this release of Cumulus Linux 3.6, we are not only driving network efficiency and simplicity, but also expanding our solution set to include data center interconnect (DCI) use cases. Additionally, to help organizations adopt these web-scale principles in networking, we are enhancing our portfolio by adding popular networking capabilities to the open Linux platform. These include:

Voyager code is available for early access

From Continue reading

Celebrating ECMP in Linux — part two

In part one of our series on ECMP, we discussed the basics of ECMP, the recent changes that have been made and Cumulus’ part in moving the ball forward for Linux networking. Now, it’s time to get a little more technical and review how advancements in ECMP development for IPv4 and IPv6 have made ECMP what it is today — and what it can be in the near future.

Setting the stage: defining our terminologies

Hashing algorithms

Hashing algorithms are the biggest component of ECMP behavior, so it makes sense for us to talk for a moment about what we specifically mean when we refer to each one.

1.) Per-packet hash
This hash was the original hashing algorithm used in the kernel’s ECMP behavior. It is trivially simple to understand as it basically uses a pseudo random number in the kernel at the time packet is being processed (jiffies) to determine which link in an ECMP bundle the traffic will use for egress. With this algorithm in place, each packet for a single flow could use a different link to get to the destination. This leads to all kinds of bad behaviors in TCP and higher level applications/protocols Continue reading

Celebrating ECMP in Linux — part one

ECMP in Linux: A brief history

Equal Cost Multi-Path (ECMP) routes are a big component of all the super-trendy data center network designs that are en vogue right now. Clos networks and the ECMP that underpins those designs are the best tools we have today to deliver high bandwidth, highly fault-tolerant networks. Clos networks are rich with multiple equal cost paths to get from Server A to Server B.

Linux kernel ECMP

2 Paths from Host to ToR * 8 Paths from ToR to Leaf * 16 Paths from Leaf to Spine * 8 Paths from Spine to Leaf * 2 Paths from Leaf to ToR
= 4096 Possible Unique Paths between Server A and Server B

FYI: The above is an actual customer network. Names have been changed to protect the innocent and colors have been added because a rainbow of links is more fun!

Cumulus has been working to improve the behavior of ECMP routes in the Linux kernel over the last several kernel releases. Now, with kernel v4.17, we have achieved the milestone we set out to attain. As of Linux kernel v4.17, Linux hosts can now leverage the “5-Tuple” style hashing used inside traditional network devices for Continue reading

Here’s how NetQ injects S.O.U.L into your network

Our passion at Cumulus is all around networking with S.O.U.L. Simple. Open. Untethered. Linux. These tenants come together to help organizations build a web-scale, modern, automated network that is necessary for the digital age. So it’s no surprise that Cumulus NetQ and networking with S.O.U.L go hand-in-hand. Let’s take a closer look at how Cumulus NetQ builds on these tenants of S.O.U.L.


NetQ is all about simplifying network operations. The deep visibility achieved through Cumulus NetQ is extremely powerful from a network validation, management and troubleshooting standpoint.

  • Simplifying rollout validation: Cumulus NetQ helps reassure networking teams that what they are rolling into production will actually work. Its validation system lets users check their configuration during production rollout. And with NetQ’s powerful tracing capability, you can validate that you have the true end-to-end connectivity you expected. Tracing saves a huge amount of time as you work to ensure that the path of the packet is working as expected. Without Tracing, you have to go box-by-box and validate all the way up and down the stack. Instead, with NetQ, you can see the pathways the packets flow. This network validation works down Continue reading

Announcing Cumulus NetQ 1.3 — now with Kubernetes!

Today, we are thrilled to announce the availability of Cumulus NetQ 1.3. With this release, Cumulus extends its leadership in container networking insight by integrating NetQ with Kubernetes, along with our previously supported integration with Docker Swarm.

This announcement aligns perfectly with Cumulus’ mission of driving web-scale networks for the digital age with automation and agility by implementing networking with S.O.U.L. Simple. Open. Untethered. Linux. NetQ is woven deep into that S.O.U.L. strategy, which we’ll get to later in a separate blog post. For now, there is a lot to dig into here with this Kubernetes integration with NetQ, so let’s begin.

The growth & challenges with containers

Container technology is all the rage in the CD/DevOps world. Nearly 70% of the companies queried in a Portworx 2017 container adoption survey invested financially in containers at some level in 2017, leaping from 52% in 2016. 451 Research predicts containers will grow to become a $2.7BN market by 2020. That’s 3.5 times greater than the $762 million container market in 2016, with a CAGR of 40%.

The popularity of these Linux-based containers stems from their ability to dramatically improve flexibility when running Continue reading

Introducing the new Cumulus VP of Engineering

It’s a new era for Cumulus technology. We’re thrilled to announce the new Cumulus VP of Engineering – Partho Mishra. Partho joins Cumulus with 25 years of experience in product definition, engineering development and technology evangelization in data science/analytics, wireless and networking.

A truly experienced leader

Prior to joining Cumulus, Partho was running the RASA Network Analytics team at Aruba where he lead as CEO and then VP after RASA was acquired by Aruba in 2016. At RASA analytics, Partho’s team was in charge of developing a product that used ML/AI techniques to analyze data collected from network infrastructure to help Aruba customers with optimizing their wireless network and improving user experience.

Previously, he was Vice President and General Manager of Cisco’s Service Provider Access Business Unit with responsibility for Metro-Ethernet and Wireless Backhaul products including the ASR 901, ASR 903, ME 3400 and CPT 50 products.

Partho is no stranger to bringing startups to the next level. Prior to being acquired, RASA was a VC-funded startup with Khosla Ventures as the lead investor. Additionally, he was involved as part of the initial technology/founding teams in two Silicon Valley start-ups: Airgo Networks (acquired by Qualcomm) and Iospan Wireless (acquired by Continue reading

NetDevOpEd: Open source principles give the workplace soul

If you’ve been tuning into Cumulus content lately, you probably know all about our initiative to bring S.O.U.L (Simple, Open, Untethered Linux) into networking — and for the uninitiated, feel free to check out our S.O.U.L page to learn all about the movement! It’s clear that our company and technology have S.O.U.L, but what does that look like in the office? Anyone who’s spent time at our office, in our bootcamps, with our people, etc. can feel that Cumulus Networks has “soul,” and this time we’re not talking about the acronym. We’ve got a passion, substance, life and feeling that pulses throughout our space like a funky bass line.

How do I know this? Well, I work at Cumulus, but it’s more than that. My former employers (before I started working at Cumulus Networks) are pretty varied, ranging from a restaurant to a university, but all of those jobs had one thing in common — they had no soul. To these Pink Floyd-ian businesses, employees were just cogs in the machine meant to forfeit passion for profit. Each day I dragged myself out of bed, drudged Continue reading

NetDevOpEd: Automation – start small, dream big

I’ve seen a number of blogs and articles describing what network automation is and what it entails, and in many cases, the descriptions end up frightening people who haven’t yet started down an automation path. The biggest question when starting any of these sorts of projects is the simplest: should you automate at all?

My answer to that first question (Spoiler alert: it’s yes, but let me explain why) is that it depends on your network itself. For years, before I was involved with networking at the operating system level, I worked on network management and automation products. Often, I’d tell my customers that if they were happy with the status quo, then I certainly wouldn’t force them down a particular path or to use a particular product. However, if you’re a bit fed up with the manual steps involved in updating a device operating system or configuring a device, then you should look into automation to save yourself time and headaches. Of course, if you only have three devices and they get updated yearly, maybe don’t bother. But if you believe automation will provide the solutions you’re looking for, there are some first steps for automation that you Continue reading

Introducing “Cooking with Cumulus” – Episode one

Alright, we know you’re hungry for more Cumulus goodness, so we’ve cooked up something new that we think will satiate your appetite for awesome technical content. It’s the perfect mix of one part technical deep-dive, one part fun and just a pinch of silliness. The wait is over — our latest project is hot, fresh and ready to serve!

Okay, enough teasing. Today we’re introducing the new Cumulus Networks video series “Cooking with Cumulus!”

“Wait, what does cooking have to do with networking??” you may think to yourself. Glad you asked. You may already know JR Rivers as the CTO and co-founder of Cumulus Networks, but did you know he’s also a master of the culinary arts? Here at the Mountain View office, we know it’s the start of a good day when JR brings in his homemade food to share. From chocolate chip cookies to paella, we’ve approved (and enjoyed) his many recipes. So, we decided that we wanted to share that gift with the rest of the Cumulus community by combining two of JR’s greatest passions — networking and cooking. The resulting brain child was a video series where we put two networking nerds in a Continue reading

Why and how to deploy Voyager

In Part I of this blog series, “What is the open packet optical switch, Voyager?”, we discussed the challenges and remedies for providing additional bandwidth for intra and inter data center connections. DWDM is a powerful technology that provides hundreds of gigabits of bandwidth over hundreds or thousands of kilometers using just a fiber pair. We also reviewed some information about DWDM networks and transponder functionality. Voyager provides all the functionality of Cumulus Linux running on a Broadcom Tomahawk based switch and integrates the transponders into the switch itself, all in 1RU. This makes it the first open and fully integrated box operating at DWDM, Layer 2 and Layer 3 in 1RU, making it extremely flexible.

Incorporating routing, switching and DWDM in one node could mean fewer boxes needed for the network, since DWDM functionality could be incorporated directly into border leafs. Because it runs Cumulus Linux (CL), all CL data center functionality, such as VXLAN Routing with EVPN, is also supported. For example, a pair of Voyager nodes can be used as VXLAN routing centralized routers with EVPN, hosting VXLAN VTEPs, running MLAG, and provide the long distance DWDM connectivity all in one box!

Voyager also Continue reading

Cumulus content roundup: April

It’s the beginning of a brand new month, and you know what that means… it’s time for the Cumulus content roundup! This month, we can’t stop talking about leveraging Linux and disaggregation — and it looks like we’re not the only ones who have white-box fever (congrats on joining the movement, Cisco)! All the webinars, videos and white papers you could ask for are included in this roundup, so grab a comfy seat and check out everything that piques your interest.

The latest from Cumulus

The S.O.U.L revolution: The era of oppressive traditional networking ends today! It’s time to add some S.O.U.L to your data center network. What does S.O.U.L stand for? Watch this video to find out and get into the movement that’s revolutionizing the way we think about networking.

Web-scale networking for telco: Cumulus Networks commissioned Heavy Reading to conduct a survey of 70+ IT leaders in the Telco and CSP space to understand what is top of mind in terms of IT priorities. Download this white paper to see what we discovered about their top concerns.

Why Linux in the data center: a fireside chat: Continue reading

Welcoming Cisco to disaggregation

Earlier this week, Cisco announced that they will be offering a disaggregated solution with their Cisco IOS XR and Nexus operating systems (1). It’s true, the same organization that claimed to have killed white-box networking is jumping on the bandwagon three years later.

Open networking is no longer just the future

It’s now a requirement in today’s innovative data centers. Cumulus was founded on the notion that the future of data center networking is disaggregation, that the industry should be open and that innovation will only prevail when open networking does. The fact that one more incumbent vendor has acknowledged this notion about where the industry is headed only validates our vision. The future of networking truly is here, and we welcome Cisco to the club — really!

In the last few years, and even last several months, we’ve seen open networking takeoff. From the moment we helped bring ONIE to the market back in 2013, we knew things were going to change in the industry. Since then, we’ve seen the list of participating hardware vendors grow like crazy and our customer base grow with them. We’ve seen web-scale companies like Facebook, Google, LinkedIn and more contribute to the Continue reading

Rewind & recap: OCP Summit 2018

As you probably know, Cumulus Networks is an active contributor and enthusiastic member of the OCP community. So naturally, we couldn’t bear to miss OCP Summit 2018! The summit was held in San Jose from March 20th – 21st, and believe us, if you’re into open source anything, it was THE place to be. From BoF sessions to engineering workshops and everything in-between, there’s so much to talk about — but we’ll spare you an essay-length article and keep this short blog limited to our absolute favorite Cumulus OCP Summit highlights from the event. So whether you’re an OCP fan that couldn’t make it or you’re an attendee that wants to reminisce, check out these stellar moments from OCP Summit 2018.

Cumulus’ OCP projects

It was great to hear Omar Baldonado from Facebook give Backpack a shoutout during his keynote address. With the support of OCP, we’ve teamed up our OS with Celestica’s hardware to bring this project to fruition, and what better place to highlight this venture than at OCP Summit? As Baldonado points out, “Cumulus also has been a very long standing partner and contributor and driver within the OCP community,” and we intend to continue those efforts. Continue reading

The Free Range Routing Project turns one: A year in review, and what to expect next

Today, we’re celebrating the one year anniversary of FRR: The Free Range Routing project, a project we at Cumulus Networks set out to collaborate on with innovators in the industry to help shape the future of web-scale networking. With FRRouting (FRR), the community has built on the foundations of Quagga and taken huge steps forward to build the most full-featured, high-performance open routing stack available — making engineers’ lives significantly easier in the process. Now, FRR is the easiest and quickest way for the community to contribute to the future of routing.

To honor its success and growth, we’d like to highlight a few key moments in time since the project began…

Increased adoption and contribution

As we set out to expand the technology, we knew we needed a team of industry leaders. Companies like 6WIND, Architecture Technology Corporation, LabN Consulting, NetDEF (OpenSourceRouting) and Orange were some of the first to collaborate with us at Cumulus Networks on the project’s mission.

At Cumulus, we knew that FRR was going to be a game-changer for our own customers, so we too adopted FRR on Cumulus Linux. Now, all 1,000+ of our customers are benefiting from a more flexible infrastructure.

Over Continue reading

What is the open packet optical switch, Voyager?

Modern web-scale data centers are thirsty for bandwidth. Popular applications such as video and virtual reality are increasing in demand, causing data centers to require higher and higher bandwidths — both within data centers and between data centers. In this blog post, we will briefly discuss the current challenges in the optics space as well as some of the key technical aspects of the Voyager’s DWDM transponder. In part two of this series, we will cover why Voyager is a unique, powerful and robust solution.

The challenges to accommodate longer distances

Within a data center, organizations are adding higher and higher bandwidth ports and connections to accommodate the need for more bandwidth. However, connections that accommodate longer distances between data centers may be limited and expensive. Therefore, a critical requirement for businesses with this challenge is how to support longer distance spans at higher bandwidths over a small amount of fiber pairs.

The optical industry solves the bandwidth problem using Dense Wave Division Multiplexing (DWDM). DWDM allows many separate connections on one fiber pair by sending them over different wavelengths. Although the wavelengths are sent on the same physical fiber, they act as “ships in the night” and don’t interact Continue reading

1 2 3 11