In June 2020 I published the first part of Redundant Server Connectivity in Layer-3-Only Fabrics article describing the target design and application-layer requirements.
During the summer I added the details of multi-subnet server and client connectivity and a few conclusions.
Today CenturyLink/Level(3), a major ISP and Internet bandwidth provider, experienced a significant outage that impacted some of Cloudflare’s customers as well as a significant number of other services and providers across the Internet. While we’re waiting for a post mortem from CenturyLink/Level(3), I wanted to write up the timeline of what we saw, how Cloudflare’s systems routed around the problem, why some of our customers were still impacted in spite of our mitigations, and what appears to be the likely root cause of the issue.
At 10:03 UTC our monitoring systems started to observe an increased number of errors reaching our customers’ origin servers. These show up as “522 Errors” and indicate that there is an issue connecting from Cloudflare’s network to wherever our customers’ applications are hosted.
Cloudflare is connected to CenturyLink/Level(3) among a large and diverse set of network providers. When we see an increase in errors from one network provider, our systems automatically attempt to reach customers’ applications across alternative providers. Given the number of providers we have access to, we are generally able to continue to route traffic even when one provider has an issue.
It’s time for a round up of Docker Talks, this time from the month of August. As you may remember, Chad Metcalf (@metcalfc) and I (@pmckee) started the weekly live-streaming video series to connect with you, our extended family of developers, and to help you succeed in your Docker journey.
In August, we held four sessions covering how to set up your local development environment with Node.js, Visual Studio remote debugging extension, the Awesome Compose project and common questions people have when starting with Docker. Below, I’ve put together the list of live streams for the month for your viewing and learning pleasure.
We live stream on our YouTube channel every Wednesday at 10 a.m. Pacific Time. You’ll find all of the past streams there and you can subscribe to get notifications. See you on the next live stream.
Docker Talks Live! Setting up your local development environment with Node.js
Chad and I explore how to set up your local development environment with Node.js and debugging inside of containers. (Streamed live Aug. 5)
Docker Live! Debugging Node.js with VSCode Docker Extension
I talk about Visual Studio remote debugging extension, do some Continue reading
The Internet Society, in partnership with South Asia Network Operators Group (SANOG) recently concluded a five-week, hands-on training course for entry level network engineers and system administrators from South Asia. The online course Introduction to Network Operations, which took place from June 15 to July 19, prepared professionals to take advantage of the new opportunities the Internet is creating. The training provided practical learning about UNIX/Linux, networking, and the Domain Name System (DNS) to over 40 participants from Research Education Networks (RENs), government institutions, network operators, universities, and private institutions. SANOG subject matter experts Thilina Pathirana from Sri Lanka and Gazi Zehadul Kabir from Bangladesh moderated via Moodle, an e-learning platform.
Skills and Knowledge for Digital Transformation
The course aimed to prepare young engineers for the future. The technical skills and hands-on knowledge enable them to build expertise to advance professionally in their chosen field of network and system administration. The course also served as a common platform for South Asia community members to actively interact, exchange knowledge, and learn from one another.
For participants, the course was a great learning experience. “It was the best online course I had yet,” said Afaq Ahmed from Pakistan. For young engineers Shreekar Continue reading
The networking community seems to have coalesced around Ansible for network device automation and Terraform for cloud. Why this divide? Are the two tools really that different? Today's Heavy Networking explores how and where you can use both these tools, with guests Ned Bellavance and Josh VanDeraa.
The post Heavy Networking 537: Ansible Vs. Terraform For Network Automation appeared first on Packet Pushers.
My friend Joshua Williams (@802DotMe) texted me today with a great quote that I wanted to share with you that made me think about certifications:
You’ve probably already thought through this extensively, and maybe even written about it, but after sitting through another 8 hour practical exam yesterday I’m more convinced than ever that expert level exams from technical companies are more analogous to a gimmicky Food Network TV show than real world application of technical acumen. They don’t care so much about my skill level as they do about what kind of meal I can prepare in 30 minutes using Tialapia, grapes, and Dr. Pepper syrup with my salt shaker taken away halfway through.
I laughed because it’s true. And then I thought about it more and realized he’s way more than right. We know for a fact that companies love to increase the level of challenge in their exams from novice to expert. It’s a way to weed out the people that aren’t committed to learning about something. However, as the questions and tasks get harder it becomes much more difficult to get a good sense of how candidates are going to perform.
Last year, we launched HTMLRewriter for Cloudflare Workers, which enables developers to make streaming changes to HTML on the edge. Unlike a traditional DOM parser that loads the entire HTML document into memory, we developed a streaming parser written in Rust. Today, we’re announcing support for asynchronous handlers in HTMLRewriter. Now you can perform asynchronous tasks based on the content of the HTML document: from prefetching fonts and image assets to fetching user-specific content from a CMS.
We designed HTMLRewriter to have a jQuery-like experience. First, you define a handler, then you assign it to a CSS selector; Workers does the rest for you. You can look at our new and improved documentation to see our supported list of selectors, which now include nth-child
selectors. The example below changes the alternative text for every second image in a document.
async function editHtml(request) {
return new HTMLRewriter()
.on("img:nth-child(2)", new ElementHandler())
.transform(await fetch(request))
}
class ElementHandler {
element(e) {
e.setAttribute("alt", "A very interesting image")
}
}
Since these changes are applied using streams, we maintain a low TTFB (time to first byte) and users never know the HTML was transformed. If you’re interested in how we’re Continue reading
The last Fallacy of Distributed Computing I addressed in the introductory part of How Networks Really Work webinar was The Network Is Homogenous. No, it’s not and it never was… for more details watch this video.
Subscribe to Kernel of Truth on iTunes, Google Play, Spotify, Cast Box and Sticher!
Click here for our previous episode.
Some of your favorites are back together on this episode of the Kernel of Truth podcast— specifically Roopa Prabhu, Brian O’Sullivan and Pete Lumbis. Things have changed a little around here since the last time the three of them were chatting together on the podcast but one thing hasn’t, how much they love to talk all things open networking. In this episode the group talks about how to navigate the open networking operating systems space. From figuring out how to choose an open network operating system, to understanding what works best for deployments, and even what resources and communities are out there for you to tap into. We have it all here for you to help you get started. All you have to do is sit back and enjoy the episode and don’t forget to also check out the links below with resources referenced in the podcast.
Guest Bios
Roopa Prabhu: Roopa is a Linux Architect at NVIDIA, formally Cumulus Networks. She and her team work on all things kernel networking and Linux system infrastructure areas. Her primary Continue reading
By Cody McCain, Senior Product Manager and Susan Wu, Senior Product Marketing Manager, Networking and Security Business Unit
Enterprises benefit from collaborative engineering and receive the latest innovations from open source projects. However, it’s a challenge for enterprise to rely solely on community support to run their operations. This is because community support is best-effort and cannot provide a pre-defined SLA.
While Kubernetes itself is open source, and part of Cloud Native Computing Foundation (CNCF), it takes an ecosystem of surrounding technologies as curated by CNCF—from the container registry and storage engine to the container network plugin to run Kubernetes.
With the new release of VMware Container Networking with Antrea, enterprises get the best of both worlds – access to the latest innovation from Project Antrea and world-class support from VMware. Container Networking with Antrea is the commercial offering consisting of and 24/7 support for Project Antrea.
Container Networking with Antrea will package the latest release of Project Antrea version 0.9.1. Antrea is a purpose-built Kubernetes networking solution for public and private clouds building upon Open vSwitch, the open source technology optimized for distributed multi-layer switching performance. Antrea is designed to run anywhere Kubernetes Continue reading