PIPEFAIL: How a missing shell option slowed Cloudflare down
At Cloudflare, we’re used to being the fastest in the world. However, for approximately 30 minutes last December, Cloudflare was slow. Between 20:10 and 20:40 UTC on December 16, web requests served by Cloudflare were artificially delayed by up to five seconds before being processed. This post tells the story of how a missing shell option called “pipefail” slowed Cloudflare down.
Background
Before we can tell this story, we need to introduce you to some of its characters.
Cloudflare’s Front Line protects millions of users from some of the largest attacks ever recorded. This protection is orchestrated by a sidecar service called dosd, which analyzes traffic and looks for attacks. When dosd detects an attack, it provides Front Line with a list of attack fingerprints that describe how Front Line can match and block the attack traffic.
Instances of dosd run on every Cloudflare server, and they communicate with each other using a peer-to-peer mesh to identify malicious traffic patterns. This decentralized design allows dosd to perform analysis with much higher fidelity than is possible with a centralized system, but its scale also imposes some strict performance requirements. To meet these requirements, we need to provide dosd with very Continue reading
Cisco BGP Weight Attribute 3 Things you must know!
BGP Weight Attribute is used in Cisco routers. In this post, with the below topology, we will look at why the BGP weight attribute is used, why it BGP weight shouldn’t be used, advantages and disadvantages of the BGP weight attribute.
Cisco BGP Weight Attribute
Let’s first define what is BGP Weight attribute. BGP selects the best path based on the BGP path attributes. Weight is considered a very important tie-breaker in BGP’s best-path selection.
When there are two paths to any BGP destination prefix, the BGP Weight attribute is compared before BGP Local Preference and many other BGP Path attributes.
Since this is not BGP’s best path selection post, and assuming you already know the process, please note, Weight attribute is compared before even BGP Local Preference.
But, let’s have a look at the below topology to understand it better.
In the above topology, we want to use the left path for the prefixes in AS1, thus we have a higher BGP Local preference value.
As the BGP Local preference value is exchanged internally between all IBGP neighbors, both left and right routers in AS65000, use the left exit point, which is Local Pref 100 to reach the Continue reading
BGP Interview Questions
BGP Interview questions and answers are shared here. In this post, we will look at some of the important BGP questions that are asked in the Interviews and some of the certification exams. You can consider this as a BGP Quiz and test your BGP knowledge.
Which of the below option is the reason to run IBGP? (Choose Two)
A. It is used for the reachability between PE devices in the MPLS network
B. It is used to carry EBGP prefixes inside an Autonomous System
C. It is used with Route Reflectors for the scalability reason in large scale networks
D. It is used to prevent failures outside your network from impacting your internal network operation
Answer: One of the correct answers to this question is to carry EBGP prefixes inside an Autonomous system. IGP is used for the reachability between PE devices in an MPLS network. Option C is valid but not the correct answer, because; the question is asking about the reasons, not the best practices.
Option D is one of the correct answers as well because, with IBGP, the internal network is protected from outside failures by separating the local failure domains.
That’s why; the answers to Continue reading
BGP Labeled Unicast Interoperability Challenges
Jeff Tantsura left me tantalizing hint after reading the BGP Labeled Unicast on Cisco IOS blog post:
Read carefully “Relationship between SAFI-4 and SAFI-1 Routes” section in RFC 8277
The start of that section doesn’t look promising (and it gets worse):
It is possible that a BGP speaker will receive both a SAFI-11 route for prefix P and a SAFI-42 route for prefix P. Different implementations treat this situation in different ways.
Now for the details:
BGP Labeled Unicast Interoperability Challenges
Jeff Tantsura left me tantalizing hint after reading the BGP Labeled Unicast on Cisco IOS blog post:
Read carefully “Relationship between SAFI-4 and SAFI-1 Routes” section in RFC 8277
The start of that section doesn’t look promising (and it gets worse):
It is possible that a BGP speaker will receive both a SAFI-11 route for prefix P and a SAFI-42 route for prefix P. Different implementations treat this situation in different ways.
Now for the details:
Tech Bytes: Why Arelion Pushes For Open Optical Networks (Sponsored)
Today on the Tech Bytes podcast we’re talking about disaggregated optical networks and what they mean for both customers and carriers when it comes to issues such as service delivery and cost. Our sponsor is Arelion.
The post Tech Bytes: Why Arelion Pushes For Open Optical Networks (Sponsored) appeared first on Packet Pushers.
Tech Bytes: Why Arelion Pushes For Open Optical Networks (Sponsored)
Today on the Tech Bytes podcast we’re talking about disaggregated optical networks and what they mean for both customers and carriers when it comes to issues such as service delivery and cost. Our sponsor is Arelion.Device Management From The Ground Up: Part 5 – External File Management
This post originally appeared on the Packet Pushers’ Ignition site on April 16, 2021. We’ve explored the various filesystems on Cisco devices, but you may have wondered how we get new files onto our devices. How do we backup files from our devices? This article explains copying files to and from a Cisco device. Additionally, […]
The post Device Management From The Ground Up: Part 5 – External File Management appeared first on Packet Pushers.
OSPF FAQ – Frequently Asked Questions
In this post, some of the frequently asked OSPF questions will be answered. Some of the answers will be from a design point of view and we will try to provide enough justification for the answer. Questions are selected randomly, not based on the order of importance.
How many routers should be placed in one OSPF area?
We don’t give any numerical number as an answer to this question. Depending on the number of routers, links, prefixes, and the topology of the network also depends on the hardware capabilities and the performance of the routers, the number change from Network to network. In some networks, you can place only a couple of hundreds, and in some networks, you can place thousands of Routers in a single OSPF Area.
What command would you use to only see the OSPF routes in the routing table?
On Cisco devices, sh ip route ospf is used to see only the OSPF routes in the routing table.
What will an OSPF router prefer to use first as a router-id?
The first preference for an OSPF router ID is an explicitly configured 32-bit address. This address is not included in the routing table and is not Continue reading
BGP Policies (Part 4)
At the most basic level, there are only three BGP policies: pushing traffic through a specific exit point; pulling traffic through a specific entry point; preventing a remote AS (more than one AS hop away) from transiting your AS to reach a specific destination. In this series I’m going to discuss different reasons for these kinds of policies, and different ways to implement them in interdomain BGP.
In this post, I’ll cover the first of a few ways to give surrounding autonomous systems a hint about where traffic should enter a network. Note this is one of the most vexing problems in BGP policy, so there will be a lot of notes across the next several posts about why some solutions don’t work all that well, or when they will and won’t work.
There are at least three reasons an operator may want to control the point at which traffic enters their network, including:
- Controlling the inbound load on each link. It might be important to balance inbound and outbound load to maintain settlement-free peering, or to equally use all available inbound bandwidth, or to ensure the quality of experience is not impacted by overusing a single link.
- Accounting for Continue reading