DNSSEC issues take Fiji domains offline
On the morning of March 8, a post to Hacker News stated that “All .fj domains have gone offline”, listing several hostnames in domains within the Fiji top level domain (known as a ccTLD) that had become unreachable. Commenters in the associated discussion thread had mixed results in being able to reach .fj hostnames—some were successful, while others saw failures. The fijivillage news site also highlighted the problem, noting that the issue also impacted Vodafone’s M-PAiSA app/service, preventing users from completing financial transactions.
The impact of this issue can be seen in traffic to Cloudflare customer zones in the .com.fj second-level domain. The graph below shows that HTTP traffic to these zones dropped by approximately 40% almost immediately starting around midnight UTC on March 8. Traffic volumes continued to decline throughout the rest of the morning.
Looking at Cloudflare’s 1.1.1.1 resolver data for queries for .com.fj hostnames, we can also see that error volume associated with those queries climbs significantly starting just after midnight as well. This means that our resolvers encountered issues with the answers from .fj servers.
This observation suggests that the problem was strictly DNS related, rather than connectivity related—Cloudflare Radar Continue reading
Segment Routing vs LDP in Hub-and-Spoke Networks
I got an interesting question that nicely illustrates why Segment Routing (the MPLS variant) is so much better than LDP. Imagine a redundant hub-and-spoke network with hundreds of spokes. Let’s settle on 500 spokes – IS-IS supposedly has no problem dealing with a link-state topology of that size.
Let’s further assume that all routers advertise only their loopbacks1 and that we’re using unnumbered hub-to-spoke links to minimize the routing table size. The global routing table thus contains ~500 entries. MPLS forwarding tables (LFIB) contain approximately as many entries as each router assigns a label to every prefix in the routing table2. What about the LDP table (LIB – Label Information Base)?
Announcing experimental DDR in 1.1.1.1
1.1.1.1 sees approximately 600 billion queries per day. However, proportionally, most queries sent to this resolver are over cleartext: 89% over UDP and TCP combined, and the remaining 11% are encrypted. We care about end-user privacy and would prefer to see all of these queries sent to us over an encrypted transport using DNS-over-TLS or DNS-over-HTTPS. Having a mechanism by which clients could discover support for encrypted protocols such as DoH or DoT will help drive this number up and lead to more name encryption on the Internet. That’s where DDR – or Discovery of Designated Resolvers – comes into play. As of today, 1.1.1.1 supports the latest version of DDR so clients can automatically upgrade non-secure UDP and TCP connections to secure connections. In this post, we’ll describe the motivations for DDR, how the mechanism works, and, importantly, how you can test it out as a client.
DNS transports and public resolvers
We initially launched our public recursive resolver service 1.1.1.1 over three years ago, and have since seen its usage steadily grow. Today, it is one of the fastest public recursive resolvers available to end-users, supporting the latest security Continue reading
Device Management From The Ground Up: Part 1 – Accessing Your First Device
This post originally appeared on the Packet Pushers’ Ignition site on April 5, 2021. In this 8-part series, I explore the basics of managing network devices. This series is geared for the novice, but I hope readers with a bit more experience will find something useful as well. I cover the following topics: Device access […]
The post Device Management From The Ground Up: Part 1 – Accessing Your First Device appeared first on Packet Pushers.
How to Address Cloud Misconfiguration-Caused Breaches
While eliminating misconfigurations may be nigh impossible, we can limit them and the potential damage they may cause.CVE-2022-26143: A Zero-Day vulnerability for launching UDP amplification DDoS attacks
A zero-day vulnerability in the Mitel MiCollab business phone system has recently been discovered (CVE-2022-26143). This vulnerability, called TP240PhoneHome, which Cloudflare customers are already protected against, can be used to launch UDP amplification attacks. This type of attack reflects traffic off vulnerable servers to victims, amplifying the amount of traffic sent in the process by an amplification factor of 220 billion percent in this specific case.
Cloudflare has been actively involved in investigating the TP240PhoneHome exploit, along with other members of the InfoSec community. Read our joint disclosure here for more details. As far as we can tell, the vulnerability has been exploited as early as February 18, 2022. We have deployed emergency mitigation rules to protect Cloudflare customers against the amplification DDoS attacks.
Mitel has been informed of the vulnerability. As of February 22, they have issued a high severity security advisory advising their customers to block exploitation attempts using a firewall, until a software patch is made available. Cloudflare Magic Transit customers can use the Magic Firewall to block external traffic to the exposed Mitel UDP port 10074 by following the example in the screenshot below, or by pasting the following expression into their Magic Firewall Continue reading
CVE-2022-26143: TP240PhoneHome reflection/amplification DDoS attack vector
Beginning in mid-February 2022, security researchers, network operators, and security vendors observed a spike in DDoS attacks sourced from UDP port 10074 targeting broadband access ISPs, financial institutions, logistics companies, and organizations in other vertical markets.
Upon further investigation, it was determined that the devices abused to launch these attacks are MiCollab and MiVoice Business Express collaboration systems produced by Mitel, which incorporate TP-240 VoIP- processing interface cards and supporting software; their primary function is to provide Internet-based site-to-site voice connectivity for PBX systems.
Approximately 2600 of these systems have been incorrectly provisioned so that an unauthenticated system test facility has been inadvertently exposed to the public Internet, allowing attackers to leverage these PBX VoIP gateways as DDoS reflectors/amplifiers.
Mitel is aware that these systems are being abused to facilitate high-pps (packets-per-second) DDoS attacks, and have been actively working with customers to remediate abusable devices with patched software that disables public access to the system test facility.
In this blog, we will further explore the observed activity, explain how the driver has been abused, and share recommended mitigation steps. This research was created cooperatively among a team of researchers from Akamai SIRT, Cloudflare, Lumen Black Lotus Labs, NETSCOUT ASERT, Continue reading
Cloud Networking Needs a New Vision
While hyperscale public clouds grab attention, the majority of workloads and cloud infrastructure will continue to remain elsewhere for the foreseeable future. Enterprise private clouds are not only NOT disappearing but growing, spanning on-premises data centers, colocation sites and increasingly distributed edge sites. Tier 2 cloud service providers cater to local markets and provide services more closely tailored to their customers’ needs. Telecom service providers operate highly distributed clouds to support their network services.
These cloud operators all have two similar goals for their network infrastructure, goals that are so critical to remaining competitive that we can even call them mandates:
- Transform cloud networks to become as agile, highly available and simple to operate as the hyperscale public clouds.
- Move rapidly toward a new, more highly distributed networking and zero-trust security architecture to address increasing cybersecurity risks.
Unfortunately, achieving these goals is far from simple. Current networking solutions are not only insufficient, they are in many ways the biggest problem. The below datapoint, from The State of Data Center Networking: 2021 Annual Report, illustrates just that. The top two challenges in achieving a highly agile and available active-active or active-hot standby data center architectures are both related to network Continue reading
International Women’s Day 2022
“I would venture to guess that Anon,
who wrote so many poems without signing them,
was often a woman.” - Virginia Woolf
Welcome to International Women’s Day 2022! Here at Cloudflare, we are happy to celebrate it with you! Our celebration is not only this blog post, but many events prepared for the month of March: our way of honoring Women’s History Month by showcasing women’s empowerment. We want to celebrate the achievements, ideas, passion and work that women bring to the world. We want to advocate for equality and to achieve gender parity. And we want to highlight the brilliant work that our women colleagues do every day. Welcome!
This is a time of celebration but also one to reflect on the current state. The global gender gap is not expected to close for another 136 years. This gap has also worsened due to the COVID-19 pandemic, which has negatively impacted the lives of women and girls by deepening pre-existing inequalities. Improving this state is a collective effort—we all need to get involved!
Who are we? Womenflare!
First, let’s introduce ourselves. We are Womenflare—Cloudflare’s Employee Resource Group (ERG) for all who identify as and advocate for Continue reading
Flow-Based Packet Forwarding
In the Cache-Based Packet Forwarding blog post I described what happens when someone tries to bypass the complexities of IP routing table lookup with a forwarding cache.
Now imagine you want to implement full-featured fast packet forwarding including ingress- and egress ACL, NAT, QoS… but find the required hardware (TCAM) too expensive. Wouldn’t it be nice if we could send the first packet of every flow to a CPU to figure out what to do with it, and download the results into a high-speed flow cache where they could be used to switch the subsequent packets of the same flow. Welcome to flow-based packet forwarding.
How To Create A Python Function You Can Call From Other Scripts
Python gives you the ability to write a bit of code and the call that code as a function. You can call the function from within the same script where the function is defined, or you can save the function in a separate script and then import the function inside of other scripts.
Writing and calling functions is a key component of the Don’t Repeat Yourself (DRY) principle of software development. Creating a function in a single script and calling that function from other scripts is preferable to performing copypasta of the same bit of code throughout several scripts. When a function lives in a single script, it only needs to be updated in that one place when it inevitably needs updating.
While Python functions can perform isolated tasks, my typical use cases send values into the function and receive a value returned from the function. In this example, I’ll import a Python function used to refresh an access token required to authenticate to a remote API endpoint. I’ll pass other tokens required to refresh the access token into the function, and the function will return the refreshed access token back to the calling script.
The Function
The names of Continue reading