Archive

Category Archives for "Networking"

Applying Essentialism to certifications and skills development in the Tech Industry

We often compare ourselves to others around us. We are impressed with the skills others possess, the content others produce, the appearances others maintain, the successes others have achieved, the feats others have conquered. This constant comparison can lead to melancholic states of ambivalence, and sometimes depression due to the artificial expectations of who we […]

The post Applying Essentialism to certifications and skills development in the Tech Industry appeared first on Overlaid.

The Serverlist: Serverless makes a splash at JSConf EU and JSConf Asia

The Serverlist: Serverless makes a splash at JSConf EU and JSConf Asia

Check out our sixth edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.

The Serverlist: Serverless makes a splash at JSConf EU and JSConf Asia

The Serverlist: Serverless makes a splash at JSConf EU and JSConf Asia

Check out our sixth edition of The Serverlist below. Get the latest scoop on the serverless space, get your hands dirty with new developer tutorials, engage in conversations with other serverless developers, and find upcoming meetups and conferences to attend.

Sign up below to have The Serverlist sent directly to your mailbox.

Verizon 和某 BGP 优化器如何在今日大范围重创互联网

大规模路由泄漏影响包括 Cloudflare 在内的主要互联网服务

事件经过

Verizon 和某 BGP 优化器如何在今日大范围重创互联网

在 UTC 时间今天( 2019年6月24号)10 点 30 分,互联网遭受了一场不小的冲击。通过主要互联网转接服务提供供应商 Verizon (AS701) 转接的许多互联网路由,都被先导至宾夕法尼亚北部的一家小型公司。这相当于  滴滴错误地将整条高速公路导航至某小巷 – 导致大部分互联网用户无法访问 Cloudflare 上的许多网站和许多其他供应商的服务。这本不应该发生,因为 Verizon 不应将这些路径转送到互联网的其余部分。要理解事件发生原因,请继续阅读。

此类不幸事件并不罕见,我们以前曾发布相关博文。这一次,全世界都再次见证了它所带来的严重损害。而Noction “BGP 优化器”产品的涉及,则让今天的事件进一步恶化。这个产品有一个功能:可将接收到的 IP 前缀拆分为更小的组成部分(称为更具体前缀)。例如,我们自己的 IPv4 路由 104.20.0.0/20 被转换为 104.20.0.0/21 和 104.20.8.0/21。就好像通往 ”北京”的路标被两个路标取代,一个是 ”北京东”,另一个是 ”北京西”。通过将这些主要 IP 块拆分为更小的部分,网络可引导其内部的流量,但这种拆分原本不允许向全球互联网广播。正是这方面的原因导致了今天的网络中断。

为了解释后续发生的事情,我们先快速回顾一下互联网基础“地图”的工作原理。“Internet”的字面意思是网络互联,它由叫做自治系统(AS)的网络组成,每个网络都有唯一的标识符,即 AS 编号。所有这些网络都使用边界网关协议(BGP)来进行互连。BGP 将这些网络连接在一起,并构建互联网“地图”,使通信得以一个地方(例如,您的 ISP)传播到地球另一端的热门网站。

通过 BGP,各大网络可以交换路由信息:如何从您所在的地址访问它们。这些路由可能很具体,类似于在 GPS 上查找特定城市,也可能非常宽泛,就如同让 GPS 指向某个省。这就是今天的问题所在。

宾夕法尼亚州的互联网服务供应商(AS33154 - DQE Communications)在其网络中使用了 BGP 优化器,这意味着其网络中有很多更具体的路由。具体路由优先于更一般的路由(类似于在滴滴 中,前往北京火车站的路线比前往北京的路线更具体)。

DQE 向其客户(AS396531 - Allegheny)公布了这些具体路由。然后,所有这些路由信息被转送到他们的另一个转接服务供应商 (AS701 - Verizon),后者则将这些"更好"的路线泄露给整个互联网。由于这些路由更精细、更具体,因此被误认为是“更好”的路由。

泄漏本应止于 Verizon。然而,Verizon 违背了以下所述的众多最佳做法,因缺乏过滤机制而导致此次泄露变成重大事件,影响到众多互联网服务,如亚马逊、Linode 和 Cloudflare

这意味着,Verizon、Allegheny 和 DQE 突然之间必须应对大量试图通过其网络访问这些服务的互联网用户。由于这些网络没有适当的设备来应对流量的急剧增长,导致服务中断。即便他们有足够的能力来应对流量剧增,DQE、Allegheny 和 Verizon 也不应该被允许宣称他们拥有访问 Cloudflare、亚马逊、Linode 等服务的最佳路由...

Verizon 和某 BGP 优化器如何在今日大范围重创互联网
涉及 BGP 优化器的 BGP 泄漏过程

在事件中,我们观察到,在最严重的时段,我们损失了大约 15% 的全球流量。

Verizon 和某 BGP 优化器如何在今日大范围重创互联网
事件中 Cloudflare 的流量水平。

如何防止此类泄漏?

有多种方法可以避免此类泄漏:

一,在配置 BGP 会话的时候,可以对其接收的路由网段的数量设置硬性限制。这意味着,如果接收到的路由网段数超过阈值,路由器可以决定关闭会话。如果 Verizon 有这样的前缀限制,这次事件就不会发生。设置前缀数量限制是最佳做法。像 Verizon 这样的供应商可以无需付出任何成本便可以设置。没有任何合理的理由可以解释为什么他们没有设置这种限制的原因,只能归咎为草率或懒惰可能就是因为草率或懒惰。

二,网络运营商防止类似泄漏的另外一种方法是:实施基于 IRR (Internet Routing Registry)的筛选。IRR 是因特网路由注册表,各网络所有者可以将自己的网段条目添加到这些分布式数据库中。然后,其他网络运营商可以使用这些 IRR 记录,在与同类运营商产生BGP 会话时,生成特定的前缀列表。如果使用了 IRR 过滤,所涉及的任何网络都不会接受更具体的网段前缀。非常令人震惊的是,尽管 IRR 过滤已经存在了 24 年(并且有详细记录),Verizon 没有在其与 Allegheny的 BGP 会话中实施任何此类过滤。IRR 过滤不会增加 Verizon 的任何成本或限制其服务。再次,我们唯一能想到的原因是草率或懒惰。

三,我们去年在全球实施和部署的 RPKI 框架旨在防止此类泄漏。它支持对源网络和网段大小进行过滤。Cloudflare 发布的网段最长不超过 20 位。然后,RPKI 会指示无论路径是什么,都不应接受任何更具体网段前缀。要使此机制发挥作用,网络需要启用 BGP 源验证。AT&T 等许多供应商已在其网络中成功启用此功能

如果 Verizon 使用了 RPKI,他们就会发现播发的路由无效,路由器就能自动丢弃这些路由。

Cloudflare 建议所有网络运营商立即部署 RPKI

Verizon 和某 BGP 优化器如何在今日大范围重创互联网
使用 IRR、RPKI 和前缀限制预防路由泄漏

上述所有建议经充分浓缩后已纳入 MANRS(共同协议路由安全规范

事件解决

Cloudflare 的网络团队联系了涉事网络:AS33154 (DQE Communications) 和 AS701 (Verizon)。联系过程并不顺畅,这可能是由于路由泄露发生在美国东海岸凌晨时间。

Verizon 和某 BGP 优化器如何在今日大范围重创互联网
发送给 Verizon 的电子邮件截图

我们的一位网络工程师迅速联系了 DQE Communications,稍有耽搁之后,他们帮助我们与解决该问题的相关人员取得了联系。DQE 与我们通过电话合作,停止向 Allegheny 播发这些“优化”路线路由。我们对他们的帮助表示感谢。采取此措施后,互联网变回稳定,事态恢复正常。

Verizon 和某 BGP 优化器如何在今日大范围重创互联网
尝试与 DQE 和 Verizon 客服进行沟通的截图

很遗憾,我们尝试通过电子邮件和电话联系 Verizon,但直到撰写本文时(事件发生后超过 8 小时),尚未收到他们的回复,我们也不清楚他们是否正在采取行动以解决问题。

Cloudflare 希望此类事件永不发生,但很遗憾,当前的互联网环境在预防防止此类事件方面作出的努力甚微。现在,业界都应该通过 RPKI 等系统部署更好,更安全的路由的时候。我们希望主要供应商能效仿 Cloudflare、亚马逊和 AT&T,开始验证路由。尤其并且,我们正密切关注 Verizon 并仍在等候其回复。

尽管导致此次服务中断的事件并非我们所能控制,但我们仍对此感到抱歉。我们的团队非常关注我们的服务,在发现此问题几分钟后,即已安排美国、英国、澳大利亚和新加坡的工程师上线解决问题。

How Verizon and a BGP Optimizer Knocked Large Parts of the Internet Offline Today

Massive route leak impacts major parts of the Internet, including Cloudflare

How Verizon and a BGP Optimizer Knocked Large Parts of the Internet Offline Today

What happened?

Today at 10:30UTC, the Internet had a small heart attack. A small company in Northern Pennsylvania became a preferred path of many Internet routes through Verizon (AS701), a major Internet transit provider. This was the equivalent of Waze routing an entire freeway down a neighborhood street — resulting in many websites on Cloudflare, and many other providers, to be unavailable from large parts of the Internet. This should never have happened because Verizon should never have forwarded those routes to the rest of the Internet. To understand why, read on.

We have blogged about these unfortunate events in the past, as they are not uncommon. This time, the damage was seen worldwide. What exacerbated the problem today was the involvement of a “BGP Optimizer” product from Noction. This product has a feature that splits up received IP prefixes into smaller, contributing parts (called more-specifics). For example, our own IPv4 route 104.20.0.0/20 was turned into 104.20.0.0/21 and 104.20.8.0/21. It’s as if the road sign directing traffic to “Pennsylvania” was replaced by two road signs, one for “Pittsburgh, PA” and Continue reading

How Verizon and a BGP Optimizer Knocked Large Parts of the Internet Offline Today

Massive route leak impacts major parts of the Internet, including Cloudflare

How Verizon and a BGP Optimizer Knocked Large Parts of the Internet Offline Today

What happened?

Today at 10:30UTC, the Internet had a small heart attack. A small company in Northern Pennsylvania became a preferred path of many Internet routes through Verizon (AS701), a major Internet transit provider. This was the equivalent of Waze routing an entire freeway down a neighborhood street — resulting in many websites on Cloudflare, and many other providers, to be unavailable from large parts of the Internet. This should never have happened because Verizon should never have forwarded those routes to the rest of the Internet. To understand why, read on.

We have blogged about these unfortunate events in the past, as they are not uncommon. This time, the damage was seen worldwide. What exacerbated the problem today was the involvement of a “BGP Optimizer” product from Noction. This product has a feature that splits up received IP prefixes into smaller, contributing parts (called more-specifics). For example, our own IPv4 route 104.20.0.0/20 was turned into 104.20.0.0/21 and 104.20.8.0/21. It’s as if the road sign directing traffic to “Pennsylvania” was replaced by two road signs, one for “Pittsburgh, PA” and Continue reading

Comment Verizon et un optimiseur BGP ont affecté de nombreuses partie d’Internet aujourd’hu

Une fuite massive de routes a eu un impact sur de nombreuses parties d'Internet, y compris sur Cloudflare

Que s'est-il passé ?

Comment Verizon et un optimiseur BGP ont affecté de nombreuses partie d’Internet aujourd’hu

Aujourd'hui à 10h30 UTC, Internet a connu une sorte de mini crise cardiaque. Une petite entreprise du nord de la Pennsylvanie est devenue le chemin privilégié de nombreuses routes Internet à cause de Verizon (AS701), un important fournisseur de transit Internet. C’est un peu comme si Waze venait à diriger le trafic d’une autoroute complète vers une petite rue de quartier : de nombreux sites Web sur Cloudflare et beaucoup d’autres fournisseurs étaient indisponibles depuis une grande partie du réseau. Cet incident n'aurait jamais dû arriver, car Verizon n'aurait jamais dû transmettre ces itinéraires au reste d’Internet. Pour en comprendre les raisons, lisez la suite de cet article.

Nous avons déjà écrit un certain nombre d’articles par le passé sur ces événements malheureux qui sont plus fréquents qu’on ne le pense. Cette fois, les effets ont pu être observés dans le monde entier. Aujourd’hui, le problème a été aggravé par l’implication d’un produit « Optimiseur BGP » de Noction. Ce produit dispose d’une fonctionnalité qui permet de diviser les préfixes IP reçus en parties contributives plus petites (appelées « Continue reading

Comment Verizon et un optimiseur BGP ont affecté de nombreuses partie d’Internet aujourd’hu

Une fuite massive de routes a eu un impact sur de nombreuses parties d'Internet, y compris sur Cloudflare

Que s'est-il passé ?

Comment Verizon et un optimiseur BGP ont affecté de nombreuses partie d’Internet aujourd’hu

Aujourd'hui à 10h30 UTC, Internet a connu une sorte de mini crise cardiaque. Une petite entreprise du nord de la Pennsylvanie est devenue le chemin privilégié de nombreuses routes Internet à cause de Verizon (AS701), un important fournisseur de transit Internet. C’est un peu comme si Waze venait à diriger le trafic d’une autoroute complète vers une petite rue de quartier : de nombreux sites Web sur Cloudflare et beaucoup d’autres fournisseurs étaient indisponibles depuis une grande partie du réseau. Cet incident n'aurait jamais dû arriver, car Verizon n'aurait jamais dû transmettre ces itinéraires au reste d’Internet. Pour en comprendre les raisons, lisez la suite de cet article.

Nous avons déjà écrit un certain nombre d’articles par le passé sur ces événements malheureux qui sont plus fréquents qu’on ne le pense. Cette fois, les effets ont pu être observés dans le monde entier. Aujourd’hui, le problème a été aggravé par l’implication d’un produit « Optimiseur BGP » de Noction. Ce produit dispose d’une fonctionnalité qui permet de diviser les préfixes IP reçus en parties contributives plus petites (appelées « Continue reading

Tech Bytes: Intent Engineering And Intelligent Networking With Gluware (Sponsored)

Gluware sponsors today's Tech Bytes episode. Greg Ferro and Gluware CEO Jeff Gray discuss network automation, how Gluware's software works, how it lets you solve immediate problems in your current environment while working toward intent, and more.

The post Tech Bytes: Intent Engineering And Intelligent Networking With Gluware (Sponsored) appeared first on Packet Pushers.

Today is a bit of a sad day for Network Collective…

It’s with quite a heavy heart that that I’m announcing that Russ White and Eyvonne Sharp are both stepping away from organizational responsibilities at Network Collective. There is no question that this is a significant loss on both accounts. Both Russ and Eyvonne have played irreplaceable roles in building Network Collective into what it is today, and those contributions cannot be understated. I’m grateful for every ounce of effort, wisdom, and perspective that they have invested into the podcast and wish nothing but success to the both of them as they move into new endeavors. Luckily both will still be around in the member’s slack and hopefully on a show or two every once in a while!

What does this mean for Network Collective?

First, Network Collective as you know it isn’t going anywhere. Clearly, I’m going to have to take a step back and evaluate how things proceed from here. We’ve made many plans with the intention of three active contributors and now that’s simply not the case. You might notice some blips in the production schedule and changes to the lineup of podcasts as I get things rearranged and streamlined for a one-person shop.

History of Networking

History Continue reading

Network Break 240: HPE ‘Cloudless’ Trolls Competitors; Riverbed Plugs SD-WAN Hole With Versa

HPE trolls the tech industry with a "cloudless" marketing campaign, Riverbed partners with Versa to bolster its SD-WAN portfolio, a startup dabbles in peer-to-peer networking and cryptocurrency, GE releases an unintentional parody video on the perils of IoT and more tech news on today's Network Break podcast.

The post Network Break 240: HPE ‘Cloudless’ Trolls Competitors; Riverbed Plugs SD-WAN Hole With Versa appeared first on Packet Pushers.

First-hand Feedback: ipSpace.net Network Automation Course

Daniel Teycheney attended the Spring 2019 Building Network Automation Solutions online course and sent me this feedback after completing it (and creating some interesting real-life solutions on the way):


I spent a bit of time the other day reflecting on how much I’ve learn’t from the course in terms of technical skills and the amount I’ve learned has been great. I literally no idea about things like Git, Jinja2, CI testing, reading YAML files and had only briefly seen Ansible before.

I’m not an expert now, but I understand these things and have real practical experience on these subjects which has given me great confidence to push on and keep getting better.

Read more ...

The Internet Is Your Oyster: MANRS at International Telecoms Week

What do oysters, clams, and mussels have in common with network operators? Hint: it’s not just that they are both in Atlanta this week, either in exhibits in the Georgia Aquarium or for the 2019 International Telecoms Week.

It’s that both bivalves and network operators play an incredibly important role for their ecosystems: they filter the bad stuff out and leave things a lot cleaner.

As water quality is vital to life in the ocean, the global routing system is vital to the smooth functioning of the Internet. The routing system’s decentralized structure, made up of thousands of independent networks tied together through business decisions and trusted relationships, provides flexibility, scalability, and overall durability.

However, despite its strengths, thousands of routing incidents occur every year. Some of these can be pretty scary, with route hijacks sending government traffic through the networks of foreign adversaries; route leaks slowing parts of the global Internet to a crawl; or hackers using spoofed traffic to take down websites in distributed denial of service (DDoS) attacks.

Network operators can help mitigate these problems by using stronger filtering policies to block spoofed traffic coming from their networks (helping guard against DDoS attacks) and filter route announcements Continue reading

The Week in Internet News: AI and IoT Could Lead to Industry 4.0

The new revolution: Advances in the Internet of Things and Artificial Intelligence are leading to a fourth industrial revolution, Ars Technica suggests. This so-called Industry 4.0 represents “huge changes in much of the technology we deal with in our daily lives. Some of those changes have been for the better, making our interaction with machines and information more natural and more powerful. Others have helped companies tap into consumers’ relationships, behaviors, locations and innermost thoughts in powerful and often disturbing ways.”

Security gaps: But all is not well in IoT land. Legacy problems related to networking technologies are making IoT security “as bad as it gets,” IoT Hub reports. In addition, staff remotely monitoring IoT sensors don’t have the experience to fix problems.

They’re keeping their eyes on you: The Indian government has found a way to track WhatsApp messages without breaking encryption, Business Insider reports. The surveillance technique involves the government asking the Facebook-owned messaging platform to implement an identifier that would allow law enforcement officials to pinpoint who sent the message without reading it.

A new way to pay: In other Facebook-related news, the company is planning to launch a new cryptocurrency called Libra, The Verge Continue reading

Join Cloudflare & Moz at our next meetup, Serverless in Seattle!

Join Cloudflare & Moz at our next meetup, Serverless in Seattle!
Photo by oakie / Unsplash
Join Cloudflare & Moz at our next meetup, Serverless in Seattle!

Cloudflare is organizing a meetup in Seattle on Tuesday, June 25th and we hope you can join. We’ll be bringing together members of the developers community and Cloudflare users for an evening of discussion about serverless compute and the infinite number of use cases for deploying code at the edge.

To kick things off, our guest speaker Devin Ellis will share how Moz uses Cloudflare Workers to reduce time to first byte 30-70% by caching dynamic content at the edge. Kirk Schwenkler, Solutions Engineering Lead at Cloudflare, will facilitate this discussion and share his perspective on how to grow and secure businesses at scale.

Next up, Developer Advocate Kristian Freeman will take you through a live demo of Workers and highlight new features of the platform. This will be an interactive session where you can try out Workers for free and develop your own applications using our new command-line tool.

Food and drinks will be served til close so grab your laptop and a friend and come on by!

View Event Details & Register Here

Agenda:

  • 5:00 pm Doors open, food and drinks
  • 5:30 pm Customer use case by Devin and Kirk
  • 6:00 pm Workers Continue reading

Join Cloudflare & Moz at our next meetup, Serverless in Seattle!

Join Cloudflare & Moz at our next meetup, Serverless in Seattle!
Photo by oakie / Unsplash
Join Cloudflare & Moz at our next meetup, Serverless in Seattle!

Cloudflare is organizing a meetup in Seattle on Tuesday, June 25th and we hope you can join. We’ll be bringing together members of the developers community and Cloudflare users for an evening of discussion about serverless compute and the infinite number of use cases for deploying code at the edge.

To kick things off, our guest speaker Devin Ellis will share how Moz uses Cloudflare Workers to reduce time to first byte 30-70% by caching dynamic content at the edge. Kirk Schwenkler, Solutions Engineering Lead at Cloudflare, will facilitate this discussion and share his perspective on how to grow and secure businesses at scale.

Next up, Developer Advocate Kristian Freeman will take you through a live demo of Workers and highlight new features of the platform. This will be an interactive session where you can try out Workers for free and develop your own applications using our new command-line tool.

Food and drinks will be served til close so grab your laptop and a friend and come on by!

View Event Details & Register Here

Agenda:

  • 5:00 pm Doors open, food and drinks
  • 5:30 pm Customer use case by Devin and Kirk
  • 6:00 pm Workers Continue reading