Almost nine years ago, Cloudflare was a tiny company and I was a customer not an employee. Cloudflare had launched a month earlier and one day alerting told me that my little site, jgc.org, didn’t seem to have working DNS any more. Cloudflare had pushed out a change to its use of Protocol Buffers and it had broken DNS.
I wrote to Matthew Prince directly with an email titled “Where’s my dns?” and he replied with a long, detailed, technical response (you can read the full email exchange here) to which I replied:
From: John Graham-Cumming
Date: Thu, Oct 7, 2010 at 9:14 AM
Subject: Re: Where's my dns?
To: Matthew Prince
Awesome report, thanks. I'll make sure to call you if there's a
problem. At some point it would probably be good to write this up as
a blog post when you have all the technical details because I think
people really appreciate openness and honesty about these things.
Especially if you couple it with charts showing your post launch
traffic increase.
I have pretty robust monitoring of my sites so I get an SMS when
anything fails. Monitoring shows I was down from 13:03:07 Continue reading
Vor etwa neun Jahren war Cloudflare noch ein winziges Unternehmen und ich war ein Kunde, kein Mitarbeiter. Cloudflare gab es erst seit einem Monat. Eines Tages wurde ich darüber benachrichtigt, dass bei meiner kleinen Website jgc.org der DNS-Service nicht mehr funktionierte. Cloudflare hat seine Verwendung von Protocol Buffers angepasst und dadurch wurde der DNS-Service unterbrochen.
Ich habe eine E-Mail mit dem Titel „Where‘s my dns?“ (Wo ist mein DNS) direkt an Matthew Prince gesendet und er hat mit einer langen, detaillierten, technischen Erklärung reagiert (Sie können den vollständigen E-Mail-Austausch hier lesen), auf die ich antwortete:
Von: John Graham-Cumming
Datum: Do., 7. Okt. 2010 um 09:14
Betreff: Re: Wo ist mein DNS?
An: Matthew Prince
Toller Bericht, danke. Ich werde auf jeden Fall anrufen, wenn es ein
Problem geben sollte. Es wäre wahrscheinlich sinnvoll, all das in
einem Blog-Beitrag festzuhalten, wenn Sie alle technischen Details haben. Ich glaube nämlich,
dass es Kunden wirklich zu schätzen wissen, wenn mit solchen Dingen offen und ehrlich umgegangen wird.
Sie könnten auch die Traffic-Zunahme nach der Implementierung mit
Diagrammen veranschaulichen.
Ich habe eine recht zuverlässige Überwachung für meine Websites eingerichtet, deshalb bekomme ich eine SMS, wenn
etwas ausfällt. Meine Daten zeigen, Continue reading
CPU 耗尽是由一个 WAF 规则引起的，该规则里包含不严谨的正则表达式，最终导致了过多的回溯。作为中断核心诱因的正则表达式是 (?:(?:\"|'|\]|\}|\\|\d|(?:nan|infinity|true|false|null|undefined|symbol|math)|\`|\-|\+)+[)]*;?((?:\s|-|~|!|{}|\|\||\+)*.*(?:.*=.*)))
Il y a près de neuf ans, Cloudflare était une toute petite entreprise dont j’étais le client, et non l’employé. Cloudflare était sorti depuis un mois et un jour, une notification m’alerte que mon petit site, jgc.org, semblait ne plus disposer d’un DNS fonctionnel. Cloudflare avait effectué une modification dans l’utilisation de Protocol Buffers qui avait endommagé le DNS.
J’ai contacté directement Matthew Prince avec un e-mail intitulé « Où est mon DNS ? » et il m’a envoyé une longue réponse technique et détaillée (vous pouvez lire tous nos échanges d’e-mails ici) à laquelle j’ai répondu :
De: John Graham-Cumming
Date: Jeudi 7 octobre 2010 à 09:14
Objet: Re: Où est mon DNS?
À: Matthew Prince
Superbe rapport, merci. Je veillerai à vous appeler s’il y a un
problème. Il serait peut-être judicieux, à un certain moment, d’écrire tout cela dans un article de blog, lorsque vous aurez tous les détails techniques, car je pense que les gens apprécient beaucoup la franchise et l’honnêteté sur ce genre de choses. Surtout si vous y ajoutez les tableaux qui montrent l’augmentation du trafic suite à votre lancement.
Je dispose d’un système robuste de surveillance de mes sites qui m’envoie un Continue reading
From: John Graham-Cumming
日時：2010/10/7（木）9:14 AM
件名：Re: 私のDNSはどうなったのでしょうか？
To: Matthew Prince
ご報告ありがとうございました。何か問題があれば
ご連絡します。 技術詳細に関する全容が判明したら、
本件をブログに記載するのはいかがでしょうか。
本件に対しての開示や誠実であることを他の人も評価すると思うのです。
特に、ローンチ後のトラフィック増加を示すグラフを
添えていただければと思います。
私は自分のサイトを厳格に監視しているので、何かあれば
SMSを受け取れます。 監視結果では13:03:07から14:04:12までダウンしていたことが
わかりました。 テストは5分おきに実行されています。
本件は大事には至らずに済んでいますし、解決していただけると確信しています。 しかしながら、ヨーロッパには本当に
誰も必要ないとお考えですか？
これに対するMatthewの返信は以下の通りです。
From: Matthew Prince
日時：2010/10/7（木）9:57 AM
件名：Re: 私のDNSはどうなったのでしょうか？
To: John Graham-Cumming
ありがとうございます。Cloudflareではいただいたメールすべてに対して返信しております。私は現在
オフィスに向かっており、ブログへの投稿またはCloudflareの掲示板システムのトップに
公式投稿をピン留めする予定です。透明性が一番だということには
全面的に同意します。
3分後、1つ目のPagerDutyページがWAFの異常を表示して停止しました。これはCloudflare外からWAFの機能を確認する模擬テストで（このようなテストは数百とあります）、正常動作を確認するためのものでした。そしてすぐにCloudflareサービスのエンドツーエンドテストの失敗、グローバルなトラフィック低下アラート、502エラーの蔓延がページに表示され、世界各都市のPoint of Presence（PoP）からCPU枯渇に関する報告を多数受けました。
上記の変更申請でご確認いただける通り、リリース計画、ロールバック計画、この種のリリース向けの内部標準業務手順書（SOP）へのリンクが記載されています。そして、ルール変更向けのSOPでは特別にグローバルなプッシュが許可されています。これはCloudflareでリリースする他のソフトウェアとは大きく異なるものです。通常SOPのプッシュ先はまず内部の試験運用版ネットワークにあるPoit of Presence（PoP）、次に独立した地域にいる少数のお客様、多数のお客様、最後に世界という順になります。
When we talk about VPNv4 prefixes – Route Distinguishers (RDs) play an incredibly important role in ensuring multipath routing. We talked about this a little bit in our last post, but I want to hit it home in this post as well as cover a couple of other items in a little great detail.
To start with, we’re going to use our same physical lab topology, but changes things up slightly. Namely….
vMX7 will remain a BGP route reflector but will also now participate in the dataplane by having it’s interfaces enabled with LDP and MPLS
vMX1 and vMX3 will act as remote PEs for our provider that are hosting an anycast service. Both will be advertising 140.10.20.0/24 into a “Provider” VPN which we will then import into our customer VPN.
vMX2 will now act like a Customer Edge (CE) router peering to vMX5 which will act as the provider edge (PE) router. It will peer with the provider in AS65000 from AS12345.
So our diagram will now look something like this…
Alright. So for the sake of thoroughness, I’ll start by including our base configurations again since they did change ever so slightly, and I Continue reading
In a culture of Internet toxicity, a question all productivity-minded people should ponder is why they are participating in social platforms. For example, Twitter has become a predominantly negative cesspool. Even the pun-loving techies I monitor from a distance seem to lean increasingly toward darkness and anger.
Few rainbows are to be found on Twitter these days. Maybe it’s just the dark mode UI talking, but I don’t go there to laugh anymore. I don’t go there to connect with friends. Instead, I put on my virtual armor, and read through comments and responses directed at me or my company. To be sure, much of what I see is fine. However, many comments are meant to start fires, even when couched in smiling niceties.
That’s the Internet for you. We’ve always had flame wars, trolls, and haters, all the way back to my dial-up days on Delphi forums and AOL. I know how to block and mute people, and of course that helps. Even so, I find that there’s something different in the tone these days. Folks are on crusades to bring others down. To shame. To burn in digital effigy.
You’re probably familiar with Cisco DevNet. If not, DevNet is the place Cisco has embraced outreach to the developer community building for software-defined networking (SDN). Though initially cautious in getting into the software developer community, Cisco has embraced their new role and really opened up to help networking professionals embrace the new software normal in networking. But where is DevNet going to go from here?
Humble Beginnings
DevNet wasn’t always the darling of Cisco’s offerings. I can remember sitting in on some of the first discussions around Cisco OnePK and thinking to myself, “This is never going to work.”
My hesitation with Cisco’s first attempts to focus on software platforms came from two places. The first was what I saw as Cisco trying to figure out how to extend the platforms to include some programmability. It was more about saying they could do software and less about making that software easy to use or program against. The second place was actually the lack of a place to store all of this software knowledge. Programmers and developers are fickle lot and you have to have a repository where they can get access to the pieces they needed.
According to a recent survey conducted by Consumers International and the Internet Society, 63% of consumers think the way Internet-connected devices collect data is “creepy.” The Trust Opportunity: Exploring Consumer Attitudes to the Internet of Things, which polled people in the US, Canada, Japan, Australia, France, and the UK, also found that 73% of consumers think people using connected devices should worry about eavesdropping. And yet, new connected devices are being introduced practically every day, and sales show no sign of slowing down.
The word “smart” is used to describe almost all of these devices. But is that right?
The marketing around the Internet of Things (IoT) has become almost non-stop. Smart-this will make your life better, happier, more efficient. If only you had smart-that, you would reap the benefits of the marvelous technological age in which we live. But this often leaves out key information consumers need to make real smart choices.
It’s really about connectivity. For instance, that smart oven is a
computer that happens to get hot in the middle. These IoT devices are able to
perform smart functions because they are connected to the Internet. And while
the marketing focuses on features and functionality, Continue reading
When using kubeadm to set up a new Kubernetes cluster, the output of the kubeadm init command that sets up the control plane for the first time contains some important information on joining additional nodes to the cluster. One piece of information in there that (until now) I hadn’t figured out how to replicate was the CA certificate hash. (Primarily I hadn’t figured it out because I hadn’t tried.) In this post, I’ll share how to calculate the CA certificate hash for kubeadm to use when joining additional nodes to an existing cluster.
When looking to figure this out, I first started with the kubeadm documentation. My searches led me here, which states:
The hash is calculated over the bytes of the Subject Public Key Info (SPKI) object (as in RFC7469). This value is available in the output of “kubeadm init” or can be calculated using standard tools.
That’s useful information, but what are the “standard tools” being referenced? I knew that a lot of work had been put into kubeadm init phase (for breaking down the kubeadm init workflow), but a quick review of that documentation didn’t reveal anything. Reviewing the referenced RFC also didn’t provide any Continue reading
The problem set-up in this paper discusses the importance of keeping web page load times low as a fundamental contributor to user satisfaction (See e.g. ‘Why performance matters’). Between client-side tools such as Google’s Lighthouse, back-end tools that can analyse ORM usage and database queries and point out issues such as N+1 selects, and the information provided by your favourite APM I was initially wondering what ground there was left to tread here. So I was pleasantly surprised when it turned out the authors were looking at the problem in a different way to most of these approaches.
Rather than accepting the current rendered view (web page) as seen by the end-user as fixed, and then asking what can be done to optimise the end-to-end loading time of that page, this paper examines the question of what changes to the current view could dramatically reduce its load time? I.e., small (or sometimes not so small) changes to what the end user ultimately sees on the page, that can have a net benefit on the overall user experience.
Developers ranked Docker as the #1 most wanted platform, #2 most loved platform, and #3 most broadly used platform in the 2019 Stack Overflow Developer Survey. Nearly 90,000 developers from around the world responded to the survey. So we asked the community why they love Docker, and here are 10 of the reasons they shared:
ROSIE the Robot at DockerCon. Her software runs on Docker containers.
It works on everyone’s machine. Docker eliminates the “but it worked on my laptop” problem.
“I love docker because it takes environment specific issues out of the equation – making the developer’s life easier and improving productivity by reducing time wasted debugging issues that ultimately don’t add value to the application.” @pamstr_
Takes the pain out of CI/CD. If there is one thing developers hate, it is doing the same thing over and over.
“Docker completely changed my life as a developer! I can spin up my project dependencies like databases for my application in a second in a clean state on any machine on our team! I can‘t not imagine the whole ci/cd-approach without docker. Automate all the stuff? Dockerize it!” @Dennis65560555
By Tom Gillis, SVP/GM of Networking and Security BU
When we first announced our intent to acquire Avi Networks, the excitement within our customer base, with industry watchers and within our own business was overwhelming. IDC analysts wrote, “In announcing its intent to acquire software ADC vendor Avi Networks, VMware both enters the ADC market and transforms its NSX datacenter and multicloud network-virtualization overlay (NVO) into a Layer 2-7 full-stack SDN fabric (1).”
Avi possesses exceptional alignment with VMware’s view of where the network is going, and how data centers must evolve to operate like public clouds to help organizations reach their full digital potential. It’s for these reasons that I am happy to announce VMware has closed the acquisition of Avi Networks and they are now officially part of the VMware family going forward.
I’ve heard Pat Gelsinger say many times that VMware wants to aggressively “automate everything.” With Avi, we’re one step closer to meeting this objective. The VMware and Avi Networks teams will work together to advance our Virtual Cloud Network vision, build out our full stack L2-7 services, and deliver the public cloud experience for on-prem environments. We will introduce the Avi platform Continue reading
But wait, there’s more! If you keep up with our podcast you may have noticed the previous episode where we talk about what open networking was, so why are we chatting about it again? Last time we talked about having open API’s and having the demarcation point between components but in this podcast, we’re extending the conversation out to show how everyone can take advantage of open networking in a wider, practical sense. Guests Rama Darbha and Roopa Prabhu join host Brian to share their thoughts, experiences and expertise on the subject. Listen, enjoy, and feel free to comment away here or on our social media channels if you have any questions or thoughts to add.
Guest Bios
Brian O’Sullivan: Brian currently heads Product Management for Cumulus Linux. For 15 or so years he’s held software Product Management positions at Juniper Networks as well as other smaller companies. Once he saw the change that was happening in the networking space, he decided to join Cumulus Networks to be a part of the open networking innovation. When not working, Brian is Continue reading
