0

Log analytics using ClickHouse

This is an adapted transcript of a talk we gave at Monitorama 2022. You can find the slides with presenter’s notes here and video here.

When a request at Cloudflare throws an error, information gets logged in our requests_error pipeline. The error logs are used to help troubleshoot customer-specific or network-wide issues.

We, Site Reliability Engineers (SREs), manage the logging platform. We have been running Elasticsearch clusters for many years and during these years, the log volume has increased drastically. With the log volume increase, we started facing a few issues. Slow query performance and high resource consumption to list a few. We aimed to improve the log consumer's experience by improving query performance and providing cost-effective solutions for storing logs. This blog post discusses challenges with logging pipelines and how we designed the new architecture to make it faster and cost-efficient.

Before we dive into challenges in maintaining the logging pipelines, let us look at the characteristics of logs.

Characteristics of logs

Unpredictable - In today's world, where there are tons of microservices, the amount of logs a centralized logging system will receive is very unpredictable. There are various reasons why capacity estimation of log volume is so difficult. Continue reading

0

Why Migrating a Database to the Cloud is Like a Heart Transplant

Migrating your organization's on-premises database to a native-cloud database is fraught with challenges and potential benefits. Here are recommendations to ensure this ‘heart transplant’ doesn't fail.

0

Google employees at loggerheads with company over defense contract, again

A group of Google employees are yet again speaking out against Google’s defense contracts, this time asking the company to shelve its $1.2 billion Project Nimbus contract for the Israeli government and military. Google partnered with Amazon to bid for the project.Under employee pressure, Google has previously dropped one US government defence contract (Project Maven), and shied away from another (JEDI).In a video posted on Youtube, a group of Google employees including Palestinian, Jewish, Muslim, and Arab staff expressed their concerns over Project Nimbus, which they claim will provide surveillance and other forms of powerful AI technology to the Israeli government and military. They are also speaking out against “the anti-Palestinian bias” they have witnessed within the company. To read this article in full, please click here

0

Google employees at loggerheads with company over defense contract, again

A group of Google employees are yet again speaking out against Google’s defense contracts, this time asking the company to shelve its $1.2 billion Project Nimbus contract for the Israeli government and military. Google partnered with Amazon to bid for the project.Under employee pressure, Google has previously dropped one US government defence contract (Project Maven), and shied away from another (JEDI).In a video posted on Youtube, a group of Google employees including Palestinian, Jewish, Muslim, and Arab staff expressed their concerns over Project Nimbus, which they claim will provide surveillance and other forms of powerful AI technology to the Israeli government and military. They are also speaking out against “the anti-Palestinian bias” they have witnessed within the company. To read this article in full, please click here

0

A Second Look at QUIC Use

A couple of months ago, in July 2022, I wrote about our work in measuring the level of use of QUIC in the Internet. Getting this measurement “right” has been an interesting exercise, and it’s been a learning experience that I’d like to relate here. We’ll start from the end of the previous article and carry on from there.

0

Video: Kubernetes Services Overview

After completing the discussion of basic Kubernetes networking with a typical inter-pod traffic scenario, Stuart Charlton tackled another confusing topic: an overview of what Kubernetes services are.

0

Video: Kubernetes Services Overview

After completing the discussion of basic Kubernetes networking with a typical inter-pod traffic scenario, Stuart Charlton tackled another confusing topic: an overview of what Kubernetes services are.

0

Experimenting TCP Congestion Control(WIP)

Introduction

I have always found TCP congestion control algorithms fascinating, and at the same time, I know very little about them. So once in a while, I will spend some time with the hope of gaining some new insights. This blog post will share some of my experiments with various TCP congestion control algorithms. We will start with TCP Reno, then look at Cubic and ends with BBR.I am using Linux network namespaces to emulate topology for experimentation, making it easier to run than setting up a physical test bed.

TCP Reno

For many years, the main algorithm of congestion control was TCP Reno. The goal of congestion control is to determine how much capacity is available in the network, so that source knows how many packets it can safely have in transit (Inflight). Once a source has these packets in transit, it uses the ACK’s arrival as a signal that packets are leaving the network, and therefore it’s safe to send more packets into the network. By using ACKs for pacing the transmission of packets, TCP is self-clocking. The number of packets which TCP can inject into the network is controlled by Congestion Window(cwnd).

Congestion Window:

Continue reading

0

FCC urged to test for ‘dangerous’ interference from Wi-Fi 6E

The National Spectrum Management Association this week warned the Federal Communications Commission that Wi-Fi 6E could cause potentially dangerous interference in networks used by first responders, utilities and others if the FCC doesn’t perform“real-world testing on its automated frequency control systems.NSMA argued in an open letter to the commission that testing facilities are already available, specifically at the Idaho National Labs spectrum test bed, and that such studies should be peer-reviewed and transparent. [ Get regularly scheduled insights by signing up for Network World newsletters. ]To read this article in full, please click here

0

FCC urged to test for ‘dangerous’ interference from Wi-Fi 6E

The National Spectrum Management Association this week warned the Federal Communications Commission that Wi-Fi 6E could cause potentially dangerous interference in networks used by first responders, utilities and others if the FCC doesn’t perform“real-world testing on its automated frequency control systems.NSMA argued in an open letter to the commission that testing facilities are already available, specifically at the Idaho National Labs spectrum test bed, and that such studies should be peer-reviewed and transparent. [ Get regularly scheduled insights by signing up for Network World newsletters. ]To read this article in full, please click here

0

McLaren Racing relies on edge computing at the racetrack’s edge

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [ Get regularly scheduled insights by signing up for Network World newsletters. ]To read this article in full, please click here

0

McLaren Racing relies on edge computing at the racetrack’s edge

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [ Get regularly scheduled insights by signing up for Network World newsletters. ]To read this article in full, please click here

0

McLaren Racing relies on edge computing at the racetrack

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [Get regularly scheduled insights by signing up for Network World newsletters]To read this article in full, please click here

0

McLaren Racing relies on edge computing at Formula 1 tracks

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [Get regularly scheduled insights by signing up for Network World newsletters]To read this article in full, please click here

0

McLaren Racing relies on edge computing at Formula 1 tracks

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [Get regularly scheduled insights by signing up for Network World newsletters]To read this article in full, please click here

0

McLaren Racing relies on edge computing at Formula 1 tracks

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [Get regularly scheduled insights by signing up for Network World newsletters]To read this article in full, please click here

0

McLaren Racing relies on edge computing at Formula 1 tracks

“Twenty-two times a year, we build a data center right down at the edge,” said Ed Green, head of commercial technology at McLaren Racing, a British motor racing team based in Surrey, England.For McLaren, the edge is wherever in the world the company’s Formula 1 racing team is competing. An IT setup at each racing site links the entire team, including mechanics, engineers, crew members, and the drivers of McLaren’s two Formula 1 racecars. [Get regularly scheduled insights by signing up for Network World newsletters]To read this article in full, please click here

0

Hedge 145: Roundtable on Professional Liability

The software world is known for overdue projects, costs overrun, lots of defects, and lots of failure all the way around. Many other engineering fields have stricter requirements to take on projects and liability insurance driving correct practice and care. The networking world, and the larger IT world, however, has neither of these things. Does this make IT folks less likely to “do the right thing,” or is the self-regulation we have today enough? Join Tom Ammon, Eyvonne Sharp, and Russ White as they discuss the possibilities of professional liability in information technology.

download

0

Know Your Adversaries: The Top Network Bad Actors

The bad guys are out to steal your data, identity, money, and anything else they can lay their digital hands on. Here's a rundown of today's major adversaries.

0

Collecting and Processing Data from Pokec using Python

Recently I have programmed Python scripts - config.py and pokec_get_stats.py, which I can use to collect statistics about the users in a particular room of the popular Slovak social network Pokec. My goal is to practice the Python programming and test the functionality of Selenium WebDriver. The Selenium WebDriver is used to emulate user interaction […]
Continue reading...