Mastodon – Part 3 – statsd and Prometheus

About this series

Mastodon

I have seen companies achieve great successes in the space of consumer internet and entertainment industry. I’ve been feeling less enthusiastic about the stronghold that these corporations have over my digital presence. I am the first to admit that using “free” services is convenient, but these companies are sometimes taking away my autonomy and exerting control over society. To each their own of course, but for me it’s time to take back a little bit of responsibility for my online social presence, away from centrally hosted services and to privately operated ones.

In my [first post], I shared some thoughts on how I installed a Mastodon instance for myself. In a [followup post] I talked about its overall architecture and how one might use Prometheus to monitor vital backends like Redis, Postgres and Elastic. But Mastodon itself is also an application which can provide a wealth of telemetry using a protocol called [StatsD].

In this post, I’ll show how I tie these all together in a custom Grafana Mastodon dashboard!

Mastodon Statistics

I noticed in the [Mastodon docs], that there’s a one-liner breadcrumb that might be easy to overlook, Continue reading

An early look at Thanksgiving 2022 Internet trends

An early look at Thanksgiving 2022 Internet trends

"The more you practice the art of thankfulness, the more you have to be thankful for."

— Norman Vincent Peale, American author  

The turkey. The sweet potatoes. The stuffing. The pumpkin pie. Yesterday, November 24, 2022, was Thanksgiving Day in the US. A time for families and loved ones to be together and thankful, according to the tradition. Last year, we saw how the US paused shopping (and browsing) for Thanksgiving. So, how was it this year? Not only did we see Internet traffic go down (by 13%) during Thanksgiving dinner, but it was much higher than usual the day before and the day after (the Black Friday effect… so far). There was also a clear, but short, Thanksgiving day effect on e-commerce DNS trends.

We'll have to wait to see what Black Friday looks like.

Let’s start with Internet traffic at the time of Thanksgiving dinner. Although every family is different, a 2018 survey of US consumers showed that for 42% early afternoon (between 13:00 and 15:00 is the preferred time to sit at the table and start to dig in). But 16:00 seems to be the “correct time” — The Atlantic explains why.

That said, Cloudflare Continue reading

Cloudflare servers don’t own IPs anymore – so how do they connect to the Internet?

Cloudflare servers don't own IPs anymore – so how do they connect to the Internet?
Cloudflare servers don't own IPs anymore – so how do they connect to the Internet?

A lot of Cloudflare's technology is well documented. For example, how we handle traffic between the eyeballs (clients) and our servers has been discussed many times on this blog: “A brief primer on anycast (2011)”, "Load Balancing without Load Balancers (2013)", "Path MTU discovery in practice (2015)",  "Cloudflare's edge load balancer (2020)", "How we fixed the BSD socket API (2022)".

However, we have rarely talked about the second part of our networking setup — how our servers fetch the content from the Internet. In this blog we’re going to cover this gap. We'll discuss how we manage Cloudflare IP addresses used to retrieve the data from the Internet, how our egress network design has evolved and how we optimized it for best use of available IP space.

Brace yourself. We have a lot to cover.

Terminology first!

Cloudflare servers don't own IPs anymore – so how do they connect to the Internet?

Each Cloudflare server deals with many kinds of networking traffic, but two rough categories stand out:

  • Internet sourced traffic - Inbound connections initiated by eyeball to our servers. In the context of this blog post we'll call these "ingress connections".
  • Cloudflare sourced traffic - Outgoing connections initiated by our servers to other Continue reading

Heavy Networking 657: New VMware Client Connects Users To SASE, SD-WAN (Sponsored)

Today on Heavy Networking, a discussion with sponsor VMware about SD-WAN and SASE. We’re diving into announcements from VMware Explore Barcelona 2022 covering a new SD-WAN client and more. With this client, you’ll be able to connect your users to the SASE cloud with software--no hardware edge box required. We dive into how it works, the network architecture, use cases, and more.

Heavy Networking 657: New VMware Client Connects Users To SASE, SD-WAN (Sponsored)

Today on Heavy Networking, a discussion with sponsor VMware about SD-WAN and SASE. We’re diving into announcements from VMware Explore Barcelona 2022 covering a new SD-WAN client and more. With this client, you’ll be able to connect your users to the SASE cloud with software--no hardware edge box required. We dive into how it works, the network architecture, use cases, and more.

The post Heavy Networking 657: New VMware Client Connects Users To SASE, SD-WAN (Sponsored) appeared first on Packet Pushers.

10 most powerful network management companies

Network management has never been easy, and the proliferation of IoT devices, the shift to remote work, and the migration of applications to multi-cloud environments have added new levels of complexity to enterprise networks.IT execs are dealing with network management tool sprawl and employee skills gaps. They are also struggling to gain visibility across increasingly distributed networks, including SaaS instances that are not under their direct control.To read this article in full, please click here

10 most powerful network management companies

Network management has never been easy, and the proliferation of IoT devices, the shift to remote work, and the migration of applications to multi-cloud environments have added new levels of complexity to enterprise networks.IT execs are dealing with network management tool sprawl and employee skills gaps. They are also struggling to gain visibility across increasingly distributed networks, including SaaS instances that are not under their direct control.To read this article in full, please click here

Linear Discriminant Analysis Rough Notes

Linear Discriminant Analysis (LDA)

stats_meme

LDA is an alternative way to predict $Y$, based on partitioning the explanatory variable into two sets: one set prediction is $\hat{Y}=1$ or $\hat{Y}=0$ in the other set. Approach here is to model the distribution of $X$ in each of the classes separately, and then use Bayes Theorem to obtain $P(Y |X)$.

Unlike Logistic regression, LDA treats explanatory variables as independent Random Variables, $X = (X_{1},…,X_{p})$. Assuming common covariance matrix for $X$ within each $Y$ category, Ronald Fisher derived the linear predictor of explanatory variables such that its observed values when $y=1$ were seperated as much as possible from its values when $y=0$, relative to the variability of the linear predictor values within each $y$ category. This linear predictor is called Linear Discriminant function. Using Gauassian distribution for each class, leads to linear or quadratic discriminant analysis. We can express the linear probabilty model as:

$ E(Y|x) = P(Y=1|x) = \beta_{0}+\beta_{1}x_{1}+…+\beta_{p}x_{p} $

We can rewrite the below Bayes Theorem:

$ P(Y=1|x) = \frac{P(x|y=1).P(Y=1)}{P(x)} $

as

$ P(Y=1|x) = \frac{\hat{f}(x|y=1)P(Y=1)}{\hat{f}(x|y=1)P(Y=1)+\hat{f}(x|y=0)P(Y=0)} $

Discriminant Analysis is useful for:

  • When the classes are well-separated, the parameter estimates for the logistic regression model are surprisingly unstable. Linear discriminant analysis does Continue reading

Why BGP communities are better than AS-path prepends

Why BGP communities are better than AS-path prepends
Why BGP communities are better than AS-path prepends

The Internet, in its purest form, is a loosely connected graph of independent networks (also called Autonomous Systems (AS for short)). These networks use a signaling protocol called BGP (Border Gateway Protocol) to inform their neighbors (also known as peers) about the reachability of IP prefixes (a group of IP addresses) in and through their network. Part of this exchange contains useful metadata about the IP prefix that are used to inform network routing decisions. One example of the metadata is the full AS-path, which consists of the different autonomous systems an IP packet needs to pass through to reach its destination.

As we all want our packets to get to their destination as fast as possible, selecting the shortest AS-path for a given prefix is a good idea. This is where something called prepending comes into play.

Routing on the Internet, a primer

Let's briefly talk about how the Internet works at its most fundamental level, before we dive into some nitty-gritty details.

The Internet is, at its core, a massively interconnected network of thousands of networks. Each network owns two things that are critical:

1. An Autonomous System Number (ASN): a 32-bit integer that uniquely identifies a network. Continue reading

How to Support Cholesterol and Cardiometabolic Health

Supporting cholesterol and cardiometabolic health is important for everyone, at any age. Generally speaking, the best way to do this involves making healthy lifestyle choices, such as exercising regularly, maintaining a healthy diet and weight, reducing stress levels and refraining from unhealthy habits like smoking. Here are some tips to help you get started on your journey towards better cholesterol and cardiometabolic health.

Exercise regularly

Regular physical activity helps increase HDL (good) cholesterol, decrease LDL (bad) cholesterol, and improve overall cardiovascular health. Aim for at least 30 minutes of exercise each day. By mixing it up with different types of exercises you can help keep your body fit and healthy.

Eat a balanced diet

Eating a balanced diet low in saturated fat, trans fats, and cholesterol can help lower LDL (bad) cholesterol. Include plenty of fruits, vegetables, lean proteins like fish and chicken, whole grains, and healthy fats like nuts and seeds.

Reduce stress levels

Stress can negatively impact cholesterol levels and cardiovascular health, so it is important to reduce or manage stress levels as much as possible. This could include activities such as mindfulness, yoga, or even just taking a few minutes each day for yourself to relax and Continue reading

Azure Networking Update (Phase 1)

Last week I completed the first part of the annual Azure Networking update. The Azure Firewall section is already online; hope you’ll find it useful. I already have the materials for the Private Link and Gateway Load Balancer services, but haven’t decided whether to schedule another live session to cover them, or just create a short video.

Then there are a half-dozen smaller things I found while processing a year worth of Azure networking News. You’ll find them (and links to documentation) in New Azure Services and Features document.

1 … 395 396 397 398 399 … 3,841