Linear Discriminant Analysis Rough Notes
Linear Discriminant Analysis (LDA)
LDA is an alternative way to predict $Y$, based on partitioning the explanatory variable into two sets: one set prediction is $\hat{Y}=1$ or $\hat{Y}=0$ in the other set. Approach here is to model the distribution of $X$ in each of the classes separately, and then use Bayes Theorem to obtain $P(Y |X)$.
Unlike Logistic regression, LDA treats explanatory variables as independent Random Variables, $X = (X_{1},…,X_{p})$. Assuming common covariance matrix for $X$ within each $Y$ category, Ronald Fisher derived the linear predictor of explanatory variables such that its observed values when $y=1$ were seperated as much as possible from its values when $y=0$, relative to the variability of the linear predictor values within each $y$ category. This linear predictor is called Linear Discriminant function. Using Gauassian distribution for each class, leads to linear or quadratic discriminant analysis. We can express the linear probabilty model as:
$ E(Y|x) = P(Y=1|x) = \beta_{0}+\beta_{1}x_{1}+…+\beta_{p}x_{p} $
We can rewrite the below Bayes Theorem:
$ P(Y=1|x) = \frac{P(x|y=1).P(Y=1)}{P(x)} $
as
$ P(Y=1|x) = \frac{\hat{f}(x|y=1)P(Y=1)}{\hat{f}(x|y=1)P(Y=1)+\hat{f}(x|y=0)P(Y=0)} $
Discriminant Analysis is useful for:
- When the classes are well-separated, the parameter estimates for the logistic regression model are surprisingly unstable. Linear discriminant analysis does Continue reading
Why BGP communities are better than AS-path prepends
The Internet, in its purest form, is a loosely connected graph of independent networks (also called Autonomous Systems (AS for short)). These networks use a signaling protocol called BGP (Border Gateway Protocol) to inform their neighbors (also known as peers) about the reachability of IP prefixes (a group of IP addresses) in and through their network. Part of this exchange contains useful metadata about the IP prefix that are used to inform network routing decisions. One example of the metadata is the full AS-path, which consists of the different autonomous systems an IP packet needs to pass through to reach its destination.
As we all want our packets to get to their destination as fast as possible, selecting the shortest AS-path for a given prefix is a good idea. This is where something called prepending comes into play.
Routing on the Internet, a primer
Let's briefly talk about how the Internet works at its most fundamental level, before we dive into some nitty-gritty details.
The Internet is, at its core, a massively interconnected network of thousands of networks. Each network owns two things that are critical:
1. An Autonomous System Number (ASN): a 32-bit integer that uniquely identifies a network. Continue reading
Quick Study: Managing and Supporting Remote Work
InformationWeek's collection of articles showcases how IT leaders and their teams made remote work happen over the last two plus years.How to Support Cholesterol and Cardiometabolic Health
Supporting cholesterol and cardiometabolic health is important for everyone, at any age. Generally speaking, the best way to do this involves making healthy lifestyle choices, such as exercising regularly, maintaining a healthy diet and weight, reducing stress levels and refraining from unhealthy habits like smoking. Here are some tips to help you get started on your journey towards better cholesterol and cardiometabolic health.
Exercise regularly
Regular physical activity helps increase HDL (good) cholesterol, decrease LDL (bad) cholesterol, and improve overall cardiovascular health. Aim for at least 30 minutes of exercise each day. By mixing it up with different types of exercises you can help keep your body fit and healthy.
Eat a balanced diet
Eating a balanced diet low in saturated fat, trans fats, and cholesterol can help lower LDL (bad) cholesterol. Include plenty of fruits, vegetables, lean proteins like fish and chicken, whole grains, and healthy fats like nuts and seeds.
Reduce stress levels
Stress can negatively impact cholesterol levels and cardiovascular health, so it is important to reduce or manage stress levels as much as possible. This could include activities such as mindfulness, yoga, or even just taking a few minutes each day for yourself to relax and Continue reading
Azure Networking Update (Phase 1)
Last week I completed the first part of the annual Azure Networking update. The Azure Firewall section is already online; hope you’ll find it useful. I already have the materials for the Private Link and Gateway Load Balancer services, but haven’t decided whether to schedule another live session to cover them, or just create a short video.
Then there are a half-dozen smaller things I found while processing a year worth of Azure networking News. You’ll find them (and links to documentation) in New Azure Services and Features document.
Mastodon – Part 2 – Monitoring
About this series
I have seen companies achieve great successes in the space of consumer internet and entertainment industry. I’ve been feeling less enthusiastic about the stronghold that these corporations have over my digital presence. I am the first to admit that using “free” services is convenient, but these companies are sometimes taking away my autonomy and exerting control over society. To each their own of course, but for me it’s time to take back a little bit of responsibility for my online social presence, away from centrally hosted services and to privately operated ones.
In the [previous post], I shared some thoughts on how the overall install of a Mastodon instance went, making it a point to ensure my users’ (and my own!) data is somehow safe, and the machine runs on good hardware, and with good connectivity. Thanks IPng, for that 10G connection! In this post, I visit an old friend, [Borgmon], which has since reincarnated and become the de facto open source observability and signals ecosystem, and its incomparably awesome friend. Hello, Prometheus and Grafana!
Anatomy of Mastodon
Looking more closely at the architecture of Mastodon, it consists of a few moving Continue reading
Time to Talk
It’s a holiday week here in the US so most people are working lighter days or just taking the whole week off. They’re looking forward to spending time with family and friends. Perhaps they’re already plotting their best strategy for shopping during Black Friday and snagging a new TV or watch. Whatever the case may be there’s lots things going on all over.
One thing that I feel needs to happen is conversation. Not just the kind of idle conversation that we make when we don’t know what to talk about. I also don’t mean the kinds of deep conversations that we need to prepare ourselves to have. I’m talking about the ones where we learn. The ones we have with friends and family where we pick up tidbits of stories and preserve them for the future.
It sounds rather morbid but these conversations aren’t going to be available forever. Our older loved ones are getting older every year. Time marches on and we never know when that time I going to run out. I have several friends that have lost loved ones this year and still others that have realized the time is growing shorter. Mortality is something that Continue reading
Network Modernization Unlocks the Power of Modern Cloud Applications
This is a guest post from IDC Analyst Brad Casemore.
Modern applications are more distributed than ever before, deployed variously across on-premises data centers, public clouds (IaaS), private clouds, and edge locations, and sometimes delivered as SaaS. While the primacy of these data-centric applications is undeniable and will only grow with the rise of artificial intelligence (AI), a failure to ensure the modernization of underlying network infrastructure can compromise and constrain an organization’s application-driven digital strategies.
Needs of today
Network modernization, especially within the context of cloud-native architectures and multi-cloud strategies, cannot be an afterthought for rapidly digitizing enterprises. As applications become the powerhouse behind digital success and competitive differentiation, organizations should consider investing in software-defined network infrastructure.
A software-defined network infrastructure provides consistent network and security policies, operational simplicity, elastic scale, and ubiquitous visibility, with support for traditional and cloud-native applications spanning on-premises environments and clouds.
Preparing for tomorrow
Special consideration also must be given to the future networking needs of the organization, particularly in relation to how modern network infrastructure will provide inherent portable application layer networking for cloud-native applications through functionality such as ingress controllers, service meshes, and visibility into workloads Continue reading
How we detect route leaks and our new Cloudflare Radar route leak service
Today we’re introducing Cloudflare Radar’s route leak data and API so that anyone can get information about route leaks across the Internet. We’ve built a comprehensive system that takes in data from public sources and Cloudflare’s view of the Internet drawn from our massive global network. The system is now feeding route leak data on Cloudflare Radar’s ASN pages and via the API.
This blog post is in two parts. There’s a discussion of BGP and route leaks followed by details of our route leak detection system and how it feeds Cloudflare Radar.
About BGP and route leaks
Inter-domain routing, i.e., exchanging reachability information among networks, is critical to the wellness and performance of the Internet. The Border Gateway Protocol (BGP) is the de facto routing protocol that exchanges routing information among organizations and networks. At its core, BGP assumes the information being exchanged is genuine and trust-worthy, which unfortunately is no longer a valid assumption on the current Internet. In many cases, networks can make mistakes or intentionally lie about the reachability information and propagate that to the rest of the Internet. Such incidents can cause significant disruptions of the normal operations of the Internet. One type Continue reading
Cloud Security is Hard Because Multi-Cloud is Complex
In today’s multi-cloud world, security-as-a-service is growing as a strategic means of standardizing security practices that span all applications, no matter where they are deployed.Day Two Cloud 173: Istio Ambient Mesh Minimizes Sidecar Proxies
Today on Day Two Cloud we examine Istio Ambient Mesh, a new option for building service meshes in a microservices environment. Istio Ambient Mesh essentially brings the concept of a load balancer to a cluster of containers. Rather than run a sidecar proxy for each pod or container, you can run Ambient Mesh per node. Our guest and guide to this open source project is Christian Posta, Global Field CTO at Solo.io.
The post Day Two Cloud 173: Istio Ambient Mesh Minimizes Sidecar Proxies appeared first on Packet Pushers.
Day Two Cloud 173: Istio Ambient Mesh Minimizes Sidecar Proxies
Today on Day Two Cloud we examine Istio Ambient Mesh, a new option for building service meshes in a microservices environment. Istio Ambient Mesh essentially brings the concept of a load balancer to a cluster of containers. Rather than run a sidecar proxy for each pod or container, you can run Ambient Mesh per node. Our guest and guide to this open source project is Christian Posta, Global Field CTO at Solo.io.Why Cloudflare’s one of the Top 100 Most Loved Workplaces in 2022
This post is also available in Français, 日本語, 简体中文, 한국어, Español.
At Cloudflare, we have strived to build a workplace where our entire team feels safe and excited to bring their whole selves to work, so they can do their best work. That’s why we are proud to share that Cloudflare has been named one of the Top 100 Most Loved Workplaces in 2022 by Newsweek and Best Practice Institute (BPI). Most Loved Workplaces recognizes companies where their workers love, and feel in sync with, the company they work for.
With this, and as we’re approaching the end-of 2022, we thought this was a good time to reflect on some of the things that go into being one of these Most Loved Workplaces and just some of what makes up our workplace and culture.
Something that really grounds our entire team is Cloudflare’s mission: to help build a better Internet. When you are solving some of the toughest challenges facing the Internet — helping make the Internet secure, fast, private, and reliable globally — you need a range of talented individuals to do this. The people at Cloudflare are exactly that, and are essential to our Continue reading
Integrated Routing and Bridging (IRB) Design Models
Imagine you built a layer-2 fabric with tons of VLANs stretched all over the place. Now the users want to exchange traffic between those VLANs, and the obvious question is: which devices should do layer-2 forwarding (bridging) and which ones should do layer-3 forwarding (routing)?
There are four typical designs you can use to solve that challenge:
- Exchange traffic between VLANs outside of the fabric (edge routing)
- Route on core switches (centralized routing)
- Route on ingress (asymmetric IRB)
- Route on ingress and egress (symmetric IRB)
This blog post is an overview of the design models; we’ll cover each design in a separate blog post.
Heavy Strategy 037 – Metaversing The Office is More Than One Thing
Are there angles on future metaverse that make sensee ? Johna and Greg dive into their perspectives on what is a metaverse and converge on the face that its a form of collaboration. Potentially it could be immersive with VR googles but more likely it’s about engaging data from external domains into the collaboration experience.
The post Heavy Strategy 037 – Metaversing The Office is More Than One Thing appeared first on Packet Pushers.
Heavy Strategy 037 – Metaversing The Office is More Than One Thing
Are there angles on future metaverse that make sensee ? Johna and Greg dive into their perspectives on what is a metaverse and converge on the face that its a form of collaboration. Potentially it could be immersive with VR googles but more likely it’s about engaging data from external domains into the collaboration experience.
Using Calico to create a Kubernetes cluster mesh for multi-cluster environments
Kubernetes has come of age with more organizations adopting a microservices architecture at scale. But scale brings a whole slew of new challenges, especially with Kubernetes, which is designed to operate as a single cluster. However, the usage of Kubernetes, especially at leading-edge organizations operating at scale, has crossed the single-cluster threshold. Organizations are building and deploying services across multiple clusters for high availability, disaster recovery, application isolation, compliance, latency concerns, staged migration, and multi-tenancy reasons.
Regardless of the reasons to deploy multiple clusters, platform and application teams must address networking, security, and observability issues related to microservices deployed across multi-clusters, sometimes spanning hybrid and multi-cloud environments.
Calico, the most widely adopted container networking and security solution (according to a recently published container adoption report by Datadog), provides an operationally simple solution to solve the networking, security, and observability challenges of running multi-cluster Kubernetes environments.
Security, observability, and networking requirements for multiple Kubernetes clusters
In simple terms, creating a multi-cluster Kubernetes environment requires stitching multiple Kubernetes clusters together to provide a common set of services. To create a single logical environment spanning multiple clusters, the key requirements are:
- Enabling inter-cluster communication – Communication across pods located in different clusters is Continue reading