Creating a Talos Linux Cluster on Azure with Pulumi

A little over a month ago I published a post on creating a Talos Linux cluster on AWS with Pulumi. Talos Linux is a re-thinking of your typical Linux distribution, custom-built for running Kubernetes. Talos Linux has no SSH access, no shell, and no console; instead, everything is managed via a gRPC API. This post is something of a “companion post” to the earlier AWS post; in this post, I’ll show you how to create a Talos Linux cluster on Azure with Pulumi.

The program I’ll share with you in this post is written in Go, but the process outlined in this post and the accompanying code is equally applicable in other languages supported by Pulumi. (TypeScript is a popular choice for lots of folks.) The code is available in this GitHub repository. It’s based on this documentation from Sidero Labs, and I also found this blog post to be helpful as well.

The Pulumi program follows this overall flow:

  1. First, the program creates the base infrastructure objects that are required—a resource group, a virtual network, some subnets, and a network security group.
  2. Next, it creates a load balancer, gets a public IP address for the load Continue reading

Upgrading one of the oldest components in Cloudflare’s software stack

Upgrading one of the oldest components in Cloudflare’s software stack
Upgrading one of the oldest components in Cloudflare’s software stack

Cloudflare serves a huge amount of traffic: 45 million HTTP requests per second on average (as of 2023; 61 million at peak) from more than 285 cities in over 100 countries. What inevitably happens with that kind of scale is that software will be pushed to its limits. As we grew, one of the problems we faced was related to deploying our code. Sometimes, a release would be delayed because of inadequate hardware resources on our servers. Buying more and more hardware is expensive and there are limits to e.g. how much memory we can realistically have on a server. In this article, we explain how we optimised our software and its release process so that no additional resources are needed.

In order to handle traffic, each of our servers runs a set of specialised proxies. Historically, they were based on NGINX, but increasingly they include services created in Rust. Out of our proxy applications, FL (Front Line) is the oldest and still has a broad set of responsibilities.

At its core, it’s one of the last uses of NGINX at Cloudflare. It contains a large amount of business logic that runs many Cloudflare products, using a variety of Continue reading

Google picks Qatar for second Middle Eastern cloud region

Google is adding a second cloud availability region in the Middle East, at Doha, to cater to demand from Qatar’s government and enterprises in the region, it said on Friday.The new cloud region will help the Qatari government achieve its Qatar National Vision 2030 plan to sustain development and provide a high standard of living for its people, according to Google Cloud’s country manager for Qatar, Ghassan Kosta.“This new region is a strong step towards building regional capacity that meets the needs of the Qatari digital economy, from availability and data residency, to digital sovereignty and sustainability,” Kosta wrote in a blog post.To read this article in full, please click here

Data center fires raise concerns about lithium-ion batteries

Fire is to blame for a small but significant number of data-center outages including a March 28 fire that caused severe damage to a data center in France, and an analysis of global incidents highlights ongoing concerns about the safety of lithium-ion batteries and their risk of combustion.The use of lithium ion (Li-ion) batteries in data centers is growing. Now commonly used in uninterruptible power supplies, they are expected to account for 38.5% of the data-center battery market by 2025, up from 15% in 2020, according to consulting firm Frost & Sullivan.To read this article in full, please click here

Data center fires raise concerns about lithium-ion batteries

Fire is to blame for a small but significant number of data-center outages including a March 28 fire that caused severe damage to a data center in France, and an analysis of global incidents highlights ongoing concerns about the safety of lithium-ion batteries and their risk of combustion.The use of lithium ion (Li-ion) batteries in data centers is growing. Now commonly used in uninterruptible power supplies, they are expected to account for 38.5% of the data-center battery market by 2025, up from 15% in 2020, according to consulting firm Frost & Sullivan.To read this article in full, please click here

4 ways to leverage existing kernel security features to set up process monitoring

The large attack surface of Kubernetes’ default pod provisioning is susceptible to critical security vulnerabilities, some of which include malicious exploits and container breakouts. I believe one of the most effective workload runtime security measures to prevent such exploits is layer-by-layer process monitoring within the container.

It may sound like a daunting task that requires additional resources, but in reality, it is actually quite the opposite. In this article, I will walk you through how to use existing Linux kernel security features to implement layer-by-layer process monitoring and prevent threats.

Threat prevention and process monitoring

Containerized workloads in Kubernetes are composed of numerous layers. An effective runtime security strategy takes each layer into consideration and monitors the process within each container, also known as process monitoring.

Threat detection in process monitoring involves integrating mechanisms that isolate workloads or control access. With these controls in place, you can effectively prevent malicious behavior, reduce your workload’s attack surface, and limit the blast radius of security incidents. Fortunately, we can use existing Kubernetes mechanisms and leverage Linux defenses to achieve this.

Kernel security features

By pulling Linux defenses closer to the container, we can leverage existing Kubernetes mechanisms to monitor processes and reduce Continue reading

Content Creation Complications

If you’ve noticed my regular blog posts have been a bit irregular as of late you’re not alone. I’m honestly working through a bit of writer’s block as of late. The irony is that I’m not running out of things to talk about. I’m actually running out of time to talk about them the way that I want.

Putting in the Work

By now you, my dear readers, know that I’m not going to put out a post of 200-300 words just to put something out during the week. I’d rather spend some time looking into a topic and creating something that informs or encourages discussion. That means having sources or doing research.

Research takes time. Ironically enough I’ve always had a much easier time writing things so long as I have the info to pull from in my head. One of the side effects of neurodivergence that I’ve learned about recently is that neurodivergent people tend to write their ‘first draft’ in their head throughout the creation process. Rather than writing and rewriting over and over again I pool all the information in my brain and work through it all to put down my final thoughts. That means what Continue reading

Kyndryl lays off staff in search of efficiency

Kyndryl, the managed IT services provider that spun out of IBM, has announced layoffs that could affect its own internal IT services.“We are eliminating some roles globally — a small percentage — to become more efficient and competitive,” said a Kyndryl spokesperson, without giving the exact number of employees affected due to the layoffs.“These actions will enable us to focus our investments in areas that directly benefit our customers and position Kyndryl for profitable growth,” the spokesperson said, adding that the company was in the process of undergoing transformation to streamline and simplify its processes and systems.Bloomberg first reported about the layoffs.To read this article in full, please click here

Kyndryl lays off staff in search of efficiency

Kyndryl, the managed IT services provider that spun out of IBM, has announced layoffs that could affect its own internal IT services.“We are eliminating some roles globally — a small percentage — to become more efficient and competitive,” said a Kyndryl spokesperson, without giving the exact number of employees affected due to the layoffs.“These actions will enable us to focus our investments in areas that directly benefit our customers and position Kyndryl for profitable growth,” the spokesperson said, adding that the company was in the process of undergoing transformation to streamline and simplify its processes and systems.Bloomberg first reported about the layoffs.To read this article in full, please click here

Kubernetes Unpacked 022: Kubernetes Networking And Abstraction With Cilium And eBPF

In this episode, Michael catches up with Stephane Karagulmez, Senior Solution Architect at Isovalent (founded by the creators of Cilium). Michael spent a lot of time working with Cilium, which is open-source software that provides networking and observability capabilities for Kubernetes workloads. Cilium is based on another open-source project, eBFP. It's important to understand the details and performance changes when implementing eBPF and removing kube-proxy.

Kubernetes Unpacked 022: Kubernetes Networking And Abstraction With Cilium And eBPF

In this episode, Michael catches up with Stephane Karagulmez, Senior Solution Architect at Isovalent (founded by the creators of Cilium). Michael spent a lot of time working with Cilium, which is open-source software that provides networking and observability capabilities for Kubernetes workloads. Cilium is based on another open-source project, eBFP. It's important to understand the details and performance changes when implementing eBPF and removing kube-proxy.

The post Kubernetes Unpacked 022: Kubernetes Networking And Abstraction With Cilium And eBPF appeared first on Packet Pushers.

Dropped packet reason codes in Linux 6+ kernels

Using sFlow to monitor dropped packets describes support for standard sFlow Dropped Packet Notications in the open source Host sFlow agent. This article describes additional capabilities in Linux 6+ kernels that clarify reasons why packets are dropped in the kernel.

The recent addition of dropreason.h in Linux 6+ kernels provides detailed reasons for packet drops. The netlink drop_monitor API has been extended to include the NET_DM_ATTR_REASON attribute to report the drop reason, see net_dropmon.h.

The following example illustrates the value of the reason code in explaining Linux packet drops.

tcp_v4_rcv+0x7c/0xef0
The value of NET_DM_ATTR_SYMBOL shown above indicates that the packet was dropped in the tcp_v4_rcv function in Linux kernel at memory location 0x7c/0xef0. While this information is helpful, there are many reasons why a TCP packet may be dropped. 
NO_SOCKET
In this case, the value of NET_DM_ATTR_REASON shown above indicates that the TCP packet was dropped because no application had opened a socket and so there was nowhere to deliver the packet.

In the case of Linux-based hardware switches or smart network adapters, where packet processing is offloaded to hardware, the netlink drop_monitor events include NET_DM_ATTR_HW_TRAP_GROUP_NAME and NET_DM_ATTR_HW_TRAP_NAME attributes and packet header information supplied by the hardware Continue reading

From IP packets to HTTP: the many faces of our Oxy framework

From IP packets to HTTP: the many faces of our Oxy framework
From IP packets to HTTP: the many faces of our Oxy framework

We have recently introduced Oxy, our Rust-based framework for proxies powering many Cloudflare services and products. Today, we will explain why and how it spans various layers of the OSI model, by handling directly raw IP packets, TCP connections and UDP payloads, all the way up to application protocols such as HTTP and SSH.

On-ramping IP packets

An application built on top of Oxy defines — in a configuration file — the on-ramps that will accept ingress traffic to be proxied to some off-ramp. One of the possibilities is to on-ramp raw IP packets. But why operate at that layer?

The answer is: to power Cloudflare One, our network offering for customers to extend their private networks — such as offices, data centers, cloud networks and roaming users — with the Cloudflare global network. Such private networks operate based on Zero Trust principles, which means every access is authenticated and authorized, contrasting with legacy approaches where you can reach every private service after authenticating once with the Virtual Private Network.

To effectively extend our customer’s private network into ours, we need to support arbitrary protocols that rely on the Internet Protocol (IP). Hence, we on-ramp Cloudflare Continue reading

Helping protect personal information in the cloud, all across the world

Helping protect personal information in the cloud, all across the world
Helping protect personal information in the cloud, all across the world

Cloudflare has achieved a new EU Cloud Code of Conduct privacy validation, demonstrating GDPR compliance to strengthen trust in cloud services

Internet privacy laws around the globe differ, and in recent years there’s been much written about cross-border data transfers. Many regulations require adequate protections to be in place before personal information flows around the world, as with the European General Data Protection Regulation (GDPR). The law rightly sets a high bar for how organizations must carefully handle personal information, and in drafting the regulation lawmakers anticipated personal data crossing-borders: Chapter V of the regulation covers those transfers specifically.

Whilst transparency on where personal information is stored is important, it’s also critically important how personal information is handled, and how it is kept safe and secure. At Cloudflare, we believe in protecting the privacy of personal information across the world, and we give our customers the tools and the choice on how and where to process their data. Put simply, we require that data is handled and protected in the same, secure, and careful way, whether our customers choose to transfer data across the world, or for it to remain in one country.

And today we are proud to announce Continue reading

1 … 322 323 324 325 326 … 3,841