Exploring AKS networking options

At Kubecon 2023 in Amsterdam, Azure made several exciting announcements and introduced a range of updates and new options to Azure-CNI (Azure Container Networking Interface). These changes will help Azure Kubernetes Services (AKS) users to solve some of the pain points that they used to face in previous iterations of Azure-CNI such as IP exhaustion and big cluster deployments with custom IP address management (IPAM). On top of that, with this announcement Microsoft officially added an additional dataplane to the Azure platform.

The big picture

Worker nodes in an AKS (Azure Kubernetes Service) cluster are Azure VMs pre-configured with a version of Kubernetes that has been tested and certified by Azure. These clusters communicate with other Azure resources and external sources (including the internet) via the Azure virtual network (VNet).

Now, let’s delve into the role of the dataplane within this context. The dataplane operations take place within each Kubernetes node. It is responsible for handling the communication between your workloads, and cluster resources. By default, an AKS cluster is configured to utilize the Azure dataplane, which Continue reading

Turbocharging host workloads with Calico eBPF and XDP

In Linux, network-based applications rely on the kernel’s networking stack to establish communication with other systems. While this process is generally efficient and has been optimized over the years, in some cases it can create unnecessary overhead that can impact the overall performance of the system for network-intensive workloads such as web servers and databases.

XDP (eXpress Data Path) is an eBPF-based high-performance datapath inside the Linux kernel that allows you to bypass the kernel’s networking stack and directly handle packets at the network driver level. XDP can achieve this by executing a custom program to handle packets as they are received by the kernel. This can greatly reduce overhead, improve overall system performance, and improve network-based applications by shortcutting the normal networking path of ordinary traffic. However, using raw XDP can be challenging due to its programming complexity and the high learning curve involved. Solutions like Calico Open Source offer an easier way to tame these technologies.

Calico Open Source is a networking and security solution that seamlessly integrates with Kubernetes and other cloud orchestration platforms. While infamous for its policy engine and security capabilities, there are many other features that can be used in an environment by installing Continue reading

What’s new in Calico v3.26

We are excited to announce the release of Calico v3.26! This latest milestone brings a range of enhancements and new features to the Calico ecosystem, delivering an optimized and secure networking solution. This release has a strong emphasis on product performance, with strengthened security measures, expanded compatibility with Windows Server 2022 and OpenStack Yoga, and notable improvements to the Calico eBPF dataplane.

As always, let’s begin by thanking our awesome community members who helped us in this release.

Community shoutout

Big thanks to our GitHub users afshin-deriv, blue-troy, and winstonu for their valuable contributions in enhancing the Kind installation and VXLAN documentation, as well as improving the code comments.

Additionally, we would like to extend our appreciation to laibe and yankay for their efforts in updating the flannel version and improving the IPtables detection mechanism. Their contributions have been instrumental in improving the overall functionality and reliability of our project.

Finally, a huge thank-you to dilyevsky, detailyang, mayurjadhavibm, and olljanat for going above and beyond in pushing Calico beyond its original scope and for generously sharing their solutions with the rest of the community.

Community-driven enhancement request: Fine-grained BGP route control

The primary responsibility Continue reading

Monitoring Kubernetes clusters activity with Azure Managed Grafana and Calico

Cloud computing revolutionized how a business can establish its digital presence. Nowadays, by leveraging cloud features such as scalability, elasticity, and convenience, businesses can deploy, grow, or test an environment in every corner of the world without worrying about building the required infrastructure.

Unlike the traditional model, which was based on notifying the service provider to set up the resources for customers in advance, in an on-demand model, cloud providers implement application programming interfaces (API) that can be used by customers to deploy resources on demand. This allows the customer to access an unlimited amount of resources on-demand and only pay for the resources they use without worrying about the infrastructure setup and deployment complexities.

For example, a load balancer service resource is usually used to expose an endpoint to your clients. Since a cloud provider’s bandwidth might be higher than what your cluster of choice can handle, a huge spike or unplanned growth might cause some issues for your cluster and render your services unresponsive.

To solve this issue, you can utilize the power of proactive monitoring and metrics to find usage patterns and get insight into your system’s overall health and performance.

In this hands-on tutorial, I will Continue reading

Calico’s 3.26.0 update unlocks high density vertical scaling in Kubernetes

Kubernetes is a highly popular and widely used container orchestration platform designed to deploy and manage containerized applications at a scale, with strong horizontal scaling capabilities that can support up to 5,000 nodes; the only limit in adding nodes to your cluster is your budget. However, its vertical scaling is restricted by its default configurations, with a cap of 110 pods per node. To maximize the use of hardware resources and minimize the need for costly horizontal scaling, users can adjust the kubelet maximum pod configuration to increase this limit allowing more pods to run concurrently on a single node.

To avoid network performance issues and achieve efficient horizontal scaling in a Kubernetes cluster that is tasked to run a large number of pods, high-speed links and switches are essential. A reliable and flexible Software Defined Networking (SDN) solution, such as Calico, is also important for managing network traffic efficiently. Calico has been tested and proven by numerous companies for horizontal scaling, but in this post, we will discuss recent improvements made to help vertical scaling of containerized applications to just work.

For example, the following chart illustrates the efficiency achieved with the improvements of vertical scaling in Calico 3. Continue reading

Navigating the security challenges of multi-tenancy in a cloud environment

Multi-tenancy can maximize the number of resources that are utilized in a cluster by sharing these resources between different groups, teams, or customers. However, boundaries must be placed to avoid problems associated with resource-sharing. On top of that, in a multi-tenant cluster, the number of security policies might gradually grow to the point where a slight misconfiguration could cause major security problems, performance issues, and service disruptions.

In this blog post, we will focus on multi-tenancy issues such as bandwidth shortage, security policy scaling, privacy impacts, and suggest a few solutions that you can deploy to solve them in your environment. We will also look at how an eBPF-based security design can offer better performance and help you navigate the complex multi-tenant environment with ease.

What is multi-tenancy?

Technologies such as virtualization, containerization, or any other technologies that allow a range of different workloads to share the underlying hardware resources, all have a common goal—allocate resources as efficiently as possible and make the most of the available hardware. However, it is common for workloads that are running in such an environment to not fully utilize all the potential power that the hardware can offer, and in many cases, leave a Continue reading

Kubernetes network monitoring: What is it, and why do you need it?

In this article, we will dive into Kubernetes network monitoring and metrics, examining these concepts in detail and exploring how metrics in an application can be transformed into tangible, human-readable reports. The article will also include a step-by-step tutorial on how to enable Calico’s integration with Prometheus, a free and open-source CNCF project created for monitoring the cloud. By the end of the article, you will be able to create customized reports and graphical dashboards from the metrics that Calico publishes to get better insight into the inner workings of your cluster and its various components. In addition, you will have the fundamental knowledge of how these pieces can fit together to establish Kubernetes network monitoring for any environment.

Background

The benefits offered by cloud computing and infrastructure as code, including scalability, easy distribution, and quick and flexible deployment, have caused cloud service adoption to skyrocket. But this rapid adoption requires checks and balances to ensure that cloud services are secure and running in their desired state. Furthermore, any security events and problems should be logged and reported for future examination.

Read our guide on Kubernetes logging: Approaches and best practices

In the past, traditional monitoring solutions such as Nagios Continue reading

Turbocharging Host Workloads with Calico eBPF and XDP

In Linux, network-based applications rely on the kernel’s networking stack to establish communication with other systems. While this process is generally efficient and has been optimized over the years, in some cases it can create unnecessary overhead that can affect the overall performance of the system for network-intensive workloads such as web servers and databases. Calico Open Source offer an easier way to tame these technologies. Calico Open Source is a networking and security solution that seamlessly integrates with Kubernetes and other cloud orchestration platforms. While infamous for its policy engine and security capabilities, there are many other features that can be used in an environment by installing Continue reading

What’s new in Calico v3.25

We’ve just released Calico v3.25! This milestone release includes a number of eBPF dataplane improvements designed to deliver an even faster upgrade experience, smaller memory footprint, and shorter eBPF networking object load time speed.

But before we get into the details of these changes, let’s welcome and thank our new community problem-solvers who got their first contribution requests merged into our beloved project.

Community shoutout

Documentation is the most essential part of any project since that is the go-to place for everyone to get a better idea about the capabilities or deployment of that project. So let’s start by giving a big shout-out to @cavcrosby, @Congrool, @chenbojian, and @gopihc for their attention to detail and fixing issues in the project documentation.

Shoutout to @OrvilleQ and @masap for extending the exclusion list of interfaces to make the automatic interface selection of Calico even faster.

Shoutout to @gregwhorley, @dlipovetsky, @nickperry, and @tamcore for their updates to `tigera-operator` that will make the installation and maintenance experience of Calico even better.

Shoutout to @ramanujadasu for enhancing the logic behind the unicast IP address hashing.

Shoutout to @chrisjohnson00 and @vitaliy-leschenko for enhancing the Calico windows installer and adding Continue reading

Securing Windows workloads

Containers are a great way to package applications, with minimal libraries required. It guarantees that you will have the same deployment experience, regardless of where the containers are deployed. Container orchestration software pushes this further by preparing the necessary foundation to create containers at scale.

Linux and Windows support containerized applications and can participate in a container orchestration solution. There is an incredible number of guides and how-to articles on Linux containers and container orchestration, but these resources get scarce when it comes to Windows, which can discourage companies from running Windows workloads.

This blog post will examine how to set up a Windows-based Kubernetes environment to run Windows workloads and secure them using Calico Open Source. By the end of this post, you will see how simple it is to apply your current Kubernetes skills and knowledge to rule a hybrid environment.

Windows containers

A container is similar to a lightweight packaging technique. Each container packages an application in an isolated environment that shares its kernel with the underlying host, making it bound by the limits of the host operating system. These days, everyone is familiar with Linux containers, a popular way to run Linux-based binary files in an Continue reading

How to build a service mesh with Istio and Calico

Microservices are loosely coupled software that provides flexibility and scalability to a cloud environment. However, securing this open architecture from vulnerabilities and malicious actors can be challenging without a service mesh.

This blog post will demonstrate how you can create an Istio and Calico integration to establish a service mesh that will manipulate HTTP traffic in the application layer. This Istio-Calico integration provides a unified way to write security policies interacting with applications and implement restrictions without disturbing the entire system.

What’s a service mesh?

A service mesh is a software layer that sits between the microservices that form your workload. After deploying and enabling a service mesh system for your workloads, an injector will add a sidecar container to each. These sidecars then collect and manipulate information via the rules you provide, allowing you to secure your cluster on an application level without requiring any change inside your software.

Without a service mesh, to ensure communication integrity and confidentiality between workloads, you must modify each to embed encryption methods. On top of that, gathering insight into the events that are happening in the application layer will require modifying the workload application itself, which all requires a good amount of Continue reading

Getting started with EKS and Calico

Cloud-native applications offer a lot of flexibility and scalability, but to leverage these advantages, we must create and deploy a suitable environment that will enable cloud-native applications to work their magic.

Managed services, self-managed services, and bare metal are three primary categories of Kubernetes deployment in a cloud environment. Our focus in this article will be on Amazon Web Service’s (AWS) managed Kubernetes service, Elastic Kubernetes Service (EKS), and capabilities that Calico Open Source adds to the EKS platform.

Managed services

A managed cluster is a quick and easy way to deploy an enterprise-grade Kubernetes cluster. In a managed cluster, mundane operations such as provisioning new nodes, upgrading the OS/Kubernetes, and scaling resources are transferred to the cloud provider, which allows you to expand your application with ease.

EKS is a managed service by AWS that offers a fault-tolerant Kubernetes control plane endpoint and automates worker node maintenance and deployment process.

Comparing popular CNI options in EKS

Most popular managed services, such as EKS, come with an official CNI that offers networking and other features for your cluster. While these CNIs are highly integrated with the underlying system, they can introduce some limitations. To remedy these limitations and unlock the Continue reading

What is new in Calico v3.24

A couple of weeks ago, TIgera engineers released the new version of Calico, as part of a community effort to drive cloud security and networking even further. But before I begin diving into the details of this new release, I want to first spotlight a few of our community members who have merged their contributions to Calico Open Source for the first time.

Shout out to @agaffney for adding configurable labels and annotations to the tigera-operator deployment in Helm charts.

Shout out to @backjo for improving the Calico Windows installation script and adding support for IMDSv2 in AWS EC2 data retrieval.

Shout out to @EugenMayer for pointing out an improvement for the calicoctl binary in a Helm chart installation and @lou-lan for making it happen.

Shout out to @joskuijpers for informing the community about the outdated ipset package in the calico-node ARM64 image and @ScOut3R for updating it.

Shout out to @juanfresia for contributing changes to enable Calico to run without programming the route table, useful when integrating with other routing mechanisms.

Shout out to @muff1nman, who added Wireguard traffic to the Calico failsafe ports, allowing us to confidently apply network security policies without worrying about accidentally cutting off Continue reading

What is eBPF and what are its use cases

With the recent advancements in service delivery through containers, Linux has gained a lot of popularity in cloud computing by enabling digital businesses to expand easily regardless of their size or budget. These advancements have also brought a new wave of attack, which is challenging to address with the same tools we have been using for non cloud-native environments. eBPF offers a new way to interact with the Linux kernel, allowing us to reexamine the possibilities that once were difficult to achieve.

In this post, I will go through a brief history of the steps that eBPF had to take to become the Swiss army knife inside the Linux kernel and point out how it can be used to achieve security in a cloud-native environment. I will also share my understanding of what happens inside the kernel that prevents BPF programs from wreaking havoc on your operating system.

BPF history

In the early days of computing, Unix was a popular solution for capturing network traffic, and using CMU/Stanford packet filter (CSPF) to capture packets using 64KB PDP-11 was gaining popularity by the second. Without a doubt, this was a pioneering work and a leap forward for its time but like Continue reading

A practical guide to container networking

An important part of any Kubernetes cluster is the underlying containers. Containers are the workloads that your business relies on, what your customers engage with, and what shapes your networking infrastructure. Long story short, containers are arguably the soul of any containerized environment.

One of the most popular open-source container orchestration systems, Kubernetes, has a modular architecture. On its own, Kubernetes is a sophisticated orchestrator that helps you manage multiple projects in order to deliver highly available, scalable, and automated deployment solutions. But to do so, it relies on having a suite of underlying container orchestration tools.

This blog post focuses on containers and container networking. Throughout this post, you will find information on what a container is, how you can create one, what a namespace means, and what the mechanisms are that allow Kubernetes to limit resources for a container.

Containers

A container is an isolated environment used to run an application. By utilizing the power of cgroup, namespace, and filesystem from the Linux kernel, containers can be allocated with a limited amount of resources and filesystems inside isolated environments.

How to maximize K3s resource efficiency using Calico’s eBPF data plane

Amazon’s custom-built Graviton processor allows users to create ARM instances in the AWS public cloud, and Rancher K3s is an excellent way to run Kubernetes in these instances. By allowing a lightweight implementation of Kubernetes optimized for ARM with a single binary, K3s simplifies the cluster initialization process down to executing a simple command.

In an earlier article, I discussed how ARM architecture is becoming a rival to x86 in cloud computing, and steps that can be taken to leverage this situation and be prepared for this new era. Following the same narrative, in this article I’ll look at an example of the Calico eBPF data plane running on AWS, using Terraform to bootstrap our install to AWS, and Rancher K3s to deploy the cluster.

A few changes to Calico are needed for ARM compatibility, including updating parts, enabling eBPF, and compiling operators for the ARM64 environment:.

• Tigera Operator – Tigera Operator is the recommended way to install Calico.
• go-build – go-build is a container environment packed with all the utilities that Calico requires in its compilation process.
• Calico-node – Calico-node is the pod that hosts Felix (i.e. it is the brain that carries control plane decisions fto Continue reading

Is ARM architecture the future of cloud computing?

Central processing units (CPUs) can be compared to the human brain in that their unique architecture allows them to solve mathematical equations in different ways. x86 is the dominant architecture used in cloud computing at the time of this writing; however, it is worth noting that this architecture is not efficient for every scenario, and its proprietary nature is causing an industry shift toward ARM.

ARM (Advanced RISC Machines) is a type of CPU architecture that powers most tablets and smartphones, as well as the fastest supercomputer in the world (supercomputer Fugaku). ARM’s low power consumption and high computational performance make it a worthy rival for x86 in cloud computing.

In this article, I will talk about a few popular ARM projects, the main difference between x86 and ARM architectures, and explore how we can prepare developers for the future by providing them with an ARM-based container environment.

ARM versus x86

Companies are increasing their pursuit to leverage ARM in order to reduce both cost and energy consumption. While x86 remains a proprietary CPU architecture, ARM provides licenses to other companies allowing them to design their own custom-built processors using ARM’s patented technology.

Amazon’s custom-designed Graviton processor is a great Continue reading

How network policies can protect your environment from future vulnerabilities like Log4j

If you have access to the internet, it’s likely that you have already heard of the critical vulnerability in the Log4j library. A zero-day vulnerability in the Java library Log4j, with the assigned CVE code of CVE-2021-44228, has been disclosed by Chen Zhaojun, a security researcher in the Alibaba Cloud Security team. It’s got people worried—and with good reason.

This is a serious flaw that needs to be addressed right away, since it can result in remote code execution (RCE) in many cases. By now, I have seen many creative ways in which this can be used to infiltrate or disturb services. The right solution is to identify and patch your vulnerable Log4j installations to the fixed versions as soon as possible. If you are using Log4j, make sure you are following this page where you can find the latest news about the vulnerability.

What else should you be doing, though, for this and similar exploits? In this blog post, I’ll look at the impact of the vulnerability in a Kubernetes cluster, and share a couple of ways that you can prevent such vulnerabilities in the future.

How does the Log4j vulnerability work?

On its own, the Log4j vulnerability Continue reading

Turbocharging AKS networking with Calico eBPF

A single Kubernetes cluster expends a small percentage of its total available assigned resources on delivering in-cluster networking. We don’t have to be satisfied with this, though—achieving the lowest possible overhead can provide significant cost savings and performance improvements if you are running network-intensive workloads. This article explores and explains the improvements that can be achieved in Microsoft Azure using Calico eBPF in AKS, including reducing CPU usage, decreasing complexity, enabling easier compliance and troubleshooting, and more.

Before going into details about how exactly Calico takes advantage of eBPF, it is important to note that in the context of this article, Calico is viewed as an additional networking layer on top of Azure CNI, providing functionality that turbocharges its performance. In particular, the standard instructions for installing Calico’s network policy engine with AKS use a version of Calico that pre-dates eBPF mode.

Accelerating network performance

Test methodology

To show how Calico accelerates AKS network performance using eBPF, the Calico team ran a series of network performance benchmarks based on the k8s-bench-suite. These performance benchmarks compared the latest Calico eBPF data plane (using the iptables data plane) with a vanilla AKS cluster.

Tests were run using Standard_D2s_v3 nodes, which are a Continue reading

Lightning-fast Kubernetes networking with Calico & VPP

Public cloud infrastructures and microservices are pushing the limits of resources and service delivery beyond what was imaginable until very recently. In order to keep up with the demand, network infrastructures and network technologies had to evolve as well. Software-defined networking (SDN) is the pinnacle of advancement in cloud networking; by using SDN, developers can now deliver an optimized, flexible networking experience that can adapt to the growing demands of their clients.

This article will discuss how Tigera’s new Vector Packet Processing (VPP) data plane fits into this landscape and share some benchmark details about its performance. Then it will demonstrate how to run a VPP-equipped cluster using AWS public cloud and secure it with Internet Protocol Security (IPsec).

Introduction to Vector Packet Processing

Project Calico is an open-source networking and security solution. Although it focuses on securing Kubernetes networking, Calico can also be used with OpenStack and other workloads. Calico uses a modular data plane that allows a flexible approach to networking, providing a solution for both current and future networking needs.

VPP is an easily extensible, kernel-independent, highly optimised, and blazing-fast open-source data plane project that operates between layer 2 and layer 4 of the OSI Continue reading