Azure Host-Based Networking: vNIC Interface Architecture – Synthetic Interface and Virtual Function

Before moving to the Virtual Filtering Platform (VFP) and Accelerated Network (AccelNet) section, let’s look at the guest OS vNIC interface architecture. When we create a VM, Azure automatically attaches a virtual NIC (vNIC) to it. Each vNIC has a synthetic interface, a VMbus device, using a netvsc driver. If the Accelerated Networking (AccelNet) is disabled on a VM, all traffic flows pass over the synthetic interface to the software switch. Azure hosts servers have Mellanox/NVIDIA Single Root I/O Virtualization (SR-IOV) hardware NIC, which offers virtual instances, Virtual Function (VF), to virtual machines. When we enable AccelNet on a VM, the mlx driver is installed to vNIC. The mlx driver version depends on an SR-IOV type. The mlx driver on a vNIC initializes a new interface that connects the vNIC to an embedded switch on a hardware SR-IOV. This VF interface is then associated with the netvsc interface. Both interfaces use the same MAC address, but the IP address is only associated with the synthetic interface. When AccelNet is enabled, VM’s vNIC forwards VM data flows over the VF interface via the synthetic interface. This architecture allows In-Service Software Updates (ISSU) for SR-IOV NIC drivers. 

Note! Exception traffic, a data flow with no flow entries on a UFT/GFT, is forwarded through VFP in order to create flow-action entries to UFT/GFT.

Figure 1-1: Azure Host-Based SDN Building Blocks.

Azure Host-Based Networking: VFP and AccelNet Introduction

Software-Defined Networking (SDN) is an architecture where the network’s control plane is decoupled from the data plane to centralized controllers. These intelligent, programmable controllers manage network components as a single system, having a global view of the whole network. Microsoft’s Azure uses a host-based SDN solution, where network virtualization and most of its services (Firewalls, Load balancers, Gateways) run as software on the host. The physical switching infrastructure, in turn, offers a resilient, high-speed underlay transport network between hosts.

Figure 1-1 shows an overview of Azure’s SDN architecture. Virtual Filtering Platform (VFP) is Microsoft’s cloud-scale software switch operating as a virtual forwarding extension within a Hyper-V basic vSwitch. The forwarding logic of the VFP uses a layered policy model based on policy rules on Match-Action Table (MAT). VFP works on a data plane, while complex control plane operations are handed over to centralized control systems. VFP layers, such as VNET, NAT, ACL, and Metering, have dedicated controllers that programs policy rules to MAT using southbound APIs.

Software switches switching processes are CPU intensive. To reduce the burden of CPU cycles, VFP offloads data forwarding logic to hardware NIC after processing the first packet of the flow and creating the flow Continue reading

Azure Host-Based SDN: Part 1 – VFP Introduction

Azure Virtual Filtering Platform (VFP) is Microsoft’s cloud-scale virtual switch operating as a virtual forwarding extension within a Hyper-V basic vSwitch. Figure 1-1 illustrates an overview of VFP building blocks and relationships with basic vSwitch. Let’s start the examination from the VM vm-nwkt-1 perspective. Its vNIC vm-cafe154 has a synthetic interface eth0 using a NetVSC driver (Network Virtual Service Client). The Hyper-V vSwitch on the Parent Partition is a Network Virtual Service Provider (NetVSP) with VM-facing vPorts. Vm-cafe154 is connected to vPort4 over the logical inter-partition communication channel VMBus. VFP sits in the data path between VM-facing vPorts and default vPort associated with physical NIC. VFP uses port-specific Layers for filtering traffic to and from VMs. A VFP Layer is a Match Action Table (MAT) having a set of policy Rules. Rules consist of Conditions and Actions and are divided into Groups. Each layer is programmed by independent, centralized Controllers without cross-controller dependencies.

Let’s take a concrete example of Layer/Group/Rule object relationship and management by examining the Network Security Group (NSG) in the ACL Layer. Each NSG has a default group for Infrastructure rules, which allows Intra-VNet traffic, outbound Internet connection, and load balancer communication (health check, etc.). We Continue reading

Azure Host-Based SDN: Part 1 – VFP Introduction

AWS Networking Fundamentals book: Table of Contents

Here is the Table of Contents of my AWS Networking Fundamentals book. I have added the figures which illustrate the example scenarios in each chapter. The book is available at Leanpub.com. It is still in progress, and there will be additional chapters soon.

AWS Networking Fundamentals: A Practical Guide to Understand How to Build a Virtual Datacenter into the AWS Cloud

 Table of Content

Table of Contents

Chapter 1: Virtual Private Cloud - VPC 1

VPC 1

VPC Introduction 1

The Structure of Availability Zone 2

Create VPC - AWS Console 4

Select Region 4

Create VPC 7

DHCP Options Set 9

Main Route Table 10

VPC Verification Using AWS CLI 12

Create VPC - AWS CloudFormation 16

Create Template 17

Uppload Template 17

Verification Using AWS Console 18

VPC Verification using AWS CLI 21

Create Subnets - AWS Console 23

Create Subnets 24

Route Tables 29

Create Subnets – AWS Console 30

Create Subnets - AWS CloudFormation 37

Create Network ACL 40

Chapter 2: VPC Control-Plane 43

VPC Control-Plane – Mapping Service 43

Introduction 43

Mapping Register 43

Mapping Request - Reply 44

Data-Plane Operation 45

References 46

Chapter 3: VPC Internet Gateway Service 47

Introduction 47

Allow Internet Access from Subnet 48

Create Internet Gateway 49

Update Subnet Route Table 54

Network Access Control List 57

Associate SG and Elastic-IP with EC2 59

Create Security Group 59

Launch an EC2 Instance 65

Allocate Elastic IP address from Amazon Ipv4 Pool 71

Reachability Analyzer 81

Billing 85

Chapter 4: VPC NAT Gateway 87

Introduction 87

Create NAT Gateway and Allocate Continue reading

AWS Networking – Part XI: VPC NAT Gateway

Introduction

Back-End EC2 instances like Application and Database servers are most often launched on a Private subnet. As a recap, a Private subnet is a subnet that doesn’t have a route to the Internet Gateway in its Route table. Besides, EC2 instances in the Private subnet don’t have Elastic-IP address association. These two facts mean that EC2 instances on the Private subnet don’t have Internet access. However, these EC2 instances might still need occasional Internet access to get firmware upgrades from the external source. We can use a NAT Gateway (NGW) for allowing IPv4 Internet traffic from Private subnets to the Internet. When we launch an NGW, we also need to allocate an Elastic-IP address (EIP) and associate it with the NGW. This association works the same way as the EIP-to-EC2 association. It creates a static NAT entry to IGW that translates  NGW’s local subnet address to its associated EIP. The NGW, in turn, is responsible for translating the source IP address from the ingress traffic originated from the Private subnet to its local subnet IP address. As an example, EC2 instance NWKT-EC2-Back-End sends packets towards the Internet to NGW. When the NGW receives these packets, it rewrites the source IP address 10.10.1.172 with its Public subnet IP address 10.10.0.195 and forwards packets to the Internet gateway. IGW translates the source IP address 10.10.0.195 to EIP 18.132.96.95 (EIP associated with NGW). That means that the source IP of data is rewritten twice, first by NGW and then by IGW.

Figure 4-1 illustrates our example NAT GW design and its configuration steps. As a pretask, we launch an EC2 instance on the Private subnet 10.10.1.0/24 (1). We also modify the existing Security Group (SG) to allow an Inbound/Outbound ICMP traffic within VPC CIDR 10.10.0.0/16 (2). We also allow an SSH session initiation from the 10.10.0.218/24. I’m using the same SG for both EC2 instances to keep things simple. Besides, both EC2 uses the same Key Pair. Chapter 3 shows how to launch an EC2 instance and how we modify the SGs, and that is why we go ahead straight to the NGW configuration.

When we have done pre-tasks, we launch an NGW on the Public subnet (3). Then we allocate an EIP and associate it with NGW (4). Next, we add a default route towards NGW on the Private subnet Route Table (5).

The last three steps are related to connectivity testing. First, verify Intra-VPC IP connectivity using ICMP (6). Then we test the Internet connectivity (7). As the last step, we can confirm that no route exists back to NWKT-EC-Backe-End from the IGW. We are using an AWS Path Analyzer for that (8).

Note! Our example doesn’t follow good design principles. AWS Availability Zones (AZ) are restricted failure domains, which means that failure in one AZ doesn’t affect the operation of other AZ. Now, if our NGW on AZ eu-west-2c fails,  Internet traffic from the Private subnet on eu-west2a fails. The proper design is to launch NGW on the AZ where unidirectional egress Internet access is needed.

AWS Networking – Part X: VPC Internet Gateway Service – Part Two

Associate SG and Elastic-IP with EC2

In the previous section, we create an Internet Gateway for our VPC. We also add a static route towards IGW into the Route Table of Subnet 10.10.0.0/24. In this section, we first create a Security Group (SG).  The SG allows SSH connection to the EC2 instance and ICMP from the EC2. Then we launch an EC2 and attach the previously configure SG to it. As the last step, we allocate an Elastic IP address (EIP) from the AWS Ipv4 address pool and associate it with the EC instance. When we are done with all the previous steps, we will test the connection. First, we take ssh connection from MyPC to EC2. Then, we ping MyPC from the EC2. We also use AWS Reachability Analyzer to validate the path from IGE to EC2 instance. The last section introduces AWS billing related to this chapter.

Figure 3-20: EC2 Instance, Elastic IP, and Security Group.

AWS Networking – Part X: VPC Internet Gateway Service – Part One

Introduction

This chapter explains what components/services and configurations we need to allow Internet traffic to and from an EC2 instance. VPCs themselves are closed entities. If we need an Internet connection, we need to use an AWS Internet Gateway (IGW) service. The IGW is running on a  Blackfoot Edge Device in the AWS domain. It performs Data-Plane VPC encapsulation and decapsulation, as well as  IP address translation. We also need public, Internet routable IP addresses. In our example, we allocate an AWS Elastic-IP (EIP) address. Then we associate it with EC2 Instance. By doing it, we don’t add the EIP to the EC2 instance itself. Instead, we create a static one-to-one NAT entry into the VPC associated IGW. The subnet Route Table includes only a VPC’s CIDR range local route. That is why we need to add a routing entry to the Subnet RT, default or more specific, towards IGW. Note that a subnet within an AWS VPC is not a Broadcast domain (VPC doesn’t even support Broadcasts). Rather, we can think of it as a logical place for EC2 instances having uniform connection requirements, like reachability from the Internet. As a next step, we define the security policy. Each Subnet has a Network Access Control List (NACL), which is a stateless Data-Plane filter. The Stateless definition means that to allow bi-directional traffic flow, we have to permit flow-specific Request/Reply data separately. For simplicity, we are going to use the Subnet Default NACL. The Security Group (SG), in turn, is a stateful EC2 instance-specific Data-Plane filter. The Stateful means that filter permits flow-based ingress and egress traffic. Our example security policy is based on the SG. We will allow an SSH connection from the external host 91.152.204.245 to EC2 instance NWKT-EC-Fron-End. In addition, we allow all ICMP traffic from the EC2 instance to the same external host. As the last part, this chapter introduces the Reachability Analyzer service, which we can use for troubleshooting connections. Figure 3-1 illustrates what we are going to build in this chapter.

Figure 3-1: Setting Up an Internet Connection for Public Subnet of AWS VPC.

AWS Networking – Part IX: AWS VPC Control-Plane – Mapping Servce

Introduction

This chapter explains the VPC Control-Plane operation when two EC2 instances within the same subnet initiate TCP session between themself. In our example, EC2 instances are launched in two different physical servers. Both instances have an Elastic Network Interface (ENI) card. The left-hand side EC2’s ENI has MAC/IP addresses cafe:0001:0001/10.10.1.11 and the right-hand side EC2’s ENI has MAC/IP addresses beef:0001:0001/10.10.1.22. Each physical server hosting EC2 instances has a Nitro Card for VPC [NC4VPC]. It is responsible for routing, data packets encapsulation/decapsulation, and Traffic limiting. In addition, Security Groups (SGs) are implemented in hardware on the  Nitro card for VPC. AWS Control-Plane relies on the Mapping Service system decoupled from the network devices. It means that switches are unaware of Overlay Networks having no state information related to VPC’s, Subnets, EC2 Instances, or any other Overlay Network components. From the Control-Plane perspective, physical network switches participate in the Underlay Network routing process by advertising the reachability information of physical hosts, Mapping Service, and so on. From the Data-Plane point of view, they forwards packet based on the outer IP header.

Mapping Register

Starting an EC2 instance triggers the Control-Plane process on a host. Figure 2-1 illustrates that Host-1 and Host-2 store information of their local EC2 instances into the Mapping cache. Then they register these instances into Mapping Service. You can consider the registration process as a routing update. We need to inform the Mapping Service about the EC2 instance’s a) MAC/IP addresses bind to ENI, b) Virtual Network Identifier (=VPC), c) the physical host IP, d) and the encapsulation mode (VPC tunnel header). If you are familiar with Locator/Id Separation Protocol LISP, you may notice that its Control-Plane process follows the same principles. The main difference is that switches in LISP-enabled networks have state information related to virtual/bare-metal servers running in a virtual network. 

Figure 2-1: VPC Control-Plane Operation: Mapping Register.

Continue reading

The Network Times 2021-09-29 10:11:00

Book Table of Contents

AWS Networking – Part VIII: AWS Network ACL (NACL)

In this section, I am going to introduce the default Network ACL for subnets in VPC NVKT-VPC-01.

Figure 1-28 shows the complete structure of our VPC NVKT-VPC-01. We have a Public subnet 10.10.0.0/24 in AZ eu-west-2c a Private subnet 10.10.1.0/24 in AZ eu-west-2a. Both subnets are protected by the default VPC’s NACL named NWKT-NACL. NACL allows all traffic to and from the subnet by default.

Figure 1-37: Complete VPC Stack.

AWS Networking – Part VII: Create Subnet and RT Using AWS CloudFormation

In this post, we create a Subnet with the set of properties and attach it to VPC. We also specify a Route Table, which we associate with the Subnet using association.

 In our YAML template (figure 1-34), we have four AWS resources (logical name within parenthesis):

    1) AWS::EC2::VPC (NwktVPC)

    2) AWS::EC2::Subnet (NwktSubnet)

    3) AWS::EC2::RouteTable (NwktPUB2RouteTable)

    4) AWS::EC2::SubnetRouteTableAssociation(NwktRouteTableAssociation)

We are using a Ref function for defining the dependencies between AWS resources when the actual AWS resource Identifier is unknown. For example, the Ref function in AWS::EC2::Subnet resource [2] refers to the resource AWS::EC2::VPC’s logical name NwktVPC (A). We have to use an intrinsic function because we don’t know which  VPC Identifier AWS generates to VPC. After creating the subnet, we specify the subnet-specific Route Table [3]. First, we need to bind it to VPC using the Ref function value NwktVPC (B). Next, we “glue” the Route Table to Subnet using RouteTableAssociation, where we use two Ref functions. The first one refers to Route Table (C), and the second to Subnet (D).

Figure 1-34: Subnet Route Table.

AWS Networking – Part VI: Subnet to Route Table Association

At this phase, we have attached subnets to their respective Availability Zones. Next, we will create subnet-specific route tables for both Public and Private subnets.

Figure 1-25: VPC Subnets: Select VPC.

AWS Networking – Part V: Create Subnet Using AWS Console

When we have created a new VPC, we can start adding subnets to it. We are going to create two subnets. Subnet 10.10.0.0/24 is a Public Subnet in Availability Zone eu-west2c, where we later add an Internet GW. Subnet 10.10.0.0/24 is a Private Subnet in Availability Zone eu-west2a that will use a NAT GW for uni-directional Internet access.

Figure 1-18: VPC Route Table: Routes.

AWS Networking – Part IV: Create VPC Using the AWZ CloudFormation

The focus of this section is to show how we can create a VPC using AWS CloudFormation service. Figure 1-12 shows our example AWS CloudFormation Templates. Its first section, AWSTemplateFormatVersion, specifies the template language format. At the time of writing, 2010-09-09 is the latest and only valid version. We can use the Description section to describe our template. Note that it must follow the AWSTemplateFormatVersion Section. AWSTemplateFormation-Version and Description are optional sections. The Resourcessection specifies the actual AWS resources and their properties. Each AWS resource is identified with a logical name. I have given the logical name NwktVPC for our example VPC. We can use resource-specific logical names for defining dependencies between resources. For example, when adding the AWS::EC2::Subnet resource to our template, we assign the VpcId value by calling it from the AWS::EC2::VPC resource using !REF intrinsinc function. I will explain the process in the Subnet section later. The resources and their properties are defined under logical names. The Resources section is the only required section in AWS CloudFormation-Template. AWS CloudFormation Templates are identified by using Stack Names in AWS Cloud Formation. Our example Stack Name is MyNetworkStack.

Figure 1-12: AWS CloudFormation: VPC.

AWS Networking – Part III: VPC Verification Using AWS CLI

VPC Verification Using AWS CLI

We can verify our VPC configuration by using AWS CLI. Example 1-1 shows the output for command aws ec2 describe-vpc in JSON format. This command lists all our VPC resources with their properties. The first one is the newest VPC NVKT-VPC-01, and the second one is the default VPC which I have named DFLT-VPC. The first VPC gets ordinal zero [0], and the second VPC gets number one [1]. Note that ordinal numbers are not shown in the output. VPC properties describe the VPC-specific CIDR Block, DHCP Options, VPC Identifier, Owner Id, CIDR Block Association, and Tags.

AWS Networking – Part II: Create VPC – AWS Console

The first thing to do when we create a VPC is to log in to the AWS console. Then we select the AWS Region where we want to launch our VPC. We are going to use VPC Region Europe (London) eu-west-2. As the last step, we give the name to VPC and associate a CIDR block 10.10.0.0/16 to it.

Figure 1-3: Virtual Private Cloud (VPC) – Example VPC.

AWS Networking – Part I: Virtual Private Cloud (VPC) Introduction

AWS Virtual Private Cloud (VPC) is a virtual network for Amazon Elastic Cloud Compute instances (EC2) within AWS Region. AWS Regions, in turn, belongs to the global AWS Cloud environment. Each AWS Region consists of three or more physical data centers, Availability Zones (AZ). At the time of writing, Seoul and Tokyo have four, and Northern Virginia has six AZs. All other regions have three AZs. VPC spans over regional AZs but not between AWS Regions. In other words, VPCs are region-specific virtual networks. 

A VPC has to have a CIDR (Classless Interdomain Routing) IP block attached to it. The VPC CIDR defines the IP range that we can use when creating subnets to VPC. CIDR range is VPC specific and can overlap with other VPC’s CIRD range. If there should be VPC-to-VPC inter-connection, VPC CIDR IP ranges have to be unique per VPC. 

We can allocate subnets for EC2 instances from the VPC’s CIDR range. Subnets are AZ-specific, and they can’t be span from one AZ to another. Subnets are classified either as Public Subnets or Private Subnets. Public Subnet has a route to Internet GW (Internet Gateway) in its Routing Table (RT). EC2 instances launched in a Public Subnet have to have a public IPv4 address in order to have an Internet connection. Note that IPv6 addresses are always assigned from the public address space. EC2 launched in a Private Subnet doesn’t need a public IPv4 address, they can have an Internet connection through the NAT GW. To allow Internet connection to EC2 instances in Private Subnet, we need to add a route to NAT GW into the Private Subnet Routing Table. We can allow a stateful egress-only Internet connection for EC2 instances with IPv6 addresses in Private Subnet by using Egress-Only Internet GW. This way EC2 instance has an Internet connection but hosts on the internet can’t initiate a connection to EC2. IP connectivity between EC2 instances within VPC is established between private IP address even if one of the EC2s is attached to Public Subnet and has a Public IP address. VPC has a main Routing Table that is used with subnets which we don’t define subnet-specific RT.

Each VPC also has a default Network Access Control List (NACL). The default NACL is bind to all subnets in VPC by default. NACL is stateless by nature, traffic to and from the subnet has to be allowed in both inbound and outbound directions. The default NACL allows all ingress/egress traffic.

Figure 1-1 illustrates our example VPC and its relationship to AWS Availability Zones, AWS Regions, and AWS Account. When we create VPC, we first have to log on to our AWS account. Next, we select an AWS Region, in our case Europe (London) eu-west-2. Then we choose Availability Zones for subnets. In our case, network 10.10.0.0/24 is a Public Subnet in the AZ eu-west-2c, and network 10.10.1.0/24 is a Private Subnet in the AZ eu-west-2a. As the last step, we create subnet-specific Routing Tables where we can later add subnet-specific routes.

Figure 1-1: Virtual Private Cloud (VPC) Basic Building Blocks.

LISP – OMP – BGP EVPN Interoperability – Part VIII: LISP, OMP, and BGP EVPN Comparison

IP reachability

Every Overlay Network solution requires IP reachability between edge devices via Underlay Network. This section explains the basic routing solution in Underlay Network from Campus Fabric, SD-WAN, and Datacenter Fabric perspectives. Figure 7-1 illustrates the IP reachability requirements for Campus Fabric, SD-WAN, and Datacenter Fabric.

Figure 7-1: IP Reachability Requirements.