Introducing the p0f BPF compiler

Two years ago we blogged about our love of BPF (BSD packet filter) bytecode.

CC BY 2.0 image by jim simonson

Then we published a set of utilities we are using to generate the BPF rules for our production iptables: the bpftools.

Today we are very happy to open source another component of the bpftools: our p0f BPF compiler!

Meet the p0f

p0f is a tool written by superhuman Michal Zalewski. The main purpose of p0f is to passively analyze and categorize arbitrary network traffic. You can feed p0f any packet and in return it will derive knowledge about the operating system that sent the packet.

One of the features that caught our attention was the concise yet explanatory signature format used to describe TCP SYN packets.

The p0f SYN signature is a simple string consisting of colon separated values. This string cleanly describes a SYN packet in a human-readable way. The format is pretty smart, skipping the varying TCP fields and keeping focus only on the essence of the SYN packet, extracting the interesting bits from it.

We are using this on daily basis to categorize the packets that we, at CloudFlare, see when we are a target Continue reading

Partial kernel bypass merged into netmap master

In a previous post we described our work on a new netmap mode called single-rx-queue.

After submitting the pull request, the netmap maintainers told us that the patch was interesting, but they would prefer something more configurable instead of a tailored custom mode.

After an exchange of ideas and some more work, our patch just got merged to mainline netmap.

Meet the new netmap

Before our patch netmap used to be an all-or-nothing deal. That is: there was no way to put a network adapter partially in netmap mode. All of the queues would have to be detached from the host network stack. Even a netmap mode called “single ring pair” didn't help.

Our final patch is extended and more generic, while still supporting the simple functionality of our original single-rx-queue mode.

First we modified netmap to leave queues that are not explicitly requested to be in netmap mode attached to the host stack. In this way, if a user requests a pair of rings (for example using nm_open(“netmap:eth0-4”)) it will actually get a reference to both the number 4 RX and TX rings, while keeping the other rings attached to the kernel stack.

But since the NIC is Continue reading

Single RX queue kernel bypass in Netmap for high packet rate networking

In a previous post we discussed the performance limitations of the Linux kernel network stack. We detailed the available kernel bypass techniques allowing user space programs to receive packets with high throughput. Unfortunately, none of the discussed open source solutions supported our needs. To improve the situation we decided to contribute to the Netmap project. In this blog post we'll describe our proposed changes.

CC BY-SA 2.0 image by Binary Koala

Our needs

At CloudFlare we are constantly dealing with large packet floods. Our network constantly receives a large volume of packets, often coming from many, simultaneous attacks. In fact, it is entirely possible that the server which just served you this blog post is dealing with a many-million packets per second flood right now.

Since the Linux Kernel can't really handle a large volume of packets, we need to work around it. During packet floods we offload selected network flows (belonging to a flood) to a user space application. This application filters the packets at very high speed. Most of the packets are dropped, as they belong to a flood. The small number of "valid" packets are injected back to the kernel and handled in the same way Continue reading