We recently gave a presentation on Programming socket lookup with BPF at the Linux Plumbers Conference 2019 in Lisbon, Portugal. This blog post is a recap of the problem statement and proposed solution we presented.
Our edge servers are crowded. We run more than a dozen public facing services, leaving aside the all internal ones that do the work behind the scenes.
Quick Quiz #1: How many can you name? We blogged about them! Jump to answer.
These services are exposed on more than a million Anycast public IPv4 addresses partitioned into 100+ network prefixes.
To keep things uniform every Cloudflare edge server runs all services and responds to every Anycast address. This allows us to make efficient use of the hardware by load-balancing traffic between all machines. We have shared the details of Cloudflare edge architecture on the blog before.
Granted not all services work on all the addresses but rather on a subset of them, covering one or several network prefixes.
So how do you set up your network services to listen on hundreds of IP addresses without driving the network stack over the edge?
Cloudflare engineers have had to ask themselves this question Continue reading