Heavy Networking 644: Building And Running Temporary Event Networks
Today's Heavy Networking podcast explores what it's like to set up a temporary network to support thousands of users at a live event. We discuss planning and preparation, defining requirements, design options, performance and security issues, and what to keep in your bag for that last-minute emergency. Our guests are Jim Troutman, Jason Davis, and Alex Latzko.
m frequently checking if CNAMEs are properly configured. CNAMEs are Canonical NAMEs, kind of like nicknames, that indicate that one domain name is a nickname for another domain name.Certification Comfort Food
I’m a big fan of comfort food. Maybe more than I should be. The idea of something simple and tasty just hits the right spot a lot of time, especially when I’m stressed or don’t have time to do something more involved. I know I really need to be better about cooking but you can’t beat a quick meal that uses something simple and gets the job done, right?
Now, before you ask yourself what I’m on about this week, I want you to think about that analogy in terms of certifications and learning. When we’re starting out in the industry or we’re learning a new skill we have to pick up basic ideas. The more advanced or radical the technology the more we need the kinds of explanations that make the concepts simple to understand. We need the equivalent of learning comfort food. Simple, digestible, and easy to prepare.
Climbing the Ladder
As our skills improve we have the choice to continue on and develop our capabilities to greater depths. Perhaps we want to learn everything there is to know about BGP and policies. We could even parlay that networking knowledge into new adjacencies that build on our skill Continue reading
Performance isolation in a multi-tenant database environment
Operating at Cloudflare scale means that across the technology stack we spend a great deal of time handling different load conditions. In this blog post we talk about how we solved performance difficulties with our Postgres clusters. These clusters support a large number of tenants and highly variable load conditions leading to the need to isolate activity to prevent tenants taking too much time from others. Welcome to real-world, large database cluster management!
As an intern at Cloudflare I got to work on improving how our database clusters behave under load and open source the resulting code.
Cloudflare operates production Postgres clusters across multiple regions in data centers. Some of our earliest service offerings, such as our DNS Resolver, Firewall, and DDoS Protection, depend on our Postgres clusters' high availability for OLTP workloads. The high availability cluster manager, Stolon, is employed across all clusters to independently control and replicate data across Postgres instances and elect Postgres leaders and failover under high load scenarios.
PgBouncer and HAProxy act as the gateway layer in each cluster. Each tenant acquires client-side connections from PgBouncer instead of Postgres directly. PgBouncer holds a pool of maximum server-side connections to Postgres, allocating those across multiple Continue reading
LastPass breach – Deja Vu all over again
Yesterday I got this notification from my Password Manager, LastPass: The key phrase here is “unusual activity within portions of the LastPass development environment.” This looked too familiar. One of the most well documented attacks of this nature is the SolarWinds supply chain attack in 2020. Although it is too early to say (the news … Continue reading LastPass breach – Deja Vu all over againOpen sourcing our fork of PgBouncer
Cloudflare operates highly available Postgres production clusters across multiple data centers, supporting the transactional workloads of our core service offerings such as our DNS Resolver, Firewall, and DDoS Protection.
Multiple PgBouncer instances sit at the front of the gateway layer per each cluster, acting as a TCP proxy that provides Postgres connection pooling. PgBouncer’s pooling enables upstream applications to connect to Postgres, without having to constantly open and close connections (expensive) at the database level, while also reducing the number of Postgres connections used. Each tenant acquires client-side connections from PgBouncer instead of Postgres directly.
PgBouncer will hold a pool of maximum server-side connections to Postgres, allocating those across multiple tenants to prevent Postgres connection starvation. From here, PgBouncer will forward backend queries to HAProxy, which load balances across Postgres primary and read replicas.
As an intern at Cloudflare I got to work on improving how our database clusters behave under load and open source the resulting code.
We run our Postgres infrastructure in non-containerized, bare metal environments which consequently leads to multitenant resource contention between Postgres users. To enforce stricter tenant performance isolation at the database level (CPU time utilized, memory consumption, disk IO operations), we’d like to configure Continue reading
Is Your Business Prepared to Operate After a Ransomware Attack?
Three best practices to ensure business continuity and minimize data loss following a ransomware attack.Ethernet Loses Most of The Time
NMEA Networking is not Ethernet