We use ClickHouse widely at Cloudflare. It helps us with our internal analytics workload, bot management, customer dashboards, and many other systems. For instance, before Bot Management can analyze and classify our traffic, we need to collect logs. The Firewall Analytics tool needs to store and query data somewhere too. The same goes for our new Cloudflare Radar project. We are using ClickHouse for this purpose. It is a big database that can store huge amounts of data and return it on demand. This is not the first time we have talked about ClickHouse, there is a dedicated blogpost on how we introduced ClickHouse for HTTP analytics.
Our biggest cluster has more than 100 nodes, another one about half that number. Besides that, we have over 20 clusters that have at least three nodes and the replication factor of three. Our current insertion rate is about 90M rows per second.
We use the standard approach in ClickHouse schema design. At the top level we have clusters, which hold shards, a group of nodes, and a node is a physical machine. You can find technical characteristics of the nodes here. Stored data is replicated between clusters. Different shards hold different parts Continue reading
Hello my friend,
as you know, Ansible is one of the leading tools for the automation of the IT and network infrastructure. We have written a lot about it earlier (e.g. CLI configs, OpenConfig with NETCONF, or VNF-M). Recently Red Hat announced the new version of Ansible (Ansible 2.10), which changes a lot the way we used to work with that.
1
2
3
4
5 No part of this blogpost could be reproduced, stored in a
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
We are here to help you. At our network automation training you learn all you need to know to be successful with such tasks in your profession:
Ansible 2.10 is more than a just another Ansible’s update. It is a new approach, paradigm shift, Continue reading
Got this question from one of the networking engineers “blessed” with rampant clueless-rush-to-the-cloud.
I plan to peer multiple VNet from different regions. The problem is that there is not any consistent deployment in regards to the private IP subnets used on each VNet to the point I found several of them using public IP blocks as private IP ranges. As far as I recall, in Azure we can’t re-ip the VNets as the resource will be deleted so I don’t see any other option than use NAT from offending VNet subnets to use my internal RFC1918 IPv4 range. Do you have a better idea?
The way I understand Azure, while you COULD have any address range configured as VNet CIDR block, you MUST have non-overlapping address ranges for VNet peering.
Ned Bellavance and Ethan Banks analyze the big announcements from two conferences the clouderati should care about: AnsibleFest and HashiConf Digital. Both of these were virtual events because there's still an pandemic on, folks. Speaking of which, how do Ned and Ethan feel about virtual events? Not great, really. Slidewhipping the attendees in multi-day webinars seems to be how vendors are running their virtual conferences, and it ain't workin'...
The post Day Two Cloud 073: AnsibleFest & HashiConf 2020 Announcements, Analysis & Awesomeness appeared first on Packet Pushers.
As Chair of the Internet Society Audit Committee, I wanted to share an update with you.
As you may know, the Audit Committee reviews the Conflict of Interest forms filed by members of the Board of Trustees and officers of the Internet Society to ensure that we are in compliance with our Conflict of Interest (“CoI”) policy.
The CoI policy states that members of the Board of Trustees cannot hold a position in the policy development process in another organization operating in the Internet Society’s areas of engagement, and we are evaluating a situation where this restriction may be relevant. One of our Trustees has been appointed as a non-voting member to the GSNO Council, the Generic Names Supporting Organization – a policy-development body that develops and recommends policies relating to generic top-level domains (gTLDs) to the ICANN Board.
The Internet Society has a long history of collaborating with our diverse community from around the world, and is committed to having vibrant and robust global engagement. We work across countries and cultures and seek diverse cross-organizational expertise. This makes us stronger—sound practices and clear policies are a critical part of that.
We recognize that the expertise of our trustees is Continue reading
One of my readers sent me a question along these lines:
Imagine you have a router with four equal-cost paths to prefix X, two toward upstream-1 and two toward upstream-2. Now let’s suppose that one of those links goes down and you want to have link protection. Do I really need Loop-Free Alternate (LFA) or MPLS Fast Reroute (FRR) to get fast (= immediate) failover or could I rely on multiple equal-cost paths to get the job done? I’m getting different answers from different vendors…
Please note that we’re talking about a very specific question of whether in scenarios with equal-cost layer-3 paths the hardware forwarding data structures get adjusted automatically on link failure (without CPU reprogramming them), and whether LFA needs to be configured to make the adjustment happen.
Recently I moved to a new house and as a lot of reconstruction was done to bring the house up to date. I took the opportunity to have something I’ve always wanted in my home: a server rack! In my previous lab set-ups they were either located in my employers lab location or placed in […]
The post Home Network 2020 first appeared on Rick Mur.If you’ve been looking at automation for your network, you should listen to this episode. This is one of those discussions that can help you understand whether you should roll your own artisanal automation, lovingly handcrafted with Python scripts flowing from your keyboard, or whether your organization would be better served with a commercially supported tool like Apstra. That’s the choice Beelastic needed to make, and we’re going to discuss why they went the Apstra direction. Apstra customer Beelastic is going to tell their tale of data center fabric, multiple data centers, automation, validation, multi-tenancy, EVPN, and more.
The post Heavy Networking 548: Automating Multi-DC EVPN With Apstra (Sponsored) appeared first on Packet Pushers.