Archive

Category Archives for "Networking"

HTTP Analytics for 6M requests per second using ClickHouse

HTTP Analytics for 6M requests per second using ClickHouse

One of our large scale data infrastructure challenges here at Cloudflare is around providing HTTP traffic analytics to our customers. HTTP Analytics is available to all our customers via two options:

In this blog post I'm going to talk about the exciting evolution of the Cloudflare analytics pipeline over the last year. I'll start with a description of the old pipeline and the challenges that we experienced with it. Then, I'll describe how we leveraged ClickHouse to form the basis of a new and improved pipeline. In the process, I'll share details about how we went about schema design and performance tuning for ClickHouse. Finally, I'll look forward to what the Data team is thinking of providing in the future.

Let's start with the old data pipeline.

Old data pipeline

The previous pipeline was built in 2014. It has been mentioned previously in Scaling out PostgreSQL for CloudFlare Analytics using CitusDB and More data, more data blog posts from the Data team.
HTTP Analytics for 6M requests per second using ClickHouse
It had following components:

  • Log forwarder - collected Cap'n Proto formatted logs from the edge, notably DNS and Nginx logs, Continue reading

BrandPost: Your Roadblock to Becoming Digital: A 1990s Server Room

Digital transformation is fundamentally changing businesses of every size. Larger enterprises no longer have a unique advantage based purely on their scale. Small and midsize businesses (SMBs) can utilize new applications and services that allow them to compete toe to toe against a business or organization of any size. Not only do your customers want the latest and greatest, but employees demand it as well. If your employees can’t get the digital tools they need to be successful, they’ll go where they’re available.The focus of management and employees today is on exciting new apps and services. However, without the appropriate IT infrastructure to support them, an organization will not be able to move forward on the digital transformation journey. New apps and new devices are dependent on the servers, storage, and network gear that provide the data and back-end services.  Unfortunately for many SMBs, IT equipment is stored in a 1990s server room that may be a repurposed conference room, cubicle, or spare office that cannot provide the physical environment the IT hardware needs to support a 21st century organization.To read this article in full, please click here

Startup ZincFive makes old battery technology new again

A startup called ZincFive is set to launch a modular uninterruptible power supply (UPS) for data center computers using the venerable nickel-zinc technology, which it claims is more efficient than lithium-ion.Nickel-zinc batteries were invented by Thomas Edison in 1901 but fell out of favor to newer designs due to their limitations, such as a low number of charge cycles and their inability to hold a charge for long.On the plus side, the batteries could hold a stronger charge and didn’t use toxic metals like other batteries that make them difficult to recycle. And they are not flammable, something lithium-ion batteries certainly can’t claim.To read this article in full, please click here

Startup ZincFive makes old battery technology new again

A startup called ZincFive is set to launch a modular uninterruptible power supply (UPS) for data center computers using the venerable nickel-zinc technology, which it claims is more efficient than lithium-ion.Nickel-zinc batteries were invented by Thomas Edison in 1901 but fell out of favor to newer designs due to their limitations, such as a low number of charge cycles and their inability to hold a charge for long.On the plus side, the batteries could hold a stronger charge and didn’t use toxic metals like other batteries that make them difficult to recycle. And they are not flammable, something lithium-ion batteries certainly can’t claim.To read this article in full, please click here

Don’t forget to tune into our CCNA Kickoff Session tomorrow!

Watch our March 2018 CCNA Kickoff Session with Keith Bogart TOMORROW at 1:30 PM EST.

 

This kickoff session for those who are interested in, or have started to study for the CCNA certification. In this free session, we will cover common trouble areas that most people experience when getting started with their certification. Topics include: how to approach making a study schedule, strategies for not becoming overwhelmed during the study process, deciding whether to take one test or two to get your CCNA, what to expect when you walk into the testing center, which topics to study and how in depth, and what study tools can be useful. Keith will also discuss the testing experience and the CCNA Certification test format.

When: March 7th at 10:30 am PST/ 1:30 pm EST

Estimated Length: 3 hours

Instructor: Keith Bogart CCIE #4923

Cost: FREE

NAT To Be Missed At SXSW

NAT To Be Missed At SXSW

We’re at the EDGE of our seats, about to LANd in Austin, Texas in route for SXSW. (TKIP, hip, hooray!)

ARP you going to be there? We R going to have three epoch sessions by Cloudflare speakers. Ifdown, seems apt you could SELECT to JOIN. Cat make it? Not a bg deal, wget it (though it mega hertz we won’t C you). All the audio from the three sessions will be recorded, you can listen to the cd.

WPS! I almost forgot to tel(net) you whois going to be there, and WAN and where to go.

On Friday, March 9, I’m moderating a panel with Emily Schechter from Google, Aaron DeVera from Deloitte and Gabe Kassel from eero about how Wi-Fi networks work and WEP happens when attackers coax people into joining insecure networks. It’s at Salon K in the Hilton at 3:30PM.

On Sunday the 11th, Nitin Rao is on a panel with Heather West from Mozilla, Stefan Lederer from Bitmovin and Fred Benenson from Unlimited Liability Corporation LLC about the impact of the recent revocation of Net Neutrality rules on online video streaming. It’s at 11AM at Salon J in the Continue reading

DFS and Low Points

On a recent history of networking episode, Alia talked a little about Maximally Redundant Trees (MRTs), and the concept of Depth First Search (DFS) numbering, along with the idea of a low point. While low points are quickly explained in my new book in the context of MRTs, I thought it worthwhile to revisit the concept in a blog post. Take a look at the following network:

On the left side is a small network with the nodes (think of these as routers) being labeled from A through G. On the right side is the same network, only each node has been numbered by traversing the graph, starting at A. This process, in a network, would either require some device which knows about every node and edge (link) in the network, or it would require a distributed algorithm that “walks” the network from one node to another, numbering each node as it is touched, and skipping any node that has already been visited (again, for more details on this, please see the book).

Once this numbering has been done, the numbers now produce this interesting property: if you remove the parent of any node, and the node can still reach Continue reading

IDG Contributor Network: What to understand about health care IoT and its security

As we have seen, the Internet of Things will disrupt and change every industry and how actors within it do business. Along with new paradigms in services and products that one can offer due to the proliferation of IoT, come business risks as well as heightened security concerns – both physical and cyber. In our prior column, we spoke about this topic in the context of the Smart Electric Grid. Today we’re taking a look at how IoT is disrupting the health care market and how we can take steps to secure it.To read this article in full, please click here

IDG Contributor Network: What to understand about health care IoT and its security

As we have seen, the Internet of Things will disrupt and change every industry and how actors within it do business. Along with new paradigms in services and products that one can offer due to the proliferation of IoT, come business risks as well as heightened security concerns – both physical and cyber. In our prior column, we spoke about this topic in the context of the Smart Electric Grid. Today we’re taking a look at how IoT is disrupting the health care market and how we can take steps to secure it.To read this article in full, please click here

BrandPost: What Is Intelligent Automation?

The operations environment for service providers is growing increasingly complex. While advancements like Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) enable on-demand services, among many other benefits, existing network management and Operational Support Systems (OSSs) lack the scale and flexibility to meet the requirements of these more dynamic network technologies.Simple automation techniques like custom scripting are often used to reduce repetitive manual tasks, but providers are looking for more robust automation capabilities to reduce operational complexity and improve efficiency, at scale. By leveraging software purpose-built for dynamic SDN/NFV environments, and incorporating recent advancements in big data analytics and machine learning, network operators can make true intelligent automation possible from end-to-end and, as a result, run their businesses smarter and much more efficiently.  To read this article in full, please click here

BrandPost: What is Intelligent Automation?

The operations environment for service providers is growing increasingly complex. While advancements like Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) enable on-demand services, among many other benefits, existing network management and Operational Support Systems (OSSs) lack the scale and flexibility to meet the requirements of these more dynamic network technologies.Simple automation techniques like custom scripting are often used to reduce repetitive manual tasks, but providers are looking for more robust automation capabilities to reduce operational complexity and improve efficiency, at scale. By leveraging software purpose-built for dynamic SDN/NFV environments, and incorporating recent advancements in big data analytics and machine learning, network operators can make true intelligent automation possible from end-to-end and, as a result, run their businesses smarter and much more efficiently.  To read this article in full, please click here

1 … 1,444 1,445 1,446 1,447 1,448 … 3,443