The problem with thread^W event loops

The problem with thread^W event loops

Back when Cloudflare was created, over 10 years ago now, the dominant HTTP server used to power websites was Apache httpd. However, we decided to build our infrastructure using the then relatively new NGINX server.

There are many differences between the two, but crucially for us, the event loop architecture of NGINX was the key differentiator. In a nutshell, event loops work around the need to have one thread or process per connection by coalescing many of them in a single process, this reduces the need for expensive context switching from the operating system and also keeps the memory usage predictable. This is done by processing each connection until it wants to do some I/O, at that point, the said connection is queued until the I/O task is complete. During that time the event loop is available to process other in-flight connections, accept new clients, and the like. The loop uses a multiplexing system call like epoll (or kqueue) to be notified whenever an I/O task is complete among all the running connections.

In this article we will see that despite its advantages, event loop models also have their limits and falling back to good old threaded architecture is sometimes Continue reading

An empirical guide to the behavior and use of scalable persistent memory

An empirical guide to the behavior and use of scalable persistent memory, Yang et al., FAST’20

We’ve looked at multiple papers exploring non-volatile main memory and its implications (e.g. most recently ‘Efficient lock-free durable sets‘). One thing they all had in common is an evaluation using some kind of simulation of the expected behaviour of NVDIMMs, because the real thing wasn’t yet available. But now it is! This paper examines the real-world behaviour of Intel’s Optane DIMM, and finds that not all of the assumptions baked into prior works hold. Based on these findings, the authors present four guidelines to get the best performance out of this memory today. Absolutely fascinating if you like this kind of thing!

The data we have collected demonstrate that many of the assumptions that researchers have made about how NVDIMMs would behave and perform are incorrect. The widely expressed expectation was that NVDIMMs would have behavior that was broadly similiar to DRAM-based DIMMs but with lower performance (i.e., higher latency and lower bandwidth)… We have found the actual behavior of Optane DIMMs to be more complicated and nuanced than the "slower, persistent DRAM" label would suggest.

Optane Continue reading

Solving “NAME is not exported by MODULE” When Using Local NPM Dependencies

This blog post will focus on a topic I don’t usually dive into (Javascript and related tooling), but I felt like others might benefit from the solution to a problem I encountered while doing local development for antidote-web, the web front-end that powers NRE Labs. A quick aside on the architecture for the front-end code for the Antidote platform - the antidote-web project is the lynchpin for everything. It’s where the general structure of the front-end app is managed.

Kustomize Transformer Configurations for Cluster API v1alpha3

A few days ago I wrote an article on configuring kustomize transformers for use with Cluster API (CAPI), in which I explored how users could configure the kustomize transformers—the parts of kustomize that actually modify objects—to be a bit more CAPI-aware. By doing so, using kustomize with CAPI manifests becomes much easier. Since that post, the CAPI team released v1alpha3. In working with v1alpha3, I realized my kustomize transformer configurations were incorrect. In this post, I will share CAPI v1alpha3 configurations for kustomize transformers.

In the previous post, I referenced changes to both namereference.yaml (to configure the nameReference transformer) and commonlabels.yaml (to configure the commonLabels transformer). CAPI v1alpha3 has changed the default way labels are used with MachineDeployments, so for v1alpha3 you may be able to get away with only changes to namereference.yaml. (If you know you are going to want/need additional labels on your MachineDeployment, then plan on changes to commonlabels.yaml as well.)

Here are the CAPI v1alpha3 changes needed to namereference.yaml:

- kind: Cluster
  group: cluster.x-k8s.io
  version: v1alpha3
  fieldSpecs:
  - path: spec/clusterName
    kind: MachineDeployment
  - path: spec/template/spec/clusterName
    kind: MachineDeployment

- kind: AWSCluster
  group: infrastructure.cluster.x-k8s.io
   Continue reading

Google Cloud Next Now Postponed Due to COVID-19

The company's headquarters in Mountain View, California, are under a "shelter in place" ordinance...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

How to Efficiently Detect Domain Generation Algorithms (DGA) in Kubernetes with Calico Enterprise

Introduction

2020 is predicted to be an exciting year with more organizations adopting Kubernetes than ever before. As critical workloads with sensitive data migrate to the cloud, we can expect to encounter various Advanced Persistent Threats (APT) targeting that environment.

Domain Generation Algorithm (DGA) – What is It?

DGA is a technique that fuels malware attacks. DGA by itself can’t harm you. But it’s a proven technique that enables modern malware to evade security products and counter-measures. Attackers use DGA so they can quickly switch the command-and-control (also called C2 or C&C) servers that they’re using for malware attacks. Security software vendors act quickly to block and take down malicious domains hard-coded in malware. So, attackers used DGA specifically to counter these actions. Now DGA has become one of the top phone-home mechanisms for malware authors to reach C2 servers. This poses a significant threat to cloud security.

Mitre defines DGA as “The use of algorithms in malware to periodically generate a large number of domain names which function as rendezvous points for malware command and control servers”. Let’s examine this definition more closely. DGA at its core generates domains by concatenating pseudo-random strings and a TLD (e.g. .com, . Continue reading

Daily Roundup: Remote Workers Strain Services

Remote worker influx stifled services; Cisco, Hitachi sliced jobs; and Red Hat, Intel bridged the...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Using Ansible Automation Platform, GitLab CE and Webhooks to Deploy IIS Website

Inside Red Hat Ansible Automation Platform, the Ansible Tower REST API is the key mechanism that helps enable automation to be integrated into processes or tools that exist in an environment. With Ansible Tower 3.6 we have brought direct integration with webhooks from GitHub and GitLab, including the enterprise on-premises versions. This means that changes in source control can trigger automation to apply changes to infrastructure configuration, deploy new services, reconfigure existing applications, and more. In this blog, I’ll run through a simple scenario and apply the new integrated webhook feature.

 

Environment

My environment consists of Ansible Tower (one component of Red Hat Ansible Automation Platform), GitLab CE with a project already created, and a code server running an IDE with the same git repository cloned. A single inventory exists on Ansible Tower with just one host, an instance of Windows 2019 Server running on a certified cloud. For this example, I’m going to deploy IIS on top of this Windows server and make some modifications to the html file that I’d like to serve from this site. 

My playbook to deploy IIS is very simple:

 ---
- name: Configure IIS
  hosts: windows

  tasks:
  - name: Install  Continue reading

A new era for Cumulus in the Cloud

When we launched Cumulus in the Cloud (CitC) over two years ago, we saw it as a way for our customer base to test out Cumulus Linux in a safe sandboxed environment. Looking back, September 2017 feels like an eternity ago.

Since then, CitC has become a place where we’ve been able to roll out new functionality and solutions to customers and Cumulus-curious alike — and we’ve done some really interesting things (some of our favs include integrating it with an Openstack demo and Mesos demo). It’s pretty much become a Cumulus technology playground.

As our CitC offering has evolved, we’ve also taken stock of the requirements from our customers and realized the direction we want to take CitC. So where is it heading? We’re excited to share that with the launch of our production-ready automation solution last week, CitC will have a new user experience and user interface.

Out with the old:

In with the new:

This redesigned UI comes with some really great enhancements:

  • Customized external connectivity to oob-mgmt-server to run user customized applications

  • Default lifetime increased to 12 hours

  • NetQ native integration within the demo

Infinera Pushes 800G Further in Latest Trial

The trial marks the successful use of a transceiver that's headline data rate can be achieved over...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Cisco, Hitachi Vantara Slicing Hundreds of Silicon Valley Jobs

The job cuts include nearly 400 Cisco employees and 151 Hitachi Vantara employees.

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Liqid Composes Inspur OCP Racks, Western Digital Storage

The OCP-compliant Inspur system includes Nvidia GPUs and enables composability via PCIe...

Read More »

© SDxCentral, LLC. Use of this feed is limited to personal, non-commercial use and is governed by SDxCentral's Terms of Use (https://www.sdxcentral.com/legal/terms-of-service/). Publishing this feed for public or commercial use and/or misrepresentation by a third party is prohibited.

Post: Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring? 

  • Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

  • Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. For the first time ever, you can now sign up for a subscription to get unlimited access to every course on the platform at a discounted price through the holiday period only. You'll also get to lock in this price as long as you're a subsciber. 

  • Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try now!

  • PA File Sight - Actively protect servers from ransomware, audit file access to see who is deleting files, reading files or moving files, Continue reading

Beyond Kube-Proxy: Project Calico Harnesses eBPF for a Faster Data Plane

Thanks to the power of the newly-introduced Calico network management software with a new data plane mode, one that can speed pod-to-pod data communication and eliminate the dependency on Kubernetes’ kube-proxy for traffic management. Tigera had started releasing work with eBPF almost a year ago, but this is the first release of Calico that fully harnesses the power of the new Linux kernel technology, Tigera co-founder and chief technology officer, said. “We wanted to derive what we were doing from fundamentals, to be confident we were building the right thing for users,” said scale Continue reading

Telnyx api/sms saves my plants! Every single time.

When its a holiday, I do some house-hold farming, mostly into Hydroponics which is based on water and associated nutrients and does not require soil as a medium.

Let me show an example.

As you can see, Plant drinks up water, and the one which you are seeing is lemon plant from its seed stage, Monitoring water level is extremely important and for me there are many more in home so cant be keeping track of everything.

So sensor gets the data, Raspberry Pi talks to AWS IOT securely since Microcontrollers are still a pain to handle SSL and Rpi makes it easy and after processing MQTT message AWS IOT will process based on the Rules, in our case if the treshold is below 500 then plant is drying up.

Why Not Inbuilt SES/SMS with AWS ?

SES works great but I check my emails only once in two days and there is no way to alert based on the email, SMS I wanted to used multiple number pool (thinking to expand) and Cost with Telnyx is Really Amazing to do any Communication services

I have tried other vendors and results are not great either my carrier has some integration problems Continue reading

Working from Home: Keeping It Secure and Private

With the global efforts to curb the spread of COVID-19 coronavirus underway, we’re seeing an unprecedented disruption of day-to-day operations across virtually all work sectors.

Coronavirus coverage is dominating headlines as more country borders, schools, major events and public venues are shut down in hopes of flattening the infection curve. Likewise, companies around the world are making tough calls to prioritize employee safety with business continuity in mind.

But thanks to the Internet, many workplaces are rapidly adapting operations to do their part and accommodate for what may soon become a new normal: remote working.

There are no shortage of web-based applications and services that allow us to continue core functions like meetings and collaboration that we’d do in an office-based setting. And not just the proverbial meeting that could have been an email.

But how do we ensure confidentiality when working remotely? There are some simple ways businesses and employees can ensure they are protecting themselves and their workplaces from malicious actors.

The answer is simple: encryption.

Encryption is the process of scrambling or enciphering data so it can be read only by someone with the means to return it to its original state.

It allows all our online Continue reading

1 … 1,034 1,035 1,036 1,037 1,038 … 3,874