Some notes on fast grep

This thread on the FreeBSD mailing discusses why GNU grep (that you get on Linux) is faster than the grep on FreeBSD. I thought I'd write up some notes on this.

I come from the world of "network intrusion detection", where we search network traffic for patterns indicating hacker activity. In many cases, this means solving the same problem of grep with complex regexes, but doing so very fast, at 10gbps on desktop-class hardware (quad-core Core i7). We in the intrusion-detection world have seen every possible variation of the problem. Concepts like "Boyer-Moore" and "Aho-Corasick" may seem new to you, but they are old-hat to us.

Zero-copy

Your first problem is getting the raw data from the filesystem into memory. As the thread suggests, one way of doing this is "memory-mapping" the file. Another option would be "asynchronous I/O". When done right, either solution gets you "zero-copy" performance. On modern Intel CPUs, the disk controller will DMA the block directly into the CPU's L3 cache. Network cards work the same way, which is why getting 10-gbps from the network card is trivial, even on slow desktop systems.

Double-parsing

Your next problem is stop with the line parsing, idiots. All these Continue reading

U.S. Marshals issue telephone scam warning

The U.S. Marshals Service today warned of a telephone scam that has some scamster calling random victims and alleging they or their family members have an active federal arrest warrant and demanding payment of fines.From the US Marshals office: “On December 7, 2015, the fraudster identified himself as a Deputy United States Marshal and informed the potential victims they or their family member had active federal warrants for their arrest. The caller then gave the potential victims a contact number and information to pay the fine. The phony law enforcement officer threatened the potential victims with arrest if the fine was not paid. The fraudster then tells the victim to buy a prepaid money card from a local grocery store in the Cincinnati area. The victim is then instructed to give the access account code for the prepaid money card to the phony law enforcement officer. “To read this article in full or to leave a comment, please click here

U.S. Marshals issue telephone scam warning

The U.S. Marshals Service today warned of a telephone scam that has some scamster calling random victims and alleging they or their family members have an active federal arrest warrant and demanding payment of fines.From the US Marshals office: “On December 7, 2015, the fraudster identified himself as a Deputy United States Marshal and informed the potential victims they or their family member had active federal warrants for their arrest. The caller then gave the potential victims a contact number and information to pay the fine. The phony law enforcement officer threatened the potential victims with arrest if the fine was not paid. The fraudster then tells the victim to buy a prepaid money card from a local grocery store in the Cincinnati area. The victim is then instructed to give the access account code for the prepaid money card to the phony law enforcement officer. “To read this article in full or to leave a comment, please click here

Homeland Security’s role in cybersecurity

CSO Contributing Writer Ira Winkler (The Irari Report) recently sat down for an interview with Alejandro N. Mayorkas, the deputy secretary of Homeland Security.We’ve separated the interview into three video segments, covering a variety of security-related topics.In the first video (above), Mayorkas describes the role of Homeland Security when it comes to cybersecurity, and how government agencies are working together to improve the overall cybersecurity of critical systems and infrastructure.In part 2, Winkler and Mayorkas discuss whether the power grid is vulnerable to cyberattack, and where opportunities exist for improving our defenses.To read this article in full or to leave a comment, please click here

Google continues enterprise push with Data Loss Prevention for Gmail

Google on Wednesday released a new tool for companies that want to make sure their sensitive information isn't shared via email.Gmail for Work now has Data Loss Prevention (DLP) capabilities, which allow administrators to set policies about what information users can send through Gmail. The goal is to protect confidential records and make sure users don't accidentally leak key data. For example, a policy could prohibit members of the accounting team from sending any emails with a spreadsheet attached. Policies could also be used to quarantine messages until an administrator can review them, or modify them to remind users not to share confidential information outside of the company. Google has tried to make crafting those policies easier with a library of predefined content detectors that help administrators build intelligent policies. For situations that aren't covered by the pre-built detectors, administrators can create their own. To read this article in full or to leave a comment, please click here

Free Red Book: Readings in Database Systems, 5th Edition

For the first time in ten years there has been an update to the classic Red Book, Readings in Database Systems, which offers "readers an opinionated take on both classic and cutting-edge research in the field of data management."

Editors Peter Bailis, Joseph M. Hellerstein, and Michael Stonebraker curated the papers and wrote pithy introductions. Unfortunately, links to the papers are not included, but a kindly wizard, Nindalf, gathered all the referenced papers together and put them in one place.

What's in it?

  • Preface 
  • Background introduced by Michael Stonebraker 
  • Traditional RDBMS Systems introduced by Michael Stonebraker 
  • Techniques Everyone Should Know introduced by Peter Bailis 
  • New DBMS Architectures introduced by Michael Stonebraker
  • Large-Scale Dataflow Engines introduced by Peter Bailis 
  • Weak Isolation and Distribution introduced by Peter Bailis 
  • Query Optimization introduced by Joe Hellerstein 
  • Interactive Analytics introduced by Joe Hellerstein 
  • Languages introduced by Joe Hellerstein 
  • Web Data introduced by Peter Bailis 
  • A Biased Take on a Moving Target: Complex Analytics by Michael Stonebraker 
  • A Biased Take on a Moving Target: Data Integration by Michael Stonebraker

Related Articles

 

IBM lets customers, partners write apps for QRadar threat intelligence platform

IBM is launching a program where customers can share apps they write to augment IBM’s QRadar platform that analyzes security data, detects behavior anomalies and sorts out high-priority risks from the mass of incidents it examines.To accomplish this, the company is opening APIs into QRadar, issuing software developer kits and creating a Security App Exchange where these custom apps can be distributed.The exchange has already been seeded with 14 apps written by IBM itself and some of its partners including Bit9 + Carbon Black, BrightPoint Security, Exabeam and Resilient Systems.Four of these apps are: User Behavior Analytics – Integrates Exabeam’s analysis of user behaviors and risk profiling into QRadar’s dashboard. Threat Intelligence – Pulls data from threat feeds and create rules about how to handle the data, such as raising the threat score for incidents involving IP addresses from a particular watch list. Carbon Black App for QRadar – Analyzes data from Carbon Black’s endpoint sensors within the QRadar interface, enabling faster responses to endpoint attacks. Incident Overview – A visualization app that uses bubbles, colors and correlation lines to help analysts quickly identify links among incidents. IBM says it will vet applications before they are made Continue reading

Reviving n3topedia

Well…

After a pretty long time no write, the big day came, when I decided to revive a project most dear to me.

For those of you who remember n3topedia, and for those of you who’ve never heard of it, a purpose statement may be worthy at this point. From a strictly educational blog, n3topedia will be transformed in a tech blog.

I am pretty certain that networking posts will be an important part of this, but my focus will also be on letting you know whatever feels interesting and useful. Both the format and the approach will be slightly different, more lively and interactive.

 

I am hoping you will all enjoy reading it as much as I enjoy writing it.

 

Cheers

Rating

Cyberspy group targets South American political figures, journalists

Since 2008, a group of attackers has used off-the-shelf remote access Trojans (RATs) to target political figures, journalists and public figures in several South American countries. The group, whose attack campaigns have been investigated by researchers working with Citizen Lab at the University of Toronto's Munk School of Global Affairs, has been dubbed Packrat. It appears mainly interested in political opposition groups and influential people from countries like Argentina, Ecuador and Venezuela.While there is insufficient evidence to link the group to a particular government or intelligence agency, the researchers believe "that the ultimate recipient of the information collected by Packrat is likely one or more governments in the region."To read this article in full or to leave a comment, please click here

Security and privacy checklist for smart devices: 50 million to be sold over holidays

When shopping for a smart device, are you most influenced by the device’s capabilities, by its coolness factor, or by holiday sales that dropped the price? Do you first review the company’s policies, terms and conditions, the potentially excessive permissions a mobile app will require to control the connected device, or with whom the manufacturer will share or sell your collected data? If you receive a smart gadget as a gift, do you think the giver was wise enough to consider the small print before purchasing, to think of security and privacy before buying the smart device?To read this article in full or to leave a comment, please click here

Notable deaths of 2015 from worlds of technology, science & inventions

RememberingThe networking and computing world, as well as the worlds of science and inventions, lost well-known pioneers as well as younger movers and shakers during 2015. Here’s a brief look back at these people and their contributions.(IDG News Service contributed to this report.)LOOK BACK: 2014’s notable deathsTo read this article in full or to leave a comment, please click here

Ansible book – Technical review

I recently did a technical review for “Mastering Ansible” book by Jesse Keating. This book covers usage of Ansible for automation with practical examples. If anyone is interested, please look at the book. Pre-requisite is to have basic Ansible knowledge. Ansible is similar to configuration management tools like Chef, Puppet. Agent-less architecture and short learning … Continue reading Ansible book – Technical review →
1 … 3,142 3,143 3,144 3,145 3,146 … 3,763