StackStorm Architecture Part I – StackStorm Core Services

A while ago, I wrote about basic concepts in StackStorm. Since then I’ve been knee-deep in the code, fixing bugs and creating new features, and I’ve learned a lot about how StackStorm is put together.

In this series, I’d like to spend some time exploring the StackStorm architecture. What subcomponents make up StackStorm? How do they interact? How can we scale StackStorm? These are all questions that come up from time to time in the StackStorm community, and there are a lot of little details that I even forget from time-to-time. I’ll be doing this in a series of posts, so we can explore a particular topic in detail without getting overwhelmed.

Also, it’s worth noting that this isn’t intended to be an exhaustive reference for StackStorm’s architecture. The best place for that is still the StackStorm documentation. My goal in this series is merely to give a little bit of additional insight into StackStorm’s inner workings, and hopefully get those curiosity juices flowing. There will be some code references, some systems-level insight, probably both.

Also note that this is a living document. This is an open source project under active development, and while I will try to keep specific Continue reading

StackStorm Architecture Part I – StackStorm Core Services

A while ago, I wrote about basic concepts in StackStorm. Since then I’ve been knee-deep in the code, fixing bugs and creating new features, and I’ve learned a lot about how StackStorm is put together.

In this series, I’d like to spend some time exploring the StackStorm architecture. What subcomponents make up StackStorm? How do they interact? How can we scale StackStorm? These are all questions that come up from time to time in the StackStorm community, and there are a lot of little details that I even forget from time-to-time. I’ll be doing this in a series of posts, so we can explore a particular topic in detail without getting overwhelmed.

Also, it’s worth noting that this isn’t intended to be an exhaustive reference for StackStorm’s architecture. The best place for that is still the StackStorm documentation. My goal in this series is merely to give a little bit of additional insight into StackStorm’s inner workings, and hopefully get those curiosity juices flowing. There will be some code references, some systems-level insight, probably both.

Also note that this is a living document. This is an open source project under active development, and while I will try to keep specific Continue reading

Cumulus Linux 3.4 REST API

The latest Cumulus Linux 3.4 release include a REST API. This article will demonstrate how the REST API can be used to automatically deploy traffic controls based on real-time sFlow telemetry. DDoS mitigation with Cumulus Linux describes how sFlow-RT can detect Distributed Denial of Service (DDoS) attacks in real-time and deploy automated controls.

The following ddos.js script is modified to use the REST API to send Network Command Line Utility - NCLU commands to add and remove ACLs, see Installing and Managing ACL Rules with NCLU:
var user = "cumulus";
var password = "CumulusLinux!";
var thresh = 10000;
var block_minutes = 1;

setFlow('udp_target',{keys:'ipdestination,udpsourceport',value:'frames'});

setThreshold('attack',{metric:'udp_target', value:thresh, byFlow:true, timeout:10});

function restCmds(agent,cmds) {
  for(var i = 0; i < cmds.length; i++) {
    let msg = {cmd:cmds[i]};
    http("https://"+agent+":8080/nclu/v1/rpc",
         "post","application/json",JSON.stringify(msg),user,password);
  }
}

var controls = {};
var id = 0;
setEventHandler(function(evt) {
  var key = evt.agent + ',' + evt.flowKey;
  if(controls[key]) return;

  var ifname = metric(evt.agent,evt.dataSource+".ifname")[0].metricValue;
  if(!ifname) return;

  var now = (new Date()).getTime();
  var name = 'ddos'+id++;
  var [ip,port] = evt.flowKey.split(',');
  var cmds = [
    'add acl ipv4 '+name+' drop udp source-ip any source-port '+port+' dest-ip '+ip+' dest-port any',
     Continue reading

BGP leak causing Internet outages in Japan and beyond.

Yesterday some Internet users would have seen issues with their Internet connectivity, experiencing slowness or parts of the Internet as unreachable. This incident hit users in Japan particularly hard and it caused the Internal Affairs and Communications Ministry of Japan to start an investigation into what caused the large-scale internet disruption that slowed or blocked access to websites and online services for dozens of Japanese companies.

In this blog post we will take a look at the root cause of these outages, who was affected and what networks were involved.

Starting at 03:22 UTC yesterday (aug 25) followers of @BGPstream would have seen an increase in alerts involving Google. The BGPstream alerts were informing us that Google was announcing the peering lan prefixes of a few well known Internet exchanges. This in itself is actually a fairly common type of incident and typically indicates something isn’t quite right within the networks hijacking those prefixes and so these alerts were the first clues that something wasn’t quite right with Google’s BGP advertisements.

  A closer look at our data shows not only BGP hijack incidents but also a high number of BGP leak events. A random example is this one: 171. Continue reading

NSX Going Wild at This Year’s VMworld

Get ready! NSX is hosting a major swag giveaway at VMworld as part of a celebration for everything our customers have accomplished in 2017! At various times throughout the conference, we’ll be on the prowl, looking for folks sporting NSX gear. If you’re spotted “in the wild” adorned with anything “NSX”, you could win some awesome swag and prizes. 

Join the hunt: show off your NSX pride (and your photography skills), and post photos of anything #NSX with the hashtag #NSXintheWild.  Winners will be chosen at random on the VMworld floor and online, so you never know when we might have you in our sights. But make no mistake – if you’re representing NSX in the wild, you’ll be a prime target for swag.

 

Pay it forward: If you happen to spot some cool NSX gear in the wild, snap a photo and tweet it out using the hashtag #NSXintheWild.  We hope you’ll join the fun and show off your NSX treasure. Your odds of winning some prizes will be much higher at VMworld if you do, as opposed to hitting the slots!

The post NSX Going Wild at This Year’s VMworld appeared first on Network Virtualization.

VMware NSX Day 1 Guide Library

Our VMware NSX Guides are authored and technically reviewed by VMware subject-matter experts and cover networking and security essentials.

Below you will find a description of the current books in our library along with a downloadable PDF link.

If you are interested in purchasing a hardcopy, you can do so at our online store.

 

Check out our Four New Releases!

 

New Release: VMware NSX Automation Fundamentals Guide

VMware NSX Automation Fundamentals delivers the roadmap to understanding networking and security automation challenges in today’s data centers.  It explains the fundamental nature of VMware NSX Data Center architecture while detailing integrated solutions for both VMware and third party offerings (such as VMware vRealize Automation, OpenStack, Puppet, Chef, PowerNSX) that assist in creating networking and security components on-demand.

Follow Caio on Twitter! And follow Thiago on Twitter too!

 

 

 

 

 

New Release: VMware NSX Network Virtualization Fundamentals

During their digital transformation process, many IT organizations still struggle with traditional networking methods and security approaches. By successfully addressing these challenges in thousands of real-world implementations, VMware NSX Data Center has established itself as the leading network virtualization platform, revolutionizing the way data center networks are designed and operated. In Continue reading

Worth Reading: Hardware and the future of databases

I foresee databases running on two-level stores and three-level stores, but I doubt they will be able to manage four-level stores because it is just too complicated to do the software. But there will be storage hierarchies and exactly what pieces will be in the storage hierarchy is yet to be determined. Main memory will be at the top and disk will be at the bottom, we know that, and there will be stuff in between for general purpose systems. For OLTP systems, there are going to be in main memory, end of story, and companies like VoltDB and MemSQL are main memory SQL engines that are blindingly fast. —Timothy Prickett Morgan @ The Next Platform

The post Worth Reading: Hardware and the future of databases appeared first on rule 11 reader.

Evidence at the cost of trust: The trouble with the Department of Justice – DreamHost case

The social and economic benefits of the Internet cannot be realized without users’ ability to communicate and organize privately, and, where appropriate, anonymously. Data collection warrants must strike a balance to protect these benefits without impeding law enforcement’s ability to enforce the law. In recent weeks, the United States Department of Justice’s (DoJ) conflict with DreamHost, a website hosting service, has underscored the importance of this balance.

A week after the 2017 U.S. presidential inauguration, the DoJ issued a warrant to DreamHost to gather evidence for almost 200 cases related to violence that occurred during Inauguration Day protests. DreamHost had provided services to a website used to coordinate protests during the presidential inauguration.

The initial warrant was broad in scope; DreamHost stated that compliance would mean handing over records relating to 1.3 million IP addresses. This July, the DoJ went even further, issuing a new warrant asking for “Files, databases, and database records” regarding the website in question. DreamHost’s filing with the court specifies that the DoJ sought: the IP addresses of visitors to the website; which website pages were viewed by visitors; and a description of the software running on visitors’ computers.

The DoJ itself appears to Continue reading

NetDevOps: important idempotence

As more and more network engineers dive into network automation, the word idempotence keeps coming up. What is it? Why is it important? Why should we care? Idempotence is often described as the ability to perform the same task repeatedly and produce the same result. I want to demonstrate a super simple example of what this means.

If I am logged into a Linux box and want to add an IP address to the loopback address, I could use something simple like a sed command.

root@leaf01:mgmt-vrf:~# sed -i '/loopback/ a address 1.1.1.1/32' /etc/network/interfaces

This produces exactly what I want!
auto lo
iface lo inet loopback
        address 1.1.1.1/32
        address 10.0.0.11/32

I have appended the address 1.1.1.1/32 to the loopback interface stanza of the /etc/network/interfaces file. Now what happens if I run that same exact command again?

Running the command again produces the following output:
auto lo
iface lo inet loopback
        address 1.1.1.1/32
        address 1.1.1.1/32
        address 10.0.0.11/32

? That is not what I wanted. I performed the same task but instead of just leaving the file alone, since the 1.1.1. Continue reading

Evidence at the cost of trust: The trouble with the Department of Justice – DreamHost case

The social and economic benefits of the Internet cannot be realized without users’ ability to communicate and organize privately, and, where appropriate, anonymously. Data collection warrants must strike a balance to protect these benefits without impeding law enforcement’s ability to enforce the law. In recent weeks, the United States Department of Justice’s (DoJ) conflict with DreamHost, a website hosting service, has underscored the importance of this balance.

Mark Buell
1 … 1,927 1,928 1,929 1,930 1,931 … 3,841