Peter

Author Archives: Peter

Secure forwarding of sFlow using ssh

Typically sFlow datagrams are sent unencrypted from agents embedded in switches and routers to a local collector/analyzer. Sending sFlow datagrams over the management VLAN or out of band management network generally provides adequate isolation and security within the site. Inter-site traffic within an organization is typically carried over a virtual private network (VPN) which encrypts the data and protects it from eavesdropping.

This article describes a simple method of carrying sFlow datagrams over an encrypted ssh connection which can be useful in situations where a VPN is not available, for example, sending sFlow to an analyzer in the public cloud, or to an external consultant.

The diagram shows the elements of the solution. A collector on the site receives sFlow datagrams from the network devices and uses the sflow_fwd.py script to convert the datagrams into line delimited hexadecimal strings that are sent over an ssh connection to another instance of sflow_fwd.py running on the analyzer that converts the hexadecimal strings back to sFlow datagrams.

The following sflow_fwd.py Python script accomplishes the task: 
#!/usr/bin/python

import socket
import sys
import argparse

parser = argparse.ArgumentParser(description='Serialize/deserialize sFlow')
parser.add_argument('-c', '--collector', default='')
parser.add_argument('-s', '--server')
parser.add_argument('-p', '--port', type=int, default=6343)
 Continue reading

Prometheus exporter

Prometheus is an open source time series database optimized to collect large numbers of metrics from cloud infrastructure. This article will explore how industry standard sFlow telemetry streaming supported by network devices (Arista, Aruba, Cisco, Dell, Huawei, Juniper, etc.) and Host sFlow agents (Linux, Windows, FreeBSD, AIX, Solaris, Docker, Systemd, Hyper-V, KVM, Nutanix AHV, Xen) can be integrated with Prometheus to extend visibility into the network.

The diagram above shows the elements of the solution: sFlow telemetry streams from hosts and switches to an instance of sFlow-RT. The sFlow-RT analytics software converts the raw measurements into metrics that are accessible through a REST API. The sflow-rt/prometheus application extends the REST API to include native Prometheus exporter functionality allowing Prometheus to retrieve metrics. Prometheus stores metrics in a time series database that can be queries by Grafana to build dashboards.

Update 19 October 2019, native support for Prometheus export added to sFlow-RT, Prometheus application no longer needed to run this example, use URL: /prometheus/metrics/ALL/ALL/txt. The Prometheus application is needed for exporting traffic flows, see Flow metrics with Prometheus and Grafana.

The Docker sflow/prometheus image provides a simple way to run the application:
docker run --name sflow-rt -p 8008:8008 -p  Continue reading

Loggly

Loggly is a cloud logging and and analysis platform. This article will demonstrate how to integrate network events generated from industry standard sFlow instrumentation build into network switches.
Loggly offers a free 14 day evaluation, so you can try this example at no cost.
ICMP unreachable describes how monitoring ICMP destination unreachable messages can help identify misconfigured hosts and scanning behavior. The article uses the sFlow-RT real-time analytics software to process the raw sFlow and report on unreachable messages.

The following script, loggly.js, modifies the sFlow-RT script from the article to send events to the Loggly HTTP/S Event Endpoint: 
var token = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx';
  
var url = 'https://logs-01.loggly.com/inputs/'+token+'/tag/http/';

var keys = [
  'icmpunreachablenet',
  'icmpunreachablehost',
  'icmpunreachableprotocol',
  'icmpunreachableport'
];

for (var i = 0; i < keys.length; i++) {
  var key = keys[i];
  setFlow(key, {
    keys:'macsource,ipsource,macdestination,ipdestination,' + key,
    value:'frames',
    log:true,
    flowStart:true
  });
}

setFlowHandler(function(rec) {
  var keys = rec.flowKeys.split(',');
  var msg = {
    flow_type:rec.name,
    src_mac:keys[0],
    src_ip:keys[1],
    dst_mac:keys[2],
    dst_ip:keys[3],
    unreachable:keys[4]
  };

  try { http(url,'post','application/json',JSON.stringify(msg)); }
  catch(e) { logWarning(e); };
}, keys);
Some notes on the script:
  1. Modify the script to use the correct token for your Loggly account.
  2. Including MAC addresses can help identify Continue reading

sFlow to JSON

The latest version of sflowtool can convert sFlow datagrams into JSON, making it easy to write scripts to process the standard sFlow telemetry streaming from devices in the network.

Download and compile the latest version of sflowtool:
git clone https://github.com/sflow/sflowtool.git
cd sflowtool/
./boot.sh 
./configure 
make
sudo make install
The -J option formats the JSON output to be human readable:
$ sflowtool -J
{
 "datagramSourceIP":"10.0.0.162",
 "datagramSize":"396",
 "unixSecondsUTC":"1544241239",
 "localtime":"2018-12-07T19:53:59-0800",
 "datagramVersion":"5",
 "agentSubId":"0",
 "agent":"10.0.0.231",
 "packetSequenceNo":"1068783",
 "sysUpTime":"1338417874",
 "samplesInPacket":"2",
 "samples":[
  {
   "sampleType_tag":"0:2",
   "sampleType":"COUNTERSSAMPLE",
   "sampleSequenceNo":"148239",
   "sourceId":"0:3",
   "elements":[
    {
     "counterBlock_tag":"0:1",
     "ifIndex":"3",
     "networkType":"6",
     "ifSpeed":"1000000000",
     "ifDirection":"1",
     "ifStatus":"3",
     "ifInOctets":"4162076356",
     "ifInUcastPkts":"16312256",
     "ifInMulticastPkts":"187789",
     "ifInBroadcastPkts":"2566",
     "ifInDiscards":"0",
     "ifInErrors":"0",
     "ifInUnknownProtos":"0",
     "ifOutOctets":"2115351089",
     "ifOutUcastPkts":"7087570",
     "ifOutMulticastPkts":"4453258",
     "ifOutBroadcastPkts":"6141715",
     "ifOutDiscards":"0",
     "ifOutErrors":"0",
     "ifPromiscuousMode":"0"
    },
    {
     "counterBlock_tag":"0:2",
     "dot3StatsAlignmentErrors":"0",
     "dot3StatsFCSErrors":"0",
     "dot3StatsSingleCollisionFrames":"0",
     "dot3StatsMultipleCollisionFrames":"0",
     "dot3StatsSQETestErrors":"0",
     "dot3StatsDeferredTransmissions":"0",
     "dot3StatsLateCollisions":"0",
     "dot3StatsExcessiveCollisions":"0",
     "dot3StatsInternalMacTransmitErrors":"0",
     "dot3StatsCarrierSenseErrors":"0",
     "dot3StatsFrameTooLongs":"0",
     "dot3StatsInternalMacReceiveErrors":"0",
     "dot3StatsSymbolErrors":"0"
    }
   ]
  },
  {
   "sampleType_tag":"0:1",
   "sampleType":"FLOWSAMPLE",
   "sampleSequenceNo":"11791",
   "sourceId":"0:3",
   "meanSkipCount":"2000",
   "samplePool":"34185160",
   "dropEvents":"0",
   "inputPort":"3",
   "outputPort":"10",
   "elements":[
    {
     "flowBlock_tag":"0:1",
     "flowSampleType":"HEADER",
     "headerProtocol":"1",
     "sampledPacketSize":"102",
     "strippedBytes":"0",
     "headerLen":"104",
     "headerBytes":"0C-AE-4E-98-0B-89-05-B6-D8-D9-A2-66-80-00-54-00-00-45-08-12-04-00-04-10-4A-FB-A0-00-00-BC-A0-00-00-EF-80-00-DE-B1-E7-26-00-20-75-04-B0-C5-00-00-00-00-96-01-20-00-00-00-00-00-01-11-21-31-41-51-61-71-81-91-A1-B1-C1-D1-E1-F1-02-12-22-32-42-52-62-72-82-92-A2-B2-C2-D2-E2-F2-03-13-23-33-43-53-63-73-1A-1D-4D-76-00-00",
     "dstMAC":"0cae4e980b89",
     "srcMAC":"05b6d8d9a266",
     "IPSize":"88",
     "ip.tot_len":"84",
     "srcIP":"10.0.0.203",
     "dstIP":"10.0.0.254",
     "IPProtocol":"1",
     "IPTOS":"0",
     "IPTTL":"64",
     "IPID":"8576",
     "ICMPType":"8",
     "ICMPCode":"0"
    },
    {
     "flowBlock_tag":"0:1001",
     "extendedType":"SWITCH",
     "in_vlan":"1",
     "in_priority":"0",
     "out_vlan":"1",
     "out_priority":"0"
    }
   ]
  }
 ]
}
The output shows the JSON representation of a single sFlow datagram containing one counter sample and one flow sample.

The Continue reading

Mininet, ONOS, and segment routing

Leaf and spine traffic engineering using segment routing and SDN and CORD: Open-source spine-leaf Fabric describe a demonstration at the 2015 Open Networking Summit using the ONOS SDN controller and a physical network of 8 switches.

This article will describe how to emulate a leaf and spine network using Mininet and configure the ONOS segment routing application to provide equal cost multi-path (ECMP) routing of flows across the fabric. The Mininet Dashboard application running on the sFlow-RT real-time analytics platform is used to provide visibility into traffic flows across the emulated network.

First, run ONOS using Docker:
docker run --name onos --rm -p 6653:6653 -p 8181:8181 -d onosproject/onos
Use the graphical interface, http://onos:8181, to enable the OpenFlow Provider Suite, Network Config Host Provider, Network Config Link Provider, and Segment Routing applications. The screen shot above shows the resulting set of enabled services.

Next, install sFlow-RT and the Mininet Dashboard application on host with Mininet:
wget https://inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz
./sflow-rt/get-app.sh sflow-rt mininet-dashboard
Start sFlow-RT:
./sflow-rt/start.sh
Download the sr.py script:
wget https://raw.githubusercontent.com/sflow-rt/onos-sr/master/sr.py
Start Mininet:
sudo env ONOS=10.0.0.73 mn --custom sr.py,sflow-rt/extras/sflow.py \
--link  Continue reading

Real-time visibility at 400 Gigabits/s

The chart above demonstrates real-time, up to the second, flow monitoring on a 400 gigabit per second link. The chart shows that the traffic is composed of four, roughly equal, 100 gigabit per second flows.

The data was gathered from The International Conference for High PerformanceComputing, Networking, Storage, and Analysis (SC18) being held this week in Dallas. The conference network, SCinet, is described as the fastest and most powerful network in the world.
This year, the SCinet network includes recently announced 400 gigabit switches from Arista networks, see Arista Introduces 400 Gigabit Platforms. Each switch delivers 32 400G ports in a 1U form factor.
NRE-36 University of Southern California network topology for SuperComputing 2018
The switches are part of 400G demonstration network connecting USC, Caltech and StarLight booths. The chart shows traffic on a link connecting the USC and Caltech booths.

Providing the visibility needed to manage large scale high speed networks is a significant challenge. In this example, line rate traffic of 80 million packets per second is being monitored on the 400G port. The maximum packet rate for 64 byte packets on a 400 Gigabit, full duplex, link is approximately 1.2 billion packet per second Continue reading

Ryu measurement based control

ONOS measurement based control describes how real-time streaming telemetry can be used to automatically trigger SDN controller actions. The article uses DDoS mitigation as an example.

This article recreates the demonstration using the Ryu SDN framework and emulating a network using Mininet. Install both pieces of software on a Linux server or virtual machine in order to follow this example.

Start Ryu with the simple_switch_13 and ryu.app.ofctl_rest applications loaded:
ryu-manager $RYU_APP/simple_switch_13.py,$RYU_APP/ofctl_rest.py
Note: The simple_switch_13.py and ofctl_rest.py scripts are part of a standard Ryu installation. The $RYU_APP variable has been set to point to the Ryu app directory.
This demonstration uses the sFlow-RT real-time analytics engine to process standard sFlow streaming telemetry from the network switches.

Download sFlow-RT:
wget https://inmon.com/products/sFlow-RT/sflow-rt.tar.gz
tar -xvzf sflow-rt.tar.gz
Install the Mininet Dashboard application: 
sflow-rt/get-app.sh sflow-rt mininet-dashboard
The following script, ryu.js, implements the DDoS mitigation function described in the previous article: 
var ryu = '127.0.0.1';
var controls = {};

setFlow('udp_reflection',
 {keys:'ipdestination,udpsourceport',value:'frames'});
setThreshold('udp_reflection_attack',
 {metric:'udp_reflection',value:100,byFlow:true,timeout:2});

setEventHandler(function(evt) {
 // don't consider inter-switch links
 var link = topologyInterfaceToLink(evt.agent,evt.dataSource);
 if(link) return;

 // get port information
 var port = topologyInterfaceToPort(evt.agent,evt.dataSource);
 if(! Continue reading

Systemd traffic marking

Monitoring Linux services describes how the open source Host sFlow agent exports metrics from services launched using systemd, the default service manager on most recent Linux distributions. In addition, the Host sFlow agent efficiently samples network traffic using Linux kernel capabilities: PCAP/BPF, nflog, and ulog.

This article describes a recent extension to the Host sFlow systemd module, mapping sampled traffic to the individual services the generate or consume them. The ability to color traffic by application greatly simplifies service discovery and service dependency mapping; making it easy to see how services communicate in a multi-tier application architecture.

The following /etc/hsflowd.conf file configures the Host sFlow agent, hsflowd, to sampling packets on interface eth0, monitor systemd services and mark the packet samples, and track tcp performance:
sflow {
  collector { ip = 10.0.0.70 }
  pcap { dev = eth0 }
  systemd { markTraffic = on }
  tcp { }
}
The diagram above illustrates how the Host sFlow agent is able to efficiently monitor and classify traffic. In this case both the Host sFlow agent and an Apache web server are are running as services managed by systemd. A network connection , shown in Continue reading

Microsoft Office 365

Office 365 IP Address and URL Web service describes a simple REST API that can be used to query for the IP address ranges associated with Microsoft Office 365 servers.

This information is extremely useful, allowing traffic analytics software to combine telemetry obtained from network devices with information obtained using the Microsoft REST API  in order to identifying clients, links, and devices carrying the traffic, as well as any issues, such as link errors, and congestion,  that may be impacting performance.
The sFlow-RT analytics engine is programmable and includes a REST client that can be used to query the Microsoft API and combine the information with industry standard sFlow telemetry from network devices. The following script, office365.js, provides a simple example:
var api = 'https://endpoints.office.com/endpoints/worldwide';

function uuidv4() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
    var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
    return v.toString(16);
  });
}

var reqid = uuidv4();

function updateAddressMap() {
  var res, i, ips, id, groups;
  try { res = http(api+'?clientrequestid='+reqid); }
  catch(e) { logWarning('request failed ' + e); }
  if(res == null) return;
  res = JSON.parse(res);
  groups  Continue reading

Northbound Networks Zodiac GX

Mininet is widely used to emulate software defined networks (SDNs). Mininet flow analytics describes how standard sFlow telemetry, from Open vSwitch used by Mininet emulate the network, provides feedback to an SDN controller, allowing the controller to adapt the network to changing traffic, for example, to mitigate a distributed denial of service (DDoS) attack.

Northbound Networks Zodiac GX is an inexpensive open source software based switch that is ideal for experimenting with software defined networking (SDN) in a physical network setting. The small fanless package makes the switch an attractive option for desktop use. The Zodiac GX is also based on Open vSwitch, making it easy to take SDN control strategies developed on Mininet.
Enabling sFlow on the Zodiac GX is easy, navigate to the System>Startup page and add the following line to the end of the startup script (before the exit 0 line):
ovs-vsctl -- --id=@sflow create sflow agent=$OVS_BR target=$IP_CONTROLLER_1 sampling=100 polling=10 -- set bridge $OVS_BR sflow=@sflow
Reboot the switch for the changed to take effect.

Use sflowtool to verify that sFlow is arriving at the controller host and to examine the contents of the telemetry stream. Running sflowtool using Docker is a simple alternative to building the software Continue reading

RDMA over Converged Ethernet (RoCE)

RDMA over Converged Ethernet is a network protocol that allows remote direct memory access (RDMA) over an Ethernet network. One of the benefits running RDMA over Ethernet is the visibility provided by standard sFlow instrumentation embedded in the commodity Ethernet switches used to build data center leaf and spine networks where RDMA is most prevalent.

The sFlow telemetry stream includes packet headers, sampled at line rate by the switch hardware. Hardware packet sampling allows the switch to monitor traffic at line rate on all ports, keeping up with the high speed data transfers associated with RoCE.

The diagram above shows the packet headers associated with RoCEv1 and RoCEv2 packets. Decoding the InfiniBand Global Routing Header (IB GRH) and InfiniBand Base Transport Header (IB BTH) allows an sFlow analyzer to report in detail on RoCE traffic.
The sFlow-RT real-time analytics engine recently added support for RoCE by decoding InfiniBand Global Routing and InfiniBand Base Transport fields. The screen capture of the sFlow-RT Flow-Trend application shows traffic associated with an RoCEv2 connection between two hosts, 10.10.2.22 and 10.10.2.52. The traffic consists of SEND and ACK messages exchanged as part of a reliable connection (RC Continue reading

ExtremeXOS 22.5.1 adds support Broadcom ASIC table utilization statistics

ExtremeXOS 22.5.1 is now available! describes added support in sFlow for "New data structures to support reporting on hardware/table utilization statistics." The feature is available on Summit X450-G2, X460-G2, X670-G2, X770, and ExtremeSwitching X440-G2, X870, X620, X690 series switches.

Figure 1 shows the packet processing pipeline of a Broadcom ASIC. The pipeline consists of a number of linked hardware tables providing bridging, routing, access control list (ACL), and ECMP forwarding group functions. Operations teams need to be able to proactively monitor table utilizations in order to avoid performance problems associated with table exhaustion.

Broadcom's sFlow specification, sFlow Broadcom Switch ASIC Table Utilization Structures, leverages the industry standard sFlow protocol to offer scaleable, multi-vendor, network wide visibility into the utilization of these hardware tables.

The following output from the open source sflowtool command line utility shows the raw table measurements (this is in addition to the extensive set of measurements already exported via sFlow by ExtremeXOS): 
bcm_asic_host_entries 4
bcm_host_entries_max 8192
bcm_ipv4_entries 0
bcm_ipv4_entries_max 0
bcm_ipv6_entries 0
bcm_ipv6_entries_max 0
bcm_ipv4_ipv6_entries 9
bcm_ipv4_ipv6_entries_max 16284
bcm_long_ipv6_entries 3
bcm_long_ipv6_entries_max 256
bcm_total_routes 10
bcm_total_routes_max 32768
bcm_ecmp_nexthops 0
bcm_ecmp_nexthops_max 2016
bcm_mac_entries 3
bcm_mac_entries_max 32768
bcm_ipv4_neighbors 4
bcm_ipv6_neighbors 0
bcm_ipv4_routes 0
bcm_ipv6_routes 0
bcm_acl_ingress_entries  Continue reading

Visualizing real-time network traffic flows at scale

Particle has been released on GitHub, https://github.com/sflow-rt/particle. The application is a real-time visualization of network traffic in which particles flow between hosts arranged around the edges of the screen. Particle colors represent different types of traffic.

Particles provide an intuitive representation of network packets transiting the network from source to destination. The animation slows time so that the particle takes 10 seconds (instead of milliseconds) to transit the network. Groups of particles traveling the same path represent flows of packets between the hosts. Particle size and frequency are used to indicate the intensity of the traffic flowing on a path.

Particles don't follow straight lines, instead following quadratic Bézier curves around the center of the screen. Warping particle paths toward the center of the screen ensures that all paths are of similar length and visible - even if the start and end points are on the same axis.

The example above is from a site with over 500 network switches carrying hundreds of Gigabits of traffic. Internet, Customer, Site and Datacenter hosts have been assigned to the North, East, South and West sides respectively.
The screen is updated 60 times per second for smooth animation. Active Continue reading

sFlow available on Juniper PTX series routers


sFlow functionality introduced on the PTX1000 and PTX10000 platforms—Starting in Junos OS Release 18.2R1, the PTX1000 and PTX10000 routers support sFlow, a network monitoring protocol for high-speed networks. With sFlow, you can continuously monitor tens of thousands of ports simultaneously. The mechanism used by sFlow is simple, not resource intensive, and accurate.  - New and Changed Features

The recent article, sFlow available on Juniper MX series routers, describes how Juniper is extending sFlow support to include routers to provide visibility across their entire range of switching and routing products.

Universal support for industry standard sFlow as a base Junos feature reduces the operational complexity and cost of network visibility for enterprises and service providers. Real-time streaming telemetry from campus switches, routers, and data center switches, provides centralized, real-time, end-to-end visibility needed to troubleshoot, optimize, and account for network usage.

Analytics software is a critical factor in realizing the full benefits of sFlow monitoring. Choosing an sFlow analyzer discusses important factors to consider when selecting from the range of open source and commercial sFlow analysis tools.

SDKLT

Logical Table Software Development Kit (SDKLT) is a new, powerful, and feature rich Software Development Kit (SDK) for Broadcom switches. SDKLT provides a new approach to switch configuration using Logical Tables.

Building the Demo App describes how to get started using a simulated Tomahawk device. Included, is a CLI that can be used to explore tables. For example, the following CLI output shows the attributes of the sFlow packet sampling table:
BCMLT.0> lt list -d MIRROR_PORT_ENCAP_SFLOW
MIRROR_PORT_ENCAP_SFLOW
  Description: The MIRROR_PORT_ENCAP_SFLOW logical table is used to specify
               per-port sFlow encapsulation sample configuration.
  11 fields (1 key-type field):
    SAMPLE_ING_FLEX_RATE
        Description: Sample ingress flex sFlow packet if the generated sFlow random
                     number is greater than the threshold. A lower threshold leads to
                     higher sampling frequency.
    SAMPLE_EGR_RATE
        Description: Sample egress sFlow packet if the generated sFlow random number is
                     greater than the threshold. A lower threshold leads to
                     higher sampling frequency.
    SAMPLE_ING_RATE
        Description: Sample ingress sFlow packet if the generated sFlow random number is
                     greater than the threshold. A lower threshold leads to
                     higher sampling frequency.
    SAMPLE_ING_FLEX_MIRROR_INSTANCE
        Description: Enable to copy ingress flex sFlow packet samples to the ingress
                     mirror member using the sFlow mirror instance configuration.
    SAMPLE_ING_FLEX_CPU
        Description: Enable to copy ingress flex  Continue reading

sFlow available on Juniper MX series routers

sFlow support on MX Series devices—Starting in Junos OS Release 18.1R1, you can configure sFlow technology (as a sFlow agent) on a MX Series device, to continuously monitor traffic at wire speed on all interfaces simultaneously. The sFlow technology is a monitoring technology for high-speed switched or routed networks.  - New and Changed Features

Understanding How to Use sFlow Technology for Network Monitoring on a MX Series Router lists the following benefits of sFlow Technology on a MX Series Router:
  • sFlow can be used by software tools like a network analyzer to continuously monitor tens of thousands of switch or router ports simultaneously.
  • Since sFlow uses network sampling (forwarding one packet from ‘n’ number of total packets) for analysis, it is not resource intensive (for example processing, memory and more). The sampling is done at the hardware application-specific integrated circuits (ASICs) and hence it is simple and more accurate.
With the addition of the MX series routers, Juniper now supports sFlow across its entire product range:
Universal support for Continue reading

ONOS measurement based control

ONOS traffic analytics describes how to run the ONOS SDN controller with a virtual network created using Mininet. The article also showed how to monitor network traffic using industry standard sFlow instrumentation available in Mininet and in physical switches.
This article uses the same ONOS / Mininet test bed to demonstrate how sFlow-RT real-time flow analytics can be used to push controls to the network through the ONOS REST API.  Leaf and spine traffic engineering using segment routing and SDN used real-time flow analytics to load balance an ONOS controlled physical network. In this example, we will use ONOS to filter DDoS attack traffic on a Mininet virtual network.

The following sFlow-RT script, ddos.js, detects DDoS attacks and programs ONOS filter rules to block the attacks:
var user = 'onos';
var password = 'rocks';
var onos = '192.168.123.1';
var controls = {};

setFlow('udp_reflection',
 {keys:'ipdestination,udpsourceport',value:'frames'});
setThreshold('udp_reflection_attack',
 {metric:'udp_reflection',value:100,byFlow:true,timeout:2});

setEventHandler(function(evt) {
 // don't consider inter-switch links
 var link = topologyInterfaceToLink(evt.agent,evt.dataSource);
 if(link) return;

 // get port information
 var port = topologyInterfaceToPort(evt.agent,evt.dataSource);
 if(!port) return;

 // need OpenFlow info to create ONOS filtering rule
 if(!port.dpid || !port.ofport) return;

 // we already have  Continue reading

ONOS traffic analytics

Open Network Operating System (ONOS) is "a software defined networking (SDN) OS for service providers that has scalability, high availability, high performance, and abstractions to make it easy to create applications and services." The open source project is hosted by the Linux Foundation.

Mininet and onos.py workflow describes how to run ONOS using the Mininet network emulator. Mininet allows virtual networks to be quickly constructed and is a simple way to experiment with ONOS. In addition, Mininet flow analytics describes how to enable industry standard sFlow streaming telemetry in Mininet, proving a simple way monitor traffic in the ONOS controlled network.

For example, the following command creates a Mininet network, controlled by ONOS, and monitored using sFlow:
sudo mn --custom ~/onos/tools/dev/mininet/onos.py,sflow-rt/extras/sflow.py \
--link tc,bw=10 --controller onos,1 --topo tree,2,2
The screen capture above shows the network topology in the ONOS web user interface.
Install Mininet dashboard to visualize the network traffic. The screen capture above shows a large flow over the same topology being displayed by ONOS, see Mininet weathermap for more examples.

In this case, the traffic was created by the following Mininet command: 
mininet-onos> iperf h1 h3
The screen capture above shows top flows, busiest Continue reading

Real-time baseline anomaly detection

The screen capture demonstrates the real-time baseline and anomaly detection based on industry standard sFlow streaming telemetry. The chart was generated using sFlow-RT analytics software. The blue line is an up to the second measure of traffic (measured in Bits per Second). The red and gold lines represent dynamic upper and lower limits calculated by the baseline function. The baseline function flags "high" and "low" value anomalies when values move outside the limits. In this case, a "low" value anomaly was flagged for the drop in traffic shown in the chart.

Writing Applications provides a general introduction to sFlow-RT programming. The baseline functionality is exposed through through the JavaScript API.

Create new baseline 
baselineCreate(name,window,sensitivity,repeat);
Where:
  • name, name used to reference baseline.
  • window, the number of previous intervals to consider in calculating the limits.
  • sensitivity, the number of standard deviations used to calculate the limits.
  • repeat, the number of successive data points outside the limits before flagging anomaly 
In this example, baseline parameter values were window=180 (seconds), sensitivity=2, and repeat=3.

Update baseline
var status = baselineCheck(name,value);
Where:
  • status, "learning" while baseline is warming up (takes window intervals),  "normal" if value is in expected range, "low" Continue reading

Flow smoothing

The sFlow-RT real-time analytics engine includes statistical smoothing. The chart above illustrates the effect of different levels of smoothing when analyzing real-time sFlow telemetry.

The traffic generator in this example creates an alternating pattern: 1.25Mbytes/second for 30 seconds followed by a pause of 30 seconds. Smoothing time constants between 1 second and 500 seconds have been applied to generate the family of charts. The blue line is the result of 1 second smoothing and closely tracks the traffic pattern. At the other extreme, the dark red line is the result of 500 second smoothing, showing a constant 625Kbytes/second (the average of the waveform).

There is a tradeoff between responsiveness and variability (noise) when selecting the level of smoothing. Selecting a suitable smoothing level depends on the flow analytics application.

Low smoothing values are appropriate when fast response is required, for example:
Higher smoothing values are appropriate when less variability is desirable, for example:

Generating the chart

The results described in this article are easily reproduced using the testbed Continue reading
1 … 4 5 6 7 8 … 13