The importance of situational awareness for network engineers

In another life, not too long ago, I spent a number of years in civilian and military law enforcement. When going through just about any kind of tactical training, one of the recurring themes they hammer into you is “situational awareness or SA.”

Wikipedia defines SA as:

Situational awareness or situation awareness (SA) is the perception of environmental elements with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event. It is also a field of study concerned with understanding of the environment critical to decision-makers in complex, dynamic areas from aviation, air traffic control, ship navigation, power plant operations, military command and control, and emergency services such as fire fighting and policing; to more ordinary but nevertheless complex tasks such as driving an automobile or riding a bicycle.

Defining the need for SA in network engineering

It’s interesting to notice that critical infrastructure such as power plants and air traffic control are listed as disciplines that train in SA, however, I’ve never seen it taught in Continue reading

The importance of situational awareness for network engineers

In another life, not too long ago, I spent a number of years in civilian and military law enforcement. When going through just about any kind of tactical training, one of the recurring themes they hammer into you is “situational awareness or SA.”

Wikipedia defines SA as:

Situational awareness or situation awareness (SA) is the perception of environmental elements with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event. It is also a field of study concerned with understanding of the environment critical to decision-makers in complex, dynamic areas from aviation, air traffic control, ship navigation, power plant operations, military command and control, and emergency services such as fire fighting and policing; to more ordinary but nevertheless complex tasks such as driving an automobile or riding a bicycle.

Defining the need for SA in network engineering

It’s interesting to notice that critical infrastructure such as power plants and air traffic control are listed as disciplines that train in SA, however, I’ve never seen it taught in Continue reading

Photos leaked for new CCR-6506-96G-48S-16S+

So photos have been leaked (we can’t say from where exactly) for the newest experimental router from MikroTik. We don’t know much about it yet except it will be the largest router MikroTik has ever built and is currently designated as CCR-6506-96G-48S-16S+.

Rough specs are:

• 6 slot chassis
• Dual redundant 720 Gbps CPU modules
• Dual power
• 96 ports of copper 1 gig
• 48 1 gig SFP ports
• 16 Ten gig SFP+ ports

Apparently this device will coincide with the release of RouterOS version 8 in 2026 [an inside source at MikroTik named “Janis” confirmed this is a realistic target date.]

Many covert mAP-quadcopters died to bring us this information…these photos are NOT for public distribution.

And if you haven’t quite figured it out yet…APRIL FOOLS DAY!!!! But seriously MikroTik….we need this router.

Photos leaked for new CCR-6506-96G-48S-16S+

So photos have been leaked (we can’t say from where exactly) for the newest experimental router from MikroTik. We don’t know much about it yet except it will be the largest router MikroTik has ever built and is currently designated as CCR-6506-96G-48S-16S+.

Rough specs are:

• 6 slot chassis
• Dual redundant 720 Gbps CPU modules
• Dual power
• 96 ports of copper 1 gig
• 48 1 gig SFP ports
• 16 Ten gig SFP+ ports

Apparently this device will coincide with the release of RouterOS version 8 in 2026 [an inside source at MikroTik named “Janis” confirmed this is a realistic target date.]

Many covert mAP-quadcopters died to bring us this information…these photos are NOT for public distribution.

And if you haven’t quite figured it out yet…APRIL FOOLS DAY!!!! But seriously MikroTik….we need this router.

ISP Architechture – MPLS Overview, Design and Implementation for WISPs

Recently, I was fortunate enough to be invited by Brian Horn with WISPA.org to teach a session at WISP America 2016 in Lousiville, KY. We had the class on Tuesday, March 15th 2016 and the turnout and response were great.  Many different people have asked for the presentation, so I decided to go ahead and post it here. Hope this helps some of you who are trying to get into MPLS and although it does have a bit of a WISP focus, almost all of the concepts in the presentation apply to wireline networks as well.

About the presentation

Scope: This session was 30 minutes long with a Q&A afterwards, so the material is really a deep dive on MPLS. The goal was to introduce WISP engineers and owners to MPLS and how it improves the network as well as revenue.

When should I put MPLS in my WISP or Service Provider network?  The answer is ASAP! I was asked this question by a small WISP earlier in the week and he said i’m just too small to be thinking about MPLS. My response to him was simply – “Do you want to get MPLS in and working Continue reading

ISP Architechture – MPLS Overview, Design and Implementation for WISPs

Recently, I was fortunate enough to be invited by Brian Horn with WISPA.org to teach a session at WISP America 2016 in Lousiville, KY. We had the class on Tuesday, March 15th 2016 and the turnout and response were great.  Many different people have asked for the presentation, so I decided to go ahead and post it here. Hope this helps some of you who are trying to get into MPLS and although it does have a bit of a WISP focus, almost all of the concepts in the presentation apply to wireline networks as well.

About the presentation

Scope: This session was 30 minutes long with a Q&A afterwards, so the material is really a deep dive on MPLS. The goal was to introduce WISP engineers and owners to MPLS and how it improves the network as well as revenue.

When should I put MPLS in my WISP or Service Provider network?  The answer is ASAP! I was asked this question by a small WISP earlier in the week and he said i’m just too small to be thinking about MPLS. My response to him was simply – “Do you want to get MPLS in and working Continue reading

Put 500,000+ BGP routes in your lab network!!! Download this VM and become your own upstream BGP ISP for testing.

[adrotate banner=”5″]

Happy New Year and welcome to the VM you can punish your routers with

Hello from stubarea51.net and Happy New Year! We are back from the holidays and recharged with lots of new stuff in the world of network engineering. If you ever thought it would be cool to put a full BGP table into a lab router, GNS3 or other virtualized router, you’re not alone.

A while back, I tackled this post and got everything up and running:

http://evilrouters.net/2009/08/21/getting-bgp-routes-into-dynamips-with-video/

First of all, thanks to evilrouters.net for figuring out the hard parts so we could build this into a VM. After basking for a while in the high geek factor of this project, it gave me an idea to build a VM that could be distributed among network engineers and IT professionals. The idea is to easily spin up one or more full BGP tables to test a particular network design or convergence speed, playing with BGP attributes, etc. After a few months of tweaking it and getting the VM ready for distribution, we finally are ready to put it out for everyone to use.

Network Diagram

Here is an overview of the topology we Continue reading

Put 500,000+ BGP routes in your lab network!!! Download this VM and become your own upstream BGP ISP for testing.

[adrotate banner=”5″]

Happy New Year and welcome to the VM you can punish your routers with

Hello from stubarea51.net and Happy New Year! We are back from the holidays and recharged with lots of new stuff in the world of network engineering. If you ever thought it would be cool to put a full BGP table into a lab router, GNS3 or other virtualized router, you’re not alone.

A while back, I tackled this post and got everything up and running:

http://evilrouters.net/2009/08/21/getting-bgp-routes-into-dynamips-with-video/

First of all, thanks to evilrouters.net for figuring out the hard parts so we could build this into a VM. After basking for a while in the high geek factor of this project, it gave me an idea to build a VM that could be distributed among network engineers and IT professionals. The idea is to easily spin up one or more full BGP tables to test a particular network design or convergence speed, playing with BGP attributes, etc. After a few months of tweaking it and getting the VM ready for distribution, we finally are ready to put it out for everyone to use.

Network Diagram

Here is an overview of the topology we Continue reading

Cisco to MikroTik – command translation – BGP

In the world of network engineering, learning a new syntax can challenging especially if you need a lot of detail quickly. The command structure for RouterOS can be a bit challenging sometimes if you are used to Cisco CLI commands.  Most of us that have been in networking for a while got our start with Cisco gear and so it is helpful to draw comparisons between the commands, especially if you are trying to build a network with a MikroTik and Cisco router.

This is the first post in a series I’ve wanted to do for a while that creates a Rosetta stone essentially between IOS and RouterOS. We plan to tackle a number of other command comparisons like OSPF, MPLS and VLANs to make it easier for network engineers trained in Cisco IOS to successfully implement MikroTik / RouterOS devices. While many commands have almost the exact same information, others are as close as possible. Since there isn’t always an exact match, sometimes you may have to run two or three commands to get the information needed.

We plan to tackle a number of other command comparisons like OSPF, MPLS and VLANs to make it easier for network Continue reading

Cisco to MikroTik – command translation – BGP

In the world of network engineering, learning a new syntax can challenging especially if you need a lot of detail quickly. The command structure for RouterOS can be a bit challenging sometimes if you are used to Cisco CLI commands.  Most of us that have been in networking for a while got our start with Cisco gear and so it is helpful to draw comparisons between the commands, especially if you are trying to build a network with a MikroTik and Cisco router.

This is the first post in a series I’ve wanted to do for a while that creates a Rosetta stone essentially between IOS and RouterOS. We plan to tackle a number of other command comparisons like OSPF, MPLS and VLANs to make it easier for network engineers trained in Cisco IOS to successfully implement MikroTik / RouterOS devices. While many commands have almost the exact same information, others are as close as possible. Since there isn’t always an exact match, sometimes you may have to run two or three commands to get the information needed.

We plan to tackle a number of other command comparisons like OSPF, MPLS and VLANs to make it easier for network Continue reading

MikroTik – CCR1072-1G-8S+ – PPPoE testing preview – 30,000 connections and queues.

[adrotate banner=”5″]

Why we chose PPPoE as the next test

First of all, thanks to everyone for all the positive feedback, comments and questions about the CCR1072-1G-8S+ testing we have been posting in the last few months.  Even MikroTik has taken an interest in this testing and we have gotten some great feedback from them as well.

We received more questions about the PPPoE capabilities of the CCR1072-1G-8S+  than any other type of request. Since we have already published the testing on BGP, throughput and EoIP, we have decided to tackle the PPPoE testing to understand where the limits of the CCR1072-1G-8S+ are. This is only a preview of the testing as we are working on different methods of testing and config, but this will at least give you a glimpse of what is possible.

30,000 PPPoE Connections !!!!

Overview of PPPoE connections and CPU load

PRTG Monitoring

We have started using PRTG in the StubArea51.net lab as it makes monitoring of resource load over time much easier when we are testing. Check it out as it is free up to 100 sensors and works very well with MikroTik

https://www.paessler.com/prtg/download

PRTG CPU Profile 

PRTG PPPoE Continue reading

MikroTik – CCR1072-1G-8S+ – PPPoE testing preview – 30,000 connections and queues.

[adrotate banner=”5″]

Why we chose PPPoE as the next test

First of all, thanks to everyone for all the positive feedback, comments and questions about the CCR1072-1G-8S+ testing we have been posting in the last few months.  Even MikroTik has taken an interest in this testing and we have gotten some great feedback from them as well.

We received more questions about the PPPoE capabilities of the CCR1072-1G-8S+  than any other type of request. Since we have already published the testing on BGP, throughput and EoIP, we have decided to tackle the PPPoE testing to understand where the limits of the CCR1072-1G-8S+ are. This is only a preview of the testing as we are working on different methods of testing and config, but this will at least give you a glimpse of what is possible.

30,000 PPPoE Connections !!!!

Overview of PPPoE connections and CPU load

PRTG Monitoring

We have started using PRTG in the StubArea51.net lab as it makes monitoring of resource load over time much easier when we are testing. Check it out as it is free up to 100 sensors and works very well with MikroTik

https://www.paessler.com/prtg/download

PRTG CPU Profile 

PRTG PPPoE Continue reading

10 Gbps of Layer 2 throughput is possible using MikroTik’s EoIP tunnel.

[adrotate banner=”5″]

[metaslider id=282]

Getting to 10 Gbps using EoIP

The EoIP tunnel protocol is one of the more popular features we see deployed in MikroTik routers.  It is useful anywhere a Layer 2 extension over a Layer 3 network is needed and can be done with very little effort / complexity.  One of the questions that seems to come up on the forums frequently is how much traffic can an EoIP tunnel handle which is typically followed by questions about performance with IPSEC turned on. Answers given by MikroTik and others on forums.mikrotik.com typically fall into the 1 to 3 Gbps range with some hints that more is possible. We searched to see if anyone had done 10 Gbps over EoIP with or without IPSEC and came up empty handed. That prompted us to dive into the StubArea51 lab and set up a test network so we could get some hard data and definitive answers.

The EoIP protocol and recent enhancements

Ethernet over IP or EoIP is a protocol that started as an IETF  draft somewhere around 2002 and MikroTik developed a proprietary implementation of it that has been in RouterOS for quite a while. Continue reading

MikroTik CCR1072-1G-8S+ Review – Part 3 – 80 Gbps Throughput testing

[adrotate banner=”5″]

[metaslider id=249]

The 80 Gbps barrier has finally been broken (and yes we are rounding up) !!!!

Well at least it has been reached by someone other than MikroTik. It’s taken us quite a while to get all the right pieces to push 80 Gbps of traffic through the CC1072 but with the latest round of servers that just got delivered to our lab, we were able to go beyond our previous high water mark of 54 Gbps all the way to just under 80 Gbps. There have been a number of questions about this particular router and what the performance will look like in the real world. While this is still a lab test, we are using non-MikroTik equipment and iperf which is considered an extremely accurate performance measuring tool for TCP and UDP.

Video of the CCR1072-1G-8S+ in action  (Turn up your volume to hear the roar of the ESXi servers as they approach 80 Gbps)

How we did it – The Hardware 

CCR1072-1G-8S+ – Obviously you can’t have a test of the CCR1072 without one to test on. Our CCR1072-1G-8S+ is a pre-production model so there are some minor differences between it and the Continue reading

MikroTik CCR1072-1G-8S+ Review – Part 3 – 80 Gbps Throughput testing

[adrotate banner=”5″]

[metaslider id=249]

The 80 Gbps barrier has finally been broken (and yes we are rounding up) !!!!

Well at least it has been reached by someone other than MikroTik. It’s taken us quite a while to get all the right pieces to push 80 Gbps of traffic through the CC1072 but with the latest round of servers that just got delivered to our lab, we were able to go beyond our previous high water mark of 54 Gbps all the way to just under 80 Gbps. There have been a number of questions about this particular router and what the performance will look like in the real world. While this is still a lab test, we are using non-MikroTik equipment and iperf which is considered an extremely accurate performance measuring tool for TCP and UDP.

Video of the CCR1072-1G-8S+ in action  (Turn up your volume to hear the roar of the ESXi servers as they approach 80 Gbps)

How we did it – The Hardware 

CCR1072-1G-8S+ – Obviously you can’t have a test of the CCR1072 without one to test on. Our CCR1072-1G-8S+ is a pre-production model so there are some minor differences between it and the Continue reading

MikroTik CCR1072-1G-8S+ Review – update on Part 3 – Throughput

[adrotate banner=”4″]

Breaking the 80 Gbps barrier!!!

The long wait for real-world 1072 performance numbers is almost over – the last two VMWARE server chassis we needed to push the full 80 Gpbs arrived in the StubArea51 lab today. Thanks to everyone who wrote in and commented on the first two reviews we did on the CCR-1072-1G-8S+.  We initially began work on performance testing throughput for the CCR1072 in late July of this year, but had to order a lot of parts to get enough 10 Gig PCIe cards, SFP+ modules and fiber to be able to push 80 Gbps of traffic through this router.

Challenges

There have been a number of things that we have had to work through to get to 80 Gbps but we are very close. This will be detailed in the full review we plan to release next week but here are a few.

• VMWARE ESXi – LACP Hashing – Initially we built LACP channels between the ESXi hosts and the 1072 expecting to load the links by using multiple source and destination IPs but we ran into issues with traffic getting stuck on one side of the LACP channel and had to move to Continue reading

MikroTik CCR1072-1G-8S+ Review – update on Part 3 – Throughput

[adrotate banner=”4″]

Breaking the 80 Gbps barrier!!!

The long wait for real-world 1072 performance numbers is almost over – the last two VMWARE server chassis we needed to push the full 80 Gpbs arrived in the StubArea51 lab today. Thanks to everyone who wrote in and commented on the first two reviews we did on the CCR-1072-1G-8S+.  We initially began work on performance testing throughput for the CCR1072 in late July of this year, but had to order a lot of parts to get enough 10 Gig PCIe cards, SFP+ modules and fiber to be able to push 80 Gbps of traffic through this router.

Challenges

There have been a number of things that we have had to work through to get to 80 Gbps but we are very close. This will be detailed in the full review we plan to release next week but here are a few.

• VMWARE ESXi – LACP Hashing – Initially we built LACP channels between the ESXi hosts and the 1072 expecting to load the links by using multiple source and destination IPs but we ran into issues with traffic getting stuck on one side of the LACP channel and had to move to Continue reading

MikroTik unveils new RouterOS development cycle

[adrotate banner=”4″]

As we all patiently await the release of RouterOS (RoS) v7 beta, MikroTik has announced a change in the way RoS development is organized.  There will now be three different tracks of development:

Bugfix only – When a current build is released, only fixes to known bugs will be added to this branch of development

Current – Current release will contain bugfixes and new features

Release Candidate – The release candidate will remain the development build for the next “current” release.

Graphical Overview of the new development cycle

Image and notes below are from here

A small addendum: the Bugfix only will only contain verified fixes, and no new features. The Current release contains the same fixes but also new features and other improvements, sometimes also less critical fixes than in Bugfix. And finally the Release Candidate is more likely to a nightly build. We will not to intensive testing before publishing these, only quick check if upgrade can be done and if most features work fine.

Origin

The idea originally came out of this thread and after a flurry of positive commentary, it became a working practice shortly therafter.

We plan to make sub-version releases Continue reading

MikroTik unveils new RouterOS development cycle

[adrotate banner=”4″]

As we all patiently await the release of RouterOS (RoS) v7 beta, MikroTik has announced a change in the way RoS development is organized.  There will now be three different tracks of development:

Bugfix only – When a current build is released, only fixes to known bugs will be added to this branch of development

Current – Current release will contain bugfixes and new features

Release Candidate – The release candidate will remain the development build for the next “current” release.

Graphical Overview of the new development cycle

Image and notes below are from here

A small addendum: the Bugfix only will only contain verified fixes, and no new features. The Current release contains the same fixes but also new features and other improvements, sometimes also less critical fixes than in Bugfix. And finally the Release Candidate is more likely to a nightly build. We will not to intensive testing before publishing these, only quick check if upgrade can be done and if most features work fine.

Origin

The idea originally came out of this thread and after a flurry of positive commentary, it became a working practice shortly therafter.

We plan to make sub-version releases Continue reading

MikroTik CCR1072-1G-8S+ Review (Part 2) – BGP Performance

[adrotate banner=”4″]

Here is Part 1 of the CCR1072-1G-8S+ review in case you missed it!

CCR1072-1G-8S+ Ultimate BGP Performance test

After many days of testing, Part 2 is finally here! Welcome to the stubarea51.net BGP gauntlet. We subjected the CCR1072 to different types of network torture stress testing. Continuing on from our initial review, we chose BGP as the first way to test the limits and capacity of the CCR1072-1G-8S+.

Here is an overview of our lab environment to test the new CCR

• CCR1072-1G-8S+
• CCR1009-8G-1S-1S+
• CRS-125-24G-1S+
• x86 VMs on ESXi 6.0 for upstream BGP peering
• (2) ESXi 6.0 Hosts with 20 Gb (4×10) connectivity
• Multimode 10 gig SFP+ using 50/125 OM3 fiber

All RouterOS devices were loaded with the latest stable code (6.30.1 at the time of testing)

Network Design of the StubArea51 LAB setup for BGP testing

For this series of testing, we took our two ESXi 6.0 hosts and built a number of VMs using RouterOS and Ubuntu to supply the 1.21 Gigawatts 3.6 Million routes we would need to beat up on the CCR1072 for a few days. If you’re not familiar with the RIPE Routing Information Service Continue reading