Unroutable

Author Archives: Unroutable

About password security and credential stuffing

Common Falsehoods About Password Security and Credential Stuffing

TechCrunch published an article that gives class-action lawyers suing 23andMe a mouthpiece to editorialize about password security practices, masquerading as a news article. The upshot of the ~article~ editorial is this:

  1. A large number of 23andMe’s customers were subject to a credential stuffing attack.
  2. Design choices in 23andMe’s site allowed other customer data to be exposed via accounts compromised in the credential stuffing attack.
  3. 23andMe’s mitigations for credential stuffing attacks were inadequate.
  4. 23andMe should be held liable for this.

I want to focus here on the third point: credential stuffing attack mitigations. I’ve worked quite a bit on analyzing large credential stuffing attacks and recommending mitigations for them. I also served as a technical escalation point for customers who had a wide variety of strongly held false beliefs about password security credential stuffing mitigation. In reading various social media responses the 23andMe case, I see all these false beliefs turning up again. Let’s have a look at some.

False: Rate limiting login attempts is a great mitigation

The first question we need to answer here is: what do you mean by rate limiting? Usually there are two main rate limits that people Continue reading

Books 2020

Books 2020

I read at least 41 books in 2020. Here are some highlights.

Intellectual History (?)

This is the category I’m using for my two overall favorites for the year, although it doesn’t quite fit. I love books like this that tell the stories of historic figures in unusual ways.

A Madman Dreams of Turing Machines - Janna Levin

This is generally described as a “philosophical novel”, a description which is both accurate and suitably vague, as the novel is difficult to describe. Janna Levin is famous for a lot of things: she’s possibly the most influential living black hole physicist, an author of a number of well-known books, and a well-known science communicator. This was her first book (I think), and doesn’t seem to be particularly well-known despite being an amazing work.

It’s a fictionalized account of the lives of Kurt Gödel and Alan Turing. The writing is alernately strange and beautiful. I can’t recommend it highly enough.

The Weil Conjectures - Karen Olsson

I read this back to back with the previous book, and they will always be connected in my mind although they don’t intersect except for a few biographical years.

The book explores the Continue reading

2019 Books

2019 Books

I read at least 32 books in 2019. The high count is due primarily to burning through a bunch of mediocre thriller novels on road trips, but I also read a number of really good books in diverse categories. Here are some highlights:

(Intellectual History) At the Existentialist Cafe - Sarah Bakewell

I love wide-ranging intellectual histories, and this fits that description completely. While I am not a huge fan of existentialism as a philosophical movement, its history and personalities are fascinating, and this book does justice to all of it. I particularly enjoyed the chapters about Simone de Beauvoir, Iris Murdoch, and Maurice Merleau-Ponty.

(Mystery) The Infinite Blacktop - Sara Gran

I don’t read a lot of mysteries, but Sara Gran’s Claire DeWitt series is certainly one of my all-time favorites. Detective noir with a touch of magical realism. This is the latest; I recommend reading them in order. I hope there are more to come.

(Micro History) Lady on the Beach - Norah Berg

This memoir is out of print, but worth finding if you have any interest in Pacific Northwest history. Battling alcoholism during the Great Depression, the author Continue reading

Book Report: Machete Season

Overview

Jean Hatzfeld’s Machete Season: the Killers in Rwanda Speak is a much different book than the Pol Pot history that I covered a couple of weeks ago. It’s harder to write about, because it’s just what the title describes: the killers in their own words, interspersed with short contextual explanations of the events surrounding the Rwandan genocide.

Hatzfeld – who has also written two books about the horrific Baltic wars of the 1990s – argues that many of what the mainstream media call genocides should be described as war crimes instead: brutal, unacceptable mass killings of defenseless humans that nonetheless take place in the the context of reducing a population’s ability to wage war. Genocide, he argues, is a term that should be reserved to describe an effort to completely exterminate a population and leave it incapable of ever recovering. In the Rwandan genocide, for example, the Hutu killers often preferred to murder women and children first, because it would leave the Tutsi population less capable of carrying on to the next generation.

History

Modern Rwanda has three main ethnic groups: the majority Hutu, the minority Tutsi, and a small population of Twa jungle-dwelling hunter-gatherers. At the time of Continue reading

Summary of Pol Pot: Anatomy of a Nightmare

Overview

I’ve decided to start a reading project on genocides and violent totalitarian dictators. Most education about these topics in the US is focused around Nazi Germany, or occasionally the Soviet Union under Stalin. While I’d like to come back to those events if I can endure the topic that long, I’m starting with non-Western events.

First up is Pol Pot and the Khmer Rouge, partly because I grew up in the 1980s around a lot of first or second-generation Cambodian and Vietnamese immigrants, but never knew much about the politics behind their flight from Southeast Asia. It’s a particularly strange case of different cultural, political, and historic influences converging in a disastrous way. The term “genocide” has been controversial with reference to the Khmer Rouge regime: while they systematically murdered or starved somewhere between 1.7 and 2.3 million people, for the most part the killings didn’t target a specific racial, ethnic, or religious group. While there were certainly elements of this – as I’ll discuss – Pol Pot’s regime was more about brutal slavery and vicious punishment of any deviance, regardless of the person.

This post is based on the book Pol Pot: Anatomy of a Nightmare Continue reading

CCIE Recert Should Be Like Wilderness Medicine Recert

Last winter I had to recertify CCIE. This time it felt like a negative, adversarial ordeal: reviewing and relearning a lot of stuff that I don’t use in order to justify the sunk costs of obtaining the certification. It’s also a zero-sum game: time spent on recertification is time not spent learning newer, more relevant things. I’ve seen a couple of blog posts (here and here) lately related to this issue. How could recertification be done better?

Outside my professional life, I’ve long been a search and rescue volunteer here in rural Colorado. As part of that, I maintain a Wilderness First Responder (WFR) certification. WFR is a certification for remote emergency medical care that starts as an 80-hour class. It’s required for most types of guiding and outdoor education careers.

Unlike with the CCIE, I always look forward to WFR recertification, even though it’s expensive and I have to take vacation time in order to do it. Why? It’s fun, cooperative, progressive, educational, and encouraging. It’s done as a 16-24 hour class that mixes classroom review, hands-on lab practice, and new material that’s been introduced or updated in the preceding years. This allows recertification candidates to interact Continue reading

Installing netmiko on Windows

Netmiko is a Python module by Kirk Byers that provides a wrapper around the Paramiko SSH module for doing screen scraping and CLI automation on network devices.

Paramiko has some dependencies that make installation on Windows a tad tricky. Here's a quick way to get it done:

  1. Install Anaconda.
  2. From the Anaconda shell, run "conda install paramiko".
  3. From the Anaconda shell, run "pip install scp".
  4. Install git for Windows.
  5. Clone netmiko with "git clone https://github.com/ktbyers/netmiko"
  6. cd into the netmiko directory and run "python setup.py install".
Done! Screen scrape away, and don't forget to hound your vendors for real APIs... :-)

Extracting Traffic from Rolling Capture Files

Every so often I need to extract a subset of traffic from a set of rolling timestamped pcap files. One common place I do this is with Security Onion; one of the great features of SO is its full-packet-capture feature: you can easily pivot from Snort, Suricata, or Bro logs to a full packet capture view, or download the associated pcap file.

But what if you don't have an associated alert or Bro log entry? Or if you're doing pcap on some system that's not as user-friendly as Security Onion, but nonetheless supports rolling captures?

The way I usually do this is with find and xargs. Here's an example of my most common workflow, using timestamps as the filtering criteria for find:

> find . -newerct "16:07" ! -newerct "16:10" | xargs -I {} tcpdump -r {} -w /tmp/{} host 8.8.8.8
> cd /tmp
> mergecap -w merged.pcap *.pcap

Translated:
  1. Find all files in the current directory created after 16:07 but not created after 16:10. This requires GNU find 4.3.3 or later. It supports many different time and date formats.
  2. Using xargs, filter each file with the "host 8.8.8. Continue reading

More ADN (Awk Defined Networking)

Want to know how many IPv4 nodes are in each of your VLANs? Use ADN:

ssh myswitch 'sh arp | i Vlan' | awk '{print $NF}' | sort | uniq -c | sort -rn

     79 Vlan38
     65 Vlan42
     58 Vlan34
     22 Vlan36
     21 Vlan32
     20 Vlan40
      9 Vlan3
      7 Vlan8
      5 Vlan6
      5 Vlan204
      5 Vlan203
      5 Vlan2
      4 Vlan74
      3 Vlan82
      3 Vlan4

ADN – Awk Defined Networking

Because I have yet to transition to a completely software-defined network in which everything configures itself (wink wink), I still have to do tasks like bulk VLAN changes.

Thanks to a recent innovation called ADN, or "AWK Defined Networking", I can do this in a shorter time window that the average bathroom break. For example, I just had a request to change all ports on a large access switch stack that  are currently in VLAN 76 to VLAN 64:

# ssh switch_name.foo.com 'show int status | i _76_' | grep Gi | awk '{print "int ",$1,"n","description PC/Phone","n","switchport access vlan 64"}'
Password: ***

int  Gi1/0/25
 description PC/Phone
 switchport access vlan 64
int  Gi1/0/26
 description PC/Phone
 switchport access vlan 64
[many more deleted]

Then I copied and pasted the results into config mode. Back to lounging on the beach.

Not even any Python skills required!

Filtering .raw fields with Python Elasticsearch DSL High-Level Client

It took me a while to figure out how to search the not_analyzed ".raw" fields created by Logstash in Elasticsearch indices, using the high-level Python Elasticsearch client. Because keyword arguments can't have attributes, Python throws an error if you try it the intuitive way (this assumes you've already set up a client as es and an index as i, as shown in the docs):

Instead, you create a dictionary with your parameters and unpack it using the ** operator:

This produces the Elasticsearch query we want:

Experimenting with the CSR1000v REST API

This all started because we occasionally want to block traffic from an IP address or two for a short time. Our firewall is a pain to configure for this sort of thing: adding a drop for a single IP address literally takes 10 minutes. You have to open a fat client, create an object, add the object to a group, save the config, verify the config, push the config, etc.

I thought that SRTBH (Source-based Real-Time Black Hole) implemented by BGP would be the ticket: fast, easy, and theoretically easy to automate with the REST API in the Cisco Cloud Services Router 1000v. SRTBH is a simple and elegant way of dropping selected traffic on BGP speaking routers. In a nutshell:
  1. You configure a "trigger router" that speaks iBGP with the rest of your BGP-speaking routers (usually your Internet edge or transit routers), but doesn't participate in traffic forwarding.
  2. On each edge/transit router you configure a static route to null0 for an unused /32, usually 192.0.2.1: ip route 192.0.2.1 255.255.255.255 null0
  3. On each edge/transit router you configure loose-mode unicast RPF filtering on your outside interfaces: ip verify source reachable-via any. Continue reading

Problems with kvm-ok in VIRL with VMWare Player

I'm installing Cisco VIRL, and despite following the instructions regarding nested virtualization settings, the kvm-ok command was still complaining. I needed to edit the .vmx file for the VIRL VM and add/edit the following:

monitor.virtual_mmu = "hardware"
monitor.virtual_exec = "hardware"
vhv.enable = "TRUE"
monitor_control.restrict_backdoor = "true"
 
 

My Network Toolkit

A while back, Chris Marget of Fragmentation Needed posted a run-down of his comprehensive and extremely clever network toolkit. Because I'm something of a weight weenie, mine is a lot more slimmed down. I thought I'd post it here:




The contents:

  1. Two random USB drives (in case I need to leave one with somebody).
  2. Single-mode and multi-mode LC fiber loopback plugs.
  3. Rack PDU plug adapter.
  4. Awesome PicQuic compact screwdriver (thanks to Chris's post).
  5. T1 loopback plug (red) (because we still have T1s out here in the boonies).
  6. Cat-6 pass-through plug (white).
  7. Crossover adapter (orange).
  8. Sharpie.
  9. Console setup:
  1. USB-to-DB9 adapter.
  2. DB9-to-RJ45 adapter.
  3. Flat Cat-6 cable.
  4. Rollover adapter.
  5. Velcro tie
  6. Flat Cat-6 cable with velcro tie.
The console setup could probably be improved by adding a DB9 null-modem adapter. The coolest thing (IMO) that I'm missing from Chris's setup is the Bluetooth console adapter -- maybe one day.

The Fenix AA light and Leatherman Skeletool CX almost always live in a pocket rather than the kit and go with me everywhere. The kit all fits into a small zippered case that used to hold a Dell laptop power supply.

My main goal here was to have all the hard-to-find professional stuff in Continue reading

Imposing Artificial Limitations to Develop Skills

I'm a big fan of imposing artificial limitations on yourself in order to aid skill development. Here are some quick ideas:

  • When troubleshooting network devices from the CLI, try not to look at the configuration. Use only "show" or "debug" commands instead. I found this enormously beneficial when practicing for CCIE.
  • When troubleshooting larger operational issues or learning a new environment, try not to log into individual devices at all. Force yourself to use only your network management system, NetFlow, packet captures, or host-based tools like ping, traceroute, or nmap.
  • When learning automation or orchestration skills, force yourself to write scripts, run API calls, or use your favorite orchestration tool to do simple things, even if it doesn't seem like they merit the extra effort.

Simple Python Syslog Counter

Recently I did a Packet Pushers episode about log management. In it, I mentioned some of the custom Python scripts that I run to do basic syslog analysis, and someone asked about them in the comments.

The script I'm presenting here isn't one of the actual ones that I run in production, but it's close. The real one sends emails, does DNS lookups, keeps a "rare messages" database using sqlite3, and a few other things, but I wanted to keep this simple.

One of the problems I see with getting started with log analysis is that people tend to approach it like a typical vendor RFP project: list some requirements, survey the market, evaluate and buy a product to fit your requirements. Sounds good, right? The problem with log analysis is that often you don't know what your requirements really are until you start looking at data.

A simple message counting script like this lets you look at your data, and provides a simple platform on which you can start to iterate to find your specific needs. It also lets us look at some cool Python features.

I don't recommend pushing this too far: once you have a decent idea Continue reading

Python Sets: Handy for Network Data

My Python-related posts seem to get the most reads, so here's another one!

A problem that comes up fairly often in networking is finding the number of occurrences of unique items in a large collection of data: let's say you want to find all of the unique IP addresses that accessed a website, traversed a firewall, got denied by an ACL, or whatever. Maybe you've extracted the following list from a log file:

1.1.1.1
2.2.2.2
3.3.3.3
1.1.1.1
5.5.5.5
5.5.5.5
1.1.1.1
2.2.2.2
...

and you need to reduce this to:

1.1.1.1
2.2.2.2
3.3.3.3
5.5.5.5

In other words, we're removing the duplicates. In low-level programming languages, removing duplicates is a bit of a pain: generally you need to implement an efficient way to sort an array of items, then traverse the sorted array to check for adjacent duplicates and remove them. In a language that has dictionaries (also known as hash tables or associative arrays), you can do it by adding each item as a key in Continue reading

Fun with Router IP Traffic Export and NSM

The Basics
I finally got around to setting up Security Onion (the best network security monitoring package available) to monitor my home network, only to discover that my Cisco 891 router doesn't support support the right form of SPAN. Here's how I worked around it. The topology looks like this:


The 891 router has an integrated 8-port switch module, so the simple case would have been a traditional SPAN setup; something like this:

! vlan 10 is the user VLAN
monitor session 1 source interface vlan 10
monitor session 1 destination interface FastEthernet0

with the server's monitoring NIC connected to FastEthernet0.

The problem is that the 891 doesn't support using a VLAN as a source interface, and because of the way the embedded WAP works, a physical source interface won't work either. Hence, I turned to an obscure feature that's helped me occasionally in the past: Router IP Traffic Export. This is a feature for IOS software platforms that enables you to enable SPAN-like functions for almost any source interface.

The configuration looks like this:

ip traffic-export profile RITE_MIRROR
  interface FastEthernet0
  bidirectional
  mac-address 6805.ca21.2ddd

interface Vlan10
 ip traffic-export apply RITE_MIRROR

This takes all traffic routed across Continue reading
1 2