dave

Author Archives: dave

Data-Driven RESTful API Testing for Java

This post is a Zombie that I'm resurrecting from my drafts. I"m not doing any Java these days, but hopefully this post might be useful to somebody

In my quest to get better code coverage for the OVSDB project in OpenDaylight I started to look at increasing coverage for the REST API. It's pretty difficult to test this in an efficient way (lines of code) and frameworks like Robot would have been easier to use. The disadvantage with using an external test framework is that code coverage (using a plugin like JaCoCo) would not be logged. Therefore I harnessed my Junit-Jitsu and found a solution that lives in the JVM

The Scenario

Lets take a very simple example REST API

GET, PUT: /v2/foo

Step 1: The Solution Components

The solution uses the following components

The parameterized runner will run run a test multiple times given a bunch of parameters. This way we can write one test, specifiy our parameters in YAML and let JUnit do the hard work!

Step 2: Writing the YAML file

Here's a sample YAML file:

---
- name: testGetAllFoo
  operation: GET
  uri Continue reading

Taming Netlink

Netlink is a Linux Kernel interface that is used to talk Networking between Kernel processes, or from userland to Kernel processes. This post documents how I tamed Netlink and make working with it in Golang easier.

If you want to create an interface, set an MTU, set an IP address etc... on a Linux machine then you have to talk Netlink to do it. For Golang, there are a number of libraries, but this one seems to be gaining a lot of traction recently. Honorable mentions go to Tenus and Docker's libcontainer.

Now, this library is good, but it isn't complete and some of the behavior I need is missing. So how does one go about implementing it you might ask?

Here's my process:

1. Browse the iproute2 source code

The iproute2 utility is insanely useful - it replaces ifconfig and route for those with memories long enough. In my case, the feature I wanted was here and I could take a look at the exact message structure required :)

2. Debugging Netlink

Once I'd written my tests and code, I wasn't surprised to see that things were not working. strace will get you part of the way, but it's Continue reading

Experimenting with Kernel-powered Open vSwitch and Docker

I've been thinking about running Docker on CoreOS and Project Atomic lately... While the deployment model would be pretty different to what we are used to, I have 50% of the work already done in docker-ovs so I was interested to see if my containers would work on a system with the Open vSwitch kernel module loaded...

As I'm a Mac User, I use boot2docker for all my docker-related things. It's also pretty easy to change the kernel config to allow the Open vSwitch module to be loaded.

  1. Install boot2docker

  2. Clone my fork

  3. git checkout openvswitch

  4. Build the iso

    docker build -t boot2docker . && docker run --rm boot2docker > boot2docker.iso

  5. Run boot2docker with the new iso

    boot2docker destroy
boot2docker init --iso="`pwd`/boot2docker.iso"
boot2docker up

  6. Load the Open vSwitch kernel module

    boot2docker ssh
sudo modprobe openvswitch
exit

  7. Run an Open vSwitch container

    docker run -t -i --privileged=true davetucker/docker-ovs:2.1.2 /bin/sh
export OVS_RUNDIR=/var/run/openvswitch
sed -i s/nodaemon=true/nodaemon=false/g /etc/supervisord.conf
supervisord

  8. Test it out

    ovs-vsctl add-br br0
ovs-vsctl show
# This didn't work before
ovs-dpctl show

This isn't a thorough test. I'd like to create some traffic and see the Continue reading

Why I gave up Networking for Software

It's now been 3 months since I transitioned from Networking to Software. This is a retrospective piece on my reasons for giving up on Networking.

Introduction

You might be reading this thinking:

"another networking guy moving to software... network engineering is doomed".

If you are, stop thinking right now. There is one important thing about my story that is very different. I've been writing software for longer than I have been doing networking albeit not in a professional capacity. Software Engineering is where my passion lies right now and let me explain why...

My Reasons

1. DevOps

DevOps for Networking is still, very slowly, becoming reality. Elsewhere DevOps is very much in full swing. Tools like:

Vagrant, Packer, Puppet, Chef, SaltStack, Ansible, Fig, Docker, Jenkins/TravisCI, Dokku, Heroku, OpenShift (the list goes on)...

have redefined how I work and being in an environment where I can build things with them day to day is a dream come true for me.

I get gersburms just thinking about building Continous Integration/Continous Delivery Pipelines, Automated creation of Dev/Test environments and Configuration as Code.

2. SDN

Software-Defined Networking was the turning point in my career. It enabled me to make the switch in career paths Continue reading

Is SDN API directionality absurd?

I was finally catching up on a number of posts I'd saved to read later and noticed the prevalent use of "Northbound" and "Southbound". I'm now starting to question whether these terms are necessary or accurate.

Dictionary definition

Let's start with the Oxford English Dictionary definition of these terms.

northbound | ˈnɔːθbaʊnd | adjective travelling or leading towards the north: northbound traffic.

southbound | ˈsaʊθbaʊnd | adjective travelling or leading towards the south: southbound traffic | the southbound carriageway of the A1.

As our interfaces are static and can't travel one can assume the intent of these adjectives in our context is to indicate that the interfaces are leading in the specified direction.

On Directionality as a descriptor

Categorizing an API by directionality is rather perplexing IMHO.

Specify directionality without a reference point is misleading For example, OVSDB is a northbound API for Open vSwitch but southbound API for an SDN controller.

For SDN controllers, there are two types of interfaces:

User-Facing or Application-Facing (formerly Northbound) This API is designed to expose higher-order functions in such a way that they can easily be consumed by humans and programmers. By this logic, we can include any " API's" or language bindings Continue reading

Response – Do We Need To Redefine Open?

Tom Hollingsworth wrote a great post on whether or not we need to redefine "Open". My response was too long for a comment, so here it is!

Open Source vs Free Software

The first item is just a point of clarification. While the terms "Open Source" and "Free Software" are often used interchangeably there is a difference.

The two terms describe almost the same category of software, but they stand for views based on fundamentally different values. Open source is a development methodology; free software is a social movement. - Richard Stallman

You can read the full article here but the TL;DR version is that while a high percentage of Open Source software is Free Software, the definition of Open Source is less strict about guaranteeing freedoms.

...with that out of the way, let's move to "open"

On "open" and "openness"

I like the Wikipedia description of "openness":

Openness is an overarching concept or philosophy that is characterized by an emphasis on transparency and free unrestricted access to knowledge and information as well as collaborative or cooperative management and decision making rather than a central authority. - Wikipedia

It highlights some key terms which our "open" things should be adhering Continue reading

Fun with Fig (and Docker)

I first heard of Fig when I read about Docker acquiring Orchard, a container hosting service, back in July. Last week I finally got to read a little more about it and it just so happens it is the missing piece of the puzzle in a couple of projects that I am working on right now!

What does Fig do?

The best way I would describe Fig is like Vagrant for Docker containers. If you don't know what Vagrant is, or aren't using it then you are missing out!

Fig lets you bring up and tear down docker containers (single or multiple) with a simple command. To do this, you express the desired configuration in a YAML file, fig.yml.

Getting started

On OSX, you'll need to have an accessible Docker environment. The easiest way to do this is with Homebrew and boot2docker

brew install docker
brew install boot2docker
boot2docker init
boot2docker start
export DOCKER_HOST=tcp://$(boot2docker ip 2>/dev/null):2375
# Install Fig
pip install fig

If you don't have Python and/or pip installed you may want to install the fig binary

Writing a Fig file for Open vSwitch

Let's say you are doing some integration Continue reading

Building a router with Open vSwitch

As part of my work in OpenDaylight, we are looking at creating a router using Open vSwitch... Why? Well OpenStack requires some limited L3 capabilities and we think that we can handle those in a distributed router.

Test Topology

My test topology looks like this:

Test Topology

We have a host in an external network 172.16.1.0/24, one host in an internal network 10.10.10.0/24 and two hosts in another internal network 10.10.20.0/24.

As such, The hosts in the 10.x.x.x range should be able to speak to each other, but should not be able to speak to external hosts.

The host 10.10.10.2 has a floating IP of 172.16.1.10 and should be reachable on this address from the external 172.16.1.0/24 network. To do this, we'll use DNAT for traffic from 172.16.1.2 -> 172.16.1.10 and SNAT for traffic back from 10.10.10.2 -> 172.16.1.2

If you'd like to recreate this topology you can checkout the OpenDaylight OVSDB project source on GitHub and:

vagrant up mininet
vagrant ssh mininet
cd /vagrant/resources/mininet
sudo mn --custom  Continue reading

Rolling back to Maven 3.0.X on OSX (Homebrew)

The current version of Maven in Homebrew at the time of writing is 3.2.2

This is great... unless one of the plugins in your project doesn't support it and then you have to downgrade :(

Fortunately it's not too painful

brew uninstall maven
brew tap homebrew/versions
brew install maven30

@dave-tucker

Creating a Vagrant base box for RHEL with Bento

I <3 the Opscode Bento project. I use the Amazon S3 hosted images for pretty much all of my Vagrant boxes. When I started to use RHEL, I didn't want to make an exception... Fortunately Bento allows you to build your own RHEL, OSX or Windows boxes using Packer. This is how I built my RHEL 6.4 x64 box, but this process should work for any other box you want to build manually...

Install Packer

If you are on OSX, you can install Packer using Homebrew:

brew tap homebrew/binary
brew install packer

If not, you can follow the instructions on the Packer's website

Pre-Reqs

Clone bento and add your RHEL Server ISO

https://github.com/opscode/bento.git
cd bento/packer
mkdir iso
#cp your rhel-server iso here... it should be named rhel-server-6.5-x86_64-dvd.iso

Build your RHEL box

packer build -only=virtualbox-iso -var 'mirror=file:///`pwd`/iso' rhel-6.5-x86_64.json

This takes a little while so go and get some coffee...

Add to Vagrant

cd ..
vagrant box add rhel65x64 builds/virtualbox/opscode_rhel-6.5_chef-provisionerless.box

Then to test it, create a new Vagrant image that uses the new rhel65x64 image

cd <a tmp dir>
vagrant init
sed -i '' 's/config.vm.box = "base"/config.vm.box = "rhel65x64"/g' Vagrantfile
vagrant up
vagrant ssh
vagrant destroy

Fin.

@dave_tucker

My pinboard bookmarks for 17 May 2014

Things of note that were discovered on the web this week.

CriticMarkup

Can't do editing in Markdown eh? Obviously you haven't seen CriticMarkup. Looks very cool indeed!

markdown

java - Excluding tests from being run in IntellIJ - Stack Overflow

This is how to exclude integration tests being run in IntelliJ I've been using this for unit testing the OpenDaylight OVSDB library with the regex ^(?!(^.*(IT).*?$)).*$ and it works well!

intellij, java, testing, junit

Maven Troubleshooting FAQs

A presentation with troubleshooting Maven issues which is a necessary skill if you plan to do any development in Java.

java, maven

OpFlex – is the abstraction in the right place?

It's been a few weeks since Cisco announced OpFlex and I've just finished gathering my thoughts...

What is OpFlex

It's a protocol for delivering policy to endpoints. Policy is declarative, based on promise theory and can therefore scale well vs. imperative models (like OpenFlow)

What does this all mean

Kyle Mestery assures us that OpFlex is not an OpenFlow killer and while I agree, I'm starting to have my doubts. Vendors have been dragging their heels when it comes to implementing OpenFlow due to its pipeline and table structures not being a good fit to current hardware. OpFlex offers them a way out as they no longer need to care "how" something is implemented, just that the "promise" is kept (read: policy is enforced).

I can see Cisco deploying OpFlex across their entire portfolio and declaring victory - we've got SDN. Who cares about OpenFlow? This begs the question of whether OpFlex is just a move from Cisco to protect it's core business?

Why I care about OpenFlow and you should too

The future of networking is "Open"

The key benefit to using OpenFlow is disaggregation. It's beneficial to everybody for the proprietary stack to be broken down to allow Continue reading

SSH for Python – In search of API perfection

My mission is simple: Establish an SSH connection to a device and run some commands in as few lines as possible. The contenders? Paramiko, Spur and Fabric.

The Scenario

I have a network device, 192.168.1.254. I want to log in via SSH with a username of dave and password of p@ssword123. Once logged in, I want to execute the command display version and print the result.

Now to the code...

The Code

Paramiko

Paramiko is the go to SSH library in Python. Let's see how it shapes up in the simple scenario:

import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy())
client.connect("192.168.1.254", username="dave", password="p@ssword123")
stdin, stdout, stderr = client.exec_command('display version')
for line in stdout:
    print line.strip('n')
client.close()

8 lines of code. The API here is very powerful, but requires me to put up some scaffolding code (Key Management) before I actually get around to connecting an executing my command. That said, it gets the job done.

Spur

Spur is a wrapper around Continue reading

NETCONF, YANG, RESTCONF and NetOps in an SDN World

I've had some great discussion with the OpenDaylight OVSDB team around NETCONF, YANG, RESTCONF and what network operations will look like in an SDN world. This post summarizes where my head is at on this subject.

Background Info

What is NETCONF

NETCONF is defined in RFC 6241 which describes it as follows:

The Network Configuration Protocol (NETCONF) defined in this document provides mechanisms to install, manipulate, and delete the configuration of network devices. It uses an Extensible Markup Language (XML)-based data encoding for the configuration data as well as the protocol messages. The NETCONF protocol operations are realized as remote procedure calls (RPCs).

It's not a new technology, as work started on this approximately 10 years ago, but what it gives us is an extensible and robust mechanism for managing network devices.

NETCONF understands the difference between configuration data and state data. As somebody who has been bitten by trying to perform a create operation and faced validation issues as I've mistakenly sent (or worse, edited) a read-only field in a request, I feel this is really valuable.

Another great thing from an operations perspective is the ability to test/validate configuration before it's applied to the device. NETCONF allows you Continue reading

Installing KVM, Libvirt and Open vSwitch on Fedora

In my NetOps to DevOps Training Plan I mentioned installing KVM, Libvirt and Open vSwitch. I did this a few weeks ago and documented it to produce this tutorial. My motivation was to replace my VMware environment at home with something Open Source. I am also a strong believer in "eat your own dog food" and as a lot of the work I am doing in the Open Source community centers around these 3 technologies, I should get used to using them every day...

Prerequsites

Before we get started, I'll assume that you already have a Fedora Minimal Installation that you are ready to work on...

Installing the packages

sudo yum install -y @standard @virtualization openvswitch

That was easy wasn't it!

@standard installs some useful utilities and @virtualization installs libvirt + KVM

I'm sure you can guess what openvswitch does.

Configuration

Now here comes the fun part!

Configure the services

# Disable NetworkManager
sudo systemctl stop NetworkManager.service
sudo systemctl disable NetworkManager.service

# Enable "Proper" Networking
sudo systemctl enable network.service
sudo systemctl start network.service

# Enable the Open vSwitch service
sudo systemctl enable openvswitch.service
sudo systemctl start openvswitch.service

Setting up Networking with Open vSwitch

Our Continue reading

Network Flavors in OpenDaylight

OpenStack uses the concept of flavors to define compute/storage configurations that vary in terms of resource consumption. When we start to consume the network as a resource pool, we need a similar concept. This post explains how this is achieved in the context of the Flavors application in the OpenDaylight project

Fallacies of Distributed Computing

If you haven't read the Fallacies of Distributed Computing you should. Specifically points 2 and 3.

2) Latency is Zero 3) Bandwidth is Infinite

The truth is that not all networks are created equally. We may have some pools of 2:1, 4:1, 8:1 oversubscribed fabric. We may links between data centers at varying bandwidths and costs to the business.

In a cloud network, we can increase our ROI by charging out portions of the network based on their actual cost to the business, but to do this we need to model the network appropriately. Applications also need to consume networking in an abstract way, and the concept of using Flavors as this abstraction is an interesting prospect.

Problem Statement

The problem is that todays cloud networking solutions (OpenDaylight included) are based on "Overlay Networks". The Overlay Network solution only assumes L3 reachability between tunnel endpoints Continue reading

My Dotfiles. Now on Github

Dotfiles are all those . files that sit in your ~ and customize your system. Here are mine.

Until a few weeks ago I had no idea that people hosted their dotfiles on GitHub, and now I am one of them... There are two reasons for this:

  1. For sharing awesome customizations with the community
  2. As a backup plan. I can now clone this repo and customize a new system.

To point 2, I've gone one step further than just including my dotfiles. I've also included all of my system customizations and installers for the packages I use most. Why a new repository and not a fork you might ask? The honest answer is that there wasn't one repo that fit my tastes well enough so I ended up taking what I considered to be the "best" elements from a number of other repos. This is still a work-in-progress and I am comitting changes every time I find somehting new and exciting, or tire of a specific setting.

What I like about my dotfiles:

  • Uses the Base16 Ocean theme
  • Nicely Organised
    • Top-level folder for each function
    • Files with extension .symlink are symlinked to the home folder
  • Multi-Platform MakeFile-based installer

A NetOps to DevOps Training Plan

In one of my rants, I asked people to kindly stop with the "All Network Guys will Need to be Programmers" FUD. My recommendation was basically for Networkers to be open to change, and to start broadening their horizons. DevOps is coming to networking and that is a FACT. You might be wondering what skills a Network DevOps Engineer needs and here I attempt to answer that.

It's still about NETWORKING

I'm going to state this upfront here. You need to be good at Networking for any of the other skills here to be useful. Continue along vendor certification tracks, follow the IETF, join NANOG, experiment with new technologies. This is all invaluable.

Software Engineering Fundamentals

A lot of the DevOps skills have roots in Software Engineering. Being a Network Guy ™ this may seem like a little bit of a paradigm shift but here's something cool. Would you believe that some of these software engineering concepts have more to do with engineering best practice than with software, and are in fact relevant to the work you are doing today? Also, your SysAdmin buddies already know this and started their DevOps pilgrimage a while ago.

Unit/Functional/Integration Testing, Version Control, Agile, Continue reading

Migrating from WordPress to Pelican on PaaS – Part 3

The final installment in this three part series. This covers installing Dokku and publishing your pelican blog to you new Docker-powere mini-Heroku.

Part 3: Publishing to PaaS with Dokku

The Plan

If you haven't read Part 1 or Part 2 yet, this should give you some background as to what I'm doing, why I'm doing it and how I built it. In this installment I'll focuse on the publishing side of things.

Hosting

My former blog was hosted on a Linode 1024 VPS, which had a healthy 1GB RAM. I've been very happy with Linode and would recommend them to anybody who needs hosting, but for the convenience of having prebuild Ubuntu images with Dokku installed, I opted to host my blog with DigitalOcean. They have a full tutorial on their website that makes this very easy to set up.

One of the big benefits of using a static site generator is that the memory requirement is a lot less than Apache+PHP or Nginx+PHP. I'm hosting my site now on a $5/month VM from DigitalOcean which is a $15/month saving on my Wordpress site.

Before publishing...

Once you have your Dokku installation set up, you can push your application to Continue reading

1 2 3