Experimenting with mozjpeg 2.0

One of the services that CloudFlare provides to paying customers is called Polish. Polish automatically recompresses images cached by CloudFlare to ensure that they are as small as possible and can be delivered to web browsers as quickly as possible.

We've recently rolled out a new version of Polish that uses updated techniques (and was completely rewritten from a collection of programs into a single executable written in Go). As part of that rewrite we looked at the performance of the recently released mozjpeg 2.0 project for JPEG compression.

To get a sense of its performance (both in terms of compression and in terms of CPU usage) when compared to libjpeg-turbo I randomly selected 10,000 JPEG images (totaling 2,564,135,285 bytes for an average image size of about 256KB) cached by CloudFlare and recompressed them using the jpegtran program provided by libjpeg-turbo 1.3.1 and mozjpeg 2.0. The exact command used in both cases was:

jpegtran -outfile out.jpg -optimise -copy none in.jpg

Of the 10,000 images in cache, mozjpeg 2.0 failed to make 691 of them any smaller compared with 3,471 for libjpeg-turbo. So mozjpeg 2.0 was significantly better at recompressing images.

On average Continue reading

What’s the Big Deal about Big Data?

Title

It goes without saying that knowledge is power. It gives one the power to make informed decisions and avoid miscalculation and mistakes. In recent years the definition of knowledge has changed slightly. This change is the result of increases in the ease and speed in computation as well as the shear volume of data that these computations can be exercised against. Hence, it is no secret that the rise of computers and the Internet has contributed significantly to enhance this capability.
The term that is often bantered about is “Big Data”. This term has gained a certain mystique that is comparable to cloud computing. Everyone knows that it is important. Unless you have been living in a cave, you most certainly have at least read about it. After all, if such big names as IBM, EMC and Oracle are making a focus of it then it must have some sort of importance to the industry and market as a whole. When pressed for a definition of what it is however, many folks will often struggle. Note that the issue is not that it deals with the computation of large amounts of data as its name implies, but more so that Continue reading

[SDN Protocols] Part 2 – OpenFlow Deep-Dive

This entry is part 3 of 4 in the series SDN Protocols

In the last post, I introduced you to the concept of control plane abstraction, specifically the OpenFlow implementation. I talked about how OpenFlow allows us to specify the flows that we want to be programmed into the forwarding plane, from outside the forwarding device itself. We can also match on fields we typically don’t have access to in traditional networking, since current hardware is optimized for destination-based forwarding.

In this post, I plan to cover quite a few bases. The goal of this post is to address the main concepts of OpenFlow’s operation, with links to find out more. With this post, you’ll be armed with the knowledge of what OpenFlow does and doesn’t do, as well as resources to dive even deeper.

NOTICE: This blog post was written referencing the specification and implementations of OpenFlow 1.3 – since this version, some aspects of the protocol may have changed (though it is likely the fundamentals discussed here will be mostly the same)

 

OpenFlow Tables

The OpenFlow specification describes a wide variety of topics. For instance, the protocol format that’s used to communicate with an OpenFlow switch Continue reading

HP Network Simulator

Is released by HP the new version of simulator for network devices (Switches and Routers) based on Comware 7 OS. This software is called HP Network Simulator.

I’m very happy about that because I have waited for a long time to run some commands and features in a lab environment.

The simulator is based on Cowmare 7 ( most commands are very similar to OS 5 version)

List below the link for download, inside there are some instructions for installing and configuring the topology that you wish.

http://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdDetails/?swItem=nw_130365_1&ac.admitted=1403627434906.876444892.199480143

HP Network Simulator

 
Despite of the software working on GUI mode, the topology design must be made via text in a configuration file (also explained in the software manual).

Enjoy it, share and comment. It’s a good time to celebrate.

If the link is broken, please leave a comment.

Show 198 – Kirk Byers on Network Automation with Python & Ansible

Kirk Byers has been doing network automation work for quite a while now. I’ve been following his Pynet mailing list, where he teaches list members in a series of structured lessons how to code in Python, harnessing the scripting language’s power for network automation. I met Kirk at Cisco Live US, and we got to […]

Author information

Ethan Banks

Co-host at Packet Pushers

Ethan Banks, CCIE #20655, has been managing networks for higher ed, government, financials and high tech since 1995. Ethan co-hosts the Packet Pushers Podcast, which has seen over 2M downloads and reaches over 10K listeners. With whatever time is left, Ethan writes for fun & profit, studies for certifications, and enjoys science fiction. @ecbanks
TwitterGoogle+LinkedIn

The post Show 198 – Kirk Byers on Network Automation with Python & Ansible appeared first on Packet Pushers Podcast and was written by Ethan Banks.

[SDN Protocols] Part 2 – OpenFlow Deep-Dive

In the last post, I introduced you to the concept of control plane abstraction, specifically the OpenFlow implementation. I talked about how OpenFlow allows us to specify the flows that we want to be programmed into the forwarding plane, from outside the forwarding device itself. We can also match on fields we typically don’t have access to in traditional networking, since current hardware is optimized for destination-based forwarding. In this post, I plan to cover quite a few bases.

[SDN Protocols] Part 2 – OpenFlow Deep-Dive

In the last post, I introduced you to the concept of control plane abstraction, specifically the OpenFlow implementation. I talked about how OpenFlow allows us to specify the flows that we want to be programmed into the forwarding plane, from outside the forwarding device itself. We can also match on fields we typically don’t have access to in traditional networking, since current hardware is optimized for destination-based forwarding. In this post, I plan to cover quite a few bases.

[SDN Protocols] Part 2 – OpenFlow Deep-Dive

In the last post, I introduced you to the concept of control plane abstraction, specifically the OpenFlow implementation. I talked about how OpenFlow allows us to specify the flows that we want to be programmed into the forwarding plane, from outside the forwarding device itself. We can also match on fields we typically don’t have access to in traditional networking, since current hardware is optimized for destination-based forwarding. In this post, I plan to cover quite a few bases.

HTIRW: DNS Security

In the last few posts on this topic, we’ve talked about the various bits and parts of the DNS system, from who pays to how it works to DNS tools. This time, we’re going to finish off DNS in this (probably record breaking for Packet Pushers) series, and talk about some various aspects of DNS […]

Author information

Russ White

Russ White
Principle Engineer at Ericsson

Russ White is a Network Architect who's scribbled a basket of books, penned a plethora of patents, written a raft of RFCs, taught a trencher of classes, and done a lot of other stuff you either already know about, or don't really care about. You want numbers and letters? Okay: CCIE 2635, CCDE 2007:001, CCAr, BSIT, MSIT (Network Design & Architecture, Capella University), MACM (Biblical Literature, Shepherds Theological Seminary). Russ is a Principal Engineer in the IPOS Team at Ericsson, where he works on lots of different stuff, serves on the Routing Area Directorate at the IETF, and is a cochair of the Internet Society Advisory Council. Russ will be speaking in November at the Ericsson Technology Day. he recently published The Art of Network Architecture, is currently working on a new book in the area Continue reading

Recognizing IP MTU Issues

At some point, Network engineers will likely face some type of issue with MTU or maximum transmittable unit. Their first experience with this may be an eye opening and time consuming effort. After resolving the issue, those with a thirst for knowledge will take the necessary time to understand the issue.

MTU problems are most often seen when Path MTU Discovery, or PMTUD, fails to function. This is the process by which one end host determines the largest possible packet size to another station on the network. Symptoms of this type of issue include two devices having proven reachability, but applications fail to work in a way that indicates a network issue. Some applications may even crash or hang the system.

Symptoms of PMTUD Failure

  • Hosts may be able to ping one another
  • Service/Port may prove accessible using telnet
  • Severe and persistent application issues
  • Partial page loads
  • Either host appearing to hang

Understanding IP MTU

To understand the problems of Path MTU Discovery, it is first necessary to understand how MTU relates to the conversation. MTU, or maximum transmittable unit, is the maximum chunk of data that a given interface can transmit. The type of data receiving our attention is IP Continue reading

Some Internet Measurements

At APNIC Labs we’ve been working on developing a new approach to navigating through some of our data sets the describe aspects of IPv6 deployment, the use of DNSSEC and some measurements relating to the current state of BGP.

Handling “Multiples” in Cisco NX-API with Python

A few weeks ago, I was working with the NX-API currently found on Cisco’s Nexus 9000 series switches, and ran into some peculiar behavior.

NX-API returns all information in terms of Tables and Rows. For a specific example, let’s look at what NX-API returns when I ask the switch for running OSPF processes:

There’s actually a lot more information in this snippet that pertains to the OSPF process itself, but I have omitted it for brevity. This specific example focuses on the section that describes the areas in this OSPF process.

{
  "ins_api": {
    "sid": "eoc",
    "type": "cli_show",
    "version": "0.1",
    "outputs": {
      "output": {
        "code": "200",
        "msg": "Success",
        "input": "show ip ospf",
        "body": {
          "TABLE_ctx": {
            "ROW_ctx": {
              ### OSPF process information omitted for brevity ###
              "TABLE_area": {
                "ROW_area": {
                  "age": "P15DT15H27M6S",
                  "loopback_intf": "1",
                  "passive_intf": "0",
                  "last_spf_run_time": "PT0S",
                  "spf_runs": "9",
                  "lsa_cnt": "5",
                  "no_summary": "false",
                  "backbone_active": "true",
                  "stub": "false",
                  "aname": "0.0.0.0",
                  "total_intf": "2",
                  "auth_type": "none",
                  "act_intf": "2",
                  "nssa": "false",
                  "lsa_crc": "0x18d91"
                }
              }
            }
          }
        }
      }
    }
  }
}

NXAPI uses a special tag that starts with TABLE, and within that, tag(s) that start with ROW, whenever it needs to describe something that would normally be Continue reading

Working with VMware NSX – Logical networking

In my last post, we wrapped up the base components required to deploy NSX.  In this post, we’re going to configure some logical routing and switching.  I’m specifically referring to this as ‘logical’ since we are only going to deal with VM to VM traffic in this post.  NSX allows you to logically connect VMs at either layer 2 or layer 3.  So let’s look at our lab diagram…

image

If you recall, we had just finished creating the transport zones at the end of the last post.  The next step is to provision logical switches.  Since we want to test layer 2 and layer 3 connectivity, we’re going to provision NSX in two separate fashions.  The first method will be using the logical distributed router functionality of NSX.  In this method, tenant 1 will have two logical switches.  One for the app layer and one for the web layer.  We will then use the logical distributed router to allow the VMs to route to one another.  The 2nd method will be to have both the web and app VMs on the same logical layer 2 segment.  We Continue reading

Quick’n’dirty Nslookup BASH Script

I’m always wondering if the addresses I’m assigning to interfaces aren’t already in DNS. So I came up with a little BASH script that takes a list of IP addresses and performs an nslookup on them to ensure they’re not in use already:

$nslookup < input-filename > output-filename

The addresses in the input file are carriage return delimited.

A better use for this would be to check if DNS entries already have an IP address assigned to them.


Handling “Multiples” in Cisco NX-API with Python

A few weeks ago, I was working with the NX-API currently found on Cisco’s Nexus 9000 series switches, and ran into some peculiar behavior. NX-API returns all information in terms of Tables and Rows. For a specific example, let’s look at what NX-API returns when I ask the switch for running OSPF processes: There’s actually a lot more information in this snippet that pertains to the OSPF process itself, but I have omitted it for brevity.

Handling “Multiples” in Cisco NX-API with Python

A few weeks ago, I was working with the NX-API currently found on Cisco’s Nexus 9000 series switches, and ran into some peculiar behavior. NX-API returns all information in terms of Tables and Rows. For a specific example, let’s look at what NX-API returns when I ask the switch for running OSPF processes: There’s actually a lot more information in this snippet that pertains to the OSPF process itself, but I have omitted it for brevity.
1 … 3,723 3,724 3,725 3,726 3,727 … 3,842