SOCKMAP – TCP splicing of the future

SOCKMAP - TCP splicing of the future

Recently we stumbled upon the holy grail for reverse proxies - a TCP socket splicing API. This caught our attention because, as you may know, we run a global network of reverse proxy services. Proper TCP socket splicing reduces the load on userspace processes and enables more efficient data forwarding. We realized that Linux Kernel's SOCKMAP infrastructure can be reused for this purpose. SOCKMAP is a very promising API and is likely to cause a tectonic shift in the architecture of data-heavy applications like software proxies.

SOCKMAP - TCP splicing of the future

Image by Mustad Marine public domain

But let’s rewind a bit.

Birthing pains of L7 proxies

Transmitting large amounts of data from userspace is inefficient. Linux provides a couple of specialized syscalls that aim to address this problem. For example, the sendfile(2) syscall (which Linus doesn't like) can be used to speed up transferring large files from disk to a socket. Then there is splice(2) which traditional proxies use to forward data between two TCP sockets. Finally, vmsplice can be used to stick memory buffer into a pipe without copying, but is very hard to use correctly.

Sadly, sendfile, splice and vmsplice are very specialized, synchronous and solve only one part Continue reading

Kubernetes, Kubeadm, and the AWS Cloud Provider

Over the last few weeks, I’ve noticed quite a few questions appearing in the Kubernetes Slack channels about how to use kubeadm to configure Kubernetes with the AWS cloud provider. You may recall that I wrote a post about setting up Kubernetes with the AWS cloud provider last September, and that post included a few snippets of YAML for kubeadm config files. Since I wrote that post, the kubeadm API has gone from v1alpha2 (Kubernetes 1.11) to v1alpha3 (Kubernetes 1.12) and now v1beta1 (Kubernetes 1.13). The changes in the kubeadm API result in changes in the configuration files, and so I wanted to write this post to explain how to use kubeadm 1.13 to set up a Kubernetes cluster with the AWS cloud provider.

I’d recommend reading the previous post from last September first. In that post, I listed four key configuration items that are necessary to make the AWS cloud provider work:

  1. Correct hostname (must match the EC2 Private DNS entry for the instance)
  2. Proper IAM role and policy for Kubernetes control plane nodes and worker nodes
  3. Kubernetes-specific tags on resources needed by the cluster
  4. Correct command-line flags added to the Kubernetes API server, controller Continue reading

Like 4G before it, 5G is being hyped

Just as it did with 4G, AT&T has once again jumped the gun and announced that it was deploying 5G (actually, they’re calling it 5G+) in 12 cities, while deploying so-called “5GE” in other cities, only to be challenged by its three major competitors, who claim that AT&T was merely re-branding a faster version of 4G as 5G and misleading the public about the technology.To read this article in full, please click here(Insider Story)

Like 4G before it, 5G is being hyped

Just as it did with 4G, AT&T has once again jumped the gun and announced that it was deploying 5G (actually, they’re calling it “5G E”) in twelve cities, only to be challenged by its three major competitors, who claim that AT&T was merely re-branding a faster version of 4G as 5G and misleading the public about the technology.To read this article in full, please click here(Insider Story)

Like 4G before it, 5G is being hyped

Just as it did with 4G, AT&T has once again jumped the gun and announced that it was deploying 5G (actually, they’re calling it “5G E”) in twelve cities, only to be challenged by its three major competitors, who claim that AT&T was merely re-branding a faster version of 4G as 5G and misleading the public about the technology.To read this article in full, please click here(Insider Story)

Last Week on ipSpace.net (2019W7)

Last Tuesday we continued the deep dive into new Ansible networking modules functionality introduced in recent software releases (up to 2.7), including a demonstration of a few simple playbooks that collect printouts from network devices and check software version or end-to-end connectivity.

In the second half of the live session we started digging into the intricacies of device configuration management, ending with the truly “fun part”: changing access control lists on Cisco IOS.

The Ansible for Networking Engineers webinar is part of standard ipSpace.net subscription and Building Network Automation Solutions online course.

The why and how of nonnegative matrix factorization

The why and how of nonnegative matrix factorization Gillis, arXiv 2014 from: ‘Regularization, Optimization, Kernels, and Support Vector Machines.’

Last week we looked at the paper ‘Beyond news content,’ which made heavy use of nonnegative matrix factorisation. Today we’ll be looking at that technique in a little more detail. As the name suggests, ‘The Why and How of Nonnegative matrix factorisation’ describes both why NMF is interesting (the intuition for how it works), and how to compute an NMF. I’m mostly interested in the intuition (and also out of my depth for some of the how!), but I’ll give you a sketch of the implementation approaches.

Nonnegative matrix factorization (NMF) has become a widely used tool for the analysis of high dimensional data as it automatically extracts sparse and meaningful features from a set of nonnegative data vectors.

NMF was first introduced by Paatero andTapper in 1994, and popularised in a article by Lee and Seung in 1999. Since then, the number of publications referencing the technique has grown rapidly:

What is NMF?

NMF approximates a matrix \mathbf{X} with a low-rank matrix approximation such that \mathbf{X} \approx \mathbf{WH}.

For the discussion in this paper, we’ll assume that \mathbf{X} is set Continue reading

Connecting Python To Slack For Testing, Development, and Chat

Plugging Python Code Into Slack, Maybe For A Chatbot

The scripting language Python can retrieve information from or publish information to the messaging app Slack. This means you can write a chatbot that puts info into Slack for you, or accepts your queries using Slack as the interface. This is useful if you spend a lot of time in Slack, as I do.

The hard work of integrating Slack and Python has been done already. Slack offers an API, and there are at least two open source Python libraries that make leveraging these APIs in your Python code easy.

When searching for Slack projects using Python, most of the top hits are using Slack’s official python-slackclient. Github reveals that python-slackclient is an active project, with recent commits. In addition, most code examples I turned up are using python-slackclient. But it’s not a preference borne of experience. Maybe you’d prefer an alternate library like slacker.

Securing The Slack App Security Token

The slackclient library is security-conscious. Some other library sample code shows putting the Slack access token right in the source code as a static variable assignment, which is a terrible, horrible, no good, very bad idea. Why? If you publish Continue reading

Cloud Notes: AWS S3

"Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance". At a high level S3 has the following characteristics: Object based storage for static files that do not change. EG: JPEG,...continue reading

Juniper AAA

Junos has a robust authentication, authorization and accounting (AAA) system ensuring authenticated users have access to only the things their permissions allow. Authentication Junos supports two categories of user authentication. Local - On box user database Remote -...continue reading

Juniper OSPFv3

3 steps to configure OSPFv3. Create a router-id (optional) Assign OSPF neighbor facing interfaces to OSPF area Inject routes into OSPF via passive interfaces Configuration Create a router-id. cmd set routing-options router-id 10.255.1.1 Assign OSPF neighbor facing...continue reading

Juniper iBGP

8 steps to configure iBGP. Configure a router-id Configure an autonomous system number Configure transport routing protocol Configure a BGP group and define the peer type Configure a BGP group local address Add neighbors to the peer group Define a routing policy to...continue reading

Worth Reading: Blockchain and Trust

One of the rules of sane social media presence should be don’t ever engage with evangelists believing in a particular technology religion, more so if their funding depends on them spreading the gospel. I was called old-school networking guru from ivory tower when pointing out the drawbacks of TRILL, and clueless incompetent (in more polite words) when retweeting a tweet pointing out the realities of carbon footprint of proof-of-work technologies.

Interestingly, just a few days after that Bruce Schneier published a lengthy essay on blockchain and trust, and even the evangelists find it a bit hard to call him incompetent on security topics. Please read what he wrote every time someone comes along explaining how blockchains will save the world (or solve whatever networking problems like VTEP-to-MAC mappings).

The Week in Internet News: Researchers Develop AI Writing App but Worry about Fake News

Too easy to fake: OpenAI, a research institute in San Francisco, has developed an Artificial Intelligence program to write news articles, but has declined to release a full-featured version of it because of fears that the AI could easily produce fake news, the MIT Technology Review says. OpenAI, associated with AI skeptic Elon Musk, will make only a simplified version publicly available. The institute will publish a research paper outlining its work.

Secure your IoT: Eleven organizations, including the Internet Society and Mozilla, have asked retailers to stop selling Internet-connected devices that don’t meet minimum security and privacy requirements, Techbizweb reports. A letter from the organizations, sent to Target, Walmart, Best Buy, and Amazon, asks them to publicly endorse minimum security and privacy guidelines for Internet of Things devices.

Competing in AI: U.S. President Donald Trump has signed an executive order meant to boost AI development in the country, The Hill reports. The order comes as some AI experts fear the U.S. is losing ground to China. Trump’s order directs federal agencies to prioritize and set aside funding for AI programs.

Broadband for all: Botetourt County in Virginia, where only about 70 percent of residents have access to Internet Continue reading