When TCP sockets refuse to die

While working on our Spectrum server, we noticed something weird: the TCP sockets which we thought should have been closed were lingering around. We realized we don't really understand when TCP sockets are supposed to time out!

In our code, we wanted to make sure we don't hold connections to dead hosts. In our early code we naively thought enabling TCP keepalives would be enough... but it isn't. It turns out a fairly modern TCP_USER_TIMEOUT socket option is equally as important. Furthermore it interacts with TCP keepalives in subtle ways. Many people are confused by this.

In this blog post, we'll try to show how these options work. We'll show how a TCP socket can timeout during various stages of its lifetime, and how TCP keepalives and user timeout influence that. To better illustrate the internals of TCP connections, we'll mix the outputs of the tcpdump and the ss -o commands. This nicely shows the transmitted packets and the changing parameters of the TCP connections.

SYN-SENT

Let's start from the simplest case - what happens when one attempts to establish a connection to a server which discards inbound SYN packets?

$  Continue reading

Stuff The Internet Says On Scalability For September 20th, 2019

Wake up! It's HighScalability time:

What could be simpler? (duckbillgroup)

Do you like this sort of Stuff? I'd love your support on Patreon. I wrote Explain the Cloud Like I'm 10 for people who need to understand the cloud. And who doesn't these days? On Amazon it has 54 mostly 5 star reviews (125 on Goodreads). They'll learn a lot and likely add you to their will.

Number Stuff: 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Weekly Wrap: VMware CEO States IBM Paid Too Much for Red Hat

Docker + Arm Virtual Meetup Recap: Building Multi-arch Apps with Buildx

Docker support for cross-platform applications is better than ever. At this month’s Docker Virtual Meetup, we featured Docker Architect Elton Stoneman showing how to build and run truly cross-platform apps using Docker’s buildx functionality. 

With Docker Desktop, you can now describe all the compilation and packaging steps for your app in a single Dockerfile, and use it to build an image that will run on Linux, Windows, Intel and Arm – 32-bit and 64-bit. In the video, Elton covers the Docker runtime and its understanding of OS and CPU architecture, together with the concept of multi-architecture images and manifests.

The key takeaways from the meetup on using buildx:

• Everything should be multi-platform
• Always use multi-stage Dockerfiles 
• buildx is experimental but solid (based on BuildKit)
• Alternatively use docker manifest — also experimental

Not a Docker Desktop user? Jason Andrews, a Solutions Director at Arm, posted this great article on how to setup buildx using Docker Community Engine on Linux. 

Check out the full meetup on Docker’s YouTube Channel:

You can also access the demo repo here. The sample code for this meetup is from Elton’s latest book, Learn Docker in a Month of Lunches, an accessible task-focused Continue reading

Community Spotlight – Terry Slattery

The post Community Spotlight – Terry Slattery appeared first on Network Collective.

It’s A Wireless Problem, Right?

How many times have your users come to your office and told you the wireless was down? Or maybe you get a phone call or a text message sent from their phone. If there’s a way for people to figure out that the wireless isn’t working they will not hesitate to tell you about it. But is it always the wireless?

Path of Destruction

During CWNP Wi-Fi Trek 2019, Keith Parsons (@KeithRParsons) gave a great talk about Tips, Techniques, and Tools for Troubleshooting Wireless LAN. It went into a lot of detail about how many things you have to look at when you start troubleshooting wireless issues. It makes your head spin when you try and figure out exactly where the issues all lie.

However, I did have to put up a point that I didn’t necessarily agree with Keith on:

Juniper JNCIE-ENT Refresh

A few months ago Juniper announced that the JNCIE-ENT lab exam would be getting a much-needed refresh. On November 1st …

Continue reading →

The post Juniper JNCIE-ENT Refresh appeared first on Fryguy's Blog.

Cascadia Code | Windows Command Line Tools For Developers

Another free and open monospaced font for code development this time from Microsoft. A key differentiator is the inclusion of ligatures for programming symbols (see below). Ligature support is rare among text editors and very rare for TTF encoded fonts. Its more common to see OTF ligatures supported. Also, no italics support yet. Creating fonts […]

The post Cascadia Code | Windows Command Line Tools For Developers appeared first on EtherealMind.

Opinionated Automation: Packaged, Extensible & Closed Systems

Network engineers for the last twenty years have created networks from composable logical constructs, which result in a network of some structure. We call these constructs “OSPF” and “MPLS”, but they all inter-work to some degree to give us a desired outcome. Network vendors have contributed to this composability and network engineers have come to expect it by default. It is absolute power from both a design and an implementation perspective, but it’s also opinionated. For instance, spanning-tree has node level opinions on how it should participate in a spanning-tree and thus how a spanning-tree forms, but it might not be the one you desire without some tweaks to the tie-breaker conditions for the root bridge persona.

Moving to the automated world primarily means carrying your existing understanding forward, adding a sprinkle of APIs to gain access to those features programmatically and then running a workflow, task or business process engine to compose a graph of those features to build your desired networks in a deterministic way.

This is where things get interesting in my opinion. Take Cisco’s ACI platform. It’s closed and proprietary in the sense of you can’t change the way it works internally. You’re lumped with a Continue reading

Privacy Regulations Are Evolving: Are Organizations Ready?

Privacy statements are both a point of contact to inform users about their data and a way to show governments the organization is committed to following regulations. On September 17, the Internet Society’s Online Trust Alliance (OTA) released “Are Organizations Ready for New Privacy Regulations?“ The report, using data collected from the 2018 Online Trust Audit, analyzes the privacy statements of 1,200 organizations using 29 variables and then maps them to overarching principles from three privacy laws around the world: General Data Protection Regulation (GDPR) in the European Union, California Consumer Privacy Act (CCPA) in the United States, and Personal Information Protection and Electronics Document Act (PIPEDA) in Canada. 

In many cases, organizations lack key concepts covering data sharing in their statements. Just 1% of organizations in our Audit disclose the types of third parties they share data with. This is a common requirement across privacy legislation. It is not as onerous as having to list all of the organizations; simply listing broad categories like “payment vendors” would suffice. 

Data retention is another area where many organizations are lacking. Just 2% had language about how long and why they would retain data. Many organizations have Continue reading

How to decommission a data center

About the only thing harder than building a data center is dismantling one, because the potential for disruption of business is much greater when shutting down a data center than constructing one.The recent decommissioning of the Titan supercomputer at the Oak Ridge National Laboratory (ORNL) reveals just how complicated the process can be. More than 40 people were involved with the project, including staff from ORNL, supercomputer manufacturer Cray, and external subcontractors. Electricians were required to safely shut down the 9 megawatt-capacity system, and Cray staff was on hand to disassemble and recycle Titan’s electronics and its metal components and cabinets. A separate crew handled the cooling system. In the end, 350 tons of equipment and 10,800 pounds of refrigerant were removed from the site.To read this article in full, please click here

How to decommission a data center

About the only thing harder than building a data center is dismantling one, because the potential for disruption of business is much greater when shutting down a data center than constructing one.The recent decommissioning of the Titan supercomputer at the Oak Ridge National Laboratory (ORNL) reveals just how complicated the process can be. More than 40 people were involved with the project, including staff from ORNL, supercomputer manufacturer Cray, and external subcontractors. Electricians were required to safely shut down the 9 megawatt-capacity system, and Cray staff was on hand to disassemble and recycle Titan’s electronics and its metal components and cabinets. A separate crew handled the cooling system. In the end, 350 tons of equipment and 10,800 pounds of refrigerant were removed from the site.To read this article in full, please click here

Video: The Need for Network Layers

After identifying some of the challenges every network solution must address (part 1, part 2, part 3) we tried to tackle an interesting question: “how do you implement this whole spaghetti mess in a somewhat-reliable and structured way?”

The Roman Empire had an answer more than 2000 years ago: divide-and-conquer (aka “eating the elephant one bite at a time”). These days we call it layering and abstractions.

In the Need for Network Layers video I listed all the challenges we have to address, and then described how you could group them in meaningful modules (called networking layers).

You need free ipSpace.net subscription to watch the video, or a paid ipSpace.net subscriptions to watch the whole webinar.

Even more amazing papers at VLDB 2019 (that I didn’t have space to cover yet)

We’ve been covering papers from VLDB 2019 for the last three weeks, and next week it will be time to mix things up again. There were so many interesting papers at the conference this year though that I haven’t been able to cover nearly as many as I would like. So today’s post is a short summary of things that caught my eye that I haven’t covered so far. A few of these might make it onto The Morning Paper in weeks to come, you never know!

Industry papers

• Tunable consistency in MongoDB. MongoDB is an important database, and this paper explains the tunable (per-operation) consistency models that MongoDB provides and how they are implemented under the covers. I really do want to cover this one at some point, along with Implementation of cluster-wide logical clock and causal consistency in MongoDB from SIGMOD 2019.
• We hear a lot from Google and Microsoft about their cloud platforms, but not quite so much from the other key industry players. So it’s great to see some papers from Alibaba and Tencent here. AliGraph covers Alibaba’s distributed graph engine supporting the development of new GNN applications. Their dataset has about 7B edges… Meanwhile, AnalyticDB Continue reading

New in Docker Hub: Personal Access Tokens

On the heels of our recent update on image tag details, the Docker Hub team is excited to share the availability of personal access tokens (PATs) as an alternative way to authenticate into Docker Hub.

Already available as part of Docker Trusted Registry, personal access tokens can now be used as a substitute for your password in Docker Hub, especially for integrating your Hub account with other tools. You’ll be able to leverage these tokens for authenticating your Hub account from the Docker CLI – either from Docker Desktop or Docker Engine: 

docker login --username <username>

When you’re prompted for a password, enter your token instead.

The advantage of using tokens is the ability to create and manage multiple tokens at once so you can generate different tokens for each integration – and revoke them independently at any time.

Create and Manage Personal Access Tokens in Docker Hub 

Personal access tokens are created and managed in your Account Settings.

From here, you can:

• Create new access tokens
• Modify existing tokens
• Delete access tokens

Note that the actual token is only shown once, at the time Continue reading

Backup the data in AWS Elastic Block Store

A recent Amazon outage resulted in a small number of customers losing production data stored in their accounts. This, of course, led to typical anti-cloud comments that follows such events. The reality is that these customers data loss had nothing to do with cloud and everything to do with them not understanding the storage they were using and backing it up.Over Labor Day weekend there was a power outage in one of the availability zones in the AWS US-East-1 region.  Backup generators came on, but quickly failed for unknown reasons. Customers’ Elastic Block Store (EBS) data is replicated among multiple servers, but the outage affected multiple servers. While the bulk of data stored in EBS was fine or was able to be easily recovered after outage, .5 percent of the data could not be recovered. Customers among the .5 percent who did not have a backup of their EBS data actually lost data.To read this article in full, please click here

How To Network To Get A Mental Health Research Job

Once you have received all of the education you need in order to get a mental health research job, the next thing you have to do is look for a job that you are most suited for. Depending on your education and your interest, a mental health research job ranges from being an actual researcher to a data analyst to a facility manager to a research moderator who monitors the way the research is being conducted. Oftentimes, it takes a great deal of networking and effort on your part to get the right mental health research job that you desire. Here are a few ways to network to help you find your dream job in this very special field.

3 Ways to Get a Mental Health Research Job

Take an Internship

Sometimes you can work as an unpaid intern in a mental health research facility while still getting your formal education. The benefit of this is the experience you’ll receive, as well as possible class credit and professional references.

In other cases, once you have your degree and take all the necessary tests, you may be able to secure a paid internship or an entry level job in mental health Continue reading

When Diverse Network ASICs Meet A Unifying Operating System

It has been two decades since Juniper Networks, then the big upstart rival to Cisco Systems and others as the dot-com boom was rising towards its crescendo several years hence, took FreeBSD Unix and turned it into a network operating system that spanned both routers and switches. …

When Diverse Network ASICs Meet A Unifying Operating System was written by Timothy Prickett Morgan at The Next Platform.

Deutsche Telekom Taps VMware for Disaster Recovery

One Service Mesh To Tie Google Anthos Together

Public clouds bring a lot of advantages to enterprises, such as more flexibility and scalability for their many of their workloads, a way to avoid expensive capital costs by using someone else’s infrastructure and having someone else manage it all, and the ability to pay only for the resources they use. …

One Service Mesh To Tie Google Anthos Together was written by Jeffrey Burt at The Next Platform.