Better pipes

In my blog post The uselessness of bash I made a tool to improve pipes in shell, to assemble a better pipeline.

It solves the problem, but it’s a bit too different, with its own language.

While complaining with some people at work that one of the main features of shell (the pipe operator) is broken, someone joked that it should be replaced by a protobuf based protocol.

But on second thought it’s not really a joke.

How about instead of this:

$ goodpipe <<EOF
[
  ["gsutil", "cat", "gs://example/input-unsorted.txt"],
  ["sort", "-S300M", "-n"],
  ["gzip", "-9"],
  ["gsutil", "cp", "-", "gs://example/input-sorted-numerically.txt.gz"]
]
EOF

how about this:

$ wp -o gsutil cat gs://example/input-unsorted.txt \
  | wp -io sort -S300M -n \
  | wp -io gzip -9 \
  | wp -i gsutil cp - gs://example/input-sorted-numerically.txt.gz

It doesn’t use protobufs, but a simpler regular protocol. This in order to avoid well known bugs types. Before implementing any protocol also see formal theory and science of insecurity.

First I hacked it together in Go, but I think the main implementation I’ll maintain is the one I made while porting it to Rust, as a way to learn Rust. The Continue reading

What is a VPN? A secure network over the internet

VPNs date back to the 1990s when the public internet lacked almost any form of security, and the technology was developed to provide secure and cost-effective connections across this insecure landscape.VPNs have become widely deployed across enterprise networks and experienced a surge during the pandemic, when companies had to scramble to provide secure remote access to employees who were suddenly working from home.VPNs remain popular today, but they are also slowly but surely being supplanted by more flexible, more secure, more granular alternatives, such as SD-WAN, Zero Trust Network Architecture (ZTNA), and SASE, a cloud-based service that includes SD-WAN, ZTNA and other security features.To read this article in full, please click here

Managing infrastructure with Terraform, CDKTF, and NixOS

A few years ago, I downsized my personal infrastructure. Until 2018, there were a dozen containers running on a single Hetzner server.1 I migrated my emails to Fastmail and my DNS zones to Gandi. It left me with only my blog to self-host. As of today, my low-scale infrastructure is composed of 4 virtual machines running NixOS on Hetzner Cloud and Vultr, a handful of DNS zones on Gandi and Route 53, and a couple of Cloudfront distributions. It is managed by CDK for Terraform (CDKTF), while NixOS deployments are handled by NixOps.

In this article, I provide a brief introduction to Terraform, CDKTF, and the Nix ecosystem. I also explain how to use Nix to access these tools within your shell, so you can quickly start using them.

CDKTF: infrastructure as code​

Terraform is an “infrastructure-as-code” tool. You can define your infrastructure by declaring resources with the HCL language. This language has some additional features like Continue reading

Looking ahead to the network technologies of 2023

What’s the single most important thing that enterprises should know about networking in 2023? Forget all that speeds-and-feeds crap you hear from vendors. The answer is that networking is now, and forever, linked to business applications, and those applications are linked now to the way that we use the Internet and the cloud. We’re changing how we distribute and deliver business value via networking, and so network technology will inevitably change too, and this is a good time to look at what to expect.Growth in Internet dependence First, the Internet is going to get a lot better because it’s going to get a lot more important. It’s not just that the top-end capacities offered will be raised, in many cases above 2 Gbps. Every day, literally, people do more online, and get more interactive, dynamic, interesting, websites to visit and content to consume. Internet availability has been quietly increasing, and in 2023 there will be a significant forward leap there, in large part because people who rely on something get really upset when it’s not working.To read this article in full, please click here

The top 12 tech stories of 2022

The year highlighted how vulnerable the technology sector is to the vagaries of geopolitics and the macroeconomy, as IT giants laid off workers, regulators cracked down on tech rule-breakers, nations negotiated data security regulations, the US-China chip war widened, and the Ukraine war disrupted business as usual.

Looking ahead to the network technologies of 2023

What’s the single most important thing that enterprises should know about networking in 2023? Forget all that speeds-and-feeds crap you hear from vendors. The answer is that networking is now, and forever, linked to business applications, and those applications are linked now to the way that we use the Internet and the cloud. We’re changing how we distribute and deliver business value via networking, and so network technology will inevitably change too, and this is a good time to look at what to expect.Growth in Internet dependence First, the Internet is going to get a lot better because it’s going to get a lot more important. It’s not just that the top-end capacities offered will be raised, in many cases above 2 Gbps. Every day, literally, people do more online, and get more interactive, dynamic, interesting, websites to visit and content to consume. Internet availability has been quietly increasing, and in 2023 there will be a significant forward leap there, in large part because people who rely on something get really upset when it’s not working.To read this article in full, please click here

The top 12 tech stories of 2022

The year highlighted how vulnerable the technology sector is to the vagaries of geopolitics and the macroeconomy, as IT giants laid off workers, regulators cracked down on tech rule-breakers, nations negotiated data security regulations, the US-China chip war widened, and the Ukraine war disrupted business as usual.

The Power of Complaining Properly

Recently I’ve started listening to a new podcast all about the brain and behaviors called Hidden Brain. It’s got a lot great content and you should totally check it out. One of the latest episodes deals with complaining and how it can make us less productive and more likely to repeat patterns or shut people out.

Complaining is as old as language. I’m sure as soon as the first person to create communications around spoken words was able to teach another person one of the first things they did was complain about the weather or something they hated. Our mind is built to express itself about things we don’t like, such as bad drivers or silly behaviors at work.

The episode explores the ways that our brain can trap us in cycles of complaining simply for the sake of complaining. It also discusses how we should try to spend more time trying to be productive in how we address complaints. I’ve experienced this a lot in IT as well as in my career after being directly involved in IT and there’s a lot of merit in changing the way we complain about things.

Airing Grievances

Complaining without a suggested solution Continue reading

Nvidia still crushing the data center market

Nvidia is playing some serious games.When Jensen Huang and his two partners established Nvidia in 1993, the graphics chip market had many more competitors than the CPU market, which had just two. Nvidia’s competitors in the gaming market included ATI Technologies, Matrox, S3, Chips & Technology, and 3DFX.A decade later, Nvidia had laid waste to every one of them except for ATI, which was purchased by AMD in 2006. For most of this century, Nvidia has shifted its focus to bring the same technology it uses to render videogames in 4k pixel resolution to power supercomputers, high-performance computing (HPC) in the enterprise, and artificial intelligence.To read this article in full, please click here

Nvidia still crushing the data center market

Nvidia is playing some serious games.When Jensen Huang and his two partners established Nvidia in 1993, the graphics chip market had many more competitors than the CPU market, which had just two. Nvidia’s competitors in the gaming market included ATI Technologies, Matrox, S3, Chips & Technology, and 3DFX.A decade later, Nvidia had laid waste to every one of them except for ATI, which was purchased by AMD in 2006. For most of this century, Nvidia has shifted its focus to bring the same technology it uses to render videogames in 4k pixel resolution to power supercomputers, high-performance computing (HPC) in the enterprise, and artificial intelligence.To read this article in full, please click here

Intel splits GPU group into two separate units

Intel announced plans to split its AXG graphics group and move the resources into two existing business units to better serve their respective markets.The consumer/gaming end of the GPU business will move to Intel’s Client Compute Group (CCG), which develops consumer computing platforms based on the company’s CPU products. The teams responsible for data center and supercomputing products such as the Ponte Vecchio and Rialto Bridge will move to the Data Center and AI (DCAI) business unit.The GPU SoC and IP design teams will also fall under the DCAI umbrella, but they will continue to support the client graphics team. Jeff McVeigh, currently the vice president and general manager of the Super Compute Group, will serve as the interim leader of this team until a permanent leader is found.To read this article in full, please click here

Intel splits GPU group into two separate units

Intel announced plans to split its AXG graphics group and move the resources into two existing business units to better serve their respective markets.The consumer/gaming end of the GPU business will move to Intel’s Client Compute Group (CCG), which develops consumer computing platforms based on the company’s CPU products. The teams responsible for data center and supercomputing products such as the Ponte Vecchio and Rialto Bridge will move to the Data Center and AI (DCAI) business unit.The GPU SoC and IP design teams will also fall under the DCAI umbrella, but they will continue to support the client graphics team. Jeff McVeigh, currently the vice president and general manager of the Super Compute Group, will serve as the interim leader of this team until a permanent leader is found.To read this article in full, please click here

Cloudflare Radar 2022 Year in Review

Cloudflare Radar 2022 Year in Review
Cloudflare Radar 2022 Year in Review

In 2022, with nearly five billion people around the world (as well as an untold number of “bots”) using the Internet, analyzing aggregate data about this usage can uncover some very interesting trends. To that end, we’re excited to present the Cloudflare Radar 2022 Year In Review, featuring interactive charts, graphs, and maps you can use to explore notable Internet trends observed throughout this past year. The Year In Review website is part of Cloudflare Radar, which celebrated its second birthday in September with the launch of Radar 2.0.

We have organized the trends we observed around three different topic areas: Traffic, Adoption, and Security. The content covered within each of these areas is described in more detail in their respective sections below. Building on the 2021 Year In Review, we have incorporated several additional metrics this year, and have also improved the underlying methodology. (As such, the charts are not directly comparable to develop insights into year-over-year changes.)

Website visualizations shown at a weekly granularity cover the period from January 2 through November 26, 2022 (the start of the first full week of the year through the end of the last full Continue reading

1 … 342 343 344 345 346 … 3,804