Watchman: monitoring dependency conflicts for Python library ecosystem

Watchman: monitoring dependency conflicts for Python library ecosystem Wang et al., ICSE ‘20

There are more than 1.4M Python libraries in the PyPI repository. Figuring out which combinations of those work well together is not always easy. In fact, we have a phrase for navigating the maze of dependencies modern projects seem to accumulate: “dependency hell”. It’s often not the direct dependencies of your project that bite you, but the dependencies of your dependencies, all the way on down to transitive closure. In today’s paper from ICSE ‘20, Wang et al. study the prevalence of dependency conflicts in Python projects and their causes. Having demonstrated (not that many of us would need much convincing) that dependency conflicts are a real problem, they build and evaluate Watchman, a tool to quickly find and report dependency conflicts, and to predict dependency conflicts in the making. Jolly useful it looks too.

Welcome to dependency hell

If you have a set of versioned dependencies that all work together, you can create a lock file to pin those versions. But if you haven’t yet reached that happy place, or you need to upgrade, add or remove a dependency, you’ll be Continue reading

Should you upgrade tape drives to the latest standard?

With the recent release of the linear tape–open 9 (LTO-9) standard, tape drives with increased capacity and speed should be available soon, but that doesn’t mean users of tape drives should rush to buy them.Here are some of the pros and cons to weigh when considering whether an upgrade is in order.Tape drives are a very reliable way to write data to storage, and are very good at holding onto data for multiple decades. They make an excellent medium for long-term storage and for shipping large amounts of data across long distances (a FedEx truck has unlimited bandwidth).[Get regularly scheduled insights by signing up for Network World newsletters.] What tape is not good at is going slow. LTO-8 has a compressed transfer speed of 900MB/s, which is significantly faster than most any backup you're going to send to it. It's definitely faster than any incremental backup that will be sent to it, and that comprises most backups. That makes tapes as the initial target of backups problematic.To read this article in full, please click here

Should you upgrade tape drives to the latest standard?

With the recent release of the linear tape–open 9 (LTO-9) standard, tape drives with increased capacity and speed should be available soon, but that doesn’t mean users of tape drives should rush to buy them.Here are some of the pros and cons to weigh when considering whether an upgrade is in order.Tape drives are a very reliable way to write data to storage, and are very good at holding onto data for multiple decades. They make an excellent medium for long-term storage and for shipping large amounts of data across long distances (a FedEx truck has unlimited bandwidth).[Get regularly scheduled insights by signing up for Network World newsletters.] What tape is not good at is going slow. LTO-8 has a compressed transfer speed of 900MB/s, which is significantly faster than most any backup you're going to send to it. It's definitely faster than any incremental backup that will be sent to it, and that comprises most backups. That makes tapes as the initial target of backups problematic.To read this article in full, please click here

Interesting: PyEnv

If you’re like me, you’re probably sick-and-tired of Python versions, environments… Every time I update Python on my MacBook Pro with Homebrew, I lose all packages I installed for the previous version of Python (because I’m installing them system-wide and they’re stored in version-specific directory).

Jon Langemak found a potential solution to this problem: PyEnv. My first reaction was: Great, just what I need… but as he described how it really works, I realized that it’s always possible to add another layer of indirection. RFC1925 strikes again.

Keepalived and unicast over multiple interfaces

Keepalived is a Linux implementation of VRRP. The usual role of VRRP is to share a virtual IP across a set of routers. For each VRRP instance, a leader is elected and gets to serve the IP address, ensuring the high availability of the attached service. Keepalived can also be used for a generic leader election, thanks to its ability to use scripts for healthchecking and run commands on state change.

A simple configuration looks like this:

vrrp_instance gateway1 {
  state BACKUP          # ❶
  interface eth0        # ❷
  virtual_router_id 12  # ❸
  priority 101          # ❹
  virtual_ipaddress {
    2001:db8:ff/64
  }
}

The state keyword in ❶ instructs Keepalived to not take the leader role when starting. Otherwise, incoming nodes create a temporary disruption by taking over the IP address until the election settles. The interface keyword in ❷ defines the interface for sending and receiving VRRP packets. It is also the default interface to configure the virtual IP address. The virtual_router_id directive in ❸ is common to all nodes sharing the virtual IP. The priority keyword in ❹ helps choosing which router will be elected as leader. If you need more information around Keepalived, be sure to check Continue reading

Syncing NetBox with a custom Ansible module

The netbox.netbox collection from Ansible Galaxy provides several modules to update NetBox objects:

- name: create a device in NetBox
  netbox_device:
    netbox_url: http://netbox.local
    netbox_token: s3cret
    data:
      name: to3-p14.sfo1.example.com
      device_type: QFX5110-48S
      device_role: Compute Switch
      site: SFO1

However, if NetBox is not your source of truth, you may want to ensure it stays in sync with your configuration management database1 by removing outdated devices or IP addresses. While it should be possible to glue together a playbook with a query, a loop and some filtering to delete unwanted elements, it feels clunky, inefficient and an abuse of YAML as a programming language. A specific Ansible module solves this issue and is likely more flexible.

Notice

I recommend that you read “Writing a custom Ansible module” as an introduction, as well as “Syncing MySQL tables” for a first simpler example.

Code

The module has the following signature and it syncs NetBox with the content of the provided YAML file:

netbox_sync:
  source: netbox.yaml
  api: https://netbox.example.com
  token: s3cret

The synchronized objects are:

Solve the Simple Problems

One thing I’ve found out over the past decade of writing is that some problems are easy enough to solve that we sometimes forget about them. Maybe it’s something you encounter once in a great while. Perhaps it’s something that needed a little extra thought or a novel reconfiguration of an existing solution. Something so minor that you didn’t even think to write it down. Until you run into the problem again.

The truth behind most of these simple problems is that the solutions aren’t always apparent. Sure, you might be a genius when it comes to fixing the network or the storage array. Maybe you figured out how to install some new software to do a thing in a way that wasn’t intended. But did you write any of it down for later use? Did you make sure to record what you’ve done so someone else can use it for reference?

Part of the reason why I started blogging was to have those written solutions to problems I couldn’t find a quick answer to. What it became was way more than I had originally intended. But the posts that I write that still get the most attention aren’t my Continue reading

MUST Read: Blockchain, the amazing solution for almost nothing

One of the weekend reads collected by Russ White contained a pointer to a hilarious description of blockchain - a solution in search of a problem. Here are a few quotes to get you started (and I had a really hard time selecting just a few):

I’ve never seen so much bloated bombast fall so flat on closer inspection.

At its core, blockchain is a glorified spreadsheet.

The only thing is that there’s a huge gap between promise and reality. It seems that blockchain sounds best in a PowerPoint slide.

Someone should use that article as a framework and replace blockchain with OpenFlow or SDN ;)

Heavy Networking 540: Sinefa Blends Network Data, Synthetics To Measure End User Experience (Sponsored)

In today's sponsored show, we talk with Sinefa about its Digital Experience Monitoring capabilities. Sinefa assembles network traffic, DPI, end point monitoring, and synthetic transactions to get clear, actionable information on network performance to measure end user experience and improve troubleshooting. Our guests are Chris Siakos, CTO; and Alex Henthorn-Iwane, VP of Product Marketing.

Heavy Networking 540: Sinefa Blends Network Data, Synthetics To Measure End User Experience (Sponsored)

In today's sponsored show, we talk with Sinefa about its Digital Experience Monitoring capabilities. Sinefa assembles network traffic, DPI, end point monitoring, and synthetic transactions to get clear, actionable information on network performance to measure end user experience and improve troubleshooting. Our guests are Chris Siakos, CTO; and Alex Henthorn-Iwane, VP of Product Marketing.

The post Heavy Networking 540: Sinefa Blends Network Data, Synthetics To Measure End User Experience (Sponsored) appeared first on Packet Pushers.

The Digital Services Act and Why the Architecture of the Internet Must Be Preserved

Earlier this year, as part of the European Digital Strategy, the European Commission announced a Digital Services Act package to develop new and revised rules to harmonize and deepen the Single Market for digital services. As a part of that initiative, it also initiated a public consultation to scope the specific issues that may require regulatory intervention.

The Internet Society submitted recommendations in response to the critical issues raised in this consultation. One of the key considerations that we hope comes across in this submission is that in order “to design better regulation for the Internet, it is important to understand two things: the first one is that today’s Internet, despite how much it has evolved, still continues to depend on its original architecture; and, the second relates to how preserving this design is important for drafting regulation that is fit for purpose.”

As noted by the Commission, the scale of digital services is substantially different from 20 years ago, when the E-Commerce Directive was introduced. New actors and new services have emerged, creating a much more complex ecosystem. This new ecosystem presents new challenges, particularly with regard to illegal and harmful activities and content online. Current discussions and regulatory initiatives Continue reading

Stuff The Internet Says On Scalability For Sep 18th, 2020

Hey, it's HighScalability time!

 

I can't wait for the duel. Just don't shoot into the air.

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. 

Know someone who could benefit from becoming one with the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 167 mostly 5 star reviews. Here's a 100% lectin-free review:

Number Stuff:

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...