Using SSH with the Pulumi Docker Provider

In August 2023, Pulumi released a version of the Docker provider that supported SSH-based connections to a Docker daemon. I’ve written about using SSH with Docker before (see here), and I sometimes use AWS-based “Docker build hosts” with my M-series Macs to make it easier/simpler (and sometimes faster) to build x86_64-based Docker images. Naturally, I’m using an SSH connection in those cases. Until this past weekend, however, I hadn’t really made the time to look deeper into how to use SSH with the Pulumi Docker provider. In this post, I’ll share some details that (unfortunately) haven’t yet made it into the documentation about using SSH with the Pulumi Docker provider.

First, let’s talk about some prerequisites to making this work.

1. You’ll need Docker installed locally. I fairly certain this is only the docker CLI (much in the same way the Pulumi Kubernetes provider requires kubectl to be installed locally), but I haven’t verified this for certain yet. I tested this from a Linux system running Docker 24.0.7; I think the earliest version that is supported is 18.09.
2. You’ll need Docker installed on the remote SSH host (obviously). I used Flatcar Container Linux (stable channel) on AWS.
3. Continue reading

Technology Short Take 173

Welcome to Technology Short Take #173! After a lull in links to share last time around, it looks like things have rebounded and folks are in full swing writing new content for me to share with you. I think I have a decent round-up of links for you; hopefully you can find something useful here. Enjoy!

Networking

• This article on running WireGuard in Docker may prove useful if that’s an approach I decide to adopt for my AWS lab infrastructure.
• Natalie Marek educates readers on VPC endpoints.
• Russ White laments some of the issues facing network engineering.

Servers/Hardware

• Alex Ellis provides some details on his workflow for booting Raspberry Pi 5 from NVMe.
• Tom Hummel finds himself veering back into a hardware-based home lab (instead of a cloud-based lab).

Security

• Rory McCune shares some information about a change in kubeadm version 1.29 pertaining to administrative credentials.
• Quintessence Anx of SPIRL shares some guidance on how to construct SPIFFE IDs.
• A set of vulnerabilities in the open source reference implementation of the UEFI specification has been uncovered. The flaws, referred to as PixieFail, specifically affect the PXE network boot process. BleepingComputer has more details.

Cloud Computing/Cloud Management

• Dean has published Continue reading

Technology Short Take 172

Welcome to Technology Short Take #172, the first Technology Short Take of 2024! This one is really short, which I’m assuming reflects a lack of blogging activity over the 2023 holiday season. Nevertheless, I have managed to scrape together a few links to share with readers. As usual, I hope you find something useful. Enjoy!

Networking

• Via this blog post, I learned that Ivan Pepelnjak has a GitHub repository of hands-on examples for learning public cloud networking (including both AWS and Azure). Ivan’s materials are always excellent, so if you’re looking for resources to help with expanding your networking skills into the public cloud, this should be on the short list. (I plan to submit a PR soon to add Pulumi examples, which the repository is currently missing.)

Cloud Computing/Cloud Management

• Jonathan Major shares his experience using Pulumi with Google Cloud APIs. I think there are some code snippets in Jonathan’s article, but my instance of Firefox wouldn’t render the code snippets (instead only showing empty black boxes).
• Nisar Ahmad explores whether Terraform or Pulumi is better for your use case.

Operating Systems/Applications

• Nick Janetakis shares another tip for determining how long a Docker container was running. He Continue reading

Selectively Replacing Resources with Pulumi

Because Pulumi operates declaratively, you can write a Pulumi program that you can safely run (via pulumi up) multiple times. If no changes are needed—meaning that the current state of the infrastructure matches what you’ve defined in your Pulumi program—then nothing happens. If only one resource needs to be updated, then it will update only that one resource (and any dependencies, if there are any). There may be times, however, when you want to force the replacement of specific resources. In this post, I’ll show you how to target specific resources for replacement when using Pulumi.

Here’s an example: I use Pulumi to manage my AWS-based lab resources, including SSH bastion hosts. However, because my code uses a dynamic AMI lookup, I’ve instructed Pulumi to ignore changes in the AMI ID for the bastion hosts (by appending pulumi.IgnoreChanges([]string{"ami"}) as a resource option). This gives me the control over when the bastion hosts get replaced, instead of Pulumi wanting to replace them every time the AMI ID changes.

With this in place, then, how do I tell Pulumi that I’m ready to replace the bastion hosts? Tearing down the entire stack isn’t an option. Fortunately, the pulumi CLI Continue reading

Dynamically Enabling the Azure CLI with Direnv

I’m a big fan of direnv, the tool that lets you load and unload environment variables depending on the current directory. It’s so very useful! Not too terribly long ago, I wanted to find a way to “dynamically activate” the Azure CLI using direnv. Basically, I wanted to be able to have the Azure CLI disabled (no configuration information) unless I was in a directory where I needed or wanted it to be active, and be able to make it active using direnv. I finally found a way to make it work, and in this blog post I’ll share how you can do this, too.

First, you’ll need both direnv and the Azure CLI installed (obviously). I’ll leave this as an exercise for the readers, but I’ll mention that if you want to use Azure CLI in a Python virtual environment you might find this article really helpful.

Next, you’ll want to create a couple of directories. I chose to “hide” these directories in a .config directory in my home directory. This directory is very commonly found (and used) on many Linux systems, but doesn’t typically exist on a macOS system. You can use this command to create the Continue reading

Conditional Git Configuration

Building on the earlier article on automatically transforming Git URLs, I’m back with another article on a (potentially powerful) feature of Git—the ability to conditionally include Git configuration files. This means you can configure Git to be configured (and behave) differently based on certain conditions, simply by including or not including Git configuration files. Let’s look at a pretty straightforward example taken from my own workflow.

Here’s a configuration stanza from my own system-wide Git configuration:

[includeIf "gitdir:~/Work/Code/Repos/"]
    path = ~/Work/Code/Repos/.gitconfig

The key here is the includeIf keyword. In this case, Git will include the referenced configuration file specified by path, if the location of the Git repository matches the path specification after gitdir. Basically, what this means is that all repositories under ~/Work/Code/Repos will trigger the inclusion of the additional configuration file.

Here’s the additional configuration file:

[user]
    email = name@work-domain.com
    name = Scott Lowe
[commit]
    gpgsign = false

As long as I group all work-relatd repositories in the specified directory path, these values override the system-wide values. This means I can specify my work e-mail address as the e-mail Continue reading

Mastering AWS with amazon.aws 7.0: A Dive into New Modules for Enhanced Security and Seamless Migrations

As the technology landscape continues to evolve, the latest release of the Red Hat Ansible Certified Content Collection for amazon.aws introduces a suite of powerful modules that redefine the boundaries of automation within Amazon Web Services (AWS) while redefining how organizations approach security deployments and seamless migrations within the AWS ecosystem.

In our previous blog post, "What's New: Cloud Automation with amazon.aws 7.0.0," we presented the latest release, outlining the changes, new features and newly supported modules. In this blog post, we embark on an exploration of two interesting use cases that highlight the capabilities of these new Ansible-supported modules included in the amazon.aws 7.0 release. Let’s dive into it!

Use Case #1: Implementing Security Best Practices and Access Control for AWS Resources

Security in AWS is more critical than ever before, and the collection for AWS, amazon.aws 7.0, is up to the challenge with a host of new Identity and Access Continue reading

What’s New: Cloud Automation with amazon.aws 7.0.0

When it comes to Amazon Web Services (AWS) infrastructure automation, the latest release of the certified amazon.aws Ansible Content Collection for Red Hat Ansible Automation Platform brings a number of enhancements to improve the overall user experience and speed up the process from development to production.

This blog post goes through changes and highlights what’s new in the 7.0.0 release of this Ansible Content Collection. We have included numerous features, plugins, bug fixes, and code quality improvements that further enhance the amazon.aws collection. Let's go through some of them! 

Forward-looking Changes

Much of our work in the 7.0.0 release has been focused on the following areas:

• Enhancing several modules from the upstream community.
• Promoting modules to being formally supported by Red Hat.
• Improving the overall quality of the collection code.
• Releasing various new enhancements and clarifying support policies.

New Modules

This release brings with it a number of new supported modules that have been Continue reading

Automatically Transforming Git URLs

Git is one of those tools that lots of people use, but few people truly master. I’m still on my own journey of Git mastery, and still have so very far to go. However, I did take one small step forward recently with the discovery of the ability for Git to automatically rewrite remote URLs. In this post, I’ll show you how to configure Git to automatically transform the URLs of Git remotes.

The key here is the url configuration stanza and the associated insteadOf keyword. Added to your Git configuration—either globally or on a per-repository basis—these configuration options will tell Git to use a different URL every time it encounters the specified original URL.

Here’s an example:

[url "[email protected]:org/"]
    insteadOf = "https://github.com/org/"

The [email protected]:org/ is the replacement URL; that is, the URL that you want Git to use. The URL specified by the insteadOf keyword is the original URL; that is, the URL you want Git to replace. As you can see in the example, it’s possible not only to transform HTTPS-based URLs to SSH URLs (or vice versa), but it’s possible to constrain this transformation to repositories belonging to a specific organization or Continue reading

Technology Short Take 171

Welcome to Technology Short Take #171! This is the next installation in my semi-regular series that shares links and articles from around the interwebs on various technology areas of interest. Let the linking begin!

Networking

The networking section this time around is focused on application level protocols…but hey, they’re still networking protocols, right?

• This post is the first of a three-part series on the core concepts of HTTP/3.
• Although the title of this article says “for Go programmers,” I still found it to be a well-written and understandable explanation of CORS (and I’m no Go programmer—yet).

Security

• Undetectable cryptomining technique on Azure? Yikes.
• Stealing cryptographic keys from SSH? Double yikes. (Fortunately, the fix for this is easy—don’t use RSA keys.)
• A UEFI flaw, affecting both Windows and Linux, that leverages malicious images? Triple yikes.
• Nick Frichette shows how to enumerate a principal ARN from an AWS unique identifier.

Cloud Computing/Cloud Management

• Veronica Gnilitska shared some thoughts on why Masterpoint investigated Crossplane but ended up sticking it out with Terraform.
• Trevor Roberts Jr. takes readers through how to allow GCP resources to assume AWS IAM roles—all using Pulumi, of course!
• Bill Shetti, a friend and former colleague of Continue reading

While you sleep, Automate resolving Dynatrace problem alerts and report them to ServiceNow!

Integrating observability tools with automation is paramount in the realm of modern IT operations, as it fosters a symbiotic relationship between visibility and efficiency. Observability tools provide deep insights into the performance, health, and behavior of complex systems, enabling organizations to proactively identify and rectify issues before they escalate. 

When seamlessly integrated with automation frameworks, these tools empower businesses to not only monitor but also respond to dynamic changes in real time. This synergy between observability and automation enables IT teams to swiftly adapt to evolving conditions, minimize downtime, and optimize resource utilization. By automating responses based on observability data, organizations can enhance their agility, reduce manual intervention, and maintain a robust and resilient infrastructure. In essence, using observability with automation is indispensable for achieving a proactive, responsive, and streamlined operational environment in the fast-paced and complex landscape of today’s technology.

In this blog post, we will look at a common use case involving the monitoring of processes on both bare metal and virtual machines. Our exploration will focus on utilizing Dynatrace's OneAgent, a deployed binary file on hosts that encompasses a suite of specialized services meticulously configured for environment monitoring. These services actively gather telemetry metrics, capturing insights into Continue reading

Saying Goodbye to the Full Stack Journey

In January 2016, I published the first-ever episode of the Full Stack Journey podcast. In October 2023, the last-ever episode of the Full Stack Journey podcast was published. After almost seven years and 83 episodes, it was time to end my quirky, eclectic, and unusual podcast that explored career journeys alongside various technologies, products, and open source projects. In this post, I wanted to share a few thoughts about saying goodbye to the Full Stack Journey.

First and foremost, let me say that I really enjoyed being the host of the Full Stack Journey podcast—far more than I expected I would, if I’m honest. While I didn’t love the logistics of producing a podcast, I did love getting to talk with folks, hear their stories, and learn about new things. So, while part of me is thankful to have a little less work to do, another part—a larger part—is sad to see it end.

That being said, some of you are probably wondering why it ended. I mentioned that I didn’t enjoy the logistics of producing a podcast; specifically, I didn’t enjoy audio editing. Some folks like it, but I didn’t. It was truly a chore for me. That was Continue reading

Agentless configuration drift detection and remediation

Over time, application owners find themselves compelled to continuously refine their applications and the underlying infrastructure to enhance the products they deliver, whether to internal or external customers. These modifications inevitably lead to changes in the configuration of both applications and infrastructure. While some of these changes may be benign, others can unintentionally steer the systems away from their securely configured state, a phenomenon commonly referred to as "configuration drift." Left unaddressed, the extent of this drift can introduce substantial risks to the organization.

Traditionally, agent-based automation configuration management tools have been favored as the primary solution for tackling configuration drift. 

However, is this approach genuinely the most effective strategy? 

According to AWS's well-architected framework, the concept of a Fault Isolation Zone (FIZ) is crucial, characterized by isolation boundaries like Availability Zones (AZ), Regions, control planes, and data planes. While this concept is centered in a cloud context, the principles behind FIZ remain relevant in traditional data centers and at the network edge. The core idea is to minimize the impact of errors, particularly human misconfigurations, that can propagate beyond a defined Fault Isolation Zone.

Are misconfigurations resulting from human error still a matter of concern?

Continue reading

5 use cases for automating your infrastructure with Ansible Automation Platform

Infrastructure automation is an area where systems administrators and  IT operations teams can see some of the biggest benefits from automation, including time savings, reducing tedious manual work, and improving the overall health of their systems. In this blog, I've identified the top 5 infrastructure automation use cases for Ansible Automation Platform that you can deploy in your own environment, and I've incorporated new capabilities like Event-Driven Ansible to make managing your infrastructure even easier. 

1. Provisioning Red Hat Enterprise Linux with Ansible Automation Platform

Before you get started with any automation project, we typically recommend using automation analytics to plan and predict the potential cost savings to help prioritize which automation projects will deliver the biggest time and cost savings.  

Then you can use Ansible Automation Platform to create a workflow to build or configure a cloud or datacenter instance for RHEL, while using surveys to gather additional user input to enable further customization to meet any system requirements. You can even introduce an IT service management ticketing option, then review the job success and confirm service availability. 

Watch this video to see it in action: 

2. Managing Configuration and Drift with Ansible Automation Platform

Ongoing Continue reading

Fact-Gathering with Event-Driven Ansible for Microsoft Windows ITSM

The use of Event-Driven Ansible to enable fact gathering from events is considered a “Getting Started” type of use case, but it can be extremely powerful. This use case is simple and it is what we consider a “Read Only” type of action, meaning that we are not making any changes, but we are using the event to trigger a fact gathering process which we can later publish to the IT Service Management system. 

The benefit with this is we are able to provide consistent automated troubleshooting and fact gathering which is used to enrich the ticketing systems, so when our engineers have a look at the incident, they have all the information they need to decide on the next steps to resolve the issue or situation. This can potentially save many hours of toil and ultimately save an organization money from reduced down time and faster resolutions. But, we are assuming that our technical teams will know what to do with this event data.

What if we could assist with filling the gap when an incident takes place, and we could receive information or even options on how to resolve the issues? This is where we could use Continue reading

Ansible can help with the Cisco IOS XE Software Web UI Privilege Escalation Vulnerability

CVE-2023-20198 

Reference: https://sec.cloudapps.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-iosxe-webui-privesc-j22SaA4z

Overview

Cisco recently published an advisory pertaining to an active exploitation of a previously unknown vulnerability in the web UI feature of Cisco IOS XE Software when exposed to the internet or to untrusted networks. This vulnerability allows a remote, unauthenticated attacker to create an account on an affected system with privilege level 15 access. The attacker can then use that account to gain control of the affected system.

Recommendations using Red Hat Ansible Automation Platform

In this blog, I will discuss a simple playbook that can help network admins quickly identify and remediate affected devices. To add additional capabilities for a large production environment, Red Hat Ansible Automation Platform could enhance the playbook run with additional capabilities (ticketing integrations, roles based access, workflow, self service, etc.).

Vulnerable Products

All Cisco IOS-XE based products are potentially at risk. The example playbook is located here.
In the example playbook we will explore its functionality using one of the Cisco Sandbox always-on routers

Determine the HTTP Server Configuration

The following portion of the playbook will determine the HTTP Server Configuration and print the results.

Cisco strongly recommends that customers disable the HTTP Continue reading

NEW!!! Getting started landing page for Ansible Automation Platform on Azure

This blog is co-authored by Tomas Znamenacek and Hicham (he-sham) Mourad

Introduction

We’re so excited to introduce you to the newest addition to Red Hat Ansible Automation Platform on Microsoft Azure – The new landing page! Now with new deployments, a single web page that consolidates all you need to know about Ansible Automation Platform on Azure, how to get started, as well as links to the Ansible Automation Platform applications, is now accessible.

Upon arriving at this getting started landing page, you will see three tiles on the overview page. You have the ability to launch each of the following Ansible applications from the tiles: automation controller, private automation hub, and automation analytics, as well as a direct link to the managed Azure product documentation.

The bottom portion of the overview page sets you up for success by providing links to all the enablement content you need. It specifically provides links to Ansible Automation Platform on Azure Knowledge Base articles, documentation, and how to contact and request support.  

Another important area is the “Managed Azure Maintenance and Feature Updates” that provides the link to the maintenance updates and feature releases to Ansible Automation Platform on Azure. Stay Continue reading

Guest Post: Moving Secrets Where They Belong

by Simen A.W. Olsen

Pulumi recently shipped Pulumi ESC, which adds the “Environment” tab to Pulumi Cloud. For us at Bjerk, this means we can move secrets into a secrets manager like Google Secrets Manager. Let me show you how we did it!

We are already rotating secrets with our own CLI tool, which works fine, meaning we are getting notifications in our Slack channel—which everyone tends to ignore until something real breaks. If you are curious how we are handling it today, we are using our own NPM package that throws an exception if a secret has expired. To ensure everything works smoothly, we utilize a GitHub Actions workflow that is scheduled to run daily for drift checking.

The secrets are shared between stacks using StackReferences, which has served us well.

Improving security

One issue with our current setup is that we publicly store encrypted secrets in our repository. Previously, we’ve thought of using Google Secrets Manager with the GetSecret function. That comes with its own territory, such as permissions to the secret and managing those permissions—not to mention that we already use multiple secret managers/vaults.

Now, with Pulumi ESC, it’s time to pick this Continue reading

Red Hat Insights Collection for Event-Driven Ansible

Event-Driven Ansible became generally available in Ansible Automation Platform 2.4. As part of the release, Red Hat Insights and Ansible teams collaborated to implement and certify a Red Hat Insights collection integrating Insights events as a source of events for Ansible Automation Platform. This provides a consistent way to receive and handle events triggered from Insights to drive and track automation within the Ansible Automation Platform. The collection is available for installation on Ansible automation hub.

In this article, we explore how to get, configure and use the Red Hat Insights collection for Event-Driven Ansible, and provide an end-to-end example of an automation flow using Ansible Automation Platform and its Event-Driven Ansible functionality. Our goal is to automate the creation of a ServiceNow incident ticket with all relevant information when malware is detected by Insights on one of our Red Hat Enterprise Linux (RHEL) systems. We show how we can track and audit all automation from the Event-Driven Ansible controller as part of Ansible Automation Platform.

Introducing a new certified  collection for Event-Driven Ansible

In a previous article, we looked at exposing and consuming Insights events to drive automation with Event-Driven Ansible. We validated that events triggered Continue reading

Seeding Ansible Automation Platform Content At Installation Time

In a previous blog, I announced the tech preview of containerized Red Hat Ansible Automation Platform. With that we introduced a new feature to enable you to pre-seed configuration and content at installation time. This blog will take you through how that works.

At Red Hat, we are very lucky to have some super talented people. Some of these great folks have worked tirelessly on producing Ansible Content Collections for managing configuration within the platform. Using this as the foundation, we have wrapped enablement around that to provide a fully automated way of doing this, in a seamless fashion.

The Benefits

Many people today already use a configuration-as-code (CaC) approach to managing the platform and laying down standards for their users. If that’s your case, then this simply extends and simplifies this capability.

If this is a new concept to you, then these examples may be of interest:

• Standardize your user’s out of the box (OOTB) experience, in one go, at installation time.
  • Are there platform artifacts that you always need to apply to each and every installation? Define those as code and simply point the installation at that code and it will pre-configure the platform

• The ability Continue reading