Creating a Talos Linux Cluster on AWS with Pulumi
Talos Linux is a Linux distribution purpose-built for running Kubernetes. The Talos web site describes Talos Linux as “secure, immutable, and minimal.” All system management is done via an API; there is no SSH access, no shell, and no console. In this post, I’ll share how to use Pulumi to automate the creation of a Talos Linux cluster on AWS.
I chose to write my Pulumi program in Go, but you could—of course—choose to write it in any language that Pulumi supports (JavaScript/TypeScript, Python, one of the .NET languages, Java, or even YAML). I’ve made the Pulumi program available via this GitHub repository. It’s based on these instructions for standing up Talos Linux on AWS.
The Pulumi program has four major sections:
- First, it creates the underlying base infrastructure needed for a Talos Linux cluster to run. This includes a VPC (and all the assorted other pieces, like subnets, gateways, routes, and route tables) and a load balancer. The load balancer is needed for the Kubernetes control plane, which we will bootstrap later in the program. This portion also creates the EC2 instances for the control plane.
- Next, it uses the Talos Pulumi provider to generate the Talos Continue reading
Using Red Hat Insights as a source of events for Event-Driven Ansible automation
One of the key announcements at AnsibleFest 2022 was the introduction of the Event-Driven Ansible developer preview. This technology is currently available on GitHub and accessible by technology providers and end users to provide feedback and drive the ecosystem. ISVs and consulting/service partners are specifically invited to create event driven automation content that makes it easy for customers to use in joint solutions.
Red Hat Insights events as a source for Event-Driven Ansible
Red Hat Insights, is a managed service that is included in every Red Hat subscription. It continuously analyzes platforms and applications to help enterprises manage hybrid cloud environments, and can trigger events through its Notifications service. Each account configures how and who can receive these events, with the ability to perform actions depending on the event type. For example, one may want to forward new recommendations found for Red Hat Enterprise Linux (RHEL) system configuration to a specific team by email, and/or create a new ticket in ServiceNow for the Operations team to handle. Others may want to forward all triggered events to Splunk for external analysis and troubleshooting. Through its Integrations service, Insights provides end-point integrations to Splunk, ServiceNow, Slack, as well as Continue reading
Red Hat Ansible Automation Platform now available on Google Cloud Marketplace
Breaking news! Red Hat just announced Ansible Automation Platform’s availability on Google Cloud Marketplace.
I’d like to take a few moments to provide some more details about this offering and why you should consider accessing Ansible Automation Platform directly from the Google Cloud Marketplace.
As organization’s hybrid cloud environments continue to grow in complexity, so does the need to increase efficiency and speed. The solution is to leverage an automation platform that can help any organization create, manage, and scale their automation efforts across the entire IT infrastructure. Ansible Automation Platform is the glue that coordinates and scales automation across all IT domains, and fosters a culture of collaboration across the disparate teams within an organization.
Google Cloud Marketplace deployment
Ansible Automation Platform deploys directly from the Google Cloud Marketplace as a self-managed application. The many benefits include:
- It deploys into your environment, where you have total control over how you deploy, configure, and operationalize the solution.
- Ongoing upgrades will be simpler because of this deployment model, so while this is a self-managed solution, it’s still simpler to maintain than if you had started from scratch yourself.
- The ability to scale out the Ansible Automation Platform environment using Continue reading
Technology Short Take 165
Welcome to Technology Short Take #165! Over the last few weeks, I’ve been collecting articles I wanted to share with readers on major areas in technology: networking, security, storage, virtualization, cloud computing, and OSes/applications. This particular Technology Short Take is a tad heavy on cloud computing, but there’s a decent mix of other articles as well. Enjoy!
Networking
- For a deeper understanding of Kubernetes networking, and in particular the role played by
kube-proxy, I highly recommend this post by Arthur Chiao. There is a ton of information here! - Denis Mulyalin shows how to use Nornir, Salt, and NetBox to template your network tests. Now if Denis’ site just had a discoverable RSS feed…
Security
- Aeva Black and Gil Yehuda tackle the conundrum of open source security.
- If you haven’t looked at Teri Radichel’s series of posts on automating cybersecurity metrics (ACM), you should. There’s quite a bit of good information there.
- This post on Cedar—a new policy language developed by AWS—is an interesting read. I’m curious as to the constraints that led AWS to develop a new policy language versus using something like Rego (part of Open Policy Agent); this isn’t something the article touches upon.
Cloud Computing/Cloud Continue reading
Enable Extensions on Azure Arc Connected Machines with Ansible Automation Platform
Last year, I blogged about how to use Red Hat Ansible Automation Platform to migrate Azure Arc-enabled servers from Azure Log Analytics Agents (MMA/OMS) to Azure Monitor Agent (AMA). Azure Arc supports a number of other extensions that can add additional value to your Arc-enabled infrastructure. Since my previous article, all of these extensions have been added to the azure.infrastructure_config_demos collection that contains a role for managing Arc-enabled server VM extensions with Ansible.
Each extension offers unique capabilities to your Arc-enabled fleet, such as logging, vulnerability scanning, key vault cert sync, update management, and more. Enabling these extensions is simple for small numbers of machines. When you need to scale out the work of enabling and configuring these extensions across hundreds or thousands of devices, then Ansible Automation Platform can help!
This article covers how to use Ansible Automation Platform to enable VM extensions supported in the azure.infrastructure_config_demos collection. Within the collection, there are a number of playbooks and roles; the following are pertinent to this post.
| File or Folder |
Description |
| Playbook that will be used as a job template to enable Azure Arc extensions. |
|
| Playbook that will be used Continue reading |
Stage Manager is Incomplete
I’ve been using macOS Stage Manager off and on for a little while now. In Stage Manager, I can see the beginnings of what might be a very useful paradigm for desktop computing. Unfortunately, in its current incarnation, I believe Stage Manager is incomplete.
Note that I haven’t yet tried Stage Manager on my iPad; my comments here apply only to the macOS implementation.
For those of you who haven’t yet tried Stage Manager yet, here’s a screenshot of my desktop, taken while I was writing this blog post:
I’ll draw your attention to the list of “recently used applications” on the left side of the screen. That’s the “Cast” (a term used by Howard Oakley in his great introductory article on Stage Manager). As you can see in this screenshot, the Cast supports application groups—like having Slack and Mail grouped together—as well as single applications. This allows you to easily switch between groups of applications simply by clicking on the preview in the Cast (which, using Howard’s terminology, moves the application or applications to the Stage).
This is the glimmer of a useful paradigm that I see in Stage Manager: being able to assemble groups of applications that Continue reading
Bringing faster updates to Ansible Automation Platform
In today’s fast moving world, schedule driven, incremental releases may not be what customers are looking for. After gathering input from both external and internal customers, there is a definite appetite for more content driven releases.
Rather than waiting weeks to get official builds with a bug fix (schedule driven), most would like to have those builds made available within days after the code has been tested and merged (content driven). Beginning with Red Hat Ansible Automation Platform 2.3, this new release mechanism will be the norm. This blog will explain what it means for you and your processes.
What is Ansible Automation Platform?
From a business perspective, Ansible Automation Platform is the solution Red Hat offers its customers to reach and unleash the full potential of strategic automation.
From a technical perspective, Ansible Automation Platform is an umbrella of many components that provide automation capabilities. Some of these well known components include automation controller, Ansible automation hub, ansible-runner and ansible-core, which also have underlying dependencies.
A parallel can be easily drawn with Red Hat Enterprise Linux, which is the sum of all its components’ capabilities to run a battle tested operating system, just like Ansible Continue reading
Installing the Prerelease Pulumi Provider for Talos
Normally, installing a Pulumi provider is pretty easy; you run pulumi up and the provider gets installed automatically. Worst case scenario, you can install the provider using pulumi plugin install. However, when dealing with prerelease providers, sometimes things have to be done manually. Such is the case with the prerelease Pulumi provider for Talos Linux. In this post, I’ll show you what the manual process looks like for installing a prerelease provider.
The GitHub repository for the prerelease Pulumi provider for Talos can be found here. As of this writing, the latest release was v0.1.0-beta.0. Currently, the prerelease provider for Talos Linux can’t be installed automatically when running pulumi up, and pulumi plugin install doesn’t work either.
The manual process for installing this provider looks like this:
- Download the latest release of the Talos provider from the GitHub Releases page. This will download a tarred and gzipped archive.
- The plugin files need to go into a specific subdirectory under
~/.pulumi/plugins. Navigate to that directory, and create a subdirectory whose name corresponds to the version of the Talos provider. For example, if the version downloaded is v0.1.0-beta.0, then the name of the new Continue reading
Creating custom Event-Driven Ansible source plugins
We’re surrounded! Our modern systems and applications are constantly generating events. These events could be generated by service requests, application events, health checks, etc. With the wealth of information from event traffic surrounding everything we do, Event-Driven Ansible allows for automated responses to incoming events.
But not only are we completely engulfed in event data, we’re also enveloped by event sources. Think about your organization or even your household for a minute and consider how many pieces of equipment or applications are generating data that could be put to use if only you were able to easily collect it.
Event source plugins within Event-Driven Ansible act as a bridge between Ansible and event generating applications and services. Event-Driven Ansible already has a handful of event plugins to consume events from a variety of sources. But what if your source plug-in isn’t represented in that list? Or what if you’re a Red Hat partner who wants to connect Event-Driven Ansible to your own solution? The good news is, developing event source plugins for Event-Driven Ansible can be a relatively painless endeavor.
What is a source plugin?
Event-Driven Ansible leverages rulebooks to codify the response to an event. Rulebooks combine Continue reading
Automating Docker Contexts with Pulumi
Since I switched my primary workstation to an M1-based MacBook Pro (see my review here), I’ve starting using temporary AWS EC2 instances for compiling code, building Docker images, etc., instead of using laptop-local VMs. I had an older Mac Pro (running Fedora) here in my home office that formerly filled that role, but I’ve since given that to my son (he’s a young developer who wanted a development workstation). Besides, using EC2 instances has the benefit of access when I’m away from my home office. I use Pulumi to manage these instances, and I extended my Pulumi code to also include managing local Docker contexts for me as well. In this post, I’ll share the solution I’m using.
For those that aren’t already aware, Docker supports SSH-based contexts, which allow you to use the docker CLI over an SSH connection to a remote Docker daemon (including one behind an SSH bastion host). This is the functionality I’m using to do remote Docker image builds on an EC2 instance. I wrote a bit about SSH-based Docker contexts here.
When I run pulumi up to create the infrastructure, the Pulumi code (written in Go) does a few things:
Ansible Automation Platform Moving Towards Smarter Inventory
TL;DR What is this?
It has been a long term ask and our desire to make Smart Inventory, well, smarter. We’ve listened to feedback, and are now addressing not only direct customer asks but also presenting solutions to make it better overall.
The current Red Hat Ansible Automation Platform Smart Inventory
The current Smart Inventory has a number of shortcomings:
- The smart inventory host_filter cannot express that a variable EQUALS a value, or do basic logic like NOT.
- Host/group/inventory variables cannot be filtered as a combined unit, as these are separate fields.
- Resultant smart inventories do not contain groups.
- The smart inventory host_filter has its own custom syntax, which isn’t the most friendly.
All of these issues stem from the original design of Smart Inventory, and the fact that Inventory Django models (Inventory, Group, and Host) save their “variables” in text form as YAML/JSON, as they appear in the UI. We then have to parse these into a dictionary form so they are in some way usable. This introduces new challenges and constraints.
A better solution: “constructed inventory”
So rather than continuing down a sub-optimal route, we’ve taken stock of the options (there were many and they got Continue reading
Ansible Automation Platform 2.3 Configuration as Code Improvements
On November 29, we launched Red Hat Ansible Automation Platform 2.3, which included new and exciting features including improvements for Configuration as Code (CaC). Ansible Automation Platform 2.3 also includes improvements to automation controller as well as the introduction of Ansible validated content. This blog post will walk you through what CaC is and the benefits it can bring to your organization, including a UI and API walkthrough of automation controller and how to take a full Configuration as Code approach to your automation infrastructure.
What is Configuration as Code (CaC) in Ansible Automation Platform?
CaC is a term generally referring to the separation of configuration settings from the actual code. The ideal being you can store that configuration data in source control, and easily run and tweak it to match different environments.
In Ansible Automation Platform terms, we can use the features within the automation controller in combination with CaC to provide a more flexible, richer experience. Essentially we’ve added ‘Prompt on Launch’ to everything within a job template, many of which will also trickle down into workflows.
‘Prompt on launch’ is our Ansible Automation Platform way of saying ‘this is the Continue reading
Technology Short Take 164
Welcome to Technology Short Take #164! I’ve got another collection of links to articles on networking, security, cloud, programming, and career development—hopefully you find something useful!
Networking
- William Morgan’s 2022 service mesh recap captures some of the significant events in service mesh in 2022, although through a Linkerd-colored lens. I do agree that the synergy between service mesh and the Gateway API was a surprise for a lot of folks, but they really are a good match.
- Back in October of last year, Tom Hollingsworth weighed in on Hedgehog, the networking company that has set out to commercialize SONiC, a Linux-based NOS used extensively in Azure.
- Ah, the bygone sounds of yesteryear…what a blast from the past!
Servers/Hardware
- What do you think of the ThinkPhone? (Hat tip to James Kane for bringing this to my attention.)
- I just found this article buried in my list of “articles to include in a future TST”: it’s a list of blade server resources from “blade server guy” Kevin Houston.
Security
- Adrian Mouat reviews some of the security-focused changes in Go 1.20.
- This article on using
osqueryfor behavioral detection of macOS malware was an interesting read.
Cloud Computing/Cloud Management
The Zen of Ansible
[This blog post is based on my presentation at AnsibleFest 2022 in Chicago and virtually.]
Recently, a suggestion was made to adopt Tim Peters’ “The Zen of Python” as an overall guiding principle for designing good automation content. That gave me pause because it didn’t seem like the right thing to me. While there is definitely some very good advice to “The Zen of Python” that can be applied to Ansible content, adopting it in its entirety would not provide the best user experience that Ansible is capable of and known for. Its presence as a guiding principle for content design gives the wrong impression and re-enforces a mindset we don't want to recommend.
This got me thinking, what is “the zen” of Ansible?
I considered the spirit of “The Zen of Python” and then I returned to the Ansible best practices talk that I first co-presented back in 2016 at Red Hat Summit and later touched upon here in this blog. In that talk, I said that Ansible was designed with a philosophy of sorts from the very beginning.
“The Ansible way” is to provide an automation tool that is simple, powerful and agentless. Ansible enables users with Continue reading
Fundamentals of Network Automation with Ansible Validated Content using the network.base collection
We introduced resource modules in Ansible 2.9, which provided a path for users to ease network management, especially across multiple different product vendors. This announcement was significant because these resource modules added a well structured representation of device configurations and made it easy to manage common network configurations.
At AnsibleFest 2022, we announced a new addition to the content ecosystem offered through the platform: Ansible validated content. Ansible validated content is use cases-focused automation that is packaged as Collections. They contain Ansible plugins, roles and playbooks that you can use as an automation job through Red Hat Ansible Automation Platform.
The Ansible validated content for network base focuses on abstract platform-agnostic network automation and enhances the experience of resource module consumption by providing production-ready content. This network base content acts as the core to the other network validated content, which I will explain more about in the examples below.
Network base use cases
The network.base Collection acts as a core for other Ansible validated content, as it provides the platform agnostic role called Resource Manager, which is the platform-agnostic entry point for managing all of the resources supported for a given network OS. It includes the Continue reading
Ansible Wisdom and ChatGPT: Putting it to the test
Image: DALL-E
Artificial intelligence (AI) is revolutionizing how we work and play in exciting ways. At first glance, AI tools, such as ChatGPT, seem to provide all the correct answers. But once we delve deeper and implement the suggestions, it often isn’t as effortless as it appears. This is especially true when generating code.
In this blog, we wanted to put ChatGPT to the test and see how it fares with developing Ansible Playbooks and share our results. We’ll also cover the experience and feedback from developers across domains.
We’ll also provide more information on our upcoming automation AI superpower, Project Wisdom.
First, let’s briefly discuss what ChatGPT is and how it works.
What is ChatGPT?
“We’ve trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.”
OpenAI ChatGPT release announcement
ChatGPT is a chatbot developed by OpenAI and built on top of their GPT (Generative Pre-trained Transformer) 3.5 large language model.
Large language models (LLM) are trained on massive amounts of data to predict the next word in a sentence. GPT 3. Continue reading
Deploy production grade automation technology at scale with Red Hat and F5
The complexity of enterprise networks is increasing with the widespread adoption of networks spanning on premises data centers and cloud. To meet the challenge of this complexity, automation technology specifically designed for a NetOpps approach for Hybrid Cloud environments has been jointly developed by Red Hat and F5. Its focus on security, scale, and speed are all goals inherent in a NetOpps strategy that allow teams to thrive in the increasingly complex world of enterprise networking.
Network teams also need to work well with groups across the IT stack while also taking advantage of the benefits of infrastructure as code. When everyone speaks the same language and uses the same revision control tools to deploy and update their infrastructure, IT infrastructure is more reliable, secure, downtime is reduced, and organizations save money.
F5 and the Red Hat Ansible Automation Platform delivers the features and functionality needed for a NetOpps team - with a powerful automation platform designed for simplicity and inter team cooperation. You will learn how F5 and Red Hat assist network teams transitioning to a NetOps workflow, while reducing risk and increasing productivity. From on-prem to the cloud, F5 and Red Hat can Continue reading
AnsibleFest 2022 Quick Take: expanding to AWS, Event-Driven Automation and Project Wisdom
AnsibleFest 2022 was our first in-person event in a few years, and it delivered some exciting news that will impact the growth and expansion of automation for our customers in the months to come. We had more than 450 organizations represented in person in Chicago. Our keynotes featured Red Hat, IBM Research, and Rockwell Automation. During the two days, we announced several new features and capabilities to make adopting automation more accessible. In addition, IDC analyst Jevin Jensen recently published his opinions and insights on AnsibleFest 2022 that we break down below .
What did we announce?
Each day of the event featured keynotes, one on the Current State of Automation and one on The Future of Automation. There was a lot of excitement over the many announcements, including:
- New Red Hat Ansible Automation Platform offering for AWS
- By offering Ansible Automation Platform as a pre-integrated service that can be quickly deployed from cloud marketplaces, we are meeting our customers where they are, while giving them the flexibility to deliver any application, anywhere, without additional overhead or complexity. Whether you are automating your hybrid cloud or multi-cloud environments, Ansible Automation Platform acts as a single platform. This platform Continue reading
AAP 2.3 Introducing Remote Execution Mesh Nodes for Openshift
Last year Ansible Automation Platform (AAP) 2 introduced major changes around the mechanics on how automation is run. The main focus was to enhance the foundational pieces of the platform while providing flexibility and simplicity for automators to automate at enterprise scale. One of those foundational pieces introduced was automation mesh.
Automation mesh provides a simple, flexible and reliable way to scale automation of large inventories across diverse network topologies, platforms and teams. It was important to evolve how the platform was developed, managed, operated to meet business demands and needs, for the large majority of customers.
Pre 2.3 Automation Mesh in OCP
Prior to AAP 2.3, the automation mesh was mostly a VM deployment base option and feature. I’ve tried to highlight the main differences between platform choices in this diagram:
We are now starting to level up the feature parity, and this brings remote mesh execution nodes to non-VM supported deployments. Before we look more at the new feature, let’s look at the options that were available before:
For VM based deployments, the automation mesh introduced horizontal scaling, the ability to scale your automation needs out, whether locally or globally. We did this by Continue reading
Addressing NetOps issues with Event-Driven Ansible
A simple example - No Shut, No Problem
Since the announcement of Event-Driven Ansible, I cannot stop thinking about potential use cases. Can I get events to automate scaling? Could I use a filesystem event to trigger filesystem integrity checks? Could I get a slackbot to trigger my choice of heavy metal playlist based on a “mood” event? It's all possible! But let’s not go too crazy, not yet.
I started having a look at the fantastic work that one of our engineers, Nilashish Chakraborty has been doing around network telemetry and Ansible. This led me down the path to explore network events and what I could potentially do with something like Event-Driven Ansible. So let’s start with a super simple interface example.
Reaching out to the team at Arista, we started discussing and looking at the mechanisms they are using to get telemetry data. With Arista we are able to use gNMI, gNMI is an open source protocol specification created by the OpenConfig working group that is used to stream data to and from network devices. The OpenConfig working group operates as an open source project with contributions from network operators, equipment vendors in providing vendor-neutral Continue reading