Adding integration tests to Ansible Content Collections
In the previous installment of our "let us create the best Ansible Content Collection ever" saga, we covered the DigitalOcean-related content migration process. What we ended up with was a fully functioning Ansible Content Collection that unfortunately had no tests. But not for long; we will be adding an integration test for the droplet module.
We do not need tests, right?
If we were able to write perfect code all of the time, there would be no need for tests. But unfortunately, this is not how things work in real life. Any modestly useful software has deadlines attached, which usually means that developers need to strike a compromise between polish and delivery speed.
For us, the Ansible Content Collections authors, having a semi-decent Collection of integration tests has two main benefits:
- We know that the tested code paths function as expected and produce desired results.
- We can catch the breaking changes in the upstream product that we are trying to automate.
The second point is especially crucial in the Ansible world, where one team of developers is usually responsible for the upstream product, and a separate group maintains Ansible content.
With the "why integration tests" behind us, we can Continue reading
Now Available: Red Hat-Maintained Content Collections on Automation Hub
Today marks an important milestone for Red Hat Ansible Automation Platform subscribers: The initial release of Red Hat-maintained Ansible Content Collections have been published to Automation Hub for automating select platforms from Arista, AWS, Cisco, IBM, Juniper, Splunk and more. The addition of these 17 Red Hat-maintained Collections on Automation Hub brings the total number to 47 Collections certified and published since September 2019. Finally, we are thrilled to have Ansible Collections for automating Red Hat Insights and Red Hat Satellite included as part of this release as well.
Why is this significant? First, it is important to understand that the Ansible project has recently completed an effort to decouple the Ansible executable from most of the content, and all migrated content now resides in new upstream repositories on GitHub. This change has had a ripple effect on backend development, testing, publishing, and maintenance on Ansible content. The good news is that now features of high quality, can be delivered more quickly, asynchronously from Ansible releases.
Today’s announcement highlights the successful culmination of the following:
- Migration of Ansible-maintained content from Ansible project to Collections.
- Releasing new features and functionality since Ansible 2.9, without having to wait Continue reading
Tolerable Ansible
Ansible Playbooks are very easy to read and their linear execution makes it simple to understand what will happen while a playbook is executing. Unfortunately, in some circumstances, the things you need to automate may not function in a linear fashion. For example, I was once asked to perform the following tasks with Ansible:
- Notify an external patching system to patch a Windows target
- Wait until the patching process was completed before moving on with the remaining playbooks tasks
While the request sounded simple, upon further investigation it would prove more challenging for the following reasons:
- The system patched the server asynchronously from the call. i.e. the call into the patching system would simply put the target node into a queue to be patched
- The patching process itself could last for several hours
- As part of the patching process the system would reboot no fewer than two times but with an unspecified maximum depending on the patches which need to be applied
- Due to the specific implementation of the patching system the only reliable way to tell if patching was completed was by interrogating a registry entry on the client
- If the patching took too long to complete additional Continue reading
Ansible Automates 2020
Today, the operational role of IT is obvious. The rapid developments enabled by automation create genuine business value. The results that can be achieved by automation have a direct link to a company’s business goals.
As a CTO or CIO, sometimes you need help articulating this to stakeholders. Translating IT departments’ performance into business prioritized KPIs. Most see efficiency gains, cost and risk reductions, for example. Automation is clearly an executive-level issue.
At first, Ansible was a classical tool that was utilized for specific automation. Ansible helps your team automate routine tasks, so that they can instead focus on what you want to do. The platform enables you to structure work by automating your processes.
Automation is a journey - start yours at Ansible Automates 2020
The global, all-day digital event – Ansible Automates 2020 – takes place on June 10. The event provides inspiration as to how the automation journey can be accelerated and taken to the next level. And no, we’re not going to discuss functionality and technology all day. We want to highlight the cultural and behavioral changes that are linked to the trend towards greater automation.
For organizations to achieve the best results, Continue reading
Installing and using collections on Ansible Tower
Ansible Collections are the new way to distribute and manage content. Ansible content can be modules, roles, plugins and even Ansible Playbooks. In my previous blog I provide a walkthrough of using Ansible Collections from Ansible Galaxy and Automation Hub. Ansible Galaxy is the upstream community for sharing Ansible Collections. Any community user can create a namespace and share content with anyone. Access to Automation Hub is included with a Red Hat Ansible Automation Platform subscription. Automation Hub only contains fully supported and certified content from Red Hat and our partners.
In this blog post we'll walk through using Ansible Collections with Ansible Tower, part of the Red Hat Ansible Automation Platform. There are a few differences between using command-line Ansible for syncing with Ansible Galaxy or the Automation Hub versus using Ansible Tower. However, it is really easy and I will show you how!
Accessing collections content from Automation Hub and Galaxy from Ansible Tower.
If the Ansible Collections are included in your project you do not need to authenticate to Automation Hub. This method is where you are downloading dynamically using a requirements file as outlined in my blog post. In general there are Continue reading
Automation Services Catalog: A Deep Dive into What’s New in Red Hat Ansible Automation Platform
Red Hat Ansible Automation Platform introduces automation services catalog, a new hosted service for Red Hat Ansible customers to extend their automation in a controlled way to the various end users who need it. This is a deep dive into the capabilities and features of the offering.
Users
The automation services catalog is designed to be a familiar experience, providing an easy and intuitive user interface for ordering products (automation resources).
Products to Order
The idea is that those using the automation services catalog may not know that what they are ordering is actually Ansible Automation. For example, a product could be a business function, like ordering a new OpenShift project or onboarding a user to a new platform.
Ordering a product will present the user with options to facilitate the order. This could be provisioning the datacenter or applying permissions for a Kubernetes project. Upon submitting the order, the user can see the progress in their order queue. Users can search for past orders and see those currently in progress indicated by statuses including: Order, Failed, Approval Pending and Completed. Orders that are pending approval can be compared with ordering a product from a website and seeing Continue reading
What’s New in Ansible Tower 3.7
We’re excited to announce the release of Ansible Tower 3.7, part of the Red Hat Ansible Automation Platform. Ansible Tower is the scalable execution framework of the Ansible Automation Platform, providing a REST API and UI framework that allows users to scale automation across their enterprise and integrate it into their processes and tools.
The focus of the Ansible Tower 3.7 release is on scalability of automation and improving the experience of our users.
Automate more without bottlenecks
These days automation often needs to work at large scales - both in terms of infrastructure, but also in terms of how many jobs are executed in parallel. With Ansible Tower 3.7 we have put in extra effort to handle job dependencies in a way that helps ensure your jobs aren’t blocked. By allowing project updates and inventory updates to happen while other jobs are running, we’ve eliminated many of the bottlenecks in job processing. This enables jobs to proceed faster and without the need to wait for each other.
Expand access to the Ansible Automation Platform without worries
Scale is not only about the IT technology itself, but also about users. As our customers Continue reading
Automation Analytics – ROI Calculator and Notifications enhancements
The Red Hat Ansible Automation Platform is continually offering enhancements through its hosted services on cloud.redhat.com. At Red Hat Summit 2020 the new automation services catalog took the spotlight, which provides lifecycle management, provisioning, retirement and cataloging of automation resources to your business. However I wanted to also talk about the additional new enhancements coming to Automation Analytics! Specifically I have two big things I want to talk about:
- Automations Calculator - a ROI (return on investment) calculator using aggregate data
- Notification improvements and a dedicated panel
If you are unfamiliar with Automation Analytics it is included as part of a Red Hat Ansible Automation Platform subscription and allows customers to analyze, aggregate, and report on data for their Red Hat Ansible Automation Platform deployments. Check out the previous blog I wrote about Getting Started with Automation Analytics, or if you have concerns around what type of data is being shared with Red Hat check out my blog Automation Analytics: Part 2 - Looking at Data Collection.
Automation Calculator
I am super excited about this new feature of Automation Analytics. A lot of customers I get to meet with are trying to figure out how to Continue reading
The latest in Red Hat Ansible Automation Platform – Using automation to effectively change your day to day
Last year, we made some significant changes to Red Hat Ansible Automation, including what is offered with it and alongside it, to bring you the first version of Red Hat Ansible Automation Platform. That change allowed users to harness the power of Ansible automation under one roof, one subscription - one platform.
Today, we are pleased to announce updates to Red Hat Ansible Automation Platform, the latest version of our enterprise-grade solution for building and operating automation at scale. The next time you log in to cloud.redhat.com, you can start utilizing the powerful new tools at your disposal. We’ve put in place the automation services catalog, a venue for developers and business users to manage, provision and retire resources. Customers told us this was a necessity and we agreed, with the automation services catalog now giving you a much deeper insight into how automation is improving efficiencies. It also gives you visibility into redundant processes that may be costing you time and resources that you may want to sunset. Accentuate what is working and eliminate what is not. More information about automation services catalog can be found here.
Another enhancement we rolled out with the Continue reading
Ansible and IBM Community Grid
The world is currently a very different place than it was only a few months ago and we have come up with some ideas on how we can help our community in dealing with this new reality. The Ansible team has started a “Here to Help” webinar series where myself and other Ansible engineers are spending time with smaller groups of people to try and help them with technical challenges: https://www.ansible.com/here-to-help-webinar-series. The goal of these webinars is strictly to help! Regardless of if folks are only using open source technologies and not Red Hat products, we want to use this time to help them solve automation challenges, and help us brainstorm use-cases that can help others.
Another idea we recently implemented is integrating IBM’s World Community Grid into our workshops. World Community Grid enables anyone with a Linux, Windows or Mac computer (or an Android smartphone for some projects) to donate their unused computing power to advance scientific research on topics related to health and sustainability. In fact, one of their projects is specifically going to help combat COVID-19. This blog post will cover what our workshops are and how we can use idle CPU time to help Continue reading
Automate App Monitoring with Ansible Platform & Dynatrace
The Dynatrace software intelligence platform automates the monitoring lifecycle. The OneAgent automatically discovers and instruments your applications, processes, containers, and log files. Smartscape topology provides real-time dependency mapping without any configuration. The Davis AI continuously analyzes metrics, traces, logs, dependencies, and more to automatically detect problems and determine the root cause. This automation helps enable organizations to monitor their IT portfolio more quickly and easily - without the headaches that can occur from manual configuration required by traditional monitoring tools.
Dynatrace is designed to work for any environment, but it’s generic. How can we automate the personalization of Dynatrace and enable monitoring as a self-service (MaaSS) Red Hat Ansible Automation Platform. With the power of automation provided by Red Hat Ansible Automation Platform and the Dynatrace API, we can automate the onboarding of applications into Dynatrace in a way that’s tailored for application stakeholders.
Dynatrace automation begins when the OneAgent is deployed on your hosts. The rollout of the OneAgent can be automated on hosts that are managed by Red Hat Ansible Automation Platform. A playbook will download and execute the OneAgent installer on your Linux hosts (via SSH) and your Windows Continue reading
AnsibleFest 2020 is now a virtual experience
Each year, AnsibleFest is one of our favorite events because it brings together our customers, partners, community members and Red Hatters to talk about the open source innovations and best practices that are enabling the future of enterprise technology and automation.
Because the safety of our attendees is our first priority, we have decided to make AnsibleFest 2020 a virtual experience. We are excited to connect with everyone virtually, and look forward to expanding our conversation across the globe. By changing our event platform, we hope to use this opportunity to collaborate, connect, and chat with more automation fans than ever before. It is exciting to think about how many more people will be able to join in on the automation conversation.
The AnsibleFest Virtual Experience will be a free, immersive multi-day event the week of October 12th, 2020, that will deliver timely and useful customer keynotes, breakout sessions, direct access to Ansible experts, and more. You will want to sign up to stay connected and up-to-date on all things AnsibleFest on the AnsibleFest page.
Call for proposals is open
We are still working through the details for the virtual event, but are very excited to announce that Continue reading
Using Cisco ACI as Inventory for Ansible Tower
Managing a software-defined networking (SDN) solution in Ansible can be tricky. In most use cases, Ansible communicates with each managed node individually. However, in a SDN scenario, Ansible is most likely managing policy on a controller appliance, which ultimately may make changes to thousands of network endpoints behind it.
But what about these endpoints behind that controller abstraction? Wouldn't it be best if Ansible Tower had visibility to every node, in addition to the controller? Not to run playbooks directly against those nodes, but for the following reasons:
- Many organizations need to perform capacity planning against current and projected license usage. Ansible Tower provides license usage data, but it's not accurate if Ansible Tower only knows about the SDN controller(s).
- Networking teams are still asked for physical node information by other IT areas, even if the focus is on a software-defined fabric. What if the best system to get that data from is Ansible Tower? You may not want a CMDB probe (for example) to communicate directly with the SDN controller, or Ansible Tower may already be home to similar data about other systems.
Inventory plugin for Cisco ACI
I wrote an Ansible inventory plugin that solves these issues Continue reading
Red Hat Ansible Tower Monitoring: Using Prometheus + Node Exporter + Grafana
A crucial piece of automation is ensuring that it runs flawlessly. Automation Analytics can help by providing insight into health state and organizational statistics. However, there is often the need to monitor the current state of Ansible Tower. Luckily, Ansible Tower does provide metrics via the API, and they can easily be fed into Grafana.
This blog post will outline how to monitor Ansible Tower environments by feeding Ansible Tower and operating system metrics into Grafana by using node_exporter & Prometheus.
To reach that goal we configure Ansible Tower metrics for Prometheus to be viewed via Grafana and we will use node_exporter to export the operating system metrics to an operating system (OS) dashboard in Grafana. Note that we use Red Hat Enterprise Linux 8 as the OS running Ansible Tower here. The data flow is outlined below:
As you see, Grafana looks for data in Prometheus. Prometheus itself collects the data in its database by importing them from node_exporters and from the Ansible Tower APIs.
In this blog post we assume a cluster of three Ansible Tower instances and an external database. Also please note that this blog post assumes an already installed instance of Prometheus and Grafana.
Continue reading
Introducing: The AWX Collection
Ansible Content Collections are a new way of distributing content, including modules, for Ansible. For detailed information on how to use collections in general, please read Colin McNaughton’s blog post about the topic.
The AWX Collection allows Ansible Playbooks to interact with AWX and Ansible Tower. Much like interacting with AWX or Red Hat Ansible Tower via the web-based UI or the API, the modules provided by the AWX Collection are another way to create, update or delete objects as well as perform tasks such as run jobs, configure Ansible Tower and more. This article will discuss new updates regarding this collection, as well as an example playbook and details on how to run it successfully.
The AWX Collection awx.awx is the upstream community distribution available on Ansible Galaxy. The downstream supported Ansible Collection ansible.tower is being targeted for mid-May on Automation Hub alongside the release of Ansible Tower 3.7. For more details on the difference between Ansible Galaxy and Automation Hub please refer to Ajay Chenampara’s blog post.
This collection is a replacement for the Ansible Tower web modules which were previously housed and maintained directly in the Ansible repo. The modules were Continue reading
Ansible Linting with GitHub Actions
Want to trigger linting to your Ansible deployment on every Pull Request?
In this blog, I will show you how to add some great automation into your Ansible code pipeline.
CI/CD is currently a pretty hot topic for developers. Operations teams can get started with some automated linting with GitHub actions. If you use GitHub you can lint your playbooks during different stages including git pushes or pull requests.
If you’re following good git flow practices and have an approval committee reviewing pull requests, this type of automated testing can save you a lot of time and keep your Ansible code nice and clean.
What is Ansible Lint?
Ansible Lint is an open source project that lints your Ansible code. The docs state that it checks playbooks for practices and behavior that could potentially be improved. It can be installed with pip and run manually on playbooks or set up in a pre-commit hook and run when you attempt a commit on your repo from the CLI.
The project can be found under the Ansible org on GitHub.
What are GitHub Actions?
From the GitHub documentation: GitHub actions enable you to create custom workflows to automate Continue reading
Getting started with Ansible and Check Point
The scale and complexity of modern infrastructures require not only that you be able to define a security policy for your systems, but also be able to apply that security policy programmatically or make changes as a response to external events. As such, the proper automation tooling is a necessary building block to allow you to apply the appropriate actions in a fast, simple and consistent manner.
Check Point has a certified Ansible Content Collection of modules to help enable organizations to automate their response and remediation practices, and to embrace the DevOps model to accelerate application deployment with operational efficiency. The modules, based on Check Point security management APIs* are also available on Ansible Galaxy, in the upstream version of Check Point Collection for the Management Server.
The operational flow is exactly the same for the API as it is for the Check Point security management GUI SmartConsole, i.e. Login > Get Session > Do changes > Publish > Logout.
Security professionals can leverage these modules to automate various tasks for the identification, search, and response to security events. Additionally, in combination with other modules that are part of Ansible security automation, existing Continue reading
Migrating Existing Content Into a Dedicated Ansible Collection
Today, we will demonstrate how to migrate part of the existing Ansible content (modules and plugins) into a dedicated Ansible Collection. We will be using modules for managing DigitalOcean's resources as an example so you can follow along and test your development setup. But first, let us get the big question out of the way: Why would we want to do that?
Ansible on a Diet
In late March 2020, Ansible's main development branch lost almost all of its modules and plugins. Where did they go? Many of them moved to the ansible-collections GitHub organization. More specifically, the vast majority landed in the community.general GitHub repository that serves as their temporary home (refer to the Community overview README for more information).
The ultimate goal is to get as much content in the community.general Ansible Collection "adopted" by a caring team of developers and moved into a dedicated upstream location, with a dedicated Galaxy namespace. Maintainers of the newly migrated Ansible Collection can then set up the development and release processes as they see fit, (almost) free from the requirements of the comunity.general collection. For more information about the future of Ansible content delivery, please Continue reading
Getting started with Ansible security automation: Investigation Enrichment
Last November we introduced Ansible security automation as our answer to the lack of integration across the IT security industry. Let's have a closer look at one of the scenarios where Ansible can facilitate typical operational challenges of security practitioners.
A big portion of security practitioners' daily activity is dedicated to investigative tasks. Enrichment is one of those tasks, and could be both repetitive and time-consuming, making it a perfect candidate for automation. Streamlining these processes can free up their analysts to focus on more strategic tasks, accelerate the response in time-sensitive situations and reduce human errors. However, in many large organizations , the multiple security solutions aspect of these activities are not integrated with each other. Hence, different teams may be in charge of different aspects of IT security, sometimes with no processes in common.
That often leads to manual work and interaction between people of different teams which can be error-prone and above all, slow. So when something suspicious happens and further attention is needed, security teams spend a lot of valuable time operating on many different security solutions and coordinating work with other teams, instead of focusing on the suspicious activity directly.
In this blog post we Continue reading
Ops by Pull Request: An Ansible GitOps Story
In a previous blog post I introduced Automation Webhooks and their uses with Infrastructure-as-Code (IaC) workflows and Red Hat Ansible Automation Platform. In this blog post, I’ll cover how those features can be applied to creating GitOps pipelines, a particular workflow gaining popularity in the cloud-native space, using Ansible and the unique benefits utilizing Ansible provides.
What is GitOps?
Like so many terms that evolve and emerge from the insights and practices of what came before it, finding a definitive meaning to the term “GitOps” is a bit elusive.
GitOps is a workflow whose conceptual roots started with Martin Fowler’s comprehensive Continuous Integration overview in 2006 and descends from Site Reliability Engineering (SRE), DevOps culture and Infrastructure as Code (IaC) patterns. What makes it unique is that GitOps is a prescriptive style of Infrastructure as Code based on the experience and wisdom of what works in deploying and managing large, sophisticated, distributed and cloud-native systems. So you can implement git-centric workflows where you treat infrastructure like it is code, but it doesn’t mean it’s GitOps.
The term GitOps was coined by Alexis Richardson, CEO and Founder of Weaveworks, so a lot of how I’m going to define Continue reading