Ansible Community – 2016 Year in Review
It's time again for the annual Ansible community review. Time flies. Our upward trajectory generally continued from 2014 and 2015 through 2016, and our growth continued to bring new challenges and new opportunities.
Let's start again, as we do every year, with a look at the numbers.
Debian Popcon
Debian’s Popularity Contest is an opt-in way for Debian users to share information about the software they’re running on their systems.
Caveats abound with this graph -- but even though it represents only a small sample of the Linux distro world, it’s useful because it’s one of the few places where we can really see an apples-to-apples comparison of install bases of the various tools. Because Ansible is agentless, we compare the Ansible package to the server packages of other configuration management tools. (Chef does not make a Debian package available for Chef server.)
We see that Ansible has continued its steady growth through the end of 2016, nearly doubling its Popcon install base again in 2016.
Github Stars
Ansible continued in 2016 to extend its already significant lead in GitHub Stars over other tools in the configuration management space, passing the 20k mark in December 2016.
Github Contributors
Why IT Automation is Becoming a Business Necessity
Automation transformed factories. It gave manufacturing the ability to perform work faster, more efficiently, at higher quality. Processes became predictable. Productivity thrived. Factories that failed to automate fell behind. Automation became a business imperative.
IT departments are the modern factories powering today’s digital businesses. And just as today’s factories can’t compete without automation, automation will soon become imperative for IT organizations. Here’s why:
- Application delivery is fuel for growth. Today every business is a software business, regardless of whether you actually sell software. Every business depends on critical systems to engage with customers and gather data. For companies competing in the knowledge economy, IT operations is oxygen. The demands businesses place on DevOps teams will only continue to increase. We’re not going back—only forward, faster.
- Automation simplifies processes. Automation never sleeps. Throwing people at the problem helps companies grow, but the law of diminishing returns limits your ability to scale. Adding people adds complexity and costs.
- IT professionals don’t want to repeat the same tasks over and over. Just as manual work in factories is tiring, endless, and thankless—repetitive work squanders the creative energy of your best and often most highly paid people.
- Automation speeds the work so your people Continue reading
How to Extend Ansible Through Plugins
Did you know a large portion of Ansible’s functionality comes from the Ansible plugin system? These important pieces of code augment Ansible’s core functionality such as parsing and loading inventory and Playbooks, running Playbooks and reading the results. Essentially, Ansible uses plugins to extend what the system is doing under the hood.
In this blog, I’ll review each of these plugins and offer a high-level overview on how to write your own plugin to extend Ansible functionality.
Action Plugins
One of the core critical plugins used by Ansible are action plugins. Anytime you run a module, Ansible first runs an action plugin.
Action plugins are a layer between the executor engine and the module and allow for controller-side actions to be taken before the module is executed. A good example of this is the template module. If you look at template.py in the modules directory, it’s basically a Python stub with documentation strings, everything is done by the action plugin. The template action plugin itself creates the template file locally as a temporary file, and then uses the copy or file modules to push it out to the target system.
If Ansible finds an action plugin with the Continue reading
Our Most Popular Blog Posts of 2016
2016, what a year. Ansible upgrades galore, Tower 3 was released, a tipping point for DevOps, and much more.
All these themes were reflected in our blog this year. From doing more with automation, working across platforms (think Windows automation), orchestrating containers at scale, to exploring all the great new features in Tower 3, we covered a lot.
Just in case you missed them, here are our 10 most viewed blog posts of the year (plus a sneaky few honorable mentions).
1. 6 Ways Ansible Makes Docker-Compose Better
Containers are an integral part of DevOps workflows. With containers you can be sure that if you build an application once, you can run it in the same way across every environment along the application lifecycle. That’s great, until one developer announces the need for a second, third, or fourth container. More of them, all doing different things, and all connecting together – somehow. But how? Docker has a tool that can help – docker-compose. But it’s limited to environments with a Docker-centric view of the world and doesn’t solve non-Docker orchestration problems. That’s where Ansible comes in. Read more
2. Testing Ansible Roles with Docker
Ansible plus Docker was a big deal in 2016, Continue reading
An Early Look at Ansible Container v0.3.0
The Ansible Container project is targeting mid-January for its next release, and so we thought now would be a good time to check in and look at the features actively under development and anticipated to ship.
With only a glance at the roadmap page, the casual visitor may think it seems a bit smallish, having only three items. However, the items represent features that are important to the project, and require a level of effort that’s anything but small, as we’ll see.
Building container images
The first item up is an image build cache. Building container images is of course a core function of the tool, and having a caching mechanism can improve the speed at which images are built.
If you’re not familiar with container images and how they’re built, think of an image as a tall building with dozens of floors, where each floor is layered on top of the previous floor, starting with the building’s foundation, and adding one floor or layer at a time until you reach the top. In the same way, a container image is a file system built in layers.
The build process starts with a base image, say Fedora 25, Continue reading
‘Tis the Season to Decorate Your Playbook
The holidays are upon us. And while we can't automate your gift wrapping, we can make your Playbook more festive.
Introducing Ansible holiday-themed cowsay!
Copy and paste this code:
---
curl "https://gist.githubusercontent.com/jlaska/14bc829af01526add07efcaa83582aaf/raw/5f4918be06ffa69ea848354c5563c6a2d7b59807/happy_holidays.sh" | bash
Decorations are popping up everywhere so why not in your Playbook?
Share your designs with #AnsibleCowsay @ansible
Tweet #AnsibleCowsayAscii sources: chris.com and ascii-code.com
Security is Hard. Why Not Automate It?
Security automation doesn't get a lot of time in the spotlight, but it's something that deserves more thought and attention. The almost constant stream of headlines about the latest data breach or large scale hack make these seem like inevitable events that can't be prevented. Nothing could be further from the truth.
In reality, most data breaches are easily prevented by applying basic security standards and fixing known vulnerabilities in a timely manner. Covering the basics frees information security teams to monitor, detect, and stop more advanced attacks. Hardened systems also make life difficult for attackers, which is always a good thing.
But where does Ansible fit into all this? Ansible is great for configuration management, continuous integration and delivery, orchestration, application deployment and even infrastructure provision. But “I solved a security problem with Ansible” might not be what you’re accustomed to hearing at the water cooler from your average information security or operations person.
To understand how Ansible fits into to the security picture, it’s worth stepping back a little to understand exactly what is information security.
Information security is a multifaceted problem
Keeping information secure isn’t easy. We all know what needs to be Continue reading
IT Automation is in the Spotlight at AWS Re:Invent (and OpsWorks Configuration Management is Just Part of the Story)
Configuration management is just the start
Automation is getting a lot of attention at AWS:ReInvent this year, as people are noticing that automation is drastically accelerating the pace of innovation within IT organizations. Whether they’re part of a DevOps initiative, attempting to modernize their existing processes, or migrating systems and applications to the cloud, infrastructure-as-code style automation is playing an increasingly bigger part in these efforts - and ‘configuration management’ is getting most of the attention.
In a recent study, IDC’s Melinda-Carol Ballou predicts that the configuration management portion of I&O spending will grow at 8% over the next several years… but the predicted growth of configuration management in public clouds is north of 31%. Similarly, in a separate report, Mary Johnston Turner and David Laing forecast the Automation component of Infrastructure spending in the public cloud to grow at almost 35% - compared to just 12% overall.
The trend is clear: configuration management is seen as critical to cloud adoption and migration. So it’s not surprising that Amazon Web Services announced that it is updating its Opsworks service offering. As environments grow in size, scope, and complexity (which is the new normal in the era of the cloud), the Continue reading
Security and Delegation with Ansible Tower, Part 2
When we talk about Ansible Tower, we talk about control, knowledge and delegation. But what does that mean?
In previous posts in this series, we've talked about the concept of 'control' for your automation and your inventory, and about the basics of security and delegation. Today we're going to show how Tower's security and delegation allows for simple self-service deployments for your users.
THE PROBLEM - SERVICE REQUESTS
If you've ever run a service desk, you've probably dealt with repetitive ticket requests. There's not a lot of thought involved - taking customer and user requests, then punching those values into commands or random scripts. You need a solution that lets you automate the boring work and get you back to the important work.You need Ansible and Ansible Tower.
THE SOLUTION - ANSIBLE, TOWER AND TOWER SURVEYS
One of Tower's key features is survey support. Tower surveys allow you to configure how a job runs via a series of questions, making it simple to customize your jobs in a user-friendly way.Say you have a team of developers. What they need is the ability to get their dev environments set up in the cloud quickly, easily and properly. What you Continue reading
The Future of Automation
Automation is a hot topic. And the automation concept that has captured our imagination the most is the idea of self-driving cars. This is the kind of automation that we can see dramatically changing what we do everyday, within grasp in our lifetime.
The automotive industry, the press, and parents of 16 year-olds will tell you that the promise of self-driving cars is all about societal benefits: people make mistakes, people don’t always have the best information, and people have to drive with…other people. We can keep everybody safe if everybody just moves to self-driving cars.
But I believe that the benefits of automation are actually much more personal. It’s about getting your life back. Let’s let the computers do the tasks that are mundane and that we shouldn’t be spending time on - like waiting in traffic - so that way we can focus on the things that are important to us.
Automation is intensely personal. It’s not necessarily for the organization, although it it does help. And it’s not just for efficiency’s sake or business sake. At the end of the day, it’s really about helping people.
Automation sometimes gets a bad rap because people think, “Well, if I automate my Continue reading
Security and Delegation with Ansible Tower, Part 1
This is part of a series of posts about how Ansible and Ansible Tower enable you to manage your infrastructure simply, securely, and efficiently.
When we talk about Tower, we often talk in terms of control, knowledge, and delegation. But what does that mean? In previous posts in this series, we've talked about the concept of 'control', as it relates to both managing your infrastructure and managing your automation. Today we're going to explain delegation, and the security aspects that go into that.
DELEGATION - THE BASICS
Ansible Playbooks, out of the box, are pretty simple - you run Ansible as a particular user, you pass it whatever inventory you want to manage, and it uses whatever credentials the executing user happens to have on hand. This is great for getting automating quickly, but what if you want to delegate automation to someone else to run as needed? You need to provide an inventory file for Ansible and Playbook to them (hopefully they don't edit them), and give them credentials (hopefully they won't use them for something else).That's where the control, knowledge, and delegation features of Ansible Tower come into play.
STEP 1: CREDENTIALS
Ansible Tower securely stores credentials for Continue readingAnsible 2.2 Network Updates
The Slack channel question seemed so innocuous at the time, “I was reviewing through the Ansible 2.2 commits related to networking. Is there a summary of the networking items that are new in 2.2?”
In a rather quick response, my first answer seemed so obvious, “Not really, mostly simplifying code, merging template with config modules and some new platforms."
As the conversation continued though, reality came crashing down with the realization that the sprint from Ansible 2.1 to Ansible 2.2 for networking modules was substantially more than a few tweaks and added platforms.
Before getting into what’s new and what’s changed, let's review the overall state of network integration with Ansible. We started this journey just about a year ago announcing that Ansible would start supporting direct integration with network devices. At the time, this was a fairly big departure from the more traditional roots where Ansible has focused on in the systems and application development worlds. There always seemed to be a natural fit between Ansible’s agentless, SSH-based architecture’s ability to adapt to automating traditional network device configurations. It didn’t take long for the initial integration of network modules to start achieving greater adoption.
In just three releases (counting Continue reading
What’s New in Tower 3: Search
In July, we released Ansible Tower 3. In this blog series, we will take a deeper dive into Tower changes designed to make our product simpler and easier to scale Ansible automation across your environments. In our last post, our Principal Software Engineer Matt Jones highlighted what’s new for Tower 3 notifications.
If you’d like to learn more about the release, our Director of Product Bill Nottingham wrote a complete overview of the Tower 3 updates.
Simplifying Search
Tower gives you the ability to manage the use of Ansible for an entire enterprise, allowing you to potentially manage hundreds of Playbooks and thousands of hosts in a single place. Tower also gives you the ability to audit all types of usage through its integrated activity stream, which can be verbose. One of the goals of Tower 3 was to provide a consistent and full-featured user interface for dealing with these large collections of data.
In Tower 2, searching these collections was pretty limited. You could only make a single query for a particular collection. In Tower 3, we developed and implemented a tag-based searching system throughout the app, allowing you to chain multiple queries together so that you can quickly Continue reading
What’s New in Tower 3: Notifications
In July, we released Ansible Tower 3. In this blog series, we will take a deeper dive into Tower changes that were all designed to make our product simpler and easier to scale Ansible automation across your environments. In our last post, Jared Tabor, Ansible Tower Software Engineer, highlights what's new in the Tower 3 user interface.
If you’d like to learn more about the release, our Director of Product Bill Nottingham for wrote a complete overview of the Ansible Tower 3 updates.
Introducing Tower Notifications
One of the exciting new features of Ansible Tower 3 is notifications. Tower notifications provide a mechanism of signaling when Tower jobs succeed or fail. This can take the form of sending a message to a Slack channel or sending an HTTP POST to another service to trigger other actions.
In Tower 3 we support the following notification types:
- Slack
- Hipchat
- Pagerduty
- Twilio
- IRC
- Webhook (POST)
Concepts
There are two important concepts to understand when it comes to notifications in Tower.
1. Notification Templates
These define the properties of where a notification will be sent and who will get notified. If you are using the Slack notification type, then this will include the token and Continue reading
Migrating the Runbook – a Journey from Legacy to DevOps
"Just type this invoice up for me will you please?" asked a sheepish looking Malcolm.
"I do have better things to do you know" I replied.
"Yes, yes, I know. But who else is going to do it?"
"Give it here then!"
In the beginning, there was a problem
That was a fairly common interaction for me as a young lad. I was fresh out of school and working my summer in the sales department of a local car dealership. My job was mostly admin related tasks, which up until that point hadn't included doing all the sales guys' typing. Our secretary had recently departed the company, and the sales guys all figured I could happily do the replacement typing jobs. The duty had fallen to me because a) I had the stereotypical 1980s glasses of a nerdy computer kid and b) they all knew I actually was a nerdy computer kid. So fair play to them for assuming I could type, I could.
The thing was I really did have better things to do, and these daily interruptions were eating into my productive time. I wanted that time back; so Continue reading
Ansible Container 0.2.0 Release
We’re excited to announce the release of Ansible Container 0.2.0. The last few months have been exciting. We’ve been working at a fever pitch to add new features, build examples, and resolve issues, while at the same time we’ve seen the interest level and participation rate of the project steadily grow. It’s been amazing, and we’re grateful to all those that helped by opening issues, contributing code, and spreading the word. Thank you!
Throughout this release cycle we heard from a number of users that being able to reuse existing Ansible content was critical. We focused on that, making Ansible roles a key part of this release. We came up with several enhancements that make it easy to access existing Ansible roles during the container build process. We added a feature to assist in retrofitting existing roles to be ‘container aware' and we looked to the future and imagined new ways roles could enhance the process of building and sharing containers.
Accessing Existing Roles
We heard several times that incorporating existing Ansible roles into the container build process needed to be easier. We solved this by creating a method for accessing roles from the local file system as Continue reading
What’s New in Tower 3: User Interface
In July, we released Ansible Tower 3. In this blog series, we will take a deeper dive into Tower changes that were all designed to make our product simpler and easier to scale Ansible automation across your environments. In our last post, our Senior Software Engineer Chris Meyers highlights what's new in the Tower 3 installer.
If you’d like to learn more about the release, our Director of Product Bill Nottingham for wrote a complete overview of the Ansible Tower 3 updates.
Simplifying the UX
The most common feedback we have received from existing Tower users concerns usability and the need to improve it. The Ansible Tower UI team was tasked to address this, along with new workflows and features, during the development of Tower 3. This was no small task as the team had to change every single page served to the user.
Tower 2.4.5 and earlier versions offered many ways of doing the same thing, often resulting in inconsistent flows and context switching. The team wanted the new interface to reflect how simple Ansible is. So the goal became offering a common flow for interacting with objects in the app and providing more context where possible.
Ansible in GitHub’s Octoverse 2016
Recently GitHub released their State of the Octoverse 2016 which shows some really nice statistics and graphs of top projects, languages and organizations working on open source.
GitHub, in the spirit of full transparency, shared the methodology and queries used to generate the report. We used this dataset to understand where Ansible stacks up. One of the drawbacks in the approach where you are just considering single repositories is that you don’t get a good idea of where a single project broken out into multiple repositories would fall. In Ansible’s case, the project is broken down into three repositories:
Let's look at GitHub’s first graphic, Repositories with the most open source contributors. When you just consider Ansible’s Core Project repo we’re just barely out of the top 10 at 11th place.1
What does that number look like if we combine all three repositories that make up the Ansible project?2 
As far as projects go that would have us in 5th just behind Patchwork.
One of the facets that was most strange to me was the inclusion of Comments Continue reading
Real-time Insights Into App Delivery with Ansible Tower Data in Splunk
Here at Ansible by Red Hat, we’re always looking for ways to make Ansible more useful when automating all the things.
That being said, most people know this UI when they see it:
When we ask Ansible users about their favorite tools, Splunk is a very common answer. Splunk software is at its most powerful when it is used to aggregate and correlate data from numerous sources across your environment. However, there hasn't been an easy way to use Splunk to analyze data from Ansible Tower job runs.
Not any longer. Today we’re happy to announce the result of our latest integration project - the Ansible Tower App for Splunk.
The value of analytics platforms such as Splunk, is the ability to collect and correlate machine data including environment events with the actions that caused them. Application lifecycle management teams need the ability to correlate deployment-related data (i.e. Tower job runs) with host events (i.e. system and service logs).
Picture the following scenario:
A development team is working to release a new version of their application. What is the easiest way for a team to validate the success of the application deployment process?
The Ansible Tower app for Splunk allows a team to deploy Continue reading
What’s New in Tower 3: Installer
In July, we released Ansible Tower 3. This blog series is a deep dive into some of the new aspects of Tower. We've reworked Tower to make it simpler and easier to automate your environments and share your automation solutions. For a complete overview of the Tower 3 updates, check out this post by Bill Nottingham, Director of Product.
Installer configuration in Tower
Before we look at what's new, let’s remember the < 3.0 installer - referred to hereafter as the legacy installer. The legacy installer configuration was designed to be ran by users without Ansible knowledge.
This requirement led to the two step process:
Step 1:
./configure prompts the user for the needed configuration information to setup Tower. This includes things like: tower mode (i.e. single machine, remote database, HA), ssh connection information, and service passwords. The Ansible variable file, tower_setup_conf.yml, is generated to be consumed by the ./setup.sh script.
tower_setup_conf.yml admin_password: password database: internal pg_password: BQgA2Z43jv86dzjDEswH7K75LAwufzSXbE7jUztq primary_machine: localhost redis_password: S3tab7QfWe2e92JEB9hNNFUunV4ircg3EdRdjpxP
Step 2:
./setup.sh wraps the Ansible install.yml, backup.yml, and restore.yml playbooks and passes in the appropriate run-time flag to include the previously generated configuration variable file and manage the generated logs. The . Continue reading