Archive

Category Archives for "Systems"

Getting started with Ansible security automation: Investigation Enrichment

Last November we introduced Ansible security automation as our answer to the lack of integration across the IT security industry. Let's have a closer look at one of the scenarios where Ansible can facilitate typical operational challenges of security practitioners.

A big portion of security practitioners' daily activity is dedicated to investigative tasks. Enrichment is one of those tasks, and could be both repetitive and time-consuming, making it a perfect candidate for automation. Streamlining these processes can free up their analysts to focus on more strategic tasks, accelerate the response in time-sensitive situations and reduce human errors. However, in many large organizations , the multiple security solutions aspect of these activities are not integrated with each other. Hence, different teams may be in charge of different aspects of IT security, sometimes with no processes in common.

That often leads to manual work and interaction between people of different teams which can be error-prone and above all, slow. So when something suspicious happens and further attention is needed, security teams spend a lot of valuable time operating on many different security solutions and coordinating work with other teams, instead of focusing on the suspicious activity directly.

In this blog post we Continue reading

Making File URLs Work Again in Firefox

At some point in the last year or so—I don’t know exactly when it happened—Firefox, along with most of the other major browsers, stopped working with file:// URLs. This is a shame, because I like using Markdown for presentations (at least, when it’s a presentation where I don’t need to collaborate with others). However, using this sort of approach generally requires support for file:// URLs (or requires running a local web server). In this post, I’ll show you how to make file:// URLs work again in Firefox.

I tested this procedure using Firefox 74 on Ubuntu, but it should work on any platform on which Firefox is supported. Note that the locations of the user.js file will vary from OS to OS; see this MozillaZine Knowledge Base entry for more details.

Here’s the process I followed:

  1. Create the user.js file (it doesn’t exist by default) in the correct location for your Firefox profile. (Refer to the MozillaZine KB article linked above for exactly where that is on your OS.)

  2. In the user.js, add these entries:

    // Allow file:// links
user_pref("capability.policy.policynames", "localfilelinks");
user_pref("capability.policy.localfilelinks.sites", "file://");
user_pref("capability.policy.localfilelinks.checkloaduri. Continue reading

Installing MultiMarkdown 6 on Ubuntu 19.10

Markdown is a core part of many of my workflows. For quite a while, I’ve used Fletcher Penny’s MultiMarkdown processor (available on GitHub) on my various systems. Fletcher offers binary builds for Windows and macOS, but not a Linux binary. Three years ago, I wrote a post on how to compile MultiMarkdown 6 for a Fedora-based system. In this post, I’ll share how to compile it on an Ubuntu-based system.

Just as in the Fedora post, I used Vagrant with the Libvirt provider to spin up a temporary build VM.

In this clean build VM, I perform the following steps to build a multimarkdown binary:

  1. Install the necessary packages with this command:

    sudo apt install gcc make cmake git build-essential

  2. Clone the source code repository:

    git clone https://github.com/fletcher/MultiMarkdown-6

  3. Switch into the directory where the repository was cloned and run these commands to build the binary:

    make
cd build
make

  4. Once the second make command is done, you’re left with a multimarkdown binary. Copy that to the host system (scp works fine). Use vagrant destroy to clean up the temporary build VM once you’ve copied the binary to your host system.

And with that, you’re good to go!

Join our new Docker Desktop Developer Preview Program!

Docker Desktop is getting ready to celebrate its fourth birthday in June this year. We have come a long way from our first version and have big plans of what we would like to do next. As part of our future plans we are going to be kicking off a new early access program for Docker Desktop called Docker Desktop Developer Preview and we need your help!

What is this Program about and what are the benefits?

This program is for a small number of heavy Docker Desktop users who want to interact with the Docker team and impact the future of Docker Desktop for millions of users around the world.

As a member of this group we will be working with you to look at and experiment with our new features. You will get direct access to the people who are building Docker Desktop everyday. You will meet with our engineering team, product manager and community leads, to share your feedback, tell us what is working in our new features and how we could improve, and also help us really dig in when something doesn’t work quite right. 

On top of that, you will have a chance to Continue reading

Setting up etcd with Kubeadm, containerd Edition

In late 2018, I wrote a couple of blog posts on using kubeadm to set up an etcd cluster. The first one was this post, which used kubeadm only to generate the TLS certs but ran etcd as a systemd service. I followed up that up a couple months later with this post, which used kubeadm to run etcd as a static Pod on each system. It’s that latter post—running etcd as a static Pod on each system in the cluster—that I’ll be revisiting in this post, only this time using containerd as the container runtime instead of Docker.

This post assumes you’ve already created the VMs/instances on which etcd will run, that an appropriate version of Linux is installed (I’ll be using Ubuntu LTS 18.04.4), and that the appropriate packages have been installed. This post also assumes that you’ve already made sure that the correct etcd ports have been opened between the VMs/instances, so that etcd can communicate properly.

Finally, this post builds upon the official Kubernetes documentation on setting up an etcd cluster using kubeadm. The official guide assumes the use of Docker, whereas this post will focus on using containerd as the container Continue reading

Update from Docker on COVID-19 Actions

As the novel coronavirus causing COVID-19 continues to spread, Docker has been taking precautionary measures to support the health, well-being, and safety of our global team members and their families, as well as ensuring our customers and community at large can continue building and shipping apps using Docker. We are also following the World Health Organization (WHO) and the Center for Disease Control and Prevention (CDC) guidelines, as well as guidelines from local public health administrations. 

Docker has always been about community, and here are the steps we have taken to ensure employees are taken care of as well as to ensure business continuity for our users worldwide:

Protecting Employees

  • On March 2, 2020, we asked all global employees to cancel or postpone any non-essential, work-related travel. 
  • Additionally, on March 9, 2020, we closed all of our offices globally to employees and visitors. We are using all available technologies like our phones, Zoom, Slack, GitHub, and Confluence now that we have transitioned to a fully remote workforce. 
  • While Docker is a geographically distributed organization, we understand this is a big shift for many of our employees and, as such, we are encouraging as much flexibility around work Continue reading

Ops by Pull Request: An Ansible GitOps Story

In a previous blog post I introduced Automation Webhooks and their uses with Infrastructure-as-Code (IaC) workflows and Red Hat Ansible Automation Platform. In this blog post, I’ll cover how those features can be applied to creating GitOps pipelines, a particular workflow gaining popularity in the cloud-native space, using Ansible and the unique benefits utilizing Ansible provides. 

 

What is GitOps?

Like so many terms that evolve and emerge from the insights and practices of what came before it, finding a definitive meaning to the term “GitOps” is a bit elusive. 

GitOps is a workflow whose conceptual roots started with Martin Fowler’s comprehensive Continuous Integration overview in 2006 and descends from Site Reliability Engineering (SRE), DevOps culture and Infrastructure as Code (IaC) patterns. What makes it unique is that GitOps is a prescriptive style of Infrastructure as Code based on the experience and wisdom of what works in deploying and managing large, sophisticated, distributed and cloud-native systems. So you can implement git-centric workflows where you treat infrastructure like it is code, but it doesn’t mean it’s GitOps.

The term GitOps was coined by Alexis Richardson, CEO and Founder of Weaveworks, so a lot of how I’m going to define Continue reading

Multi-Platform Docker Builds

This is a guest post from Docker Captain Adrian Mouat who is Chief Scientist at Container Solutions, a cloud-native consultancy and Kubernetes Certified Service Provider. Adrian is the author of “Using Docker,” published by O’Reilly Media. He is currently developing Trow, a container image registry designed to securely manage the flow of images in a Kubernetes cluster. Adrian is a regular conference speaker and trainer and he has spoken at several events including KubeCon EU, DockerCon, CraftConf, TuringFest and GOTO Amsterdam.

Docker images have become a standard tool for testing and deploying new and third-party software. I’m the main developer of the open source Trow registry and Docker images are the primary way people install the tool. If I didn’t provide images, others would end up rolling their own which would duplicate work and create maintenance issues.

By default, the Docker images we create run on the linux/amd64 platform. This works for the majority of development machines and cloud providers but leaves users of other platforms out in the cold. This is a substantial audience – think of home-labs built from Raspberry Pis, companies producing IoT devices, organisations running on IBM mainframes and clouds utilising low-power arm64 chips. Users of Continue reading

HA Kubernetes Clusters on AWS with Cluster API v1alpha3

A few weeks ago, I published a post on HA Kubernetes clusters on AWS with Cluster API v1alpha2. That post was itself a follow-up to a post I wrote in September 2019 on setting up HA clusters using Cluster API v1alpha1. In this post, I’ll follow up on both of those posts with a look at setting up HA Kubernetes clusters on AWS using Cluster API v1alpha3. Although this post is similar to the v1alpha2 post, be aware there are some notable changes in v1alpha3, particularly with regard to the control plane.

If you’re not yet familiar with Cluster API, take a look at this high-level overview I wrote in August 2019. That post will provide an explanation of the project’s goals as well as provide some terminology.

In this post, I won’t discuss the process of establishing a management cluster; I’m assuming your Cluster API management cluster is already up and running. (I do have some articles in the content pipeline that discuss creating a management cluster.) Instead, this post will focus on creating a highly-available workload cluster. By “highly available,” I mean a cluster with multiple control plane nodes that are distributed across multiple availability zones (AZs). Continue reading

Awesome-Compose: Application samples for project development kickoff

Software systems have become quite complex nowadays. A system may consist of several distributed services, each one providing a specific functionality and being updated independently. Starting the development of a project of such complexity is sometimes time consuming, in particular when you are not already familiar with the software stack you are going to work on. This may be because, most of the time, we need to follow rigorous steps to put together the entire project and if we make a mistake in-between, we may have to start all over again.

As a developer, getting a quick understanding of how all the stack is wired together and having an easy to manage project skeleton may be a very good incentive to use that particular stack for future projects. 

Furthermore, there are plenty of open-source software stacks that developers can use for their own setups. Providing them with a straightforward way to deploy these software stacks in order to check them out is an important matter when looking to simplify software development and allow developers to explore different options. 

To tackle this, we have put together a Github repository with application samples that can be easily deployed with Docker Continue reading

Hands On With Ansible collections

Ansible collections have been introduced previously through two of our blogs Getting Started with Ansible Content Collections and The Future of Ansible Content Delivery. In essence, Ansible Automation content is going to be delivered using the collection packaging mechanism.  Ansible Content refers to Ansible Playbooks, modules, module utilities and plugins. Basically all the Ansible tools that users use to create their Ansible Automation. Content is divided between two repositories:

  1. Ansible Galaxy (https://galaxy.ansible.com)
  2. Automation Hub (https://cloud.redhat.com/ansible/automation-hub)

Ansible Galaxy is the upstream community for sharing Ansible Collections.  Any community user can create a namespace and share content with anyone. Access to Automation Hub is included with a Red Hat Ansible Automation Platform subscription.  Automation Hub only contains fully supported and certified content from Red Hat and our partners. This makes it easier for Red Hat customers to determine which content is the official certified, and importantly supported, content.  This includes full content from partners such as Arista, Cisco, Checkpoint, F5, IBM, Microsoft and NetApp. 

In this blog post we'll walk through a use case wherein, the user would like to use a Red Hat certified collection from Automation Hub Continue reading

#myDockerBday Discounts on Docker Captain Docker + Kubernetes Content

If your #myDockerBday celebration included wanting to learn more about Docker or Kubernetes, you are in luck. In honor of Docker’s 7th birthday, Docker Captains have extended some fantastic deals on their learning content. Take this opportunity to level up your skills and learn Docker with excellent instructors. 

Books and EBooks

Through the end of March, you can get Elton Stoneman’s Learn Docker in a Month of Lunches and/or Jeff Nickeloff’s Docker in Action 2nd Edition for 40% off using the code mtpdocker20.

Learn Docker in a Month of Lunches

Elton Stoneman
Go from zero to production readiness with Docker in 22 bite-sized lessons! Learn Docker in a Month of Lunches is an accessible task-focused guide to Docker on Linux, Windows, or Mac systems. In it, you’ll learn practical Docker skills to help you tackle the challenges of modern IT, from cloud migration and microservices to handling legacy systems. There’s no excessive theory or niche-use cases– just a quick-and-easy guide to the essentials of Docker you’ll use every day.

Docker in Action 2nd Edition

Jeff Nickeloff
Docker in Action, Second Edition teaches you the skills and knowledge you need to create, deploy, and manage applications hosted in Docker containers. Continue reading

Technology Short Take 125

Welcome to Technology Short Take #125, where I have a collection of articles about various data center and cloud technologies collected from around the Internet. I hope I have managed to find a few useful things for you! (If not, contact me on Twitter and tell me how I can make this more helpful for you.)

Networking

Servers/Hardware

Nothing this time around. I’ll try hard to find something useful for the next Technology Short Take.

Security

Using KinD with Docker Machine on macOS

I’ll admit right up front that this post is more “science experiment” than practical, everyday use case. It all started when I was trying some Cluster API-related stuff that leveraged KinD (Kubernetes in Docker). Obviously, given the name, KinD relies on Docker, and when running Docker on macOS you generally would use Docker Desktop. At the time, though, I was using Docker Machine, and as it turns out KinD doesn’t like Docker Machine. In this post, I’ll show you how to make KinD work with Docker Machine.

By the way, it’s worth noting that, per the KinD maintainers, this isn’t a tested configuration. Proceed at your own risk, and know that while this may work for some use cases it won’t necessarily work for all use cases.

Prerequisites/Assumptions

These instructions assume you’ve already installed both KinD and Docker Machine, along with an associated virtualization solution. I’ll be using VirtualBox, but this should be largely the same for VMware Fusion or Parallels (or even HyperKit, if you somehow manage to get that working). I’m also assuming that you have jq installed; if not, get it here.

Making KinD work with Docker Machine

Follow the steps below to make Continue reading

Kustomize Transformer Configurations for Cluster API v1alpha3

A few days ago I wrote an article on configuring kustomize transformers for use with Cluster API (CAPI), in which I explored how users could configure the kustomize transformers—the parts of kustomize that actually modify objects—to be a bit more CAPI-aware. By doing so, using kustomize with CAPI manifests becomes much easier. Since that post, the CAPI team released v1alpha3. In working with v1alpha3, I realized my kustomize transformer configurations were incorrect. In this post, I will share CAPI v1alpha3 configurations for kustomize transformers.

In the previous post, I referenced changes to both namereference.yaml (to configure the nameReference transformer) and commonlabels.yaml (to configure the commonLabels transformer). CAPI v1alpha3 has changed the default way labels are used with MachineDeployments, so for v1alpha3 you may be able to get away with only changes to namereference.yaml. (If you know you are going to want/need additional labels on your MachineDeployment, then plan on changes to commonlabels.yaml as well.)

Here are the CAPI v1alpha3 changes needed to namereference.yaml:

- kind: Cluster
  group: cluster.x-k8s.io
  version: v1alpha3
  fieldSpecs:
  - path: spec/clusterName
    kind: MachineDeployment
  - path: spec/template/spec/clusterName
    kind: MachineDeployment

- kind: AWSCluster
  group: infrastructure.cluster.x-k8s.io
   Continue reading

Using Ansible Automation Platform, GitLab CE and Webhooks to Deploy IIS Website

Inside Red Hat Ansible Automation Platform, the Ansible Tower REST API is the key mechanism that helps enable automation to be integrated into processes or tools that exist in an environment. With Ansible Tower 3.6 we have brought direct integration with webhooks from GitHub and GitLab, including the enterprise on-premises versions. This means that changes in source control can trigger automation to apply changes to infrastructure configuration, deploy new services, reconfigure existing applications, and more. In this blog, I’ll run through a simple scenario and apply the new integrated webhook feature.

 

Environment

My environment consists of Ansible Tower (one component of Red Hat Ansible Automation Platform), GitLab CE with a project already created, and a code server running an IDE with the same git repository cloned. A single inventory exists on Ansible Tower with just one host, an instance of Windows 2019 Server running on a certified cloud. For this example, I’m going to deploy IIS on top of this Windows server and make some modifications to the html file that I’d like to serve from this site. 

My playbook to deploy IIS is very simple:

 ---
- name: Configure IIS
  hosts: windows

  tasks:
  - name: Install  Continue reading

First Docker GitHub Action is here!

We are happy to announce that today Docker has released its first Github Action! We’ve been working with GitHub, looking into how developers have been using GitHub Actions with Docker to set up their CI/CD workflows. The standard flows you’ll see if you look around are what you’d expect: building an image, tagging it, logging into Hub, and pushing the image. This is the workflow we’ve aimed to support with our Docker build-push action.

Simplify CI/CD workflows

At Docker traditionally much of our CI/CD workflows has been handled through Jenkins using a variety of products to set up and maintain it. For some things this is the best solution like when we are testing Docker Desktop on a whole variety of different hosts and configurations. For others it’s a bit overkill. Like many, we at Docker have been looking at how we can leverage GitHub Actions to simplify our workflows, including how we use Docker itself.

GitHub Actions already leverages Docker in a lot of its workflows. From having Docker pre-installed and configured on the cloud runners to having first class support for containerized actions allows developers to easily use the same Docker workflows they use locally to configure Continue reading

DockerCon LIVE with theCUBE: Call for Papers is Open

CFP Deadline: March 27th at 11:59 PM PST

The beauty of Docker is in the ways that developers are using it to positively impact their lives, industries, and day-to-day workflows. From sending rockets to space, to running some of the biggest apps on Earth, Docker helps developers build and share containerized apps – from the boring to the apps that change the world. DockerCon is the place where the community comes together to connect and share stories, best practices, and use cases. 

Back in December, we announced that DockerCon would not be a physical event but instead was evolving into a digital event. At the time, that decision was made in order to make attending DockerCon an option for any and all developers and members of the community. And now with the current state of the global COVID-19 pandemic, we are extra thankful to have already been planning for a virtual-only gathering. This change to DockerCon is the safest and healthiest option for our community, and we are excited to still bring everyone together to learn and share from one another.  

This year, DockerCon will be a virtual event on May 28th from 9am to 5pm GMT -8. Continue reading

Configuring Kustomize Transformers for Cluster API

In November 2019 I wrote an article on using kustomize with Cluster API (CAPI) manifests. The idea was to use kustomize to simplify the management of CAPI manifests for clusters that are generally similar but have minor differences (like the AWS region in which they are running, or the number of Machines in a MachineDeployment). In this post, I’d like to show a slightly different way of using kustomize with Cluster API that involves configuring the kustomize transformers.

If you aren’t familiar with kustomize, I’d recommend having a look at the kustomize web site and/or reading my introductory post. A transformer in kustomize is the part that is responsible for modifying a resource, or gathering information about a resource over the course of a kustomize build process. This page has some useful terminology definitions.

Looking back at the earlier article on using kustomize with CAPI, you can see that—due to the links/references between objects—modifying the name of the AWSCluster object also means modifying the reference to the AWSCluster object from the Cluster object. The same goes for the KubeadmConfigTemplate and AWSMachineTemplate objects referenced from a MachineDeployment. Out of the box, the namePrefix transformer will change the names of these Continue reading

Updating Visual Studio Code’s Kubernetes API Awareness

After attempting (and failing) to get Sublime Text to have some of the same “intelligence” that Visual Studio Code has with certain languages, I finally stopped trying to make Sublime Text work for me and just went back to using Code full-time. As I mentioned in this earlier post, now that I’ve finally solved how Code handles wrapping text in brackets and braces and the like I’m much happier. (It’s the small things in life.) Now I’ve moved on to tackling how to update Code’s Kubernetes API awareness.

Code’s awareness of the Kubernetes API comes via the YAML extension by Red Hat, and uses the yaml-language-server project found in this GitHub repository. (This is the same language server I was trying to get working with Sublime Text via LSP.) Note that I tested this procedure with version 0.7.2 of the extension; files may be in different locations or have different contents in other versions.

The information in this post is based heavily on this article by Josh Rosso, who some of you may know from his guest appearances on TGIK. I’ve adapted the information in Josh’s post to apply to the macOS version of Continue reading

1 … 30 31 32 33 34 … 125