Introduction to Docker: Featured Video of January 2017
The featured webinar in January 2017 is the Introduction to Docker webinar, and in the featured video Matt Oswalt explains the basic Docker tasks. Other videos in this webinar cover Docker images, volumes, networking, and Docker Compose and Swarm.
To view the featured video, log into my.ipspace.net, select the webinar from the first page, and watch the video marked with star.
Read more ...Container Namespaces – Deep dive into container networking
Of late, there have been various Open source projects to manage networking for containers. Docker implemented "libnetwork". I've written in the past about using Calico with Docker containers. Debugging and low level tweaks to performance isn't easy without an in-depth understanding of how the network stack works for a container. This post tries to explain network namespaces and its role in container networking using default networking that comes out-of-box with docker.
As you probably already know containers use namespaces to isolate resources and rate limit their use. Linux's network namespaces are used to glue container processes and the host networking stack. Docker spawns a container in the containers own network namespace (use the CLONE_NEWNET flag defined in sched.h when calling the clone system call to create a new network namespace for the subprocess) and later on runs a veth pair (a cable with two ends) between the container namespace and the host network stack. If you are new to network namespaces this blog post by Scott gives a quick overview and serves as a good 101 refresher if you are already familiar with these concepts but haven't used them for a while.
As you probably already know containers use namespaces to isolate resources and rate limit their use. Linux's network namespaces are used to glue container processes and the host networking stack. Docker spawns a container in the containers own network namespace (use the CLONE_NEWNET flag defined in sched.h when calling the clone system call to create a new network namespace for the subprocess) and later on runs a veth pair (a cable with two ends) between the container namespace and the host network stack. If you are new to network namespaces this blog post by Scott gives a quick overview and serves as a good 101 refresher if you are already familiar with these concepts but haven't used them for a while.
Now lets see how to Continue reading
oVirt System Tests to the Rescue!—How to Run End-to-End oVirt Tests on Your Patch
Today, when an oVirt developer pushes a patch to review on oVirt Gerrit, various validations are triggered in CI via the 'check-patch' job, as defined by the project maintainers. Usually these jobs includes 'unit-tests', 'db tests', static analysis checks, and even an occasional 'functional test'. While it might seem that it covers alot and gives a good indication that the patch is good to be merged, unfortunately it is not always the case.
The reason it's not enough lies in oVirt's complexity and the fact it's a Virtualization project, which means the only real way to know if your patch didn't break things is to install oVirt and try running a few basic commands, like 'adding host', 'adding vm', 'creating snapshots', and other tasks you can only do if you have a full oVirt system up and running. Here is where OST comes in!
oVirt System Tests
oVirt system tests is a testing framework written in Python, using 'python-nose' and oVirt Python SDK and runs on auto-generated VMs created by Lago. It is used by the oVirt CI to run post merge end-to-end testing that runs on a fully deployed oVirt environment and has been proven to detect multiple regressions Continue reading
Technology Short Take #75
Welcome to Technology Short Take #75, the final Technology Short Take for 2016. Fortunately, it’s not the final Technology Short Take ever, as I’ll be back in 2017 with more content. Until then, here’s some data center-related articles and links for your enjoyment.
Networking
- Ajay Chenampara has some observations about running Ansible at scale against network devices.
- Andrey Khomyakov shares some information on automating the setup of whitebox switches running Cumulus Linux in part 2 of this series on learning network automation.
- Russell Bryant has shared the results of some testing comparing ML2+OVS and OVN as backends for OpenStack networking. As Russell indicates in his post, some additional analysis is needed to truly understand what’s happening, but early looks at the results of his tests show performance improvements in OVN versus ML2+OVS when it comes to total time required to boot a VM.
- Ivan Pepelnjak shares a Python script that creates Ansible inventory from Vagrant’s SSH configuration. Handy.
Servers/Hardware
Nothing this time around!
Security
- James Green shares some information on Docker Bench, a tool designed to help secure Docker environments.
- Via Mike Foley, the community now has a new resource to help educate on VM escapes.
- Gordon Turner Continue reading
brvirt: when brctl meets virsh
Hypervisors diversity is definitely one of the benefits of having Nuage managing your next-generation network. That means that we, as Nuage engineers, have to play with all kinds of hypervisors — like KVM, ESXi and Hyper-V to be more precise. As to me, I love to work with KVM most, simply because it gives you that feel that youCI Please Build—How to build your oVirt project on-demand
All projects in oVirt CI are built today post merge, using the 'build-artifacts' stage from oVirt's CI standards. This ensures that all oVirt projects are built and deployed to oVirt repositories and can be consumed by CI jobs, developers or oVirt users.
However, on some occasions a developer might need to build his project from an open patch. Developers need this capability in order to to examine the effects of their changes on a full oVirt installation before merging those changes. On some cases developers may even want to hand over packages based on un-merged patches to the QE team to verify that a given change will fix some complex issue or to preview a new feature on its early stages of development.
The Current Build Option
Until now, to build rpms from a patch, a developer needed to use a custom Jenkins job, which was only available to ovirt-engine and only for master branch. Another option was to try and build it locally using standard CI 'mock runner.sh' script which will use the same configuration as in CI. For full documentation on how to use 'mock-runner', checkout the Standard CI page.
The New Build Option
To ease Continue reading
IP Networking – generational shift in the industry to pure-L3 network stack
 |
| © Calico project (http://docs-archive.projectcalico.org/en/1.4.3/addressing.html) |
Docker networking eco-system begun with libnetwork and after that has propagated with multitude of solutions by both community and networking vendors. Unlike Flannel (CoreOS) and VXLAN which leverage tunneling and L2 protocols, Calico is a distributed L3 networking solution that works both with Virtual Machines and Continue reading
How to Setup HA for RHV-M pt3
Hi Folks.. the last time I left off, we had just finished setting up the initial pieces for hosted engine (RHEL configuration) in order to provide HA for RHV-M. Today we’re going to add an additional host and test things out.
Let’s get going!Once hosted engine is setup, adding another RHEL host to the configuration is almost identical to a standard configuration. There is only 1 additional step that involves ensuring that the HA pieces specific to hosted engine get installed along with the RHV packages.
The workflow itself is simple:
- Deploy a RHEL host, subscribe it to Red Hat CDN or Satellite and update it
- Add the Cockpit package and enable it (we won’t highlight that here, but you can see it in the previous post)
- Log into RHV-M, go to the “Hosts” tab, select “New”, fill in the relevant information, be sure to visit the “Hosted Engine” tab and select the “deploy” radio button.
- Once it finishes installing, give everything a minute or so to sync up, then test!
As usual, I recorded a walk through so that things are more clear. I sped things up significantly as my home lab is ~very~ slow.. Maybe I’ll get Continue reading
Using OVN with KVM and Libvirt
In this post, I’m going to discuss how to use OVN (Open Virtual Network; part of the Open vSwitch project) with KVM and Libvirt to provide virtual networking for KVM-based virtual machines. This post will build on some concepts around OVS and Libvirt that I’ve discussed previously; be sure to review the OVS posts and Libvirt posts on this site for more details and prerequisite knowledge.
I’ll structure this discussion around 2 key steps:
- Setting up OVN
- Integrating KVM/Libvirt into OVN
Note that I’m not going to discuss setting up KVM/Libvirt, as that’s something I’ve covered previously and is well-documented.
Ready? Let’s jump in!
Setting up OVN
The biggest “challenge” here is package availability—many Linux distributions don’t have packages available for OVS 2.6.0, which is the first release with non-experimental support of OVN. If you’re an Ubuntu user, then you can use the Ubuntu Cloud Archive for the OpenStack “Newton” release, which includes OVS/OVN 2.6.0 packages. For other distributions, you’ll probably need to compile from source. In that case, the OVS installation documentation is quite accurate and usable.
For the purposes of this post, I’ll assume you’re using Ubuntu 16.04 and will pull packages Continue reading
Technology Short Take #74
Welcome to Technology Short Take #74! The end of 2016 is nearly upon us, and it looks as if there will be only one more Technology Short Take before the end of the year. So, let’s get on with the content—time is short!
Networking
- If you haven’t heard of Apstra, David Varnum has a great introduction to Apstra available on his site.
- Will Robinson talks about how to structure your Ansible playbooks in the context of using Ansible to control your network gear.
- This is an interesting project to watch, I think—it’s porting OVN (Open Virtual Network) from a “traditional” OvS back-end to an IOVisor-based back-end (IOVisor implements the data plane in eBPF).
- If you’re interested in playing around with OVN, I’ve built a Vagrant-based environment running OVS/OVN 2.6.0 on Ubuntu 16.04. Have a look here.
Servers/Hardware
Nothing this time, but I’ll stay alert for content to include in the future.
Security
- Jake Bennett discusses strengthening your AWS security posture by protecting application credentials and rotating EC2 and IAM keys.
- Gert van Dijk has a great article on upgrading your SSH keys to take advantage of new key types and enhanced security.
- In case you’re interested, here’s Continue reading
The Need for Speed—Coming Changes in oVirt’s CI Standards
oVirt's CI standards have been in use for a while in most oVirt projects and have largely been a success.
These standards have put the control of what the CI system does in the hands of the developers without them
having to learn about Jenkins and the tooling around it. The way the standards were implemented, with the mock_runner.sh script, also enabled developers to easily emulate the CI system on their own machines to debug and diagnose issues.
From the oVirt infra team's point of view, the CI standards have removed the need to constantly maintain build dependencies on the Jenkins slaves and also eliminated most of the situations where jobs running on the same slave influenced one another.
The CI standards implementation we have has one shortcoming, it is not particularity fast.
We started seriously looking at this after one of the VDSM maintainers reported that the check_patch jobs for his project are running for far too long a time. In the end it turned out that a major reason for the delay was in the way the tests themselves worked, but still, we looked at mock_runner.sh and managed to speed it up quite a Continue reading
Get all the Docker talks from Tech Field Day 12
As 2016 comes to a close, we are excited to have participated in a few of the Tech Field Day and inaugural Cloud Field Day events to share the Docker technology with the IT leaders and evangelists that Stephen Foskett and Tom Hollingsworth have cultivated into this fantastic group. The final event was Tech Field Day 12 hosting in Silicon Valley.
In case you missed the live stream, check out videos of the sessions here.
Session 1: Introduction to Docker and Docker Datacenter
Session 2: Securing the Software Supply Chain with Docker
Session 3: Docker for Windows Server and Windows Containers
Session 4: Docker for AWS and Azure
Session 5: Docker Networking Fabric
These are great overviews of the Docker technology applied to enterprise app pipelines, operations, and diverse operating systems and cloud environments. And most importantly, this was a great opportunity to meet some new people and get them excited about what we are excited about.
+1!!! #TFD12 #Docker https://t.co/Zdsuw1emlo
— Alex Galbraith (@alexgalbraith) November 17, 2016
Visit the Tech Field Day site to watch more videos from previous events, read articles written by delegates or view the conversation online.
New #Docker videos from #TFD12 @TechFieldDay w/ @SFoskett @GestaltIT Continue reading
Еxtension of iptables Rules on oVirt 4.0 Hosts
In one of my last articles I described the example of installing HP System Management Tools to the physical server HP ProLiant DL360 G5 with CentOS Linux 7.2. After a while, the same exact server was used as a virtualization host and the oVirt Hosted Engine components were deployed on it. The host was put into maintenance mode recently, all packages were upgraded from the online repository, including the HP tool pack installed on it.
After the installation, I decided to check the workability of the upgraded tools. I also tried to open the web page of HP System Management homepage, but I didn’t succeed, because the host was simply blocking TCP port 2381.
Firewalld service was stopped on the host and the iptables was loaded with a set of rules, which was typical for oVirt. Moreover, the rules on all oVirt hosts, which I was deploying with the oVirt Engine web console, were the same.
In order to edit the rules, which are shared and centralized to all hosts from the oVirt Engine, we need to use the engine-config tool within the Engine server.
The engine-config tool has a large set of keys, which set the oVirt infrastructure Continue reading
New oVirt Project Underway
As oVirt continues to grow, the many projects within the broader oVirt community are thriving as well. Today, the oVirt community is pleased to announce the addition of a new incubator subproject, Vagrant Provider, as well as the graduation of another subproject, moVirt, from incubator to full project status!
According to maintainer Marc Young, Vagrant Provider is a provider plugin for the Vagrant suite that enables command-line ease of virtual machine provisioning and lifecycle management.
More on Vagrant Provider
The Vagrant provider plugin will interface with the oVirt REST API (version 4 and higher) using the oVirt provided ruby SDK 'ovirt-engine-sdk-ruby'. This allows users to abstract the user interface and experience into a set of command-line abilities to create, provision, destroy and manage the complete lifecycle of virtual machines. It also allows the use of external configuration management and configuration files themselves to be committed into code.
As Young explains in his project proposal, the "trend in configuration management, operations, and devops has been to maintain as much of the development process as possible in terms of the virtual machines and hosts that they run on. With software like Terraform the tasks of creating the underlying infrastructure such as Continue reading
How to Setup HA for RHV-M pt2
Hi folks, I’m finally getting around to the high availability for RHV-M (hosted engine) walk through demo that I promised. The truth is that due to unforeseen circumstances, I had to go to “plan b”. The end result is still the same, and the workflows are almost identical, but the “in betweens” are just a bit different.
Allow me to illuminate..
So when I last left off, I was explaining the virtues of both the lightweight virtualization host (RHVH) as well as the hosted engine configuration for use as a means of providing high availability for RHV-M, the management piece for RHV. Hosted engine can support either (not both at the same time) RHVH or RHEL hosts as the hypervisor nodes.. While I really wanted to show you how get things up and running with RHVH first, I’m going to show you the “RHEL way” first. I’ll come back around the RHVH, I promise.
The workflow for getting things up and ready is very similar when comparing RHVH and RHEL – hosts, networks, and storage all get setup. DNS (forward and reverse, FQDN for hosts and RHV-M) is configured. Subscriptions are set and hosts are updated. The biggest differences are Continue reading
oVirt Newsletter—November 2016
oVirt's development is continuing on pace, as the calendar year draws to a close and we get ready for a new year of development, evangelism, and making virtual machine management a simple process for everyone.
Here's what happened in November of 2016.
Software Releases
oVirt 4.0.6 Third Release Candidate is now available
oVirt 4.1.0 First Beta Release is now available for testing
In the Community
Testing ovirt-engine changes without a real cluster
Request for oVirt Ansible modules testing feedback
Deep Dives and Technical Discussions
Important Open Source Cloud Products [German]
Red Hat IT runs OpenShift Container Platform on Red Hat Virtualization and Ansible
Keynote: Blurring the Lines: The Continuum Between Containers and VMs [Video]
Quick Guide: How to Plan Your Red Hat Virtualization 4.0 Deployment
Your Agenda for HPE Discover London 2016
Next week HPE will host more than 10,000 top IT executives, architects, engineers, partners and thought-leaders from across Europe at Discover 2016 London, November 29th – December 1st in London.
Come visit Docker in Booth #208 to learn how Docker’s Containers-as-a-Service platform is transforming modern application infrastructures, allowing business to benefit from a more agile development environment.
Docker experts will be on-hand to for in-booth demos, hands-on-labs, breakout sessions and Transformation Zone sessions to demonstrate how Docker’s infrastructure platform, provides businesses with a unifying framework to embrace hybrid infrastructures and optimize resource utilization across legacy and modern Linux and Windows applications.
Not attending Discover London? Don’t miss a thing and “Save the Date” for the live streaming of keynotes and top sessions beginning November 29th at 11:00 GMT and through the duration of the event.
Be sure to add these key Docker sessions to your HPE Discover London agenda:
Ongoing: Transformation Zone Hours Show Floor
DEMO315: HPE IT Docker success stories
Supercharge your container deployments on bare metal and VMs by orchestrating large workloads using simple Docker mechanisms. See how the HPE team automated Continue reading
Video: Docker Networking Options
After introducing the fundamentals of Docker networking, Dinesh Dutt focused on various Docker networking options, including multi-host networking with overlays.
After watching the video, you might also want to listen to Episode 49 of Software Gone Wild with Brent Salisbury, Dave Tucker and Madhu Venugopal.
Technology Short Take #73
Welcome to Technology Short Take #73. Sorry for the long delay since the last Technology Short Take; personal matters have been taking quite the toll (if you follow me on Twitter, you’ll know to what personal matters I’m referring). In any case, enough of that—here’s some data center-related content that I hope you find useful!
Networking
- Ansible has made some good progress in supporting network automation in the latest release (2.2), according to this blog post. This is an area where I hope to spend more time in the coming weeks before years’ end.
- Tomas Fojta shows how to use a PowerShell script to monitor the health of NSX Edge gateways.
- Jeremy Stretch mulls over the (perceived) problem of getting traffic into and out of overlay networks. I recommend reading this article, as well as reading the comments. Many commenters suggest just using L3 and having the hosts participate in a routing protocol like BGP, but as Jeremy points out many switches don’t have the capacity to handle that many routes. (Or, if they do, they’re quite expensive.) Seems like there’s this company in Palo Alto making a product that handles this issue pretty decently…(hint).
- Cumulus Continue reading
Can VMware NSX and Cisco ACI Interoperate over VXLAN?
I got a long list of VXLAN-related questions from one of my subscribers. It started with an easy one:
Does Cisco ACI use VXLAN inside the fabric or is something else used instead of VXLAN?
ACI uses VXLAN but not in a way that would be (AFAIK) interoperable with any non-Cisco product. While they do use some proprietary tagging bits, the real challenge is the control plane.
Read more ...