Integrating Kibana/Elasticsearch on Top of OpenShift with oVirt Engine SSO
oVirt Engine provides a powerful way to manage users and domains using the oVirt Engine AAA extensions. oVirt Engine supports many different LDAP server types for authentication using the ovirt-engine-extension-aaa-ldap extension and supports managing internal users using the ovirt-engine-extension-aaa-jdbc extension. Clients can use the powerful oVirt Engine user management in their applications by using the OAuth2 or OpenId Connect end points provided by oVirt Engine SSO to authenticate users in their applications.
Below is step-by-step instructions on how to integrate Kibana/Elasticsearch on top of OpenShift with oVirt Engine SSO. The instructions should work for any client application that can be configured to use a OAuth2 or OpenID Connect server to authenticate its users.
The goal is to integrate Kibana/Elasticsearch on top of OpenShift with oVirt Engine SSO, so existing engine users can access Kibana/Elasticsearch without reauthentication (we don't need to maintain authentication configuration separately for oVirt Engine and Kibana/Elasticsearch).
The integration requires a fully working and configured oVirt Engine instance on oVirt Engine host and a fully working and configured instance of Kibana/Elasticsearch on top of OpenShift on the OpenShift host.
Installing Kibana/Elasticsearch and OpenShift Backend
Install Kibana/Elasticsearch/OpenShift on CentOS7 or RHEL 7.3 as described in https://github.com/ViaQ/Main/blob/master/README-mux.md
Continue reading
A Warm Welcome to the Summer Internship Students
The oVirt project is glad to announce that five talented students will be joining the oVirt community over the summer period, as part of the 2017 Google Summer of Code (GSoC) and Outreachy internship programs.
Both GSoC and Outreachy focus on getting more student developers interested in open source software development, as well as providing opportunities for talented people, underrepresented in the tech world, to gain valuable technology experience. The students will spend their summer break writing code, learning about open source development and documentation, and earning a stipend.
Joining us as part of GSoC are:
Tasdik Rahman will be working on adding Ansible roles for oVirt-utilities, for easier testing and automated redeployment. His mentor will be Lukas Svaty.
Shubham Dubey will be working on configuring backup storage for Ovirt. The idea is to replace the need for a dedicated storage domain for backup and disaster recovery. Shubham's mentor will be Maor Lipchuk.
Joining us as part of Outreachy, are:
Anastasia Antsiferova will be working on the oVirt log analyser. Her mentor will be Milan Zamazal.
Leni Kadali will be working on documetation. His mentor will be Jason Brooks.
Valentina Makarova will be working on implementing oVirt integration tests using Continue reading
Up and Running with oVirt 4.1 and Gluster Storage
Last month, the oVirt Project shipped version 4.1 of its open source virtualization management system. With a new release comes an update to this howto for running oVirt together with Gluster storage using a trio of servers to provide for the system's virtualization and storage needs, in a configuration that allows you to take one of the three hosts down at a time without disrupting your running VMs.
If you're looking instead for a simpler, single-machine option for trying out oVirt, your best bet is the oVirt Live ISO page. This is a LiveCD image that you can burn onto a blank CD or copy onto a USB stick to boot from and run oVirt. This is probably the fastest way to get up and running, but once you're up, this is definitely a low-performance option, and not suitable for extended use or expansion.
Read on to learn about my favorite way of running oVirt.
oVirt, Glusterized
Prerequisites
Hardware: You’ll need three machines with 16GB or more of RAM and processors with hardware virtualization extensions. Physical machines are best, but you can test oVirt using nested KVM as well. I've written this howto using VMs running on my "real" Continue reading
Up and Running with oVirt 4.1 and Gluster Storage
Last month, the oVirt Project shipped version 4.1 of its open source virtualization management system. With a new release comes an update to this howto for running oVirt together with Gluster storage using a trio of servers to provide for the system's virtualization and storage needs, in a configuration that allows you to take one of the three hosts down at a time without disrupting your running VMs.
If you're looking instead for a simpler, single-machine option for trying out oVirt, your best bet is the oVirt Live ISO. This is a LiveCD image that you can burn onto a blank CD or copy onto a USB stick to boot from and run oVirt. This is probably the fastest way to get up and running, but once you're up, this is definitely a low-performance option, and not suitable for extended use or expansion.
Read on to learn about my favorite way of running oVirt.
oVirt, Glusterized
Prerequisites
Hardware: You’ll need three machines with 16GB or more of RAM and processors with hardware virtualization extensions. Physical machines are best, but you can test oVirt using nested KVM as well. I've written this howto using VMs running on my "real" oVirt+Gluster Continue reading
Say Hello to oVirt 4.1.1
On March 22, the oVirt project released version 4.1.1, available for Red Hat Enterprise Linux 7.3, CentOS Linux 7.3, or similar.
oVirt is the open source virtualization solution that provides an awesome KVM management interface for multi-node virtualization. This maintenance version is super stable and there are some nice new features.
So what's new in oVirt 4.1.1?
Storage Team
- LUNs can be removed from a block data domain, provided that there is enough free space on the other domain devices to contain the data stored on the LUNs being removed.
- Support for NFS version 4.2 connections (when supported by storage).
Integration Team
- oVirt-hosted-engine-setup now works with NetworkManager enabled.
Network Team
- NetworkManager keeps running when a host is added to oVirt. This allows users to review networking configurations in cockpit whenever they want.
Infra Team
- A new tool, engine-vacuum, performs a vacuum on the PostgreSQL database in order to reclaim disk space on the operating system. It also updates and removes garbage from tables.
- Alerts for all data centers and clusters that are not upgraded to the highest compatibility version.
- Time zones are now shown in log records to make it easier to correlate Continue reading
oVirt Gamification–The oVirt Game You Didn’t Know you Were Playing
Gamification is the concept of applying game mechanics and game design techniques to engage and motivate people to achieve their goals.
It taps into the basic desires and needs of the users impulses which revolve around the idea of Status and Achievement.
To put it in other words, it is turning day-to-day tasks, the kind you might do at home or work, into a game which you can earn points, badges and compete with other people that are doing the same things.
oVirt & Gamification
You probably didn't know, but this isn't the first time oVirt gamification is being used. A few years ago there was an initiative to use oVirt UI plugins system to add Gamification to the project, there was even a "space invaders" game written and available to play inside oVirt!
So What is New?
The oVirt infra team recently reached out to 'GetBadges', a company which provides 'Gamification as a Service'. Luckily for us, open source projects get to have a free game! So oVirt was rewarded with its own oVirt Open Source Game.
The game works automagically every time you contribute to the project. Current integrations are only active on specific projects like 'ovirt-engine' and Continue reading
Using oVirt and Vagrant
Introducing oVirt virtual machine management via Vagrant.
In this short tutorial I'm going to give a brief introduction on how to use vagrant to manage oVirt with the new community developed oVirt v4 Vagrant provider.
Background
Vagrant is a way to tool to create portable and reproducible environments. We can use it to provision and manage virtual machines in oVirt by managing a base box (small enough to fit in github as an artifact) and a Vagrantfile. The Vagrantfile is the piece of configuration that defines everything about the virtual machines: memory, cpu, base image, and any other configuration that is specific to the hosting environment.
Prerequisites
- A fully working and configured oVirt cluster of any size
- A system capable of compiling and running the oVirt ruby SDK gem
- Vagrant 1.8 or later
- The oVirt vagrant plugin installed via
$ vagrant plugin install vagrant-ovirt4
The Vagrantfile
To start off, I'm going to use this Vagrantfile:
Vagrant.configure("2") do |config|
config.vm.box = 'ovirt4'
config.vm.hostname = "test-vm"
config.vm.box_url = 'https://github.com/myoung34/vagrant-ovirt4/blob/master/example_box/dummy.box?raw=true'
config.vm.network :private_network,
:ip => '192.168.56.100', :nictype => 'virtio', :netmask Continue readingoVirt Community Newsletter December 2016
It's a new year with new opportunities for oVirt to show up its virtualization features! We're getting ready for DevConf.CZ in Brno next week, and FOSDEM in Brussels the week after that! We look forward to meeting European developers and sysadmins to share your experiences!
Here's what happened in December of 2016.
Software Releases
oVirt 4.0.6 Release is now available
In the Community
oVirt System Tests to the Rescue!—How to Run End-to-End oVirt Tests on Your Patch
CI Please Build—How to build your oVirt project on-demand
The Need for Speed—Coming Changes in oVirt's CI Standards
Еxtension of iptables Rules on oVirt 4.0 Hosts
Deep Dives and Technical Discussions
KVM/Linux Nested Virtualization Support For ARM
Virtual Machines in Kubernetes? How and what makes sense?
ANNOUNCE: New libvirt project Go XML parser model
Using OVN with KVM and Libvirt
New libvirt project Go language bindings
CI tools testing lab: Making it do useful work
CI tools testing lab: Integrating Jenkins and adding Zuul UI
CI tools testing lab: Adding Zuul Merger
CI tools testing lab: Setting up Zuul Server
Happy New Documentation!
The oVirt Project is pleased to announce the availability of all-new principal documentation for the oVirt 4.0 branch.
There are many people out there who are content to use software without documentation, preferring to muddle through the software based on past experience with similar software or just the desire to put the software through its paces.
We all do this; I could not tell you the last time I looked at documentation for Firefox or Chrome, because I've been using browsers for over 20 years and seriously, what else is there to learn? Until I learn about a cool new feature from a friend or a web site.
In a software community project, one of the biggest things a community must do is to provide proper onboarding to the project's result. This means:
-
Explaining what the software is
-
Providing a clear path to getting the software
-
Demonstrating how to use the software
All three of these onboarding requirements must be done right in order for onboarding to work successfully. Documenation, then, fulfills the third requirement: showing how software can be used. Not every one will need it, but for those users who do need it, it is very nice Continue reading
oVirt System Tests to the Rescue!—How to Run End-to-End oVirt Tests on Your Patch
Today, when an oVirt developer pushes a patch to review on oVirt Gerrit, various validations are triggered in CI via the 'check-patch' job, as defined by the project maintainers. Usually these jobs includes 'unit-tests', 'db tests', static analysis checks, and even an occasional 'functional test'. While it might seem that it covers alot and gives a good indication that the patch is good to be merged, unfortunately it is not always the case.
The reason it's not enough lies in oVirt's complexity and the fact it's a Virtualization project, which means the only real way to know if your patch didn't break things is to install oVirt and try running a few basic commands, like 'adding host', 'adding vm', 'creating snapshots', and other tasks you can only do if you have a full oVirt system up and running. Here is where OST comes in!
oVirt System Tests
oVirt system tests is a testing framework written in Python, using 'python-nose' and oVirt Python SDK and runs on auto-generated VMs created by Lago. It is used by the oVirt CI to run post merge end-to-end testing that runs on a fully deployed oVirt environment and has been proven to detect multiple regressions Continue reading
CI Please Build—How to build your oVirt project on-demand
All projects in oVirt CI are built today post merge, using the 'build-artifacts' stage from oVirt's CI standards. This ensures that all oVirt projects are built and deployed to oVirt repositories and can be consumed by CI jobs, developers or oVirt users.
However, on some occasions a developer might need to build his project from an open patch. Developers need this capability in order to to examine the effects of their changes on a full oVirt installation before merging those changes. On some cases developers may even want to hand over packages based on un-merged patches to the QE team to verify that a given change will fix some complex issue or to preview a new feature on its early stages of development.
The Current Build Option
Until now, to build rpms from a patch, a developer needed to use a custom Jenkins job, which was only available to ovirt-engine and only for master branch. Another option was to try and build it locally using standard CI 'mock runner.sh' script which will use the same configuration as in CI. For full documentation on how to use 'mock-runner', checkout the Standard CI page.
The New Build Option
To ease Continue reading
The Need for Speed—Coming Changes in oVirt’s CI Standards
oVirt's CI standards have been in use for a while in most oVirt projects and have largely been a success.
These standards have put the control of what the CI system does in the hands of the developers without them
having to learn about Jenkins and the tooling around it. The way the standards were implemented, with the mock_runner.sh script, also enabled developers to easily emulate the CI system on their own machines to debug and diagnose issues.
From the oVirt infra team's point of view, the CI standards have removed the need to constantly maintain build dependencies on the Jenkins slaves and also eliminated most of the situations where jobs running on the same slave influenced one another.
The CI standards implementation we have has one shortcoming, it is not particularity fast.
We started seriously looking at this after one of the VDSM maintainers reported that the check_patch jobs for his project are running for far too long a time. In the end it turned out that a major reason for the delay was in the way the tests themselves worked, but still, we looked at mock_runner.sh and managed to speed it up quite a Continue reading
Еxtension of iptables Rules on oVirt 4.0 Hosts
In one of my last articles I described the example of installing HP System Management Tools to the physical server HP ProLiant DL360 G5 with CentOS Linux 7.2. After a while, the same exact server was used as a virtualization host and the oVirt Hosted Engine components were deployed on it. The host was put into maintenance mode recently, all packages were upgraded from the online repository, including the HP tool pack installed on it.
After the installation, I decided to check the workability of the upgraded tools. I also tried to open the web page of HP System Management homepage, but I didn’t succeed, because the host was simply blocking TCP port 2381.
Firewalld service was stopped on the host and the iptables was loaded with a set of rules, which was typical for oVirt. Moreover, the rules on all oVirt hosts, which I was deploying with the oVirt Engine web console, were the same.
In order to edit the rules, which are shared and centralized to all hosts from the oVirt Engine, we need to use the engine-config tool within the Engine server.
The engine-config tool has a large set of keys, which set the oVirt infrastructure Continue reading
New oVirt Project Underway
As oVirt continues to grow, the many projects within the broader oVirt community are thriving as well. Today, the oVirt community is pleased to announce the addition of a new incubator subproject, Vagrant Provider, as well as the graduation of another subproject, moVirt, from incubator to full project status!
According to maintainer Marc Young, Vagrant Provider is a provider plugin for the Vagrant suite that enables command-line ease of virtual machine provisioning and lifecycle management.
More on Vagrant Provider
The Vagrant provider plugin will interface with the oVirt REST API (version 4 and higher) using the oVirt provided ruby SDK 'ovirt-engine-sdk-ruby'. This allows users to abstract the user interface and experience into a set of command-line abilities to create, provision, destroy and manage the complete lifecycle of virtual machines. It also allows the use of external configuration management and configuration files themselves to be committed into code.
As Young explains in his project proposal, the "trend in configuration management, operations, and devops has been to maintain as much of the development process as possible in terms of the virtual machines and hosts that they run on. With software like Terraform the tasks of creating the underlying infrastructure such as Continue reading
oVirt Newsletter—November 2016
oVirt's development is continuing on pace, as the calendar year draws to a close and we get ready for a new year of development, evangelism, and making virtual machine management a simple process for everyone.
Here's what happened in November of 2016.
Software Releases
oVirt 4.0.6 Third Release Candidate is now available
oVirt 4.1.0 First Beta Release is now available for testing
In the Community
Testing ovirt-engine changes without a real cluster
Request for oVirt Ansible modules testing feedback
Deep Dives and Technical Discussions
Important Open Source Cloud Products [German]
Red Hat IT runs OpenShift Container Platform on Red Hat Virtualization and Ansible
Keynote: Blurring the Lines: The Continuum Between Containers and VMs [Video]
Quick Guide: How to Plan Your Red Hat Virtualization 4.0 Deployment
Testing ovirt-engine changes without a real cluster
The ovirt-engine component of oVirt is the brain of oVirt and is responsible for managing attached systems; providing the webadmin UI and REST interfaces; and other core tasks. The process of setting up a real cluster on which to deploy the project is a time-consuming task that greatly increases patch turnaround time and can provide a significant barrier of entry to those wanting to contribute to the project.
Development Environment
There are couple of preparation steps you must take to create your development environment. I am using CentOS 7 as my development machine so I will use that system to describe everything, but it should be pretty straightforward to adapt the article to Fedora.
We first need the source code for the ovirt-engine itself. You can get it from the project's code review tool: gerrit.ovirt.org. Just execute the following command and wait for it to finish:
# git clone git://gerrit.ovirt.org/ovirt-engine.git
You will also need a directory for the development deployments, so create a directory somewhere. Mine is in ~/Applications/ovirt-engine-prefix. I have set the$OVIRT_PREFIX environment variable to point to that path, so when you see it used throughout this article, substitute the path for your own Continue reading
oVirt Software Defined Networking, The OVN Network Provider
oVirt offers not only its own internal networking, but also an API for external network providers. This API enables using external network management software inside environments managed by oVirt and takes advantage of their extended capabilities. One of such solutions is OVN: Open Virtual Network. OVN is an OVS (Open vSwitch) extension that brings Software Defined Networking to OVS.
OVN enables support for virtual networks abstraction by adding native OVS support for virtual L2 and L3 overlays. This allows the user to create as many VM networks as required, without troubling the adminstrator with vlan requests or infrastructure changes.
The oVirt provider for OVN consists of two parts: * The oVirt OVN driver * The oVirt OVN provider
oVirt OVN Driver
The oVirt OVN driver is the Virtual Interface Driver placed on oVirt hosts that handle the wiring of VM NICs to OVN networking.
The driver allows Vdsm, libvirt, and OVN to interact whenever a NIC is plugged in such a way that the VM NIC is added to an appropriate OVN Logical Switch and the appropriate OVN overlays on all the hosts in the oVirt environment.
The oVirt OVN driver rpm is now available for testing. The latest version Continue reading
Setting Community Values
The oVirt community is made up of a diverse mix of individuals using and contributing to all aspects of the project from all over the world, and we want to make sure that the community is a safe and friendly place for everyone.
This code of conduct applies equally to founders, mentors, and those seeking help and guidance. It applies to all spaces managed by the oVirt project, including IRC, mailing lists, GitHub, Gerrit, oVirt events, and any other forums created by the project team which the community uses for communication.
While we have contribution guidelines for specific tools, we expect all members of our community to follow these general guidelines and be accountable to the community. This isn’t an exhaustive list of things that you can’t do. Rather, take it in the spirit in which it’s intended—a guide to make it easier to enrich all of us and the technical communities in which we participate.
To that end, some members of the oVirt community have put together a new Community Code of Conduct to help guide everyone through what it means to be respectful and tolerant in a global community like the oVirt Project.
We're not looking for a Continue reading