The libvirt library is a Linux API over the virtualization capabilities of Linux that supports a variety of hypervisors, including Xen and KVM, as well as QEMU and some virtualization products for other operating systems. This article explores libvirt, its use, and its architecture.

When it comes to scale-out computing (such as cloud computing), libvirt may be one of the most important libraries you‘ve never heard of. Libvirt provides a hypervisor-agnostic API to securely manage guest operating systems running on a host. Libvirt isn‘t a tool per se but an API to build tools to manage guest operating systems. Libvirt itself is built on the idea of abstraction. It provides a common API for common functionality that the supported hypervisors implement. Libvirt was originally designed as a management API for Xen, but it has since been extended to support a number of hypervisors.

Basic architecture

Join the green groups on My developerWorks

Discuss topics and share resources about energy, efficiency, and the environment on the GReen IT Report space and the Green computing group on My developerWorks.

Let‘s start our discussion of libvirt with a view of the use model, then dig into its architecture and use. Libvirt exists as a set of APIs designed to be used by a management application (see Figure 1). Libvirt, through a hypervisor-specific mechanism, communicates with each available hypervisor to perform the API requests. I explore how this is done with QEMU later in the article.

Figure 1. Comparison and use model of libvirt

Also shown is a comparison of the terminology that libvirt uses. This terminology is important, as these terms are used in API naming. The two fundamental differences are that libvirt calls the physical host a node, and the guest operating system is called a domain. Note here that libvirt (and its application) runs in the domain of the host Linux operating system (domain 0).

Means of control

With libvirt, you have two distinct means of control. The first is demonstrated in Figure 1, where the management application and domains exist on the same node. In this case, the management application works through libvirt to control the local domains. The other means of control exist when the management application and the domains are on separate nodes. In this case, remote communication is required (see Figure 2). This mode uses a special daemon called libvirtd that runs on remote nodes. This daemon is started automatically when libvirt is installed on a new node and can automatically determine the local hypervisors and set up drivers for them (to be discussed shortly). The management application communicates through the local libvirt to the remote libvirtd through a custom protocol. For QEMU, the protocol ends at the QEMU monitor. QEMU includes a monitor console that allows you to inspect a running guest operating system as well as control various aspects of the virtual machine (VM).

Figure 2. Control of remote hypervisors with libvirtd

Hypervisor support

To support extensibility over a wide variety of hypervisors, libvirt implements a driver-based architecture, which allows a common API to service a large number of underlying hypervisors in a common fashion. This means that certain specialized functionality of some hypervisors is not visible through the API. Additionally, some hypervisors may not implement all API functions, which are then defined as unsupported within the specific driver. Figure 3 illustrates the layering of the libvirt API and associated drivers. Note also here that libvirtd provides the means to access local domains from remote applications.

Figure 3. Driver-based architecture of libvirt

As of this writing, libvirt implements drivers for the hypervisors listed in Table 1. Other drivers will no doubt be available as new hypervisors emerge from the open source communities.

Table 1. Hypervisors that libvirt supports

Hypervisor
Description

Xen
Hypervisor for IA-32, IA-64, and PowerPC 970 architectures

QEMU
Platform emulator for various architectures

Kernel-based Virtual Machine (KVM)
Linux platform emulator

Linux Containers (LXC)
Linux (lightweight) containers for operating system virtualization

OpenVZ
Operating system-level virtualization based on the Linux kernel

VirtualBox
Hypervisor for x86 virtualization

User Mode Linux
Linux platform emulator for various architectures

Test
Test driver for a fake hypervisor

Storage
Storage pool drivers (local disk, network disk, iSCSI volume)

Back to top

Libvirt and the virtualization shell

Now that I‘ve covered some of the architecture of libvirt, let‘s look at some examples of the use of the libvirt virtualization API. I start by using an application called virsh (virtualization shell), which is built on top of libvirt. This shell permits use of much of the libvirt functionality but in an interactive (shell-based) fashion. In this section, I demonstrate some of the aspects of VM manipulation using virsh.

The first step is to define the domain configuration file (shown in Listing 1, below). This code specifies all the necessary options for defining a domain—from the hypervisor (emulator) to the resources that the domain uses and peripheral configuration (such as the network). Note that this is a very simple configuration: the actual attributes that libvirt supports are much more diverse. For example, you can specify a BIOS and host bootloader, resources to be used by the domain, and devices to be used—from floppy disks and CD-ROMs to USB and PCI devices.

The domain configuration file defines some of the basic metadata to be used for this QEMU domain, including the domain name, maximum memory, and initially available memory (current) as well as the number of virtual processors to be made available to this domain. You don‘t assign a Universally Unique Identifier (UUID); instead, you allow libvirt to assign one. You define the type of machine to emulate for this platform—in this case, a 686 processor that is fully virtualized (hvm). You define the location of the emulator (in case you need to support multiple of the same type) and the virtual disk for the domain. Note here that you indicate the VM, which is a ReactOS operating system in Virtual Machine Disk (VMDK) format. Finally, you specify the default networking configuration, and you use Virtual Network Computing (VNC) for graphics.

Listing 1. Domain configuration file

<xml version="1.0"?><domain type=‘qemu‘>  <name>ReactOS-on-QEMU<name>  <uuid<uuid>  <memory>131072<memory>  <currentMemory>131072<currentMemory>  <vcpu>1<vcpu>  <os>    <type arch=‘i686‘ machine=‘pc‘>hvm<type>  <os>  <devices>    <emulator>usr/bin/qemu<emulator>    <disk type=‘file‘ device=‘disk‘>      <source file=‘/home/mtj/libvtest/ReactOS.vmdk‘/>      <target dev=‘hda‘/>    <disk>    <interface type=‘network‘>      <source network=‘default‘/>    <interface>    <graphics type=‘vnc‘ port=‘-1‘/>  <devices><domain>

Now, with the domain configuration file complete, let‘s start a domain with the virsh tool. The virsh tool takes a command argument for the particular action to be taken. In the case of starting a new domain, you use the create command and the domain configuration file:

Listing 2. Starting a new domain

mtj@mtj-desktop:~/libvtest$ virsh create react-qemu.xmlConnecting to uri: qemu:///systemDomain ReactOS-on-QEMU created from react-qemu.xmlmtj@mtj-desktop:~/libvtest$

Note here the Universal Resource Indicator (URI) used to attach to the domain (qemu:///system). This local URI attaches to the system mode daemon for the local QEMU driver. To attach to a remote QEMU hypervisor over the Secure Shell (SSH) protocol on host shinchan, you could use the URI qemu+ssh://shinchan/.

Next, you can list the active domains on a given host using the list command within virsh. Doing so lists the active domains, their domain IDs, and their state, as shown below:

Listing 3. Listing active domains

mtj@mtj-desktop:~/libvtest$ virsh listConnecting to uri: qemu:///system Id Name                 State----------------------------------  1 ReactOS-on-QEMU      runningmtj@mtj-desktop:~/libvtest$

Note that the name defined here is the name you defined in your domain configuration file metadata. You can see that this domain has a domain ID of 1 and is currently running.

You can also suspend a domain with the suspend command. This command stops the domain from being scheduled, but the domain continues to reside in memory and can be quickly resumed. The following example illustrates suspending the domain, performing a list to see status, and then restarting the domain:

Listing 4. Suspending a domain, checking status, and restarting

mtj@mtj-desktop:~/libvtest$ virsh suspend 1Connecting to uri: qemu:///systemDomain 1 suspendedmtj@mtj-desktop:~/libvtest$ virsh listConnecting to uri: qemu:///system Id Name                 State----------------------------------  1 ReactOS-on-QEMU      pausedmtj@mtj-desktop:~/libvtest$ virsh resume 1Connecting to uri: qemu:///systemDomain 1 resumedmtj@mtj-desktop:~/libvtest$

The virsh utility also supports a number of other commands, such as saving a domain (save), restoring a saved domain (restore), rebooting a domain (reboot), and many others. You can also create a domain configuration file from a running domain (dumpxml).

So far, you‘ve started and manipulated a domain. But what about attaching to it so that you can see the domain in action. You can do this using VNC. To create a window representing the graphical desktop of the particular domain, you can use VNC as:

Listing 5. Attaching to a domain

mtj@mtj-desktop:~/libvtest$ xvnc4viewer 127.0.0.1 0

Back to top

Libvirt and Python

The previous example illustrated the control of domains using the command-line utility virsh. Let‘s now look an example of domain control using Python. Python was the libvirt-supported scripting language and provides a clean, object-oriented interface to the libvirt API.

In this example, I explore some of the same operations that I demonstrated with the virsh utility (list, suspend, resume, and so on). The Python example script is provided in Listing 6. In this example, you begin by importing the libvirt module. You then connect to the local QEMU hypervisor. From here, you iterate through the domain IDs that are available; for each one, you create a domain object, and then suspend, resume, and finally destroy the domain.

Listing 6. Sample Python script for domain control (libvtest.py)

import libvirtconn = libvirt.open(‘qemu:///system‘)for id in conn.listDomainsID():	dom = conn.lookupByID(id)	print "Dom %s  State %s" % ( dom.name(), dom.info()[0] )	dom.suspend()	print "Dom %s  State %s (after suspend)" % ( dom.name(), dom.info()[0] )	dom.resume()	print "Dom %s  State %s (after resume)" % ( dom.name(), dom.info()[0] )	dom.destroy()

Although this is a simple example, you can see the power that libvirt provides through Python. Through a simple script, you are able to iterate through all of the local QEMU domains, emit some information about the domain, and then control the domain. The output of this script is shown in Listing 7.

Listing 7. Output from the Python script in Listing 6

mtj@mtj-desktop:~/libvtest$ python libvtest.pyDom ReactOS-on-QEMU  State 1Dom ReactOS-on-QEMU  State 3 (after suspend)Dom ReactOS-on-QEMU  State 1 (after resume)mtj@mtj-desktop:~/libvtest$

Back to top

API overview

At a high level, the libvirt API can be divided into five API sections: the hypervisor connection API, the domain API, the network API, the storage volume API, and finally the storage pool API.

All libvirt communication occurs after a connection is created for a given hypervisor (for example, as shown with the open call in Listing 6). The connection provides a path for all other APIs to work through. In the C API, this behavior is provided through the virConnectOpen call (as well as others for authentication). The response of these functions is a virConnectPtr object, which represents a connection to a hypervisor. This object serves as the basis for all other management functionality and is therefore a required argument for subsequent API calls to a given hypervisor. Important subsequent calls are virConnectGetCapabilities, which returns the capabilities of the hypervisor and driver, and virNodeGetInfo, which retrieves information about the node. This information is returned as an XML document that can be parsed to understand which behaviors are possible.

Now, having access to a hypervisor, you can iterate through the various resources on that hypervisor with a set of API calls. The virConnectListDomains API call returns a list of domain identifiers representing the active domains on that hypervisor.

The API implements a large number of functions targeted toward domains. To explore or manage a domain, you first need a virDomainPtr object. You can get this handle in a number of ways (using either the ID, UUID, or domain name). Continuing with the example of iterating domains, you can use the index list that this function returns and call virDomainLookupByID to get the domain handle. With the domain handle in hand, you can now perform a large number of operations, from exploring the domain (virDomainGetUUID, virDomainGetInfo, virDomainGetXMLDesc, virDomainMemoryPeek) to controlling the domain (virDomainCreate, virDomainSuspend, virDomainResume, virDomainDestroy, and virDomainMigrate).

You can also use the API to manage and inspect virtual networks and storage resources. Following the model of the API, a virNetworkPtr object is necessary to manage and inspect virtual networks, and a virStoragePoolPtr (storage pool) or virStorageVolPtr (volume) object is necessary to manage these resources.

The API also supports an event mechanism with which you can register to be notified of particular events (such as a domain being booted, suspended, resumed, or stopped).

Back to top

Language bindings

The libvirt library was implemented in C (supporting C++) and includes direct support for Python. But it also supports a number of language bindings. Bindings have been implemented for Ruby, the Java? language, Perl, and OCaml. Work has also been done for calling libvirt from C#. Libvirt supports the most popular system programming languages (C and C++), a variety of scripting languages, and even a unified functional language (Objective caml). So whatever your language focus, libvirt provides a path to control your domains.

Back to top

Applications using libvirt

From just the small amount of capabilities that I‘ve demonstrated in this article, you can see the power that libvirt provides. And as you can expect, there are a number of applications that are being successfully built on libvirt. One of the interesting applications is virsh (demonstrated here), which is a virtualization shell. There‘s also virt-install, which can be used to provision new domains from operating system distributions. The utility virt-clone can be used to clone a VM from another VM (covering both operating system and disk replication). Some of the higher-level applications include virt-manager, which is a general-purpose desktop-management tool, and virt-viewer, which is a lightweight tool for securely attaching to the graphical console of VMs.

One of the most important tools built on libvirt is called oVirt. The oVirt VM management application was designed to manage a single VM on a single node or thousands of VMs over hundreds of hosts. In addition to simplifying management of large numbers of hosts and VMs, it can be used to automate clustering and load balancing and works across platforms and architectures.

Back to top

Going further

As you can see from this short article, libvirt is a great library for building applications that manage domains in many different hypervisor environments over large networks of systems. Given the growing popularity of cloud computing, libvirt will no doubt grow along with it, finding new applications and users. As of this writing, libvirt is only just over four years old, so it‘s relatively new in the massively scalable computing space. More is most certainly to come.

Resources

Learn

• Check out the libvirt Web site for the latest news about libvirt and to download the latest version. You‘ll also find a complete API reference manual that covers the core interfaces and error-handling interfaces.
• Libvirt supports a large number of hypervisors, including:
  • Xen
  • QEMU
  • KVM
  • LXC
  • OpenVZ
  • VirtualBox
  • User-Mode Linux
• In "Virtual Linux: An overview of virtualization methods, architectures, and implementations" (developerWorks, December 2006), learn more about the various types of virtualization. Cloud computing relies on virtualization for optimal use of server-available resources. With virtualization, servers can be used to host multiple operating systems and application sets.
• In "LXC: Linux Container Tools" (developerWorks, February 2009), you can learn more about the management tools built specifically for LXC. You‘ll notice some parallels here with the management methods for libvirt.
• Virtual Network Computing, or VNC, is a method for sharing graphical desktops over a network. This abstraction is ideal for massive scale-out computing, as it means a single station can manage many distributed clients.
• The Linux subsystem for managing file systems is large and complex. You can learn more about the larger file system subsystem in "Anatomy of the Linux file system" (developerWorks, October 2007).
• In the developerWorks Linux zone, find more resources for Linux developers, and scan our most popular articles and tutorials.
• See all Linux articles and Linux tutorials on developerWorks.
• Stay current with developerWorks technical events and webcasts.

Get products and technologies

• Red Hat‘s oVirt open VM management platform is a user of libvirt and demonstrates what kinds of applications can be built. You can use oVirt to manage large numbers of hosts, and the platform easily scales to support thousands of VMs.
• In this article, you used the ReactOS as a means to demonstrate a domain on QEMU. This article provides more information on ReactOS (a free Windows? clone), which is easily executed on the QEMU platform.
• With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.