Xen Orchestra 5.83

In this month's Xen Orchestra update, we spotlight enhancements to self-service capabilities and our interface, progress in XO Lite, and the introduction of the new 'Mirror Backup' feature. With advancements in REST API and ongoing development in Project Pyrgos, we continue to elevate your user experience. Dive in to discover these developments in detail!

💾 Backup

This release brings a wealth of new features to the backup side of things. Let's go!

Backup nomenclature

As announced, we started to rename/clarify all our backups:

It's now visible in XO UI for this latest release. As a reminder:

• The first run of an Incremental Backup is called the 'key backup'.
• All subsequent runs of an Incremental Backup are referred to as 'delta'.
• 'Full backup' specifically refers to a backup made using the Full Backup feature.

For the Replication:

• A job previously known as a 'Disaster Recovery' job is now termed Full Replication.
• The initial run of an Incremental Replication is called the 'key replication'.
• All following runs of an Incremental Replication are known as 'delta'.

Mirror backup

We have introduced a feature allowing you to back up a Backup Repository (BR) directly! But let's first understand its necessity, given that you can already include multiple BRs in a single backup job.

Multi BR job

Within a single backup job, it was already possible to configure two distinct BRs simultaneously. Imagine executing backups on hosts from two separate datacenters, and sending all those backups to two distinct BRs (A and B). Here, the data is transmitted concurrently to different backup repositories, which is quite beneficial.

However, since the data is synchronously dispatched to both BRs, it's important to note that the speed of the entire backup job will be dictated by the slower BR. Another limitation is that this multi-BR backup job will utilize the same schedule and retention on both backup repositories, as they are perfectly synchronized.

Introducing Mirror backup

This is where mirror backup comes into play. You can now create a specific backup job for an existing BR, entirely asynchronously. Essentially, mirror backup allows you to independently synchronize all your "direct" backups to another BR, but with your personal schedule and retention. There are numerous use cases: you might carry out all your nightly backups to a local BR (same datacenter) with a broad bandwidth, and then create a mirror to a remote site with a smaller bandwidth, but with an entirely different schedule and retention. The mirrored BR could utilize slower hardware (like spinning disks with more capacity) -while your original BR is fast for your daily backups- providing an ideal "archive" scenario for longer retention!

Here is a diagram:

This implies the possibility to manage a completely different location, schedule, retention, encryption, and type of backup repository, based on an existing BR!

To use mirror backup, navigate to "Backup"/"New"/"Mirror Backup". You can then configure the Backup Repository ("BR", formerly "remote") to backup to another BR:

Planned improvements

We're planning to refine and broaden this feature further. Initially, we're focusing on the technical aspects. For instance, by creating a synthetic VHD, we can reduce the quantity of files to be transferred.

The second improvement we're planning is related to filter capabilities: you'll be able to mirror only under specific conditions (like mirroring only VMs with the "Archive" tag). This dynamic approach will contribute to providing a high level of protection and automation.

🛍️ Self service

The current release brings substantial upgrades to our self-service capabilities.

Tag selector

The new release allows for a default tag configuration in self-service, increasing efficiency especially when linking these self-service created VMs to a designated backup job. An illustration of this would be establishing a self-service titled customer1 that carries a default customer1 tag on creation. Consequently, a "smart backup job" will initiate automatic backups for ALL VMs associated with this tag, completely eliminating any need for manual intervention.

Default sharing of newly created VMs

Previously, a VM created within a self-service platform was exclusively visible to its creator. With our latest release, we provide the option to automatically share all VMs among every user in a particular self-service. This feature proves particularly useful when looking to instantly share VMs within a user group such as developers or customers.

📡 REST API

This new REST API release is all about updates. Our goal was to equip you with everything necessary to monitor updates and even enable a degree of automation to keep your infrastructure fully updated!

Check updates

Checking for all available updates on an entire pool is now a breeze with this simple HTTP call. We also made the effort to make the new endpoint discoverable! For example, if you just request a pool object with a regular:

curl \
  -b authenticationToken=KQxFkTbs \
  'https://xo.company.lan/rest/v0/pools/<POOL UUID>'

You'll have this new entry, see the missing_patches_href:

{
  "current_operations": {},
  "default_SR": "86a9757d-9c05-9fe0-e79a-8243cb1f37f3",
  "HA_enabled": false,
  "haSrs": [
    "86a9757d-9c05-9fe0-e79a-8243cb1f37f3"
  ],
  "master": "0aea61f4-c9d1-4060-94e8-4eb2024d082c",
  "tags": [],
  "name_description": "",
  "name_label": "XO Lab",
[...]
  "cpus": {
    "cores": 120,
    "sockets": 6
  },
  "missing_patches_href": "/rest/v0/pools/<POOL UUID>/missing_patches"
}

So if you request this URL:

curl \
  -b authenticationToken=KQxFkTbs \
  'https://xo.company.lan/rest/v0/pools/<POOL UUID>/missing_patches'

Then, you'll have the available updates list, in a proper JSON array:

[
  {
    "url": "https://github.com/tianocore/edk2",
    "version": "20180522git4b8552d",
    "name": "edk2",
    "license": "BSD and MIT",
    "changelog": {
      "date": 1665144000,
      "description": "- Correct the usage of memory barriers to fix an occasional hang",
      "author": "Andrew Cooper <andrew.cooper3@citrix.com> - 20180522git4b8552d-1.4.6"
    },
    "release": "1.4.6.xcpng8.2",
    "size": 2382792,
    "description": "EFI Development Kit II"
  },
  {
    "url": "https://github.com/xcp-ng/xe-guest-utilities",
    "version": "8.2.0",
    "name": "xcp-ng-pv-tools",
    "license": "BSD and GPLv2",
    "changelog": {
      "date": 1662984000,
      "description": "- Switch RPMs to systemd by default and provide legacy RPMs for chkconfig.\n- Merge separate -xenstore RPMs back into xe-guest-utilities RPMs",
      "author": "Gael Duperrey <gduperrey@vates.fr> - 8.2.0-11"
    },
    "release": "11.xcpng8.2",
    "size": 15129256,
    "description": "ISO with the Linux PV Tools"
  }
]

Rolling Pool Updates

Now, you can even trigger the convenient Rolling Pool Updates mechanism directly with a simple HTTP call. It couldn't be easier:

curl \
  -X POST \
  -b authenticationToken=KQxFkTbs \
  'https://xo.company.lan/rest/v0/pools/<POOL UUID>/actions/rolling_update'

This paves the way for some truly interesting capabilities. Consider setting up a CRON job (or a Systemd service) to automatically fire this call to the REST API, every Saturday night, for instance. This ensures that your cluster is always up to date!

☸️ Project Pyrgos (k8s)

For those who might have overlooked our initial announcement, here's a recap on Project Pyrgos:

Announcing project Pyrgos

Have you ever wanted to easily create and update Kubernetes clusters on top of your on-prem infrastructure? That’s exactly what Pyrgos project is.

Xen Orchestra BlogOlivier Lambert

It's now possible to create Kubernetes clusters with multiple control-planes, improving the resiliency of your cluster:

With a solid foundational cluster in place, the next stages of development are set to bring more enhancements. Keep an eye out for our forthcoming release!

🔭 XO Lite

In the previous month, we mentioned the intensive work on multiple components without immediate visible progress. This behind-the-scenes work is crucial in crafting superior user experience (UX) and user interface (UI) – there are simply no shortcuts!

This month, we are thrilled to present tangible outcomes of our efforts! For an in-depth understanding of our approach towards UX and UI, do not miss this insightful article by Clemence, our accomplished UX/UI designer. It provides a sneak peek into the upcoming interface for both XO Lite and XO 6:

Unleashing the Power of Atomic Design System

Exploring the Atomic Design System in open-source virtual infrastructure: enhancing user experiences, inclusivity, and accessibility while empowering teams.

Xen Orchestra BlogClémence Barthoux

Better consoles

Now, all our consoles are a lot better to use! First, the space usage of the console is really fitting the screen regardless the size:

The big bonus is that now you have a button to detach the console in a dedicated window:

Other improvements

And there's also many other improvements, like:

• Better tool-tips components
• Tabs added for the VM view
• Better modals (easier to read and can be closed with escape key)
• New form component
• Better spinner
• More actions available in bulk (copy, remove, power state)
• Dynamic page titles

An example of copying a VM:

And in bonus, here is a nice preview of the next XO 6 Dashboard:

🆕 Misc

While the new additions to backup, XO Lite, REST API, and Pyrgos are substantial, we haven't overlooked the need for quality-of-life enhancements for the existing interface.

Introducing XO Tasks in UI

We've taken the first step to incorporate our bespoke tasks, known as "XO tasks". As you might be aware, we've been displaying pool tasks in the "Task" view. However, as Xen Orchestra extends its orchestration features and performs increasingly complex asynchronous tasks, sometimes spanning different clusters, we've found the need to create dedicated XO tasks. This is a critical part of our major architectural revamp for XO 6.

The first application of this will be our V2V import system, enabling a seamless transition of VMware to Vates (the XCP-ng+XO stack).

Over time, this will prove invaluable for other use cases such as backups, Rolling Pool Updates, and more, enabling easy tracking of overall progress.

Oracle VM OVA import

While VMware and HyperV users are our most common converts to XCP-ng, Oracle VM is also being phased out from an inscreasing number of our new customers. This marks a notable shift in the virtualization market! Oh and by the way, don't forget our recent article on how Vates with XCP-ng and Xen Orchestra is making an meaningful impact in this revived market:

Vates’ Continued Recognition in Gartner’s Server Virtualization Guide

We’re thrilled to announce, just as we did last year, that Vates has once again been recognized in the Gartner Market Guide for Server Virtualization.

Vates BlogMarc-André Pezin

Back to the topic: The OVA import from Oracle VM had some peculiarities, including a discrepancy in size between metadata and actual data (disks). I don't think it was meant on purpose, but anyway, we had to adapt our import code to make it work with their OVAs. Now it works well, so if you're looking to migrate from Oracle VM, you're more than welcome!

Plugin Filter by Name

A minor but convenient improvement: with an ever-growing list of plugins, we've enabled a simple filter to focus on a specific plugin. All it takes is a click on the plugin name.

Editable Proxy Address

Yet another life improvement: for users who make network modifications after setting up several proxies, we've made the proxy IP address editable directly from XO. This will inform the proxy appliance to use the new address!

Warning on hosts lacking HVM

Now, if you have a host that doesn't support HVM mode (hardware assisted virtualization), you'll see a warning in the host view. It's an important heads-up, as there's little reason to not have an HVM-capable host today:

If you really know what you are doing, you can simple ignore this message.