Xen Orchestra 5.81

We're excited to present our latest 5.81 monthly release, packed with a multitude of improvements and new features. While our team has been hard at work refining "under the hood" aspects of the platform, we've still managed to deliver a substantial update that enhances your overall experience.

This release showcases our commitment to continuous development and innovation, and we're thrilled to share these updates with you. So, without further ado, let's dive into the enhancements and additions you'll find in Xen Orchestra 5.81!

🐦 VMware migration tool (V2V)

Over the past three months, we have been diligently enhancing our V2V tool to simplify the migration process from VMware to XCP-ng. With this release, our VMware-to-Vates V2V tool has become even more efficient and user-friendly!

Multi VM import

We are excited to introduce the ability to import multiple VMs simultaneously (either in parallel or sequentially) using our V2V web interface. You'll have access to a list of each VM to transfer, including its name, description, and settings. In other words, you can initiate a large batch transfer and then sit back and enjoy your coffee! This feature is fully compatible with our unique warm migration system, minimizing downtime to just a few minutes per VM.

Better error handling

We have now a better error message when importing a VM is failing. We covered 2 extra cases:

• if the *.vmx file missing on the VMware side
• if we don't support warm migration because of the VMware version (6.5+), giving you also the solution to do a normal conversion (consolidate and having the VM halted)

💾 Improved backup

This release brings a multitude of enhancements to XO backups. We've introduced the ability to set a transfer speed limit, making the platform more aligned with your infrastructure requirements. Additionally, we've expanded the use of NBD to accelerate backups, irrespective of whether a "block-based" system is being utilized or not. Overall, these improvements significantly boost the performance, flexibility and reliability of our platform.

Backup Transfer Rate Control

When configuring your backup job, you now have the option to set a speed limit for the entire process. XO will enforce the maximum speed you specify, irrespective of the number of hosts, disks, and VMs involved. If no speed limit is provided, the platform will continue to operate as before, with no restrictions on the transfer speed.

Faster backups

In our quest of faster backups, we started, few months ago, to enable NBD-capable backups. You can read more about our initial work in this blog post:

Xen Orchestra 5.76

Xen Orchestra 5.76 has landed. Discover what’s new!

Xen Orchestra BlogOlivier Lambert

We managed to add NBD "lower" in the stack, meaning that any disk export, regardless its origin (VHD, VMDK, OVA, Continuous Replication), will be exported via NBD if it's possible. Also, it works now even if you rely on existing VHD file storage, not only for "Block mode". In other words: faster backups for everyone!

There's only one downside: right now, the export task doesn't work for VBD, so it will be up to Xen Orchestra to compute the progress and "feed" XAPI with the progress. This will come in a next release!

You can see if your backup is using NBD for the transfer inside the backup log:

To enable this option, you need to edit your config.toml file, in the following way:

[xapiOptions]
preferNbd = true

And then restart xo-server.

More checks on the VHD stream

We have enhanced XO's backup feature by introducing a more efficient method for identifying the root cause of incomplete backups. Although we previously detected these issues, our latest update provides a more comprehensive approach to troubleshooting.

Now, we cross-check the anticipated size of a VHD file with the actual size read from the XAPI and the size written on the remote. This enables us to determine whether any errors occurred during the transfer process, either when pulling data from the host or pushing it to a remote location.

Additionally, our new feature offers more precise error reporting, as it identifies the specific size transferred before a failure. This allows us to pinpoint unique cases, such as issues occurring during metadata or block transfers, or at the end of the data stream.

This update effectively replaces the previous error message footer1 !== footer2, offering a more streamlined and accurate diagnostic tool for our users.

Improved S3 Compatibility

Originally, we developed S3 backup functionality with a focus on AWS S3. As more users started utilizing alternative providers, we received valuable feedback that allowed us to eliminate certain AWS S3-specific checks, enhancing our compatibility with other providers.

📡 REST API

We added 2 endpoints for the REST API, mostly linked to the backup jobs and logs. You can list all the enpoints with a simple GET on the API URL, eg https://myxoa.example.org/rest/v0. The new ones are:

[
  "/rest/v0/backups",
  "/rest/v0/restore"
]

You can see everything related to the backup by also fetching the endpoint "backup", eg https://myxoa.example.org/rest/v0/backups

Backup logs

You can now list all the backup logs on https://myxoa.example.org/rest/v0/backups/logs. Example:

[
  "/rest/v0/backups/logs/1583325989421",
  "/rest/v0/backups/logs/1583396152258",
  "/rest/v0/backups/logs/1583396182726"
]

If you want to take a look at a specific logs, just request its unique ID:

{
  "data": {
    "mode": "delta",
    "reportWhen": "always"
  },
  "id": "1583325989421",
  "jobId": "d3d8e979-d02a-448c-88a6-c2ca21b54b71",
  "jobName": "QA",
  "message": "backup",
  "scheduleId": "0767e335-9b7a-478c-85c7-62e502d4d101",
  "start": 1583325989421,
  [...]
}

VM restore logs

You can also check backup restore logs! Like our previous example, with https://myxoa.example.org/rest/v0/restore/logs. Same principle, you'll have a list of unique restore job, and you can pick one to get all the details:

{
  "data": {
    "backupId": "e37988bd-2dd4-423f-b8d9-a01dca0913de//xo-vm-backups/123e4f2b-498e-d0af-15ae-f835a1e9f59f/20230306T165313Z.json",
    "jobId": "d3d8e979-d02a-448c-88a6-c2ca21b54b71",
    "srId": "86a9757d-9c05-9fe0-e79a-8243cb1f37f3",
    "time": 1678121593431
  },
  "id": "1678121829279",
  "message": "restore",
  "start": 1678121829279,
  "status": "success",
[...]
}

Backup job

You can also list your current backup jobs at any time on the https://myxoa.example.org/rest/v0/backup/jobs endpoint. It will list all the jobs, and then you can request one specifically to get all the details:

{
  "remotes": {
    "id": {
      "__or": []
    }
  },
  "name": "test",
  "mode": "full",
  "settings": {
    "bbc34094-aa92-4879-beca-bd0b9474f7e2": {
      "snapshotRetention": 1
    }
  },
  "type": "backup",
  [...]
}

🆕 Misc

We are releasing various qualify of life improvements for XO 5.

Enhanced Network Configuration

When creating a network, you can choose a physical network card, known as a PIF in XAPI terminology. However, if multiple VLANs are associated with the same PIF, the selection menu wouldn't display this information, leading to a confusing list of seemingly identical NICs without their corresponding VLANs.

We have now resolved this issue by displaying the PIF VLAN alongside the network card. For example, for eth2, you will now see the associated VLAN clearly displayed, making network configuration more intuitive and user-friendly:

Grouping icons on pool and host view

Xen Orchestra is providing more and more information for each host or pool: NTP out of sync, updates available among many other things. It's now grouped into one icon, providing the details when clicked.

For example, on an old XenServer version, it was like a christmas tree:

Now, it's a lot better:

Better experience for supported hosts

Previously, we only displayed if you had XCP-ng Pro support at the pool level. However, it wasn't clear when browing on the host view. Now, have a clear information about the support status for a specific host:

OIDC improvements

Our feature released last month is already improved on two aspects:

• the plugin will use the standard well-known suffix for auto-discovery
• email field is now supported as username

Add Suse icon for all their distros

Sometimes, we endure the joy of marketing from some software companies. Especially when you want to match the distro name with its icon. Suse is probably the "best" for that. Let's take a look on how we match the Ubuntu icon from the tools: ubuntu: ['ubuntu']. Easy right? Now for Suse, every year there's another name to add… See for yourself:

suse: ['sles', 'suse', 'opensuse-leap', 'opensuse-microos']

I asked GPT4 about what's next:

While I cannot predict the future naming decisions of SUSE, I can provide a playful suggestion based on their naming pattern. The next name could be something like "OpenSuse Nano OS" or "OpenSuse Quantum OS," emphasizing the notion of a compact, lightweight, and efficient operating system.

Get ready for Nano OS then…

🔭 XO Lite

Despite a lot of our work is mostly done "behind the scene" (see below), we managed to get new features in XO Lite for this release.

Also, don't miss our previous article on the way we are building our XO Lite components!

XO Lite components

Explore the inner workings of XO Lite’s component-based design and how it enhances user experience in virtualization management through our in-depth blog post.

Xen Orchestra BlogThierry Goettelmann

VM Filters and Sort

In the Pool/VMs view, you can now create filters and/or sort your VMs. It's still a work in progress, but it's already a very powerful tool, that we will likely re-use on XO 6.

The filter is now visible:

If you click on it, it's easy to update it:

Sort is very similar: you can sort per name, description and power status. To revert between ascending and descending, you just have to click on the created sort:

Enhanced Console Response Time

We've made significant improvements to the console response time after a VM reboot, ensuring you won't miss the Grub menu or the virtual BIOS/UEFI during the boot process. This enhancement proves valuable in various situations, and although it may not sound remarkable, the results genuinely contribute to more efficient VM management!

Multiple pages and modals improved

We've made numerous enhancements across multiple pages and modals in our ongoing effort to provide an exceptional user experience. Although some elements are still missing in the current interface, these improvements aim to address key areas. Here are some examples:

CPU provisioning

You can now track the total number of vCPUs in use across your pool, compared to the available number of cores. This feature provides a clearer understanding of your resource utilization:

Available updates

You can now see the available updates from XO Lite:

☄️ Improved deploy script

If you don't want to use our web deploy form, you can -as an alternative- our deploy script in Bash. We updated it to be more user friendly, with more detailed steps and also some friendly emojis 😜

🚀 Other stuff to check out

Don't forget to catch up on some cool articles from the XCP-ng and Vates blog. Here are a few highlights:

Vates joins the Xen Project

Vates, the European system management and virtualization stack company, announced today that the company has joined the Xen Project.

Vates BlogMarc-André Pezin

March 2023 Security Update

We published an update that fixes XSA-[427-428-429]. At the same time, we’ve included bugfixes and enhancements for Xen such as an update to version 4.13.5 and Initial Sapphire Rapids support.

XCP-ng BlogGaël Duperrey

Bringing Rust to the Xen Project

Bringing the Rust language to the Xen Project? Yes! But how? And where to start? Discover more in our first article in a future series dedicated to our journey in the Rust and Xen world!

XCP-ng BlogOlivier Lambert

New Guest Tools in Rust

Discover the latest blog on integrating Rust into Xen Project. We’re building a flexible Rust-based agent for VM-Dom0 communication, showing promise in supporting various guest OS with room to grow!

XCP-ng BlogYann Dirson

🎯 Conclusion

In conclusion, we're delighted with the progress and updates in the Xen Orchestra 5.81 release, and we believe you'll appreciate the enhancements as well. As we look forward to our April release, we're excited to share that it will be even more impressive, with further advancements and news about our Project Pyrgos, which focuses on Kubernetes integration. Stay tuned for more details, and as always, thank you for your continued support and enthusiasm for Xen Orchestra!