Release

Xen Orchestra 5.78

Time to see what's inside this December release and what's next.

Olivier Lambert

Dec 20, 2022 • 7 min read

Merry Christmas and welcome to XO 5.78, the last release of the year! 🎄🎄🎄

We are doing our release a little bit earlier than usual, so we can have some time to rest before a new year 🎅

📸 VM snapshot ignoring disks

Thanks to a contribution made by XCP-ng team to the XAPI Project (see below), we can now snapshot partially a VM while calling VM.snapshot method. Partially means if the VM got 2 or more disks, we can ignore any disk we want. Why is it a great thing? Because previously, with the [NOBAK] name of the disk, we indeed ignored the tagged disk in the export/backup process.

However, the disk was still snapshot on the storage, using space for nothing. With this new feature, XO detecting [NOBAK] will automatically make the new call asking XAPI to ignore this specific disk. No more "useless" disk snapshot!

Our contribution in XAPI Project for this feature is visible here:

☝️

Note: it works with a recent version of the XAPI, already available in XCP-ng 8.3 alpha. This is also 100% transparent for you: when you'll migrate to 8.3, it will work "out-of-the-box" with your existing [NOBAK] disks!

🐦 VMware warm migration tool

Do you remember our previous blog post on how to migrate to VMware to XCP-ng? We announced something in there but now it's coming:

Yes, warm migration from VMware to XCP-ng! We are really excited with this feature. In short, we are working on a tool to apply the concept of "warm migration" implemented in our previous release (see the dedicated blog post below). However, instead of migrating between two XCP-ng hosts, this time it's between an EXSi/vSphere host to… XCP-ng!

You might ask "how?", and well, even if we use the same principle, it's a bit harder to implement as you can imagine.

Phase 1: metadata. First, our tool will create a similar VM on XCP-ng (same name, disk, network…). We'll replicate automatically all the metadata: you just have to choose on which storage repository, which network to create the disks and the virtual interfaces respectively. The rest is handled automatically!

Phase 2: initial replication (full). Then, we snapshot the VMware guest and we export this "full" toward the XCP-ng VM, injecting directly the content in the previously created disks. This phase is done without shutdown the current VM on VMware side. Yes, this is taking some time, but your service stays up during this first phase. Alternatively, you can decide to keep the VM shutdown if you don't care, meaning you won't have to go for the next phase.

Phase 3: send the diff (delta). Now we have an initial sync between those two sides, we will shutdown the VMware guest and immediately export all the modified blocks (diff/delta) since the first snapshot. And we finally merge them on XCP-ng side "on the fly".

As soon it's done (should be few minutes only!), we can boot the VM on XCP-ng side, and… that's it! Sure, you might need to change some details inside the VM (like the network configuration, installing XCP-ng tools), but most things should work roughly out of the box.

Current status

For this release, we managed to get a preview working up to the Phase 2. The VMware compatibility is relatively broad, most versions on the market should be compatible with our tool (from VMware 4.x to 8.x). However, we'd like to test it outside our lab and get some feedback from it. That's why we opened a forum thread so you can try it on your side:

XO Forum: VMware migration feedback

Also, since we could only export the "raw blocks" from VMware (see below), a direct transfer into VHD would have created a "thick VHD". For example, on a 40GiB disk with Ubuntu inside using only 2GiB, the copy on XCP-ng would have use 40GiB on the storage, regardless the usage of thin or thick SR.

🤷

It seems impossible to export a VM disk in a VMDK format from the VMware API. The only way we found is to pass the VM file path in the URL, having acess to the raw blocks only. It works, but it requires some tricks to make the transer on used blocks only. Obviously, if you know better, please tell us!

That's why we had to make a "double read" solution: the first will be used to count and note the used blocks on the whole disk, and the second to only transfer them, without any empty blocks! It works well, meaning your destination VHD will be now only 2GiB (per our example).

Testing it

To test this new tool, you need to switch your XOA to the vmware release channel. If you are using it from the sources, switch to the vmware branch and rebuild.

At this point, you can use xo-cli to call the migration tool:

xo-cli vm.importFromEsxi host=<VSPHERE_IP> user=<VSPHERE_USER> password=<VSHPERE_PWD> sslVerify=<true|false> vm=<VSPHERE_VM_ID> sr=<SR_UUID> network=<NETWORK_UUID>

As you can see, it's pretty obvious. Just gather all the information needed to contact the VMware side (IP, credentials, ssl check or not and the ID of the VM). On the XCP-ng side, just provide the destination SR and network.

💡

On the VMware side: in order to get a working transfer, you need either to have one snapshot of your VM OR to get it halted.

Now, you can see the transfer progress in the "Task" view of your Xen Orchestra UI. As soon it's done, you can directly boot the VM!

☸️ Kubernetes recipe with Flannel

It's been a while, but we decided to update our Kubernete recipe. The things evolved a bit in this world: we now must rely on a Network Plugin, using the "Container Network Interface" (CNI). We made the choice of Flannel, because it's very simple and works with our Kubernetes automated deployment.

It's entirely transparent for you, so if you want to make a test with a small k8s cluster, go ahead!

🔭 XO Lite: new features

Reminder: XO Lite is meant to be the XCP-ng embedded UI to manage/bootstrap your host locally. Here is the new components available.

Graphs

Our first graph is related to CPU usage. More graphs will come soon after, since most of the work is to create the initial component.

Our first CPU graph, used with the dark theme (default)

404 or missing data page

It's part of building a nice tool: managing the error cases and accessing missing resources. That's why we added 2 pages to handle those cases:

🔄 XO tasks: a work in progress

One of the big feature of the next big Xen Orchestra release, "XO 6", will be a complete new task system. If you want an overview on XO 6, you can read this previous article:

Anyway, at the moment, we only rely on XCP-ng/XenServer tasks. Which is fine for "simple" action (VM migration, export…) but not great when you trigger more complex operations, like "Rolling Pool Upgrade" or "Warm migration". The new goal is to create an "XO task", including other tasks inside, so you can follow a complex process easily.

For example, a "warm migration" contains at least 7 steps: snapshot, VM creation on destination, transfer, shutdown source VM, re-snapshot, delta transfer, boot on destination. So we can have a precise "current step" tracked with this system, so you know exactly what's going on!

We recently made good progress on this new task system. We expect it to be used first with the warm migration in the next releases, early next year.

And also, this "generic" task system will be exposed in the UI in XO 6, with some extra bonus features (like cancellation), and a "history" of them so you can get details on them (success, failure, at which step, how long…)

⭐ 2022: a recap

It's our tradition: we release an infographic providing a recap of the year for Xen Orchestra, XCP-ng and Vates in general. Go take a look to our article so get a sum up of what happened!