Xen Orchestra 5.88
October release is here 🎃 And it's unreal 👻 How it's possible to cram like one quarter of work in one month? I really wondered when writing this blog post.
As usual, here is the video log of this release:
You can also listen to it in a podcast version on Spotify.
🧑🚀 Project & Community
Since Xen Orchestra is a lot more than code, but an entire project with a vibrant community, we wanted to show you what's up in this area.
SFSCON 2023: Elevating Open Source Innovation in Europe
SFSCON 2023 is an important Open Source event in Europe focused on new ideas, working together, and leading the way in the Open Source world. The conference, happening in Bolzano, Italy, will have more than 100 speakers from all around Europe. One of them will be our own Chief Strategy Officer, Charles Schulz. He'll talk about why open digital infrastructure is crucial and how we can keep control of our digital future. This year's conference is all about discussing the potential of free software and how we can work together to keep our digital freedom.
Read more about Charles' talk in our Vates blog post:
Marc-André Pezin
CloudStack Collaboration Conference 2023
This time, it's an event dedicated to CloudStack, in Paris on 23-24 November. I will be there to host a workshop called "Zero to Cloud Hero: Crafting a Private Cloud from Scratch with XCP-ng, Xen Orchestra, and CloudStack."
Charles will also make a talk on Open Source specifically, while Andrei will present "Enabling DPU Hardware Accelerators in the XCP-ng Cloud Platform Environment." More details here:
Marc-André Pezin
A new card preview and README file for XO
We added a new card (in your Github project settings by the way), to display a nice image preview when you post it on social networks and such:
vatesfrAlso, if you are browsing Github to find new exciting projects, the first thing you will notice will be the README file. We neglected it (in favor of our official documentation) but now it's a lot better!
I mean, take a look by yourself. Before and after. I think we can tell it's clearly better now!
XO team is growing
We welcome Mélissa in the XO team, and it's not the only one joining the team. Two more developers will be joining us in the coming months. So keep an eye out—our team is expanding, and it couldn't come at a better time with all the exciting new projects we have on the horizon!
A community investigation on Netbox
Sometimes, users expect unknown behaviors, and in this case, the right move is to open a thread in our forums. And that's exactly what was done for this issue: during a Netbox sync, we had an error message telling:
The selected cluster is not assigned to this site (None).
After various investigations (thanks to Pierre!) we managed to track down the problem, due to a behavior change in Netbox between multiple versions:
- Prior to Netbox v3.3.0: no "site" field on VMs
- v3.3.0: "site" is REQUIRED and MUST be the same as cluster's site
- v3.3.5: "site" is OPTIONAL (auto-assigned in UI, not in API).
nulland cluster's site are accepted. - v3.4.8: "site" is OPTIONAL and AUTO-ASSIGNED with cluster's site. If passed: ignored except if site is different from cluster's, then error.
So updating Netbox to v3.4.8 or later fixed the issue. Obviously, we also managed to get a patch to workaround the problem if you are running on an older Netbox version. But you know, it's always important to keep your software up to date, mostly for potential security flaws.
A new external contribution
We are please to report another external (outside Vates) contribution. Every time someone is contributing, we do our best to help her/him, and we never forget to credit the person!
This month, it's a big thanks to Malcolm Scott! See below for details on the feature "Preserving current page across reauth".
💾 Backup
There's always a lot of work in this area. This month, it was mostly improving the existing code and fixing bugs. But we also managed to deliver an important optimization for those using S3 with full backups (large XVA files).
S3 full backup improvements
As soon you sent XVAs larger than 50GiB, you could had various issues, depending on your S3 provider, who usually wanted to split the files or at least sending some error messages about that. We found a way to get it in a better way, that will also make it more robust and consume less XO memory!
🌍 Terraform
Many new things and improvements for our Terraform provider! The latest release (v0.25.1) and the one before are offering the management of XenServer/XCP-ng bonded network, support for destroyCloudConfigVdiAfterBoot and source_pif_device. Among many other things, feel free to check the changlogs at https://github.com/terra-farm/terraform-provider-xenorchestra/releases
terra-farmIt was also the opportunity to improve XO internal API (the JSON RPC one, used by the Terraform plugin) to expose xenstore attributes. This is useful for a Citrix/Cloud Software Group developper to do some nice automation with Citrix VDI with Terraform on top of Xen Orchestra!
🔭 XO Lite
More actions can be achieved in XO Lite this month! You can now clone and/or snapshot your VM:
We also added the very much needed "Ctrl Alt Del" button in the console view so you can finally login to your Windows guest:
Various improvements also in the UI, like nice icons when loading a graph, putting the title instead of the XO Lite logo on the header:
And finally a simple but nice extra, a "remember me" checkbox on the login form:
🛰️ Xen Orchestra 6
We wanted to keep you posted on the progress we are making for XO 6. It's hard to juggle between bug fixes, XO 5 and XO Lite improvements but we are still moving forward! Most of the work is mostly on designing that new UI (see the "New mockups" paragraph below) but also re-designing on the technical aspects.
We started to build a basic bootstrapping for the new xo-web, which will serve as the cornerstone for our entirely revamped user interface. That's also where we start to work on a truly poweruly new subscription system, fetching only the required objects. This will solve so many limitations or performances issues, we are pretty excited to demonstrate something before the end of the year. Stay tuned and enjoy the new mockups in the meantime!
New mockups
XO 6 will focus on providing a massively upgraded experience on what we think is the most important stuff to improve: backup management.
Obviously, we'll also have a fresh/new dashboard, which we refined:
But the real deal will be the backup management experience. For example, here is the new "flat" backup view without specific grouping of your backups:
That's already many times better than the current XO 5 view. It's a lot clearer, and directly build with the capability to interact with all your backups, that you have one or hundreds of them.
Check the "Add group" button on the top right. That's the real deal: it's a very efficient way to restore your entire site or a large chunk of your infrastructure. You can group all your backups by BR (Backup Repository), VMs, jobs etc.
Finally, the file level restore view is also vastly improved, with many extra information without cluterring the UI:
As you can see, this will give a far more pleasant experience when dealing with your backups. And it will be the first part of XO 6 that will be available in a dedicated path of your UI in Xen Orchestra virtual Appliance, so you can play with it as soon it's there.
☄️ First XOSTOR interface
As we are moving forward with XOSTOR (and still having our share of new bugs but hey! better now than later!), we wanted to provide a simple UI to create our hyperconverged storage, so we can gather more feedback from our XOA users.
As a first step, we just finished to provide the UI to create a new XOSTOR storage. It wasn't trivial, since we needed to expose the physical disks (inside your hosts) to the Xen Orchestra web UI. After writing some new plugins on the host side, and a some glue in XO, here is the first result:
We tried to make the UI as simple as possible, hiding pools and disks where you can't create any XOSTOR (missing requirements), make it easier to interact with, limiting the potential issues. It's even possible we'll make it even simpler in the next iteration.
After that, we'll have a dedicated view (like we had in XOSAN) to manage your XOSTOR, with many interesting health data directly displayed in XOA.
All feedback is welcome, we also have a 100% dedicated section to XOSTOR in our forums!
🚀 XCP-ng 8.3 features
This section is dedicated to new features we added in Xen Orchestra, that are only compatible with XCP-ng 8.3: most of the new modifications we are doing in XCP-ng are not backported to the LTS version, for obvious stability reasons.
vTPM management in the UI
Most of the vTPM work was done by XenServer (we wanted to contribute but we can understand they were already advanced enough to need some help, even if we think it would have been helpful for review but hey, you can't force someone to accept contributions).
But at least, we "finished" the work by exposing the features in a centralized and web UI (and API, and CLI): you guessed… Xen Orchestra!
You can now do the entire "cycle" of using a vTPM to run a Windows 11 VM, or any other OS by the way. Just create the vTPM for your VM (in the Advanced tab):
And that's it!
You can delete it anytime if needed.
host.evacuate max limit
When doing a "maintenance mode" on a host in Xen Orchestra (or a Rolling Pool Update), at some point it will call the host.evacuate XAPI method from XCP-ng. This call will live migrate all VMs from a host to any other available host in the same pool. It will do that by concurrently migrating 10 VMs at once to speed up the evacuation.
However, if your storage system is pretty under stress already (or your host), this might affect the "freezing" time of a VM in the process. For example in a very crowded and noisy environment, in some cases, one VM can be paused around 30 seconds, far from the usual hundreds of milliseconds in a "normal" case. There's a workaround already: modify the xapi.conf file to set a lower value. However, this requires a modification on all your hosts, so it's really not a definitive solution. We wanted something integrated!
To answer that requirement, we decided to upgrade the XAPI method with an optional parameter, the "batch size":
xapi-projectThanks to a very reactive XAPI team, our modification was merged pretty quickly. But that's not it! Now you have an optional parameter in the method, Xen Orchestra is also using it, with a concurrency at 3 by default. 3 is a sweet spot between speed to migrate and avoiding long pause time in live migrations. And it doesn't change any default configuration on your hosts!
This feature is another great demonstration how great is the fact we -at Vates- can deliver the whole stack at once: when we modify the API of the host (XCP-ng) we can also leverage it to apply the thing we wanted directly in Xen Orchestra!
🆕 Misc
Despite all the "behind-the-scenes" work done for new XCP-ng 8.3 features, XOSTOR, XO Lite & XO 6 (that's a lot 🥵), we also have many improvements in the existing Xen Orchestra version.
Preserve current page after re-authentication
This one is not that big in terms of lines of code, but really handy for both our users and developers. First, mostly for developers: if your restart xo-server, you won't lose your current page after re-login.
And for most users: if you need to re-log (eg after one night), you will be re-transferred to the previous page you were before. Very very handy, pretty simple to achieve but… we never really managed to work on that. Thanks a lot to our external contributor to push us to make it real!
vatesfrNetbox
Thanks to a nice investigation lead on the forum and Pierre's dedication (see the community section before), we managed to solve the synchronization issues one could encounter if not running on a recent version of Netbox. But keep up on the updates, because like Xen Orchestra, the great Netbox project is releasing often and fixing security issues on new versions!
Show network name in PIF
Sounds trivial, but in fact it is very helpful. Now, we show the network name in PIF selectors as it can help differentiate PIFs that have the same device/deviceName.
Check version for source users
Our documentation is pretty clear about this:
But it's a common issue we have in the community: people are eager to report a problem, but forgeting -at first- to check if your XO is correctly up-to-date on the latest commit on our master branch.
That's why we decided to help our source users to display if they are up-to-date or not, directly in the "About" view:
Self service improvement
In the self service view, you can now have a precise number of VMs using a specific self service (or resource set).
And by clicking on the link, you'll go to an automaticallyt filtered view of these VMs:
This is pretty useful if you administrate multiple self services and want to keep an eye on who is doing what.
VDI health display improvements
This month, we managed to generate various improvements related to your VM disks, or VDIs.
First, in the Dashboard/health view, where we a displaying "Orphaned VDIs", in other words, VDI not connected to any VM will appear here. However, some VDIs are special and shouldn't be displayed there (like some virtual iDRAC devices). Those devices are sized at "0", so we used that to filter them out.
We also improved the computation of the number of VDI to coalesce, because we were a bit pessimistic on the number, since you can have the same chain with multiple branches. And since we now have a correct number, we also provided the exact amount: before it was "more than 10" for example.
Token management revision
Token management could have a dedicated blog post, since the topic is pretty large. But in short, we previously generated many many tokens from the original one created (for example) with XO CLI. And cluttered the interface and was confusing.
Now, your freshly created token will stay unique and the UI to see your token will be a lot more understandable!
