XOSAN

XOSAN Beta Phase 1

Olivier Lambert

• 5 min read

The first beta stage for XOSAN is now open!

To activate your XOSAN beta, just go in the "XOSAN" menu in your (up-to-date) XOA, and click on "Register for Beta".

As soon we unlocked your beta, you'll receive an email! This could take some time, we'll start with a few testers first, then unlock more and more people. Be patient ;)

XenServer Hyperconvergence

The goal of this beta is to provide a first scale test of XOSAN, a thin-provisioned storage that turns all your local disks into a virtual SAN. No SPOF, simple to use and resilient.

WARNING: Don't use XOSAN SR for production VMs yet.

This phase 1 will allow us to test:

  • Adaptability: deploy XOSAN on a lot of various situations (ie your own XenServer infrastructure, which is de facto unique in terms of hardware/network/software setup)
  • Resilience: please shutdown one host and see what happens. It shouldn't be possible to lose data or even lose service if you are in the redundancy zone. At worst (ie outside redundancy number), you should expect read-only behavior.
  • Automation: if XOSAN auto-deployment works correctly

XOSAN creation guide

It's pretty simple. There is mainly 2 steps.

Step 1: pre-flight check

XOSAN will need a pool with:

  • XenServer 7 (7.1 compatibility coming soon)
  • Local LVM SR with at least 15 GiB of free space on each host
  • 2 GiB free RAM on each host for the XOSAN VM
  • a working connection with the updater (exactly like when you are in trial)
  • Our XOSAN pack installed on each hosts

The pack will just install user-space packages and add a new SR type ("xosan"). No other modification is done. Toolstack has to be restarted to be able to deal with XOSAN (no VM/service interruption).

Optional requirements
  • if you have an extra physical network for storage, that would be better
  • 10G networks will deliver better perfs
  • SSDs too, obviously

Step 2: XOSAN creation

After all the requirement are met, you can install XOSAN itself. First step is to select all local SR you want to participate to the new SR:

Here, we selected 3x SRs on 3 hosts, with each SR having 70GiB free

For this small setup (3 nodes), XOSAN will suggest you 2 modes:

As you can see, "disperse" will offer you almost 120GiB, with the possibility to lose 1 host entirely. "Replica" for 3 hosts will mean to "triplicate" data on 3 hosts, so only 60 GiB available, but you can lose 2 hosts!

We also generate a picture that recap the situation, respectively for "disperse" and "replicate", with in red the disk/nodes/hosts that can fail without losing availability:

Disperse mode for 3 hosts: one can be lost.

Replicate 3 mode for 3 hosts: two can be lost.

You have also to select on which physical interface it will communicate:

Then, just click on "Create XOSAN" and wait a bit: it will do everything until the Storage is ready:

You can now enjoy XOSAN!

Step 3: test it!

Now your XOSAN is created, it's "like" any other SR (pretty close than a NFS SR, but better: no SPOF)

You'll have running VMs for the storage:

What can I expect?

Probably removing/reinstalling XOSAN multiple times to test various settings, no promises on perfs for this first stage.

This beta won't be able to update and modify the initially created XOSAN, so it's really "create and destroy" for now.

What about perfs?

It's not the priority right now: tuning will come after we are sure of reliability and easy deployment. We can always improve things after, and maybe even distribute a common benchmark template to deploy on XOSAN to centralize/compare results.

Current limitations

  • you can't add new hosts to an existing XOSAN
  • you can't replace faulty hosts
  • you can't extend an existing XOSAN
  • some storage "combo" (redundancy/number of disks) are not yet available

For all of those limitations, for now it's simple: just remove and recreate it.

Need to add a local storage?

You need a local LVM SR on each host you want to participate to XOSAN. How to do this?

If you already have a free disk but if it's not a LVM SR already:

  • get the device name (eg with fdisk -l)
  • add the SR in XO (Add/SR/Local/Path of the device)

Eg you have a free SSD into your host, called /dev/sdc, use this path to create the local SR.

It will be used then by XOSAN to create the "shared SR" with all other local SR on all other hosts.

What's next?

This is Phase I. But then, after the first feedback and some time to develop it, another phases are in the pipes:

  • Phase II: introduce flexibility (remove/replace failed nodes, extend XOSAN when adding new nodes)
  • Phase III: performance tuning, cache system support (HDD+SSD), multiple disks per hosts…

GlusterFS under the hood

We'll create a blog post on how XOSAN works in details. We want to thanks the GlusterFS community for their assistance and availability to answer quickly all our questions.