Delta File Restore - LVM



  • As you can see, it's not the same error. This time, the mount command fails because your XFS has a dirty log.

    So in order to mount it, XFS "driver" need to replay the log and therefore need to write inside it. However, we mount it in read only, because we don't want to modify the data inside your VHD.

    That's why it doesn't work in this case.

    The real struggle on our side, is to be able to detect which FS is existing on a disk, and to pass specific options to mount. In theory, mount can handle this itself, but there is some case where it's not possible (like this one). Also, if we modify the VHD, we need to handle this by creating a new delta to avoid writing directly on the backup VHD. As you can see, it evolves a LOT of stuff.

    BTW, we could avoid this issue in ext FS because we can discard this error by passing an extra flag to the mount command. Seems not doable with XFS.

    edit: maybe norecovery flag can do it. See https://serverfault.com/questions/839898/cannot-mount-block-device-dev-loop-read-only

    edit2: checking the code right now to see if a fix is doable quickly



  • Oh well another one to the list of tasks for a weekend.



  • Good news: we used noload for ext filesystems, but it seems the norecovery option would work for both ext AND xfs.

    If you are using it from sources @dlucas46 , please switch to the branch xfs-norecovery of xo-server (don't forget to yarn just after) and tell me if it works now.



  • Whats the command to change to that release branch?



  • git checkout xfs-norecovery



  • That gave me :

    fatal: Not a git repository (or any of the parent directories): .git
    


  • @dlucas46 Were you in the correct subdirectory (/opt/xo-server)?



  • @Danp

    Yes.

    I can change to next release.

    but xfs-norecovery gives me this

    root@xen-orc:/opt/xo-server# git checkout xfs-norecovery
    error: pathspec 'xfs-norecovery' did not match any file(s) known to git.
    root@xen-orc:/opt/xo-server#
    
    


  • @dlucas46 Then git fetch origin first.



  • @dlucas46 Try issuing git fetch first.



  • @olivierlambert

    OK

    That switched but it does not seem to have checked anything out.

    Backup.js is showing the old code still:

     // `noload` option is used for ext3/ext4, if it fails it might
      // `be another fs, try without
      return mount([ ...options, 'noload' ]).catch(() =>
        mount(options)
      ).then(() => ({
        path,
        unmount: once(() => execa('umount', [ '--lazy', path ]))
      }), error => {
        console.log(error)
    
        throw error
      })
    })
    


  • So you are not on the right branch or not uptodate with our repo on GitHub…

    See https://github.com/vatesfr/xo-server/blob/xfs-norecovery/src/xo-mixins/backups.js#L225

    git pull to be sure you are OK

    edit: git branch will show you your active branch.



  • @olivierlambert

    git branch shows xfs-norecovery as active.

    Since this is a testing VM, I have deleted the directory and am cloning the xfs-norecovery branch fresh.

    Currently running yarn



  • OK

    That seems to have cured the mount error and replaced it with:

    backup.scanFiles
    {
      "remote": "b39d9031-bd3d-4ca5-b513-35a8d8b3dd9b",
      "disk": "vm_delta_Backup_59b8a473-aa4c-25ef-39c7-e381331f0bed/vdi_e748211b-3959-4506-a124-594469dd34a4/20170508T103255Z_full.vhd",
      "path": "/",
      "partition": "000f0648-05/dc01-vg/root"
    }
    {
      "message": "Command failed: vgchange -ay dc01-vg
    File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000019.log) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 18 (/var/lib/xo-server/data/leveldb/MANIFEST-000017) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    ",
      "stack": "Error: Command failed: vgchange -ay dc01-vg
    File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000019.log) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 18 (/var/lib/xo-server/data/leveldb/MANIFEST-000017) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    
        at Promise.all.then.arr (/opt/xo-server/node_modules/execa/index.js:210:11)
        at tryCatcher (/opt/xo-server/node_modules/bluebird/js/release/util.js:16:23)
        at Promise._settlePromiseFromHandler (/opt/xo-server/node_modules/bluebird/js/release/promise.js:512:31)
        at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:569:18)
        at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Promise._fulfill (/opt/xo-server/node_modules/bluebird/js/release/promise.js:638:18)
        at PromiseArray._resolve (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:126:19)
        at PromiseArray._promiseFulfilled (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:144:14)
        at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:574:26)
        at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Async._drainQueue (/opt/xo-server/node_modules/bluebird/js/release/async.js:133:16)
        at Async._drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:143:10)
        at Immediate.Async.drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:17:14)",
      "code": 5,
      "killed": false,
      "stdout": "",
      "stderr": "File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000019.log) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
    File descriptor 18 (/var/lib/xo-server/data/leveldb/MANIFEST-000017) leaked on vgchange invocation. Parent PID 21356: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    ",
      "failed": true,
      "signal": null,
      "cmd": "vgchange -ay dc01-vg",
      "timedOut": false
    }
    

    More problems with LVM on my end?



  • @dlucas46 Can you reboot? I just want to be sure we don't have previous failed tasks running in background.



  • @olivierlambert

    Have rebooted.

    Same error

    backup.scanFiles
    {
      "remote": "b39d9031-bd3d-4ca5-b513-35a8d8b3dd9b",
      "disk": "vm_delta_Backup_59b8a473-aa4c-25ef-39c7-e381331f0bed/vdi_e748211b-3959-4506-a124-594469dd34a4/20170508T110035Z_delta.vhd",
      "path": "/",
      "partition": "000f0648-05/dc01-vg/root"
    }
    {
      "message": "Command failed: vgchange -ay dc01-vg
    File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000028.log) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 15 (/var/lib/xo-server/data/leveldb/MANIFEST-000027) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    ",
      "stack": "Error: Command failed: vgchange -ay dc01-vg
    File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000028.log) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 15 (/var/lib/xo-server/data/leveldb/MANIFEST-000027) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    
        at Promise.all.then.arr (/opt/xo-server/node_modules/execa/index.js:210:11)
        at tryCatcher (/opt/xo-server/node_modules/bluebird/js/release/util.js:16:23)
        at Promise._settlePromiseFromHandler (/opt/xo-server/node_modules/bluebird/js/release/promise.js:512:31)
        at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:569:18)
        at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Promise._fulfill (/opt/xo-server/node_modules/bluebird/js/release/promise.js:638:18)
        at PromiseArray._resolve (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:126:19)
        at PromiseArray._promiseFulfilled (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:144:14)
        at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:574:26)
        at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Async._drainQueue (/opt/xo-server/node_modules/bluebird/js/release/async.js:133:16)
        at Async._drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:143:10)
        at Immediate.Async.drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:17:14)",
      "code": 5,
      "killed": false,
      "stdout": "",
      "stderr": "File descriptor 12 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 14 (/var/lib/xo-server/data/leveldb/000028.log) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
    File descriptor 15 (/var/lib/xo-server/data/leveldb/MANIFEST-000027) leaked on vgchange invocation. Parent PID 1547: /usr/local/bin/node
      Volume group \"dc01-vg\" not found
      Cannot process volume group dc01-vg
    ",
      "failed": true,
      "signal": null,
      "cmd": "vgchange -ay dc01-vg",
      "timedOut": false
    }
    
    


  • We'll push the fix in XOA so you could test in a validated environment.



  • Fix integrated into XOA. Please test! 🙂



  • Hi,

    I have just tested this in my trial XOA Vm.

    Instead of not mounting the virtual drive file because of the file system error. I get the following

    backup.scanFiles
    {
      "remote": "c2eb9cc3-d3ea-4c03-90c1-d60c357be68a",
      "disk": "vm_delta_Backup_59b8a473-aa4c-25ef-39c7-e381331f0bed/vdi_e748211b-3959-4506-a124-594469dd34a4/20170508T110035Z_delta.vhd",
      "path": "/",
      "partition": "000f0648-05/dc01-vg/root"
    }
    {
      "message": "Command failed: mount --options=loop,ro --source=/dev/dc01-vg/root --target=/tmp/tmp-13797pt7gkxM1XAdy
    mount: unknown filesystem type 'xfs'
    ",
      "stack": "Error: Command failed: mount --options=loop,ro --source=/dev/dc01-vg/root --target=/tmp/tmp-13797pt7gkxM1XAdy
    mount: unknown filesystem type 'xfs'
    
        at Promise.all.then.arr (/usr/local/lib/node_modules/xo-server/node_modules/execa/index.js:210:11)
        at tryCatcher (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:16:23)
        at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:512:31)
        at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:569:18)
        at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Promise._fulfill (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:638:18)
        at PromiseArray._resolve (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise_array.js:126:19)
        at PromiseArray._promiseFulfilled (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise_array.js:144:14)
        at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:574:26)
        at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:614:10)
        at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:693:18)
        at Async._drainQueue (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:133:16)
        at Async._drainQueues (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:143:10)
        at Immediate.Async.drainQueues (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:17:14)",
      "code": 32,
      "killed": false,
      "stdout": "",
      "stderr": "mount: unknown filesystem type 'xfs'
    ",
      "failed": true,
      "signal": null,
      "cmd": "mount --options=loop,ro --source=/dev/dc01-vg/root --target=/tmp/tmp-13797pt7gkxM1XAdy",
      "timedOut": false
    }
    


  • @dlucas46 said in Delta File Restore - LVM:

    mount: unknown filesystem type 'xfs'

    Can you tell me the output of sudo lsmod | grep xfs?


Log in to reply