File Level Restore Issue



  • And just to make things even stranger, now ALLLLLL of them work since I took the one new Delta backup.



  • @BRRABill said:

    Upgrading to 16.10 did indeed fix that issue.

    However, I am now getting errors on other VMs

    "message": "Command failed: vgchange -ay graylog-vg
    File descriptor 11 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 862: /usr/local/bin/node
    File descriptor 12 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 862: /usr/local/bin/node
    File descriptor 13 (/var/lib/xo-server/data/leveldb/000086.log) leaked on vgchange invocation. Parent PID 862: /usr/local/bin/node
    File descriptor 15 (/var/lib/xo-server/data/leveldb/MANIFEST-000084) leaked on vgchange invocation. Parent PID 862: /usr/local/bin/node
    Volume group "graylog-vg" not found
    Cannot process volume group graylog-vg

    FWIW, I'm seeing similar errors.



  • Also seeing this --

    Feb 01 14:13:50 xo-5 xo-server[15394]: { Error: Command failed: mount --options=loop,ro --source=/dev/centos/root --target=/tmp/tmp-15394LhJ5uBDdCp5K
    Feb 01 14:13:50 xo-5 xo-server[15394]: mount: cannot mount /dev/loop1 read-only
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise.all.then.arr (/opt/xo-server/node_modules/execa/index.js:201:11)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at tryCatcher (/opt/xo-server/node_modules/bluebird/js/release/util.js:16:23)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromiseFromHandler (/opt/xo-server/node_modules/bluebird/js/release/promise.js:510:31)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:567:18)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:612:10)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:691:18)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._fulfill (/opt/xo-server/node_modules/bluebird/js/release/promise.js:636:18)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at PromiseArray._resolve (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:125:19)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at PromiseArray._promiseFulfilled (/opt/xo-server/node_modules/bluebird/js/release/promise_array.js:143:14)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromise (/opt/xo-server/node_modules/bluebird/js/release/promise.js:572:26)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromise0 (/opt/xo-server/node_modules/bluebird/js/release/promise.js:612:10)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Promise._settlePromises (/opt/xo-server/node_modules/bluebird/js/release/promise.js:691:18)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Async._drainQueue (/opt/xo-server/node_modules/bluebird/js/release/async.js:133:16)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Async._drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:143:10)
    Feb 01 14:13:50 xo-5 xo-server[15394]:     at Immediate.Async.drainQueues (/opt/xo-server/node_modules/bluebird/js/release/async.js:17:14)
    Feb 01 14:13:50 xo-5 xo-server[15394]:   code: 32,
    Feb 01 14:13:50 xo-5 xo-server[15394]:   killed: false,
    Feb 01 14:13:50 xo-5 xo-server[15394]:   stdout: '',
    Feb 01 14:13:50 xo-5 xo-server[15394]:   stderr: 'mount: cannot mount /dev/loop1 read-only\n',
    Feb 01 14:13:50 xo-5 xo-server[15394]:   failed: true,
    Feb 01 14:13:50 xo-5 xo-server[15394]:   signal: null,
    Feb 01 14:13:50 xo-5 xo-server[15394]:   cmd: 'mount --options=loop,ro --source=/dev/centos/root --target=/tmp/tmp-15394LhJ5uBDdCp5K',
    Feb 01 14:13:50 xo-5 xo-server[15394]:   timedOut: false }
    Feb 01 14:13:51 xo-5 xo-server[15394]: Wed, 01 Feb 2017 20:13:51 GMT xo:perf blocked for 23ms
    Feb 01 14:13:51 xo-5 xo-server[15394]: Wed, 01 Feb 2017 20:13:51 GMT xo:api dan@mydomain.com | backup.scanFiles(...) [1s] =!> Error: Command failed: mount --options=loop,ro --source=/dev/centos/root --target=/tmp/tmp-15394LhJ5uBDdCp5K
    Feb 01 14:13:51 xo-5 xo-server[15394]: mount: cannot mount /dev/loop1 read-only
    


  • Anyone have a valid XOA license to test against the same backup repository? Mine has expired.



  • @Danp said:

    Anyone have a valid XOA license to test against the same backup repository? Mine has expired.

    I'm sure @olivierlambert will renew it for a few days for you.

    All seems to be working fine here.



  • @Danp I extended your trial 😉



  • @olivierlambert Thanks!



  • Tested in XOA. Still getting errors. This is what I get when I select the disk --

    Feb  1 17:18:01 xoa xo-server[5596]: Wed, 01 Feb 2017 22:18:01 GMT xo:api admin@admin.net | resourceSet.getAll(...) [7ms] ==> array
    Feb  1 17:18:05 xoa systemd[1]: Starting LVM2 PV scan on device 7:0...
    Feb  1 17:18:05 xoa systemd[1]: Started LVM2 PV scan on device 7:0.
    Feb  1 17:18:05 xoa systemd-udevd[5675]: failed to execute '/bin/systemd-run' '/bin/systemd-run /sbin/lvm pvscan --cache 7:0': No such file or directory
    Feb  1 17:18:05 xoa systemd[1]: Stopping LVM2 PV scan on device 7:0...
    Feb  1 17:18:05 xoa systemd[1]: Stopped LVM2 PV scan on device 7:0.
    Feb  1 17:18:05 xoa xo-server[5596]: Wed, 01 Feb 2017 22:18:05 GMT xo:perf blocked for 29ms
    Feb  1 17:18:05 xoa xo-server[5596]: [Warn] Possibly unhandled rejection: Error: Command failed: pvs --noheading --nosuffix --nameprefixes --unbuffered --units b -o vg_name /dev/loop0
    Feb  1 17:18:05 xoa xo-server[5596]: File descriptor 12 (/var/lib/xo-server/data/leveldb/000019.log) leaked on pvs invocation. Parent PID 5596: node
    Feb  1 17:18:05 xoa xo-server[5596]: File descriptor 13 (/var/lib/xo-server/data/leveldb/LOG) leaked on pvs invocation. Parent PID 5596: node
    Feb  1 17:18:05 xoa xo-server[5596]: File descriptor 14 (/var/lib/xo-server/data/leveldb/MANIFEST-000017) leaked on pvs invocation. Parent PID 5596: node
    Feb  1 17:18:05 xoa xo-server[5596]: File descriptor 16 (/var/lib/xo-server/data/leveldb/LOCK) leaked on pvs invocation. Parent PID 5596: node
    Feb  1 17:18:05 xoa xo-server[5596]: Failed to find device "/dev/loop0"
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise.all.then.arr (/usr/local/lib/node_modules/xo-server/node_modules/execa/index.js:201:11)
    Feb  1 17:18:05 xoa xo-server[5596]: at tryCatcher (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:16:23)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:510:31)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:567:18)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:612:10)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:691:18)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._fulfill (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:636:18)
    Feb  1 17:18:05 xoa xo-server[5596]: at PromiseArray._resolve (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise_array.js:125:19)
    Feb  1 17:18:05 xoa xo-server[5596]: at PromiseArray._promiseFulfilled (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise_array.js:143:14)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:572:26)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:612:10)
    Feb  1 17:18:05 xoa xo-server[5596]: at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:691:18)
    Feb  1 17:18:05 xoa xo-server[5596]: at Async._drainQueue (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:138:16)
    Feb  1 17:18:05 xoa xo-server[5596]: at Async._drainQueues (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:148:10)
    Feb  1 17:18:05 xoa xo-server[5596]: at Immediate.Async.drainQueues (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:17:14)
    Feb  1 17:18:05 xoa xo-server[5596]: Wed, 01 Feb 2017 22:18:05 GMT xo:api admin@admin.net | backup.scanDisk(...) [195ms] ==> object
    


  • Are you sure you have a recent XOA?



  • @olivierlambert Yes, just updated today using the built-in upgrade process.



  • No I mean a recent Appliance, not if you are updated.



  • I believe I downloaded it in late December.



  • @olivierlambert Is the XOA from December timeframe supposed to work or should I redownload?



  • It should be the right XOA yes.



  • So, as the log indicated, there is no systemd-run in /bin. Appears to actually reside in /usr/bin on my copy of XOA.

    Thoughts?



  • Never saw this message before, I don't understand. I suppose you have rebooted your XOA?



  • @Danp said:

    So, as the log indicated, there is no systemd-run in /bin. Appears to actually reside in /usr/bin on my copy of XOA.

    Thoughts?

    It is not on my compiled version, either.



  • @BRRABill said:

    @Danp said:

    So, as the log indicated, there is no systemd-run in /bin. Appears to actually reside in /usr/bin on my copy of XOA.

    Thoughts?

    It is not on my compiled version, either.

    Not that that maters, just adding info. 🙂



  • Thanks for the feedback guys!

    We are investigating, it's strange that it works for others 😕



  • @olivierlambert said:

    Thanks for the feedback guys!

    We are investigating, it's strange that it works for others 😕

    What's strange is that it didn't work for me until I ran a backup on the upgraded 16.10 system.

    But I was getting a different error than @Danp

    @Danp ... have you tried taking a new, fresh backup with the XOA and seeing if that worked?


Log in to reply