Snapshots not being deleted after delta



  • Just tested on a freshly downloaded XOA, got this in the log at the end of the backup process:

    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:xapi Deleting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:xapi Deleting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:api admin@admin.net | vm.rollingDeltaBackup(...) [20s] ==> string
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:api admin@admin.net | job.runSequence(...) [20s] ==> boolean
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: event.from(...) [76ms] ==> object
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: VDI.destroy(...) [149ms] ==> string
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: event.from(...) [124ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [374ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [76ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [86ms] ==> object
    
    

    You can see the Deleting order and the VDI.destroy command. So you have a problem somewhere in your setup 😞



  • @olivierlambert said:

    Just tested on a freshly downloaded XOA, got this in the log at the end of the backup process:

    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:xapi Deleting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:xapi Deleting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:api admin@admin.net | vm.rollingDeltaBackup(...) [20s] ==> string
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xo:api admin@admin.net | job.runSequence(...) [20s] ==> boolean
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: event.from(...) [76ms] ==> object
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: VDI.destroy(...) [149ms] ==> string
    Mar 16 14:02:26 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:26 GMT xen-api root@192.168.100.1: event.from(...) [124ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [374ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [76ms] ==> object
    Mar 16 14:02:27 xoa xo-server[1308]: Wed, 16 Mar 2016 18:02:27 GMT xen-api root@192.168.100.1: event.from(...) [86ms] ==> object
    
    

    You can see the Deleting order and the VDI.destroy command. So you have a problem somewhere in your setup 😞

    Maybe in XOA, but not in source version.
    Here's what i've done:

    1. Created a new machine with fresh install of Debian 8.3 minimal
    2. Install N and XenOrchestra (manually, using Github).
    3. Connected to one host/Pool with a fresh install of Dundee (TP2) running 4 machines.
    4. Created a remote store in Xen Orchestra using NFS
    5. Created a Delta Backup with only one machine (running) and depth=2.
      Then, I tried 4 'one shot' backups. No VDI delete (snapshot). I've setup this same backup to run every 3 minutes (backup time is 1 minute). Same problem: no VDI delete, no errors. All fresh installs, no copy of old config or files.
    root@XENORCHESTRA:~# tail -f /opt/xo/xo-server/*.log | grep VDI
    Wed, 16 Mar 2016 19:15:03 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 19:15:04 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [420ms] ==> string
    Wed, 16 Mar 2016 19:15:04 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:15:04 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Wed, 16 Mar 2016 19:15:59 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:18:03 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 19:18:04 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [428ms] ==> string
    Wed, 16 Mar 2016 19:18:04 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:18:04 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Wed, 16 Mar 2016 19:18:59 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:21:03 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 19:21:04 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [426ms] ==> string
    Wed, 16 Mar 2016 19:21:04 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:21:04 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Wed, 16 Mar 2016 19:21:58 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:24:03 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 19:24:04 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [443ms] ==> string
    Wed, 16 Mar 2016 19:24:04 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 19:24:04 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    


  • Can you show me what's the result on the remote? Do you have delta files on only full VHDs?



  • [root@paloma vm_delta_VPN_ecfb057a-65ef-240f-174e-f57572a31921]# ls -lasht && ls -lasht vdi_2c70af85-8105-4ec4-942f-21b8312ccdd1
    total 44K
    4.0K drwxr-xr-x 3 root root 4.0K Jun 11 05:13 .
     16K -rw-r--r-- 1 root root  13K Jun 11 05:13 20160316T194626Z_VPN Novo.json
    4.0K drwxr-xr-x 2 root root 4.0K Jun 11 05:13 vdi_2c70af85-8105-4ec4-942f-21b8312ccdd1
     16K -rw-r--r-- 1 root root  13K Jun 11 05:13 20160316T194620Z_VPN Novo.json
    4.0K drwxrwxrwx 4 root root 4.0K Jun 11 05:07 ..
    total 10G
    4.0K drwxr-xr-x 3 root root 4.0K Jun 11 05:13 ..
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:13 20160316T194501Z_full.vhd
    4.0K drwxr-xr-x 2 root root 4.0K Jun 11 05:13 .
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:13 20160316T194445Z_full.vhd
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:10 .nfs000000000c6400050000000c
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:10 .nfs000000000c6400060000000d
    [root@paloma vm_delta_VPN_ecfb057a-65ef-240f-174e-f57572a31921]#
    
    


  • They are keeping 2 'full' and creating '.nfs******a', 'b', 'c,' etc..



  • There is definitively a problem with your source install. Which are your npm and node version?

    edit: or a problem with Dundee Beta 2, I am trying to make tests on my side too.



  • @olivierlambert said:

    There is definitively a problem with your source install. Which are your npm and node version?

    edit: or a problem with Dundee Beta 2, I am trying to make tests on my side too.

    root@XENORCHESTRA:~# npm -v
    3.7.3
    root@XENORCHESTRA:~# node -v
    v5.8.0
    

    could be.....but I have the same problem on XS 6.5. I didn't tried with this new freshinstall, but is the same problem: no error or anything related to VDI delete on logs.



  • Try on Node LTS version please (as asked in the documentation). Don't forget to rebuild.



  • I'll try tomorow. Instructions on XO Github page says to use stable: https://github.com/vatesfr/xo/blob/master/doc/installation/manual_installation.md



  • You are not on the right documentation: https://xen-orchestra.com/docs/from_the_sources.html



  • Still same proble, even using node LTS

    root@XENORCHESTRA:/opt/xo/xo-server# tail -f *.log | grep VDI
    Wed, 16 Mar 2016 20:25:10 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 20:25:25 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 20:25:25 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [392ms] ==> string
    Wed, 16 Mar 2016 20:25:25 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 20:25:25 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Wed, 16 Mar 2016 20:26:17 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 20:27:22 GMT xen-api root@10.107.205.107: VDI.snapshot(...) [4s] ==> string
    Wed, 16 Mar 2016 20:27:23 GMT xen-api root@10.107.205.107: VDI.set_name_label(...) [470ms] ==> string
    Wed, 16 Mar 2016 20:27:23 GMT xo:xapi task created: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    Wed, 16 Mar 2016 20:27:23 GMT xo:xapi exporting VDI XO_DELTA_BASE_VDI_SNAPSHOT
    Wed, 16 Mar 2016 20:28:13 GMT xo:xapi task destroyed: VDI Export (XO_DELTA_BASE_VDI_SNAPSHOT)
    ^C
    root@XENORCHESTRA:/opt/xo/xo-server# node -v
    v4.4.0
    
    

    Looks liek all files are getting the same size of full backup....

    [root@paloma vm_delta_VPN_ecfb057a-65ef-240f-174e-f57572a31921]# ls -lasht && ls -lasht vdi_2c70af85-8105-4ec4-942f-21b8312ccdd1
    total 44K
    4.0K drwxr-xr-x 3 root root 4.0K Jun 11 05:55 .
    4.0K drwxr-xr-x 2 root root 4.0K Jun 11 05:55 vdi_2c70af85-8105-4ec4-942f-21b8312ccdd1
     16K -rw-r--r-- 1 root root  13K Jun 11 05:55 20160316T202813Z_VPN Novo.json
     16K -rw-r--r-- 1 root root  13K Jun 11 05:53 20160316T202617Z_VPN Novo.json
    4.0K drwxrwxrwx 4 root root 4.0K Jun 11 05:43 ..
    total 10G
    4.0K drwxr-xr-x 3 root root 4.0K Jun 11 05:55 ..
    4.0K drwxr-xr-x 2 root root 4.0K Jun 11 05:55 .
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:55 20160316T202719Z_full.vhd
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:53 20160316T202521Z_full.vhd
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:52 .nfs000000000c64000300000002
    2.5G -rw-r--r-- 1 root root 2.5G Jun 11 05:49 .nfs000000000c64000700000001
    
    

    For these last tests, i've deleted node_modules and installed && build again (after chaning to node LTS), also rebooted server and releted old backups on remote store. All backups are tried manually using 'one shot' backup in this last tests.



  • So I have no idea why it doesn't work on your side.

    We changed at little bit the delta algorithm for 4.15 which is coming soon (Friday?). Retry there.



  • @olivierlambert said:

    So I have no idea why it doesn't work on your side.

    We changed at little bit the delta algorithm for 4.15 which is coming soon (Friday?). Retry there.

    Would you like to take a look? I can give you access.
    This machine (XO) is not in production, just some VM's that are on XS.



  • Sadly, I don't have enough time to go that far for community support. Just retry when the 4.15 is out with the new algorithm, and keep us posted!



  • anybody else reading this topic is having the same problem? Except for "webguyz", that alread contacted me by chat with same problem.



  • Sorry... Not me.



  • Some news....

    Dundee TP2 = Didn't work
    6.5 update 1021 = Didn't work
    6.5 update 1024 = Work



  • 4.15 is out, feel free to test!



  • @olivierlambert said:

    4.15 is out, feel free to test!

    Can't checkout node_modules from xo-web anymore?

    root@XENORCHESTRA:/opt/xo/xo-web# git checkout node_modules
    error: pathspec 'node_modules' did not match any file(s) known to git.
    


  • Nothing more in node_modules, you can continue without bothering this step.


Log in to reply