Error Continous Replications 5.18.0



  • Well tried again with backup legacy

    none of them has worked.

    Production Pool. (inside the same SR)
    at first I received a time out error, changed the value to 120 seconds and now the error is " VDI_IO_ERROR(Device I/O errors)"
    as much I retried

    Development pool (inside the same SR)
    I'm receiving a "interrupted" error every time I try.



  • Something that fails almost everytime would have been detected if reproducible 😕

    I suspect a XenServer issue, please check your XS logs.



  • but the backup-legacy also fails with XCP-ng 7.4

    where can I see this logs ?



  • XenServer and XCP-ng are 99.99% similar.

    edit: in /var/log/xensource.log and /var/log/SMlog



  • @olivierlambert said in Error Continous Replications 5.18.0:

    /var/log/xensource.log

    thanks.

    do you want me to copy & paste it here ?



  • I'm pretty busy today, and the log could be really big 😕 Try to pinpoint evens that's time related to errors you saw in XO.



  • yes, you're right.
    well I think I catch something

    this is a develop server while doing a backup-legacy, when the backup stoped at 42% more or less

    Apr  6 15:26:34 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] Processing warnings
    Apr  6 15:26:34 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] Done with warnings
    Apr  6 15:26:34 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] The node we think is the master is still alive and marked as master; this is OK
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] session.login_with_password D:1f68466fceda failed with exception Server_error(HOST_IS_SLAVE, [ 192.168.222.230 ])
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] Raised Server_error(HOST_IS_SLAVE, [ 192.168.222.230 ])
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 1/8 xapi @ XCP1 Raised at file ocaml/xapi/xapi_session.ml, line 383
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 2/8 xapi @ XCP1 Called from file ocaml/xapi/xapi_session.ml, line 39
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 3/8 xapi @ XCP1 Called from file ocaml/xapi/xapi_session.ml, line 39
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 4/8 xapi @ XCP1 Called from file ocaml/xapi/server_helpers.ml, line 69
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 5/8 xapi @ XCP1 Called from file ocaml/xapi/server_helpers.ml, line 91
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 6/8 xapi @ XCP1 Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 22
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 7/8 xapi @ XCP1 Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 26
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace] 8/8 xapi @ XCP1 Called from file lib/backtrace.ml, line 177
    Apr  6 15:26:42 XCP1 xapi: [error|XCP1|4262 INET :::80|dispatch:session.login_with_password D:c8f25c93214a|backtrace]
    Apr  6 15:26:47 XCP1 xapi: [ info|XCP1|4263 UNIX /var/lib/xcp/xapi||cli] xe host-ha-xapi-healthcheck username=root password=(omitted)
    Apr  6 15:26:47 XCP1 xapi: [debug|XCP1|4263 UNIX /var/lib/xcp/xapi|session.slave_local_login_with_password D:822808a40866|xapi] Add session to local storage
    Apr  6 15:26:49 XCP1 xapi: [debug|XCP1|4261 ||xmlrpc_client] stunnel pid: 18843 (cached = true) returned stunnel to cache
    Apr  6 15:26:49 XCP1 xapi: [debug|XCP1|4264 ||mscgen] xapi=>xapi [label="event.from"];
    Apr  6 15:26:49 XCP1 xapi: [debug|XCP1|4264 ||xmlrpc_client] stunnel pid: 14677 (cached = true) connected to 192.168.222.230:443
    Apr  6 15:26:49 XCP1 xapi: [debug|XCP1|4264 ||xmlrpc_client] with_recorded_stunnelpid task_opt=None s_pid=14677
    Apr  6 15:26:54 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] Liveset: online 11d55d91-2e0d-4008-803d-42db55ca4cf7 [ L  A ]; 55185571-6dfa-429a-9415-2dacb9ff1f3a [*L  A ]; a16383be-39c6-4bba-8709-75653eacd759 [ LM A ];
    Apr  6 15:26:54 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] Processing warnings
    Apr  6 15:26:54 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] Done with warnings
    Apr  6 15:26:54 XCP1 xapi: [debug|XCP1|6 ha_monitor|HA monitor D:bc6ea1becaa8|xapi_ha] The node we think is the master is still alive and marked as master; this is OK
    

    copy and paste on another editor, big screen is more legible.

    and now I can see this on tasks
    0_1523017912310_Selecció_054.png



  • It seems you have issue with HA, can you disable it?



  • yes, of course
    on develop pool
    xe pool-ha-disable

    tried again the backup Continuous replication backup-legacy.

    received the same error "interrupted"

    the log:

    Apr  6 15:56:17 XCP1 xapi: [debug|XCP1|4367 INET :::80|handler:http/rrd_updates D:0b2544c2a79f|xmlrpc_client] stunnel pid: 16092 (cached = true) connected to 192.168.222.230:443
    Apr  6 15:56:17 XCP1 xapi: [debug|XCP1|4367 INET :::80|handler:http/rrd_updates D:0b2544c2a79f|xmlrpc_client] with_recorded_stunnelpid task_opt=None s_pid=16092
    Apr  6 15:56:17 XCP1 xapi: [debug|XCP1|4367 INET :::80|handler:http/rrd_updates D:0b2544c2a79f|xmlrpc_client] stunnel pid: 16092 (cached = true) returned stunnel to cache
    Apr  6 15:56:17 XCP1 xapi: [debug|XCP1|4367 INET :::80|Get RRD updates. D:df56aef76915|xapi] hand_over_connection GET /rrd_updates to /var/lib/xcp/xcp-rrdd.forwarded
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] session.login_with_password D:bd4561f87253 failed with exception Server_error(HOST_IS_SLAVE, [ 192.168.222.230 ])
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] Raised Server_error(HOST_IS_SLAVE, [ 192.168.222.230 ])
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 1/8 xapi @ XCP1 Raised at file ocaml/xapi/xapi_session.ml, line 383
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 2/8 xapi @ XCP1 Called from file ocaml/xapi/xapi_session.ml, line 39
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 3/8 xapi @ XCP1 Called from file ocaml/xapi/xapi_session.ml, line 39
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 4/8 xapi @ XCP1 Called from file ocaml/xapi/server_helpers.ml, line 69
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 5/8 xapi @ XCP1 Called from file ocaml/xapi/server_helpers.ml, line 91
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 6/8 xapi @ XCP1 Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 22
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 7/8 xapi @ XCP1 Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 26
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace] 8/8 xapi @ XCP1 Called from file lib/backtrace.ml, line 177
    Apr  6 15:56:32 XCP1 xapi: [error|XCP1|4368 INET :::80|dispatch:session.login_with_password D:4f8aeaec976e|backtrace]
    Apr  6 15:56:35 XCP1 xapi: [debug|XCP1|63 heartbeat|Heartbeat D:b70ceab1b744|mscgen] xapi=>xapi [label="host.tickle_heartbeat"];
    Apr  6 15:56:35 XCP1 xapi: [debug|XCP1|63 heartbeat|Heartbeat D:b70ceab1b744|stunnel] stunnel start
    Apr  6 15:56:35 XCP1 xapi: [debug|XCP1|63 heartbeat|Heartbeat D:b70ceab1b744|xmlrpc_client] stunnel pid: 29155 (cached = false) connected to 192.168.222.230:443
    Apr  6 15:56:35 XCP1 xapi: [debug|XCP1|63 heartbeat|Heartbeat D:b70ceab1b744|xmlrpc_client] with_recorded_stunnelpid task_opt=None s_pid=29155
    


  • Please use three backtick around your text, otherwise it's hard to read it



  • Sorry. Didn't know that. Corrected



  • Can you double check that you connect to this pool with only one server in XOA?



  • @olivierlambert hi. What do you mean? Only 1 XO? At this moment I have 2 XO (one created today i began with problems 3 days ago when first installed) in each pool. Also I connect with XenCenter.

    Or do you mean in Settgins->servers ?
    in this option, I have 2 servers... and I have 3 in the pool, gonna add this one and check.
    or should only be 1 server connected ?



  • No I mean, in the "Settings/Server" view. You should have only 1 server added for one pool.



  • @olivierlambert thanks for your patience, this is what I have now there , the three servers of the pool. its only needed one here ? which one ? 0_1523031460864_Selección_012.jpg



  • You need to ONLY have the pool master. Remove the others and restart xo-server.



  • ok I removed all of them, only master remained. restarted EVERYTHING.
    tested again, and de continuous replication fails.
    in one specific vm, i can see that exporting is at 100% and importing stops at 42%, then it stops wit error. this time is VDI_IO_ERROR(Device I/O errors)



  • well, may be its a problem on my side, I will keep trying.
    If I find it I will tell you.
    Thanks for your time !



  • @bsastre Have you tried this where both hosts are running the same version of XS?



  • hi @Danp yes, I've done a lot of combinations.

    nevermind, maybe in summer I will try to reinstall everything from scratch and try again.

    thanks again for your time and effort.


Log in to reply