CTDB asymetric (non-)recovery
Nicolas Ecarnot
nicolas at ecarnot.net
Thu Jun 7 07:39:36 MDT 2012
Le 07/06/2012 14:45, ronnie sahlberg a écrit :
> What does 'ctdb scriptstatus all' show.
> Sounds like it is stuck, failing to start up the services proeprly.
>
> Also, check a 'pstree -p -a' and see what child processes ctdbd has
> during the start. It should probably be a shell script there as a
> child that is failing.
Hi Ronnie,
Below will you find the pstree -pal for node0 then for node1 (relevant
part).
Then you will find the result of ctdb scriptstatus all for node0 (the
failing one) then for node1.
* Node 0 : pstree -pal
├─ctdbd,4609 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,4654 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,4655 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,4656 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ └─ctdbd,4657 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
There is no smbd/nmbd/winbindd running (as the node has not recovered
yet, this is normal).
You'll notice there is no script child process.
* Node 1 : pstree -pal
├─ctdbd,14586 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,14634 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,17683 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ ├─ctdbd,17684 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
│ └─ctdbd,17685 --reclock=/ctdb/.ctdb.lock --nlist=/etc/ctdb/nodes
--socket=/tmp/ctdb.socket --public-addresses=/etc/ctdb/public_addresses
--public-interface=bond0 --dbdir=/var/ctdb
--dbdir-persistent=/var/ctdb/persistent
--event-script-dir=/etc/ctdb/events.d --transport=tcp -d 3
Same comment : no script.
* Node 0 : ctdb scriptstatus all
root at node0:/etc/init# ctdb scriptstatus all
18 scripts were executed last init cycle
00.ctdb Status:OK Duration:0.010 Thu Jun 7 15:26:25 2012
01.reclock Status:OK Duration:0.006 Thu Jun 7 15:26:25 2012
10.interface Status:OK Duration:0.006 Thu Jun 7 15:26:25 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:26:25 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:26:25 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:26:25 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last setup cycle
00.ctdb Status:OK Duration:0.013 Thu Jun 7 15:26:25 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:26:25 2012
10.interface Status:OK Duration:0.006 Thu Jun 7 15:26:25 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:26:25 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:26:25 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:26:25 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
startup cycle never run
18 scripts were executed last startrecovery cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:31:13 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:31:13 2012
10.interface Status:OK Duration:0.005 Thu Jun 7 15:31:13 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:31:13 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:31:13 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:31:13 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
recovered cycle never run
takeip cycle never run
releaseip cycle never run
stopped cycle never run
monitor cycle never run
status cycle never run
shutdown cycle never run
reload cycle never run
updateip cycle never run
ipreallocated cycle never run
* Node 1 : ctdb scriptstatus all
root at node1:/etc# ctdb scriptstatus all
18 scripts were executed last init cycle
00.ctdb Status:OK Duration:0.010 Thu Jun 7 15:24:58 2012
01.reclock Status:OK Duration:0.006 Thu Jun 7 15:24:58 2012
10.interface Status:OK Duration:0.007 Thu Jun 7 15:24:58 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:24:58 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:24:58 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:24:58 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last setup cycle
00.ctdb Status:OK Duration:0.008 Thu Jun 7 15:24:58 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:24:58 2012
10.interface Status:OK Duration:0.006 Thu Jun 7 15:24:58 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:24:58 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:24:58 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:24:58 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last startup cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:25:20 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:25:20 2012
10.interface Status:OK Duration:0.028 Thu Jun 7 15:25:20 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:25:20 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:25:20 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.112 Thu Jun 7 15:25:20 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last startrecovery cycle
00.ctdb Status:OK Duration:0.006 Thu Jun 7 15:33:33 2012
01.reclock Status:OK Duration:0.006 Thu Jun 7 15:33:33 2012
10.interface Status:OK Duration:0.006 Thu Jun 7 15:33:33 2012
11.routing Status:OK Duration:0.006 Thu Jun 7 15:33:33 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.007 Thu Jun 7 15:33:33 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:33:33 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last recovered cycle
00.ctdb Status:OK Duration:0.006 Thu Jun 7 15:25:55 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:25:55 2012
10.interface Status:OK Duration:0.005 Thu Jun 7 15:25:55 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:25:55 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:25:55 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:25:55 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last takeip cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:25:54 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:25:54 2012
10.interface Status:OK Duration:0.017 Thu Jun 7 15:25:54 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:25:54 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:25:54 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.013 Thu Jun 7 15:25:54 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
18 scripts were executed last releaseip cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:30:53 2012
01.reclock Status:OK Duration:0.006 Thu Jun 7 15:30:53 2012
10.interface Status:OK Duration:0.043 Thu Jun 7 15:30:53 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:30:53 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.007 Thu Jun 7 15:30:53 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.013 Thu Jun 7 15:30:53 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
stopped cycle never run
18 scripts were executed last monitor cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:28:44 2012
01.reclock Status:OK Duration:0.014 Thu Jun 7 15:28:44 2012
10.interface Status:OK Duration:0.021 Thu Jun 7 15:28:44 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:28:44 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:28:44 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.023 Thu Jun 7 15:28:44 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
status cycle never run
shutdown cycle never run
reload cycle never run
updateip cycle never run
18 scripts were executed last ipreallocated cycle
00.ctdb Status:OK Duration:0.007 Thu Jun 7 15:26:05 2012
01.reclock Status:OK Duration:0.005 Thu Jun 7 15:26:05 2012
10.interface Status:OK Duration:0.006 Thu Jun 7 15:26:05 2012
11.routing Status:OK Duration:0.005 Thu Jun 7 15:26:05 2012
11.natgw Status:DISABLED
13.per_ip_routing Status:OK Duration:0.006 Thu Jun 7 15:26:05 2012
20.multipathd Status:DISABLED
31.clamd Status:DISABLED
40.fs_use Status:DISABLED
40.vsftpd Status:DISABLED
41.httpd Status:DISABLED
50.samba Status:OK Duration:0.007 Thu Jun 7 15:26:05 2012
50.samba.dpkg-old Status:DISABLED
60.ganesha Status:DISABLED
60.nfs Status:DISABLED
62.cnfs Status:DISABLED
70.iscsi Status:DISABLED
91.lvs Status:DISABLED
--
Nicolas Ecarnot
More information about the samba-technical
mailing list