CTDB and Glusterfs setup

Morten Bøhmer Morten.Bohmer at pilaro.no
Mon Oct 8 01:12:46 MDT 2012


Hi all

I am working on a ctdb setup and would like to use Glusterfs for a shared volume for ctdb.

I have setup a simple replicated volume with glusterfs:

gluster volume create lock replicate 2 lock 172.16.0.1:/mnt/gluster/lock 172.16.0.2:/mnt/gluster/lock

I have started this volume and can mount it successfully on /mnt/lock

I have configured files for ctdb;

[root at gluster1 lock]# ls -la
total 48
drwxr-xr-x  2 root root 4096 Oct  7 23:09 .
drwxr-xr-x. 4 root root 4096 Oct  7 00:21 ..
-rw-r--r--  1 root root  165 Oct  7 23:09 ctdb
-rw-r--r--  1 root root   22 Oct  7 23:09 nodes
-rw-r--r--  1 root root   42 Oct  7 23:09 public_addresses
-rw-r--r--  1 root root  417 Oct  7 23:09 smb.conf
-rw-------  1 root root    3 Oct  7 23:09 test.dat

And tried to run the ping_pong test software:

[root at gluster1 lock]# /tmp/ping_pong test.dat 1
  2934 locks/sec

When I run ping_pong on node 2 while node 1 is  running it, the systems halts and  it is time for a reboot.

Secondly I cannot  get ctdb to start it complains about being unable to lock the files in the volume, even though the files are fully accessible from both nodes by  using common tools like vi,touch,less and so on:

2012/10/08 08:59:45.072196 [ 8399]: Starting CTDBD as pid : 8399
2012/10/08 08:59:45.072640 [ 8399]: Unable to set scheduler to SCHED_FIFO (Operation not permitted)
2012/10/08 08:59:45.207976 [ 8399]: Freeze priority 1
2012/10/08 08:59:45.208166 [ 8399]: Freeze priority 2
2012/10/08 08:59:45.208286 [ 8399]: Freeze priority 3
2012/10/08 08:59:48.212279 [recoverd: 8451]: Taking out recovery lock from recovery daemon
2012/10/08 08:59:48.212553 [recoverd: 8451]: Take the recovery lock
2012/10/08 08:59:48.212731 [recoverd: 8451]: ctdb_recovery_lock: Unable to open '/mnt/lock/lockfile' --nlist='/mnt/lock/nodes' --public-addresses='/mnt/lock/public_addresses' - (No such file or directory)
2012/10/08 08:59:48.212850 [recoverd: 8451]: Unable to get recovery lock - aborting recovery
2012/10/08 08:59:49.213604 [recoverd: 8451]: Taking out recovery lock from recovery daemon
2012/10/08 08:59:49.213823 [recoverd: 8451]: Take the recovery lock
2012/10/08 08:59:49.213975 [recoverd: 8451]: ctdb_recovery_lock: Unable to open '/mnt/lock/lockfile' --nlist='/mnt/lock/nodes' --public-addresses='/mnt/lock/public_addresses' - (No such file or directory)
2012/10/08 08:59:49.214095 [recoverd: 8451]: Unable to get recovery lock - aborting recovery
2012/10/08 08:59:50.214874 [recoverd: 8451]: Taking out recovery lock from recovery daemon
2012/10/08 08:59:50.215338 [recoverd: 8451]: Take the recovery lock
2012/10/08 08:59:50.215616 [recoverd: 8451]: ctdb_recovery_lock: Unable to open '/mnt/lock/lockfile' --nlist='/mnt/lock/nodes' --public-addresses='/mnt/lock/public_addresses' - (No such file or directory)
2012/10/08 08:59:50.215727 [recoverd: 8451]: Unable to get recovery lock - aborting recovery
2012/10/08 08:59:51.216524 [recoverd: 8451]: Taking out recovery lock from recovery daemon
2012/10/08 08:59:51.216749 [recoverd: 8451]: Take the recovery lock
2012/10/08 08:59:51.216906 [recoverd: 8451]: ctdb_recovery_lock: Unable to open '/mnt/lock/lockfile' --nlist='/mnt/lock/nodes' --public-addresses='/mnt/lock/public_addresses' - (No such file or directory)
2012/10/08 08:59:51.217030 [recoverd: 8451]: Unable to get recovery lock - aborting recovery
2012/10/08 08:59:52.217836 [ 8399]: Banning this node for 300 seconds
2012/10/08 08:59:52.218095 [recoverd: 8451]: Taking out recovery lock from recovery daemon
2012/10/08 08:59:52.218228 [recoverd: 8451]: Take the recovery lock
2012/10/08 08:59:52.218359 [recoverd: 8451]: ctdb_recovery_lock: Unable to open '/mnt/lock/lockfile' --nlist='/mnt/lock/nodes' --public-addresses='/mnt/lock/public_addresses' - (No such file or directory)
2012/10/08 08:59:52.218519 [recoverd: 8451]: Unable to get recovery lock - aborting recovery


I have done tests with both FC17+stock Glusterfs and with 3.3 version from RPM. Anyone got a clue of how to get this up and running ?


Morten


More information about the samba-technical mailing list