Problems with CTDB and GlusterFS

[Michael Adam]

On 2016-06-21 at 00:23 +1000, Amitay Isaacs wrote:
> Hi,
> 
> On Mon, Jun 20, 2016 at 11:41 PM, Sebastian Stellingwerff <
> sebastian at otherstuff.nl> wrote:
> 
> > Hi,
> >
> > I'm setting up a Samba+CTDB+GlusterFS setup. GlusterFS seems to work
> > fine and we have it in use in production for some time without problems
> > it seems. However when setting up CTDB with this, one node just keeps
> > complaining about:
> >
> > 2016/06/20 13:32:29.606345 [recoverd: 7079]: Taking out recovery lock
> > from recovery daemon
> > 2016/06/20 13:32:29.606360 [recoverd: 7079]: Take the recovery lock
> > 2016/06/20 13:32:29.607962 [recoverd: 7079]: ctdb_recovery_lock: Failed
> > to get recovery lock on '/mnt/gv0/ctdblock'
> > 2016/06/20 13:32:29.608002 [recoverd: 7079]: Unable to get recovery lock
> > - retrying recovery
> >
> > It seems to be the node that's started last. I've searched around the
> > internet, but no luck. An old thread suggested enabling direct-io, but
> > this is the default these days and giving it as an explicit option
> > doesn't solve the problem.
> >
> > I've also tested locking with ping_pong which does not give me any
> > trouble, al results are as expected.
> >
> > Anyone here who can help me debug this?
>
> From the log messages it appears you are either running ctdb 4.2.x or
> 2.5.x.  The code around recovery lock has changed quite a bit in the recent
> releases.  I would recommend testing with the latest stable release of ctdb
> 4.4.x.

From my experience I can tell you that ctdb 4.2 and 2.5 generally
work quite well with gluster.

What version of gluster are you running?

Can you share longer parts of the ctdb logs?

Can you describe your setup?
E.g. how many nics do you have on the nodes,
and for what purposes (gluster / ctdb / samba..)
What does your gluster setup and ctdb config look like?

Etc.

Cheers - Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160620/7eaa62f0/signature.sig>