Split brain with Gluster (was: [PATCH] CTDB event script error (and unusual) path fixes (bug 12180))

Ira Cooper ira at wakeful.net
Tue Sep 6 13:03:17 UTC 2016


On Mon, Sep 5, 2016 at 11:38 PM, Martin Schwenke <martin at meltin.net> wrote:
> On Thu, 1 Sep 2016 05:53:19 +0000, 趙 良基 <hijk123452001 at hotmail.com>
> wrote:
>
>> I use gluster replica 3 volume as CTDB recovery lockfile path,but when split brain happend (split to 2 nodes and 1 node) the 1 node also can open the lockfile so that 1 node can takeover all IPs.
>>
>> 2016/08/31 23:49:22.296820 [set_recmode: 5604]: ctdb_recovery_lock:fd=5 - (No such file or directory)
>> 2016/08/31 23:52:19.963449 [recoverd:26006]: ctdb_recovery_lock:fd=37 - (Cannot assign requested address)
>>
>> Volume Name: IFTSwiftShareVol
>> Type: Replicate
>> Volume ID: 2501be2f-90be-477f-947a-2832198c982c
>> Status: Started
>> Number of Bricks: 1 x 3 = 3
>> Transport-type: tcp
>> Bricks:
>> Brick1: CS174119:/export/IFT_lvol_LujgjV35tD/fs
>> Brick2: CS7dc2b:/export/IFT_lvol_RhVVs85UEk/fs
>> Brick3: CS81876:/export/IFT_lvol_RgEJJyCnZs/fs
>> Options Reconfigured:
>> cluster.quorum-type: auto
>> cluster.server-quorum-type: server
>> nfs.disable: on
>> performance.readdir-ahead: on
>> cluster.server-quorum-ratio: 51%
>
> It sounds like lock coherence is not working on your cluster filesystem,
> so it can not support the CTDB recovery lock. This is probably a
> Gluster-specific issue.  Please see:
>
>   https://wiki.samba.org/index.php/Ping_pong
>
> for information about how to test your filesystem's lock coherence.
>
> You also have not mentioned which version of CTDB you are trying to use.
>
> By the way, you should not reply to existing thread to try to get help
> with a new issue.  You should start a new thread...  :-)

A modern version of Gluster should pass ping_pong, and other things
should stop this bug.

Can you please tell me what version of Gluster is in use here, and we
may want to shift to the gluster-devel mailing list to get this
solved.

Thanks,

-Ira



More information about the samba-technical mailing list