about active/active clustered nfs with ctdb
风无名
wuming_81 at 163.com
Thu Jan 28 01:55:29 UTC 2021
martin, thanks for your reply.
No, I did not modify 70.iscsi. Maybe I need to make full understanding of it.
after many days reading/debuging the source code of ctdb and its shell scripts, I found the key point in the script 10.interface.
my modification is:
1 create nfs share(mount fs, modify /etc/exports, restart nfs service ..) before any public ip is added to some interface
2 delete the corresponding nfs share after any public ip is removed from some interface
I tested many shutdown-reboot cycles (of node in a ctdb cluster), and the results are the same as my expectation.
I think I need more tests and more scenario tests.
Thanks for anyone's advice.
At 2021-01-27 17:46:13, "Martin Schwenke" <martin at meltin.net> wrote:
>Hi,
>
>On Fri, 22 Jan 2021 15:55:54 +0800 (CST), 风无名 via samba-technical
><samba-technical at lists.samba.org> wrote:
>
>> I want to build a nfs cluster:
>>
>> 1) the nfs cluster consists of three nodes(linux servers)
>>
>> 2) each node has logined an iscsi lun, i.e.
>>
>> node_1 -> lun_1
>>
>> node_2 -> lun_2
>>
>> node_3 -> lun_3
>>
>> 3) make xfs file system on each lun
>>
>> 4) export each xfs file system via NFS
>>
>> node_1 -> lun_1-> /share-1
>>
>> node_2 -> lun_2-> /share-2
>>
>> node_3 -> lun_3-> /share-3
>>
>> 5) ctdb distributes public ips to the nodes
>>
>>
>>
>>
>> if one node is failed, ctdb redistributes its public ip to another living node.
>>
>> and the shell scripts executed by ctdb on the node, will mount the file system, restart nfs service etc.
>>
>> if the failed node restarts, similar steps execute.
>>
>>
>>
>>
>> I have written some shell scripts to implement the above process.
>>
>> I hope that during the lun/service moving process,
>>
>> the file io on the mount points of the corresponding lun is blocked for one or two minutes and succeeds after the moving process completed.
>>
>>
>>
>>
>> My test result is:
>>
>> scenario 1: one node failed, another takes over its service
>>
>> the file io on the mount point will be blocked almost everytime
>>
>>
>>
>>
>> scenario 2: failed node restarts
>>
>> the file io on the mount point sometimes is blocked, sometimes the io(open, write) will encounter some error.
>>
>>
>>
>>
>> could I achieve my goal just through modifying/rewriting the shell script?
>>
>> or must I modify the kernel nfs or ctdb?
>
>Are you using CTDB's 70.iscsi event script for this and adding the
>extra scripts that it suggests?
>
>Unfortunately, although I am one of the CTDB developers, I don't think
>I have ever tested this script so I am not sure how reliable it is. :-(
>
>I'm CC:ing Ronnie, who originally wrote this event script, to see if he
>has any advice for you... if he can remember this at all... :-)
>
>peace & happiness,
>martin
More information about the samba-technical
mailing list