about active/active clustered nfs with ctdb

风无名 wuming_81 at 163.com
Thu Jan 28 01:55:29 UTC 2021


martin, thanks for your reply.
No, I did not modify 70.iscsi. Maybe I need to make full understanding of it.


after many days reading/debuging the source code of ctdb and its shell scripts, I found the key point in the script 10.interface. 
my modification  is:
1 create nfs share(mount fs, modify /etc/exports, restart nfs service ..) before any public ip is added to some interface
2 delete the corresponding nfs share after any public ip is removed from some interface


I tested many shutdown-reboot cycles (of node in a ctdb cluster), and the results are the same as my expectation.
I think I need more tests and more scenario tests.

Thanks for anyone's advice.








At 2021-01-27 17:46:13, "Martin Schwenke" <martin at meltin.net> wrote:
>Hi,
>
>On Fri, 22 Jan 2021 15:55:54 +0800 (CST), 风无名 via samba-technical
><samba-technical at lists.samba.org> wrote:
>
>> I want to build a nfs cluster:
>> 
>> 1) the nfs cluster  consists of three nodes(linux servers)
>> 
>> 2) each node has  logined an iscsi lun, i.e.
>> 
>> node_1 -> lun_1
>> 
>> node_2 -> lun_2
>> 
>> node_3 -> lun_3
>> 
>> 3) make xfs file system on each lun
>> 
>> 4) export each xfs file system via NFS
>> 
>> node_1 -> lun_1-> /share-1
>> 
>> node_2 -> lun_2-> /share-2
>> 
>> node_3 -> lun_3-> /share-3
>> 
>> 5) ctdb distributes public ips to the nodes
>> 
>> 
>> 
>> 
>> if one node is failed, ctdb redistributes its public ip to another living node.
>> 
>> and the shell scripts executed by ctdb on the node, will mount the file system, restart nfs service etc.
>> 
>> if the failed node restarts, similar steps execute.
>> 
>> 
>> 
>> 
>> I have written some shell scripts to implement the above process.
>> 
>> I hope that during the lun/service moving process, 
>> 
>> the file io on the mount points of the corresponding lun is blocked for one or two minutes and succeeds after the moving process completed. 
>> 
>> 
>> 
>> 
>> My test result is:
>> 
>> scenario 1: one node failed, another takes over its service
>> 
>> the file io on the mount point will be blocked almost everytime
>> 
>> 
>> 
>> 
>> scenario 2: failed node restarts
>> 
>> the file io on the mount point sometimes is blocked, sometimes the io(open, write)  will encounter some error.
>> 
>> 
>> 
>> 
>> could I achieve my goal just through modifying/rewriting the shell script? 
>> 
>> or must I modify the kernel nfs or ctdb? 
>
>Are you using CTDB's 70.iscsi event script for this and adding the
>extra scripts that it suggests?
>
>Unfortunately, although I am one of the CTDB developers, I don't think
>I have ever tested this script so I am not sure how reliable it is.  :-(
>
>I'm CC:ing Ronnie, who originally wrote this event script, to see if he
>has any advice for you... if he can remember this at all...  :-)
>
>peace & happiness,
>martin


More information about the samba-technical mailing list