about active/active clustered nfs with ctdb

Martin Schwenke martin at meltin.net
Wed Jan 27 09:46:13 UTC 2021


Hi,

On Fri, 22 Jan 2021 15:55:54 +0800 (CST), 风无名 via samba-technical
<samba-technical at lists.samba.org> wrote:

> I want to build a nfs cluster:
> 
> 1) the nfs cluster  consists of three nodes(linux servers)
> 
> 2) each node has  logined an iscsi lun, i.e.
> 
> node_1 -> lun_1
> 
> node_2 -> lun_2
> 
> node_3 -> lun_3
> 
> 3) make xfs file system on each lun
> 
> 4) export each xfs file system via NFS
> 
> node_1 -> lun_1-> /share-1
> 
> node_2 -> lun_2-> /share-2
> 
> node_3 -> lun_3-> /share-3
> 
> 5) ctdb distributes public ips to the nodes
> 
> 
> 
> 
> if one node is failed, ctdb redistributes its public ip to another living node.
> 
> and the shell scripts executed by ctdb on the node, will mount the file system, restart nfs service etc.
> 
> if the failed node restarts, similar steps execute.
> 
> 
> 
> 
> I have written some shell scripts to implement the above process.
> 
> I hope that during the lun/service moving process, 
> 
> the file io on the mount points of the corresponding lun is blocked for one or two minutes and succeeds after the moving process completed. 
> 
> 
> 
> 
> My test result is:
> 
> scenario 1: one node failed, another takes over its service
> 
> the file io on the mount point will be blocked almost everytime
> 
> 
> 
> 
> scenario 2: failed node restarts
> 
> the file io on the mount point sometimes is blocked, sometimes the io(open, write)  will encounter some error.
> 
> 
> 
> 
> could I achieve my goal just through modifying/rewriting the shell script? 
> 
> or must I modify the kernel nfs or ctdb? 

Are you using CTDB's 70.iscsi event script for this and adding the
extra scripts that it suggests?

Unfortunately, although I am one of the CTDB developers, I don't think
I have ever tested this script so I am not sure how reliable it is.  :-(

I'm CC:ing Ronnie, who originally wrote this event script, to see if he
has any advice for you... if he can remember this at all...  :-)

peace & happiness,
martin



More information about the samba-technical mailing list