[Samba] CTDB+GFS2+CMAN. clean_start="0" or clean_start="1"?

[Yauheni Labko]

Thank you for the answer, Michael.

As far as I understood clean_star="1" is absolutely ok for GFS/GFS2?

CTDB is not going to work without Red Hat Cluster manage. CMAN starts 
dlm_controld and gfs_controld. ccsd handles node-to-node communication. I 
think GPFS has the similar manager like CMAN. The clean_start="1" is the only 
setting which can provide the necessary access to the GFS/GFS2 partitions as 
CTDB required. Correct me if I'm wrong.

Btw, i thought OCFS2 is not ready to use with CTDB due to the lacks of some 
features. This was primary reason why I started  with GFS.

I left manual fencing for testing only. I was going to use iLO in production.

Yauheni Labko (Eugene Lobko)
Junior System Administrator
Chapdelaine & Co.
(212)208-9150

> CTDB is pretty ignorant of CMAN as such.
> It just relies on a cluster file system, like GFS2.


> So you should only start ctdbd when the cluster is up
> and the gfs2 file system is mounted. I think you should
> not start ctdbd as a cluster service managed by cman,
> since ctdbd can be considered a cluster manager for
> certain services (like samba...) itself. Apart from
> that, ctdb should be considered pretty much independent
> of the red hat cluster manager.
>
> CTDB needs a file in the cluster file system, the
> recovery lock file. The location of this file (or a
> directory, in which such a file can be created) should
> be specified in the CTDB_RECOVERY_LOCK=... setting
> in /etc/sysconfig/ctdb.
>
> At a glance, your cluster.conf looks sane, but
> I think manual fencing can be a real problem with
> cman.
>
> GPFS is very well tested with ctdb.
> I think there are many people testing ctdb with gfs2.
> I have heard positive feedback of people using ctdb
> with GlusterFS and lustre (and recently with ocfs2).
>
> You might want to join the #ctdb irc channel on freenode.
> There are ususally some people around with more expertise
> in gfs2 than me.
>
> Cheers - Michael
>
> Yauheni Labko wrote:
> > Hi everybody,
> >
> > I have tested CTDB+GFS2+CMAN under Debian. It works good but I do not
> > understand some points.
> > It is possible to run the CTDB defining it under services section in
> > cluster.conf but running it on the second node shuts down the process at
> > the first one. My CTDB configuration implies 2 active-active nodes.
> >
> > Does CTDB care if the node starts with clean_start="0" or
> > clean_start="1"? man fenced says this is a safe way especially during
> > startup because it prevents a data corruption if a node was dead for some
> > reason. From my understanding CTDB uses CMAN only as "module" to get
> > access to gfs/gfs2 partitions. Or maybe it is better to look at GPFS and
> > LustreFS?
> >
> > Could anybody show the working configuration of cluster.conf for
> > CTDB+GFS2+CMAN?
> >
> > I used the following cluster.conf and ctd conf:
> >
> > <?xml version="1.0"?>
> > <cluster name="smb-cluster" config_version="8">
> >   <fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
> >   <cman expected_votes="1" two_node="1"/>
> >   <cman cluster_id="101"/>
> >   <clusternodes>
> >     <clusternode name="smb01" votes="1" nodeid="1">
> >       <fence>
> >         <!-- Handle fencing manually -->
> >         <method name="human">
> >           <device name="human" nodename="smb01"/>
> >         </method>
> >       </fence>
> >     </clusternode>
> >     <clusternode name="smb02" votes="1" nodeid="2">
> >       <fence>
> >         <!-- Handle fencing manually -->
> >         <method name="human">
> >           <device name="human" nodename="smb02"/>
> >         </method>
> >       </fence>
> >     </clusternode>
> >   </clusternodes>
> >   <fencedevices>
> >     <!-- Define manual fencing -->
> >     <fencedevice name="human" agent="fence_manual"/>
> >     <!-- Define ilo fencing -->
> >     <fencedevice name="ilo" agent="fence_ilo" login="admin"
> > password="foo"/> </fencedevices>
> > </cluster>
> >
> > # Options to ctdbd. This is read by /etc/init.d/ctdb
> > CTDB_RECOVERY_LOCK="/smb-ctdb/.ctdb_locking"
> > CTDB_PUBLIC_INTERFACE=eth2
> > CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
> > CTDB_MANAGES_SAMBA=yes
> > CTDB_INIT_STYLE=ubuntu
> > CTDB_NODES=/etc/ctdb/nodes
> > CTDB_NOTIFY_SCRIPT=/etc/ctdb/notify.sh
> > CTDB_DBDIR=/var/ctdb
> > CTDB_DBDIR_PERSISTENT=/var/ctdb/persistent
> > CTDB_SOCKET=/tmp/ctdb.socket
> > CTDB_LOGFILE=/var/log/ctdb.log
> > CTDB_DEBUGLEVEL=2
> >
> > Yauheni Labko (Eugene Lobko)
> > Junior System Administrator
> > Chapdelaine & Co.
> > (212)208-9150