Setting up CTDB on OCFS2 and VMs ...

Michael Adam obnox at samba.org
Thu Jan 1 11:30:21 MST 2015


Oops, I replied before realizing that Martin Schwenke
and Stefan Kania had already replied to your mail,
and that Martin has nailed your issues to the
corresponding debian backports packaging issues
already. So most of my mail can be ignored. :-)

Still: You need to get your clusterfs setup right
so that you can use a reclock....

Michael

On 2015-01-01 at 19:25 +0100, Michael Adam wrote:
> Happy new year!
> 
> On 2014-12-31 at 18:40 +0000, Rowland Penny wrote:
> > On 31/12/14 17:59, Michael Adam wrote:
> > >On 2014-12-31 at 15:46 +0000, Rowland Penny wrote:
> > >>OK, I have been having another attempt at the ctdb cluster, I cannot get
> > >>both nodes healthy if I use a lockfile in /etc/default/ctdb,
> > >This can't be expected to work, since a recovery lock file needs
> > >to be on shared storage (clustered file system, providing posix
> > >fcntl byte range lock semantics), and /etc/default/ is not
> > >generally such a place, unless you are building a fake cluster
> > >with multiple ctdb instances running on one host.
> > 
> > The lockfile *is* on the shared area i.e. I am sharing /cluster and that is
> > what I have in /etc/default/ctdb:
> > 
> > CTDB_RECOVERY_LOCK=/cluster/lockfile
> 
> Oh, ok... I misread your words above.
> 
> > >>so I have commented it out, both nodes are now showing OK.
> > >It is possible to run w/o recovery lock, but it is not
> > >recommended in a production setup at least.
> > 
> > I am aware of this, but it seems to be the only way of getting ctdb to
> > start.
> 
> Which probably means that your clustered fs setup is still
> not correct. Or there is still a flaw in your ctdb setup.
> Could you (re-)post your /etc/default/ctdb and /etc/ctdb/nodes
> and also your network config (ip a l) on the nodes?
> 
> It is really important to get this right before starting to
> seriously play with clustered samba on top.
> 
> > >>Why have very similar data in 3 places ? why have the conf (which
> > >>incidentaly isn't called a conf file) in a different place from the other
> > >>ctdb files in /etc ?
> > >That's essentially two places, one hierarchy under /var/ctdb
> > >(old ctdb versions) and one hierarchy under /var/lib/ctdb (new
> > >ctb versions) so my guess is that this stems from earlier
> > >installs of older versions.
> > >
> > >If you stop ctdb, remove both these directory trees, and then
> > >restart ctdb, do both trees reappear?
> > 
> > No idea, I have only installed ctdb *once*, there is no earlier version.
> 
> Ok. Does that mean that you performed the above steps and
> both directory trees reappeared?
> 
> > >>More to the point, Why, oh why doesn't it work.
> > >Has the samba version been compiled against the used ctdb
> > >version. One possible source of such problems is that
> > >samba might have beenm compiled against an older version
> > >of ctdb and then you install the latest version of ctdb.
> > 
> > Again, no idea, I am using the samba4 & ctdb packages from backports,
> > versions 4.1.9 & 2.5.3
> 
> > >The problem that could explain the "Could not initialize ..."
> > >message would be that samba tries to access CTDB under the
> > >socket file /tmp/ctdbd.socket (default in old ctdb versions)
> > >and the new ctdbd uses /var/run/ctdb/ctdbd.socket by default.
> > 
> > Now that is interesting, because if I do not put a line in smb.conf saying
> > where ctdbd.socket is, it tries to use /tmp.
> 
> That confirms my theory. And it means that
> the samba and ctdv packages from backports simply don't match.
> Samba has apparently been compiled against an oder version
> of ctdb that still used /tmp.
> 
> > With the line in smb.conf, it
> > just errors with: connect(/var/lib/ctdb/ctdb.socket) failed: No such file or
> > directory
> 
> Er strange, and you have entered "/var/run/ctdb/..." into
> smb.conf and not by accient "/var/lib/ctdb/... ?
> 
> > >So you could (without needing to recompile) test if things
> > >work out more nicely if you set:
> > >
> > >"ctdbd socket = /var/run/ctdb/ctdbd.socket"
> > >in smb.conf
> > 
> > No, but finding out where the socket is and altering the line to: ctdbd
> > socket = /var/lib/run/ctdb/ctdbd.socket
> > 
> > and running: net ads join -U Administrator at EXAMPLE.COM -d5
> > 
> > Gets me (after a lot of output)
> > 
> > Using short domain name -- EXAMPLE
> > Joined 'SMBCLUSTER' to dns domain 'example.com'
> > Not doing automatic DNS update in a clustered setup.
> > return code = 0
> > 
> > Good Grief!!!! It actually seems to have worked =-O
> 
> Yay!
> 
> That path is strange.
> Are there some symlinks involved or maybe missing?
> Or does the debian ctdb package have an altered
> path for the socket? That may well be.
> 
> Which debian version are you using? (I could inspect it locally.)
> 
> > Now to try altering the conf file to get it start smbd, nmbd and winbind.
> > 
> > >and (for the sake of explicitness):
> > >"CTDB_SOCKET=/var/run/ctdb/ctdbd.socket"
> > >in /etc/default/ctdb.
> > 
> > I have tried similar lines in /etc/default/ctdb, but whatever I tried, it
> > just wouldn't let ctdb start.
> 
> Er, that should not be. Could you post the exact
> /etc/default/ctdb file used and the error messages
> that ctdbd prints?
> 
> Cheers - Michael


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20150101/ebdbb556/attachment.pgp>


More information about the samba-technical mailing list