Setting up CTDB on OCFS2 and VMs ...

[Michael Adam]

Happy new year!

On 2014-12-31 at 18:40 +0000, Rowland Penny wrote:
> On 31/12/14 17:59, Michael Adam wrote:
> >On 2014-12-31 at 15:46 +0000, Rowland Penny wrote:
> >>OK, I have been having another attempt at the ctdb cluster, I cannot get
> >>both nodes healthy if I use a lockfile in /etc/default/ctdb,
> >This can't be expected to work, since a recovery lock file needs
> >to be on shared storage (clustered file system, providing posix
> >fcntl byte range lock semantics), and /etc/default/ is not
> >generally such a place, unless you are building a fake cluster
> >with multiple ctdb instances running on one host.
> 
> The lockfile *is* on the shared area i.e. I am sharing /cluster and that is
> what I have in /etc/default/ctdb:
> 
> CTDB_RECOVERY_LOCK=/cluster/lockfile

Oh, ok... I misread your words above.

> >>so I have commented it out, both nodes are now showing OK.
> >It is possible to run w/o recovery lock, but it is not
> >recommended in a production setup at least.
> 
> I am aware of this, but it seems to be the only way of getting ctdb to
> start.

Which probably means that your clustered fs setup is still
not correct. Or there is still a flaw in your ctdb setup.
Could you (re-)post your /etc/default/ctdb and /etc/ctdb/nodes
and also your network config (ip a l) on the nodes?

It is really important to get this right before starting to
seriously play with clustered samba on top.

> >>Why have very similar data in 3 places ? why have the conf (which
> >>incidentaly isn't called a conf file) in a different place from the other
> >>ctdb files in /etc ?
> >That's essentially two places, one hierarchy under /var/ctdb
> >(old ctdb versions) and one hierarchy under /var/lib/ctdb (new
> >ctb versions) so my guess is that this stems from earlier
> >installs of older versions.
> >
> >If you stop ctdb, remove both these directory trees, and then
> >restart ctdb, do both trees reappear?
> 
> No idea, I have only installed ctdb *once*, there is no earlier version.

Ok. Does that mean that you performed the above steps and
both directory trees reappeared?

> >>More to the point, Why, oh why doesn't it work.
> >Has the samba version been compiled against the used ctdb
> >version. One possible source of such problems is that
> >samba might have beenm compiled against an older version
> >of ctdb and then you install the latest version of ctdb.
> 
> Again, no idea, I am using the samba4 & ctdb packages from backports,
> versions 4.1.9 & 2.5.3

> >The problem that could explain the "Could not initialize ..."
> >message would be that samba tries to access CTDB under the
> >socket file /tmp/ctdbd.socket (default in old ctdb versions)
> >and the new ctdbd uses /var/run/ctdb/ctdbd.socket by default.
> 
> Now that is interesting, because if I do not put a line in smb.conf saying
> where ctdbd.socket is, it tries to use /tmp.

That confirms my theory. And it means that
the samba and ctdv packages from backports simply don't match.
Samba has apparently been compiled against an oder version
of ctdb that still used /tmp.

> With the line in smb.conf, it
> just errors with: connect(/var/lib/ctdb/ctdb.socket) failed: No such file or
> directory

Er strange, and you have entered "/var/run/ctdb/..." into
smb.conf and not by accient "/var/lib/ctdb/... ?

> >So you could (without needing to recompile) test if things
> >work out more nicely if you set:
> >
> >"ctdbd socket = /var/run/ctdb/ctdbd.socket"
> >in smb.conf
> 
> No, but finding out where the socket is and altering the line to: ctdbd
> socket = /var/lib/run/ctdb/ctdbd.socket
> 
> and running: net ads join -U Administrator at EXAMPLE.COM -d5
> 
> Gets me (after a lot of output)
> 
> Using short domain name -- EXAMPLE
> Joined 'SMBCLUSTER' to dns domain 'example.com'
> Not doing automatic DNS update in a clustered setup.
> return code = 0
> 
> Good Grief!!!! It actually seems to have worked =-O

Yay!

That path is strange.
Are there some symlinks involved or maybe missing?
Or does the debian ctdb package have an altered
path for the socket? That may well be.

Which debian version are you using? (I could inspect it locally.)

> Now to try altering the conf file to get it start smbd, nmbd and winbind.
> 
> >and (for the sake of explicitness):
> >"CTDB_SOCKET=/var/run/ctdb/ctdbd.socket"
> >in /etc/default/ctdb.
> 
> I have tried similar lines in /etc/default/ctdb, but whatever I tried, it
> just wouldn't let ctdb start.

Er, that should not be. Could you post the exact
/etc/default/ctdb file used and the error messages
that ctdbd prints?

Cheers - Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20150101/86922d24/attachment.pgp>