[Samba] CTDB and nfs-ganesha

[Martin Schwenke]

Hi Max,

On Tue, 1 Oct 2019 18:57:43 +0000, Max DiOrio via samba
<samba at lists.samba.org> wrote:

> Hi there – I seem to be having trouble wrapping my brain about the
> CTDB and ganesha configuration.  I thought I had it figured out, but
> it doesn’t seem to be doing any checking of the nfs-ganesha service.

> I put nfs-ganesha-callout as executable in /etc/ctdb

> I create nfs-checks-ganesha.d folder in /etc/ctdb and in there I have
> 20.nfs_ganesha.check

You may want to symlink some of the others across, such as
00.portmapper.check.

> In my ctdbd.conf file I have:

Aha!  In >=4.9 nothing looks at that file anymore.  If you run

  https://git.samba.org/?p=samba.git;a=blob_plain;f=ctdb/doc/examples/config_migrate.sh;h=8479aeb39f383bf9d3a05d79b9357a7e07a6a836;hb=refs/heads/v4-9-stable

on it then you'll get some useful suggestions...  but keep reading
below...

> # Options to ctdbd, read by ctdbd_wrapper(1)
> #
> # See ctdbd.conf(5) for more information about CTDB configuration variables.
> 
> # Shared recovery lock file to avoid split brain.  No default.
> #
> # Do NOT run CTDB without a recovery lock file unless you know exactly
> # what you are doing.
> CTDB_RECOVERY_LOCK=/run/gluster/shared_storage/.CTDB-lockfile

This should be in ctdb.conf:

[cluster]

  recovery lock = /run/gluster/shared_storage/.CTDB-lockfile

> # List of nodes in the cluster.  Default is below.
> CTDB_NODES=/etc/ctdb/nodes
> 
> # List of public addresses for providing NAS services.  No default.
> CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses

These are no longer used.  The above defaults in /etc/ctdb/ are now
hardwired.  Symlinks can be used if necessary.

> # What services should CTDB manage?  Default is none.
> # CTDB_MANAGES_SAMBA=yes
> # CTDB_MANAGES_WINBIND=yes
> CTDB_MANAGES_NFS=yes

Gone.  Now just enable the event scripts.

> # Raise the file descriptor limit for CTDB?
> # CTDB_MAX_OPEN_FILES=10000

Gone.  Either do the right thing in the systemd unit file or put a
ulimit command in /etc/sysconfig/ctdb or /etc/default/ctdb (depending
on distro).

> # Default is to use the log file below instead of syslog.
> CTDB_LOGGING=file:/var/log/log.ctdb

ctdb.conf:

[logging]
  location = file:/var/log/log.ctdb

However, this is the default so you can just omit it.

If you can use syslog instead, then I strongly suggest you do that
(see ctdb.conf(5) manpage).  CTDB's file logging has no useful way of
rotating the logs. There's an open bug, there are plans, but it is
complicated.

> # Default log level is NOTICE.  Want less logging?
> CTDB_DEBUGLEVEL=DEBUG

ctdb.conf:

[logging]
  log level = DEBUG

> CTDB_NFS_CALLOUT=/etc/ctdb/nfs-ganesha-callout
> CTDB_NFS_CHECKS_DIR=/etc/ctdb/nfs-checks-ganesha.d
> CTBS_NFS_STATE_FS_TYPE=glusterfs
> CTDB_NFS_STATE_MNT=/run/gluster/shared_storage
> CTDB_NFS_SKIP_SHARE_CHECK=yes
> NFS_HOSTNAME=hq-6pnfs

Move all of these to /etc/ctdb/script.options.

> But in the logs, I see nothing relating to nfs-ganesha, and when the
> ganesha service fails, the IP’s don’t get migrated and/or the service
> doesn’t get restarted.

> Any ideas?

Yeah, complete rewrite of configuration handling.  :-)

At some stage where will be another change some time in the future when:

* service event scripts go to /etc/ctdb/events/service/ - not many
  related options to change here...

* failover event scripts go to /etc/ctdb/events/failover/ and most of
  the failover-related tunables become script options

* ...

After that, things should be very easy to understand...  :-)

peace & happiness,
martin