[Samba] CTDB and nfs-ganesha

[Max DiOrio]

Hi Marin - again thank you for the help.  I can't believe I coundn't find any info about this big configuration change.  Even the Samba WIKI doesn't really spell this out at all in instructs you to use ctdbd.conf. 

Do I need to enable the 20.nfs_ganesha.check script file at all, or will the config itself take care of that?  Also, are there any recommendations on which nfs-checks.d checks should still be used in the nfs-checks-ganesha?

Gluster, nfs-ganesha, ctdb seems like a great combination that's sorely lacking on proper documentation.  I know there's storhaug that is supposed to tie this together, but that flat out didn't work for me at all, has poor documentation and needed manual edits just to get it to work.



On 10/1/19, 5:46 PM, "Martin Schwenke" <martin at meltin.net> wrote:

    NOTE: This email originated from outside of the organization.
    
    
    Hi Max,
    
    On Tue, 1 Oct 2019 18:57:43 +0000, Max DiOrio via samba
    <samba at lists.samba.org> wrote:
    
    > Hi there – I seem to be having trouble wrapping my brain about the
    > CTDB and ganesha configuration.  I thought I had it figured out, but
    > it doesn’t seem to be doing any checking of the nfs-ganesha service.
    
    > I put nfs-ganesha-callout as executable in /etc/ctdb
    
    > I create nfs-checks-ganesha.d folder in /etc/ctdb and in there I have
    > 20.nfs_ganesha.check
    
    You may want to symlink some of the others across, such as
    00.portmapper.check.
    
    > In my ctdbd.conf file I have:
    
    Aha!  In >=4.9 nothing looks at that file anymore.  If you run
    
      https://git.samba.org/?p=samba.git;a=blob_plain;f=ctdb/doc/examples/config_migrate.sh;h=8479aeb39f383bf9d3a05d79b9357a7e07a6a836;hb=refs/heads/v4-9-stable
    
    on it then you'll get some useful suggestions...  but keep reading
    below...
    
    > # Options to ctdbd, read by ctdbd_wrapper(1)
    > #
    > # See ctdbd.conf(5) for more information about CTDB configuration variables.
    >
    > # Shared recovery lock file to avoid split brain.  No default.
    > #
    > # Do NOT run CTDB without a recovery lock file unless you know exactly
    > # what you are doing.
    > CTDB_RECOVERY_LOCK=/run/gluster/shared_storage/.CTDB-lockfile
    
    This should be in ctdb.conf:
    
    [cluster]
    
      recovery lock = /run/gluster/shared_storage/.CTDB-lockfile
    
    > # List of nodes in the cluster.  Default is below.
    > CTDB_NODES=/etc/ctdb/nodes
    >
    > # List of public addresses for providing NAS services.  No default.
    > CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
    
    These are no longer used.  The above defaults in /etc/ctdb/ are now
    hardwired.  Symlinks can be used if necessary.
    
    > # What services should CTDB manage?  Default is none.
    > # CTDB_MANAGES_SAMBA=yes
    > # CTDB_MANAGES_WINBIND=yes
    > CTDB_MANAGES_NFS=yes
    
    Gone.  Now just enable the event scripts.
    
    > # Raise the file descriptor limit for CTDB?
    > # CTDB_MAX_OPEN_FILES=10000
    
    Gone.  Either do the right thing in the systemd unit file or put a
    ulimit command in /etc/sysconfig/ctdb or /etc/default/ctdb (depending
    on distro).
    
    > # Default is to use the log file below instead of syslog.
    > CTDB_LOGGING=file:/var/log/log.ctdb
    
    ctdb.conf:
    
    [logging]
      location = file:/var/log/log.ctdb
    
    However, this is the default so you can just omit it.
    
    If you can use syslog instead, then I strongly suggest you do that
    (see ctdb.conf(5) manpage).  CTDB's file logging has no useful way of
    rotating the logs. There's an open bug, there are plans, but it is
    complicated.
    
    > # Default log level is NOTICE.  Want less logging?
    > CTDB_DEBUGLEVEL=DEBUG
    
    ctdb.conf:
    
    [logging]
      log level = DEBUG
    
    > CTDB_NFS_CALLOUT=/etc/ctdb/nfs-ganesha-callout
    > CTDB_NFS_CHECKS_DIR=/etc/ctdb/nfs-checks-ganesha.d
    > CTBS_NFS_STATE_FS_TYPE=glusterfs
    > CTDB_NFS_STATE_MNT=/run/gluster/shared_storage
    > CTDB_NFS_SKIP_SHARE_CHECK=yes
    > NFS_HOSTNAME=hq-6pnfs
    
    Move all of these to /etc/ctdb/script.options.
    
    > But in the logs, I see nothing relating to nfs-ganesha, and when the
    > ganesha service fails, the IP’s don’t get migrated and/or the service
    > doesn’t get restarted.
    
    > Any ideas?
    
    Yeah, complete rewrite of configuration handling.  :-)
    
    At some stage where will be another change some time in the future when:
    
    * service event scripts go to /etc/ctdb/events/service/ - not many
      related options to change here...
    
    * failover event scripts go to /etc/ctdb/events/failover/ and most of
      the failover-related tunables become script options
    
    * ...
    
    After that, things should be very easy to understand...  :-)
    
    peace & happiness,
    martin