[SCM] CTDB repository - branch 2.5 updated - ctdb-2.5-47-gd0f7f95

Amitay Isaacs amitay at samba.org
Mon Nov 25 22:02:38 MST 2013

The branch, 2.5 has been updated
       via  d0f7f95f95f6bfcd9139b4d21feb08cb48366527 (commit)
       via  258a49e7bd0a81593c895217a0db4b0fdae21807 (commit)
       via  f42f07e607e2910126f13ec5b91031d60700a452 (commit)
       via  9af57ced4ba3b0964706f026bdcf529ada1b8309 (commit)
       via  552218ae0b0ecc62f9f0c7fbafe681a0b26a7480 (commit)
       via  62d823bfbe2da1437a355ffc6bd5e604a31f7c65 (commit)
       via  3ec711b713edba522f302474d506f0b3e3e51b19 (commit)
       via  a0ba1b57121f740d2d38021a81238c4aeb825ec1 (commit)
      from  e6df02968fe0ff26eb5ce5e3409e0f3276a47634 (commit)


- Log -----------------------------------------------------------------
commit d0f7f95f95f6bfcd9139b4d21feb08cb48366527
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Nov 25 19:28:10 2013 +1100

    Update NEWS
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>

commit 258a49e7bd0a81593c895217a0db4b0fdae21807
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Nov 26 15:41:50 2013 +1100

    scripts: Be careful when generating unique pids for stack traces
    sort expects the data to be line based, so make it so.
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

commit f42f07e607e2910126f13ec5b91031d60700a452
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Nov 26 14:38:58 2013 +1100

    config: Simplify the default CTDB configuration file
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Pair-programmed-with: Martin Schwenke <martin at meltin.net>

commit 9af57ced4ba3b0964706f026bdcf529ada1b8309
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Nov 26 14:29:52 2013 +1100

    scripts: Replace hard-coded /var/ctdb with CTDB_VARDIR
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

commit 552218ae0b0ecc62f9f0c7fbafe681a0b26a7480
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Nov 26 13:27:46 2013 +1100

    scripts: Set defaults for CTDB_DBDIR and CTDB_DBDIR_PERSISTENT
    If these configuration variables are not defined, then there should
    a default fallback.  This is a workaround till CTDB compile time
    configuration can be accessed at runtime.
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

commit 62d823bfbe2da1437a355ffc6bd5e604a31f7c65
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Nov 26 11:39:54 2013 +1100

    eventscripts: Perform share check before NFS RPC checks in 60.ganesha
    If NFS RPC checks do restart Ganesha, then it's possible that share
    check can fail prematurely.
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

commit 3ec711b713edba522f302474d506f0b3e3e51b19
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Nov 22 13:57:31 2013 +1100

    tools/ctdb: Improve error checking when parsing node string
    If a node isn't numeric then it is silently converted to 0.
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit a0ba1b57121f740d2d38021a81238c4aeb825ec1
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Nov 22 13:57:03 2013 +1100

    recoverd: Only respond to currently queued ipreallocated requests
    Otherwise new requests can come in during the latter parts of the
    takeover run when the IP allocation algorithm has already run, and the
    new requests will be dequeued even though they haven't really be
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>


Summary of changes:
 NEWS                       |   38 +++++
 config/ctdb.sysconfig      |  343 +++-----------------------------------------
 config/debug_locks.sh      |    7 +-
 config/events.d/00.ctdb    |    4 +-
 config/events.d/60.ganesha |   12 +-
 server/ctdb_recoverd.c     |   11 ++-
 tools/ctdb.c               |    9 +-
 7 files changed, 90 insertions(+), 334 deletions(-)

Changeset truncated at 500 lines:

diff --git a/NEWS b/NEWS
index ae4cff6..6dee537 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,41 @@
+Changes in CTDB 2.5.1
+Important bug fixes
+* The locking code now correctly implements a per-database active
+  locks limit.  Whole database lock requests can no longer be denied
+  because there are too many active locks - this is particularly
+  important for freezing databases during recovery.
+* The debug_locks.sh script locks against itself.  If it is already
+  running then subsequent invocations will exit immediately.
+* ctdb tool commands that operate on databases now work correctly when
+  a database ID is given.
+* Various code fixes for issues found by Coverity.
+Important internal changes
+* statd-callout has been updated so that statd client information is
+  always up-to-date across the cluster.  This is implemented by
+  storing the client information in a persistent database using a new
+  "ctdb ptrans" command.
+* The transaction code for persistent databases now retries until it
+  is able to take the transaction lock.  This makes the transation
+  semantics compatible with Samba's implementation.
+* Locking helpers are created with vfork(2) instead of fork(2),
+  providing a performance improvement.
+* config.guess has been updated to the latest upstream version so CTDB
+  should build on more platforms.
 Changes in CTDB 2.5
diff --git a/config/ctdb.sysconfig b/config/ctdb.sysconfig
index 35bf5f8..d96fb8e 100644
--- a/config/ctdb.sysconfig
+++ b/config/ctdb.sysconfig
@@ -1,336 +1,33 @@
-# Options to ctdbd. This is read by /etc/init.d/ctdb
-# You must specify the location of a shared lock file across all the
-# nodes for split brain prevention to work.
-# This must be on shared storage.
-# CTDB can operate without a reclock file, but this means that there is no
-# protection against a split brain.
-# It is strongly suggested to NOT run ctdb without a reclock file.
-# when doing IP takeover you also may specify what network interface
-# to use by default for the public addresses. Otherwise you must
-# specify an interface on each line of the public addresses file
-# there is no default
-# Should ctdb do IP takeover? If it should, then specify a file
-# containing the list of public IP addresses that ctdb will manage
-# Note that these IPs must be different from those in $NODES above
-# there is no default.
-# The syntax is one line per public address of the form :
-#   <ipaddress>/<netmask> <interface>
-# Example: eth0
+# Options to ctdbd, read by ctdbd_wrapper(1)
-# CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
-# Should CTDB present the cluster using a single public ip address to clients
-# and multiplex clients across all CONNECTED nodes ?
-# This is based on LVS 
-# When this is enabled, the entire cluster will present one single ip address
-# which clients will connect to.
+# See ctdbd.conf(5) for more information about CTDB configuration variables.
+# Shared recovery lock file to avoid split brain.  No default.
+# Do NOT run CTDB without a recovery lock file unless you know exactly
+# what you are doing.
-# should ctdb manage starting/stopping the Samba service for you?
-# default is to not manage Samba
-# If there are very many shares it may not be feasible to check that all
-# of them are available during each monitoring interval.
-# In that case this check can be disabled
+# List of nodes in the cluster.  Default is below.
+# CTDB_NODES=/etc/ctdb/nodes
-# specify which ports we should check that there is a daemon listening to
-# by default we use testparm and look in smb.conf to figure out.
+# List of public addresses for providing NAS services.  No default.
-# should ctdb manage starting/stopping Winbind service?
-# if left comented out then it will be autodetected based on smb.conf
+# What services should CTDB manage?  Default is none.
-# should ctdb manage starting/stopping the VSFTPD service
-# should ctdb manage starting/stopping the ISCSI service
-# should ctdb manage starting/stopping the NFS service
-# should ctdb manage starting/stopping the Apache web server httpd?
-# The init style (redhat/suse/debian...) is usually auto-detected.
-# The names of init scripts of services managed by CTDB are set
-# based on the detected init style. You can override the init style
-# auto-detection here to explicitly use a scheme. This might be
-# useful when you have installed a packages (for instance samba
-# packages) with a different init script layout.
-# There is no default.
-# The following are specific Samba init scripts / services that you
-# can override from auto-detection.
-# There are no defaults.
-# you may wish to raise the file descriptor limit for ctdb
-# use a ulimit command here. ctdb needs one file descriptor per
-# connected client (ie. one per connected client in Samba)
-#  ulimit -n 10000
-# the NODES file must be specified or ctdb won't start
-# it should contain a list of IPs that ctdb will use
-# it must be exactly the same on all cluster nodes
-# defaults to /etc/ctdb/nodes
-# CTDB_NODES=/etc/ctdb/nodes
-# a script to run when node health changes
-# CTDB_NOTIFY_SCRIPT=/etc/ctdb/notify.sh
-# a script to collect data when an eventscript has hung
-# CTDB_DEBUG_HUNG_SCRIPT=/etc/ctdb/debug-hung-script.sh
-# the directory to put the local ctdb database files in
-# defaults to /var/ctdb
-# CTDB_DBDIR=/var/ctdb
-# the directory to put the local persistent ctdb database files in
-# defaults to /var/ctdb/persistent
-# CTDB_DBDIR_PERSISTENT=/var/ctdb/persistent
-# the directory where service specific event scripts are stored
-# defaults to /etc/ctdb/events.d
-# CTDB_EVENT_SCRIPT_DIR=/etc/ctdb/events.d
-# the location of the local ctdb socket
-# defaults to /var/run/ctdb/ctdbd.socket
-# CTDB_SOCKET=/var/run/ctdb/ctdbd.socket
-# what transport to use. Only tcp is currently supported
-# defaults to tcp
-# These setting allow monitoring for low/out-out of memory conditions.
-# If set, once available memory drops below CTDB_MONITOR_FREE_MEMORY_WARN
-# ctdb will start logging messages that memory is low, but will not
-# take any further action.
-# If the amount of free memory drops below CTDB_MONITOR_FREE_MEMORY 
-# ctdb will fail all service over to a different node and finally shutdown.
-# Once this occurs, the administrator needs to find the reason for the OOM
-# situation, rectify it and restart ctdb with "service ctdb start"
-# The unit is MByte
-# Should the 60.nfs monitor event try to correct the number of nfsd
-# threads?  This works around a limitation in some NFS initscripts
-# where some threads can be stuck in host filesystem calls (perhaps
-# due to slow storage), a restart occurs, some threads don't exit, the
-# start only adds the missing number of threads, the stuck threads
-# exit, and the result is a lower than expected thread count.  Note
-# that if you must also set $RPCNFSDCOUNT (RedHat/Debian) or
-# $USE_KERNEL_NFSD_NUMBER (SUSE) in your NFS configuration so the
-# monitoring code knows how many threads there should be - if neither
-# of these are set then this option will be ignored.  The default is
-# to not do this check.
-# The number of nfsd threads to dump stack traces for if some are
-# still alive after stopping NFS during a restart.  The default is to
-# dump no stack traces.
-# Host to use for rpcinfo checks.  Using "localhost" causes
-# /etc/services to be read serveral times by rpcinfo, so this allows
-# optimisation.
-# When set to yes, the CTDB node will start in DISABLED mode and not host
-# any public ip addresses. The administrator needs to explicitely enable
-# the node with "ctdb enable"
+# Raise the file descriptor limit for CTDB?
+# ulimit -n 10000
-# LMASTER and RECMASTER capabilities.
-# By default all nodes are capable of both being LMASTER for records and
-# also for taking the RECMASTER role and perform recovery.
-# These parameters can be used to disable these two roles on a node.
-# Note: If there are NO available nodes left in a cluster that can perform
-# the RECMASTER role, the cluster will not be able to recover from a failure
-# and will remain in RECOVERY mode until an RECMASTER capable node becomes
-# available. Same for LMASTER.
-# These parametersd are useful for scenarios where you have one "remote" node
-# in a cluster and you do not want the remote node to be fully participating
-# in the cluster and slow things down.
-# For that case, set both roles to "no" for the remote node on the remote site
-# but leave the roles default to "yes" on the primary nodes in the central
-# datacentre.
-# NAT-GW configuration
-# Some services running on nthe CTDB node may need to originate traffic to
-# remote servers before the node is assigned any IP addresses,
-# This is problematic since before the node has public addresses the node might
-# not be able to route traffic to the public networks.
-# One solution is to have static public addresses assigned with routing
-# in addition to the public address interfaces, thus guaranteeing that
-# a node always can route traffic to the external network.
-# This is the most simple solution but it uses up a large number of 
-# additional ip addresses.
-# A more complex solution is NAT-GW.
-# In this mode we only need one additional ip address for the cluster from
-# the exsternal public network.
-# One of the nodes in the cluster is elected to be hosting this ip address
-# so it can reach the external services. This node is also configured
-# to use NAT MASQUERADING for all traffic from the internal private network
-# to the external network. This node is the NAT-GW node.
-# All other nodes are set up with a default rote with a metric of 10 to point
-# to the nat-gw node.
-# The effect of this is that only when a node does not have a public address
-# and thus no proper routes to the external world it will instead
-# route all packets through the nat-gw node.
-# CTDB_NATGW_NODES is the list of nodes that belong to this natgw group.
-# You can have multiple natgw groups in one cluster but each node
-# can only belong to one single natgw group.
-# CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
-# Normally any node in the natgw group can act as the natgw master.
-# In some configurations you may have special nodes that is a part of the
-# cluster/natgw group, but where the node lacks connectivity to the 
-# public network.
-# For these cases, set this variable to make these nodes not able to
-# become natgw master.
-# PER_IP_ROUTING configuration
-# Some setups have multiple network interfaces connected to the
-# same network. By default all traffic for a network is routed
-# through only one interface, while the others are idle.
-# On Linux it possible to use policy based routing to spread the load
-# across all interfaces. The is implemented by using a separate
-# routing table per public ip address.
-# The configuration file configured by CTDB_PER_IP_ROUTING_CONF
-# contains the list of additional routes. The routes are bound to the
-# interface that is holding the public ip address.
-# The format of the config file looks like this:
-# <public_ip_address> <network> [<gateway>]
-# and it's possible to have multiple routes per public ip address.
-# If the special value "__auto_link_local__" is used, the config
-# file autogenerated. Each public ip address gets a special route
-# for its own subnet bound to it's current interface.
-# E.g. will result in a config file line
-# The CTDB_PER_IP_ROUTING_RULE_PREF option needs to be configured.
-# The value will be passed as "pref" argument of "ip rule".
-# The value should be between 1 and 32765. So that the rule
-# comes after the rule for "local" routing table and before
-# the rule for the "main" routing table. This way the specific
-# routing table just overloads the "main" routing table,
-# this is useful because with the "__auto_link_local__" setup
-# the default route still comes from the "main" routing table.
-# The routing table ids are automaticly allocated. On
-# Linux the routing table ids must be in the range of 0 to 255.
-# But some are reserved values, see /etc/iproute2/rt_tables.
-# You need to configure a range (CTDB_PER_IP_ROUTING_TABLE_ID_LOW
-# and CTDB_PER_IP_ROUTING_TABLE_ID_HIGH) from which the table ids can be taken.
-# The default value for CTDB_PER_IP_ROUTING_CONF is "",
-# which means the feature is disabled by default.
-# CTDB_PER_IP_ROUTING_CONF="/etc/ctdb/per_ip_routing.conf"
-# CTDB_PER_IP_ROUTING_CONF="__auto_link_local__"
-# Make offline interfaces not a reason for being UNHEALTHY.
-# the behavior of the 10.interface monitor event.
-# In some setups it's desired that interfaces without
-# an active link don't change the node to unhealthy.
-# ctdbd is just informed about the interface status
-# and "ctdb status" dislays the node as "PARTIALLYONLINE".
-# where to log messages
-# the default is /var/log/log.ctdb
+# Default is to use the log file below instead of syslog.
 # CTDB_LOGFILE=/var/log/log.ctdb
-# what debug level to run at. Higher means more verbose
-# the default is ERR
-# whether to suppress core files.  Default is no.
-# Write debug messages to syslog instead of logfile?
-# The default is not to use syslog.
-# Should the 99.timeout monitor event script be run?
-# This event script just sleeps long enough to trigger the
-# event script timeout. Might be useful for debugging.
-# The default is "no".
-# Should ctdbd start with corrupted/unhealthy persistent databases?
-# This parameter specifies the max error count for persistent health
-# checks before the "startup" event. The value must be a positive
-# interger value, "0" or "-1".
-# The default is "0", which means ctdbd will not start.
-# "-1" means wait forever.
-# All log entries up to level 9 are also collected into a in-memory ringbuffer
-# in addition to the log that is written to the log file.
-# This parameter controls how many entries we allow for this in memory log
-# Monitor filesystem useage.
-# when set, and the 40.fs_use eventscript is enabled, this variable
-# allows one to monitor the filesystem use and flag a node as unhealthy when
-# the filesystem becomes too full.
-# This is useful for example when /var grows too big.
-# Example: monitor both / and /var and make the node unhealthy when eitehr go
-# above 90%
-# CTDB_CHECK_FS_USE="/:90 /var:90"
+# Default log level is ERR.  NOTICE is a little more verbose.
-# Should CTDB automatically start and stop services when it is told to
-# newly manage or no longer manage them?
-# set any default tuning options for ctdb
-# use CTDB_SET_XXXX=value where XXXX is the name of the tuning
-# variable
-# for example
-# you can get a list of variables using "ctdb listvars"
+# Set some CTDB tunable variables during CTDB startup?
+# CTDB_SET_TraverseTimeout=60
diff --git a/config/debug_locks.sh b/config/debug_locks.sh
index 72a8917..54b52ed 100755
--- a/config/debug_locks.sh
+++ b/config/debug_locks.sh
@@ -12,6 +12,11 @@
 . "$CTDB_BASE/functions"
+# Default fallback location for database directories.
+# These can be overwritten from CTDB configuration
 loadconfig ctdb
@@ -46,7 +51,7 @@ loadconfig ctdb
 	    pids=$(echo "$out" | grep -v "W$" | grep "$db" | grep -v ctdbd | awk '{print $1}')
 	    all_pids="$all_pids $pids"
-	pids=$(echo $all_pids | sort -u)
+	pids=$(echo $all_pids | tr " " "\n" | sort -u)
 	# For each process waiting, log stack trace
 	for pid in $pids ; do
diff --git a/config/events.d/00.ctdb b/config/events.d/00.ctdb
index 880d07f..719bee8 100755
--- a/config/events.d/00.ctdb
+++ b/config/events.d/00.ctdb
@@ -63,7 +63,7 @@ check_tdb ()
 check_persistent_databases ()
-    _dir="${CTDB_DBDIR_PERSISTENT:-${CTDB_DBDIR:-/var/ctdb}/persistent}"
+    _dir="${CTDB_DBDIR_PERSISTENT:-${CTDB_DBDIR:-${CTDB_VARDIR}}/persistent}"
     mkdir -p "$_dir" 2>/dev/null
     [ "${CTDB_MAX_PERSISTENT_CHECK_ERRORS:-0}" = "0" ] || return 0
@@ -78,7 +78,7 @@ check_persistent_databases ()
 check_non_persistent_databases ()
-    _dir="${CTDB_DBDIR:-/var/ctdb}"
+    _dir="${CTDB_DBDIR:-${CTDB_VARDIR}}"
     mkdir -p "$_dir" 2>/dev/null
     for _db in $(ls "${_dir}/"*.tdb.*[0-9] 2>/dev/null) ; do
diff --git a/config/events.d/60.ganesha b/config/events.d/60.ganesha
index 242321a..89b63ff 100755
--- a/config/events.d/60.ganesha
+++ b/config/events.d/60.ganesha
@@ -187,6 +187,12 @@ case "$1" in
+	# Check that directories for shares actually exist.
+	[ "$CTDB_NFS_SKIP_SHARE_CHECK" = "yes" ] || {
+	    grep Path /etc/ganesha/$CTDB_CLUSTER_FILESYSTEM_TYPE.ganesha.exports.conf |
+ 	    cut -f2 -d\" | sort -u | ctdb_check_directories
+	} || exit $?
 	update_tickles 2049
 	# check that statd responds to rpc requests
@@ -209,12 +215,6 @@ case "$1" in
 	# not a critical service so we dont flag the node as unhealthy
 	nfs_check_rpc_service "rquotad" \
 	    -gt 0 "verbose restart:b"
-	# Check that directories for shares actually exist.
-	[ "$CTDB_NFS_SKIP_SHARE_CHECK" = "yes" ] || {
-	    grep Path /etc/ganesha/$CTDB_CLUSTER_FILESYSTEM_TYPE.ganesha.exports.conf |
- 	    cut -f2 -d\" | ctdb_check_directories
-	} || exit $?
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index 70ed87e..c5cd36c 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -2606,9 +2606,18 @@ static void process_ipreallocate_requests(struct ctdb_context *ctdb,
 	TDB_DATA result;
 	int32_t ret;
 	uint32_t culprit;
+	struct srvid_requests *current;
 	DEBUG(DEBUG_INFO, ("recovery master forced ip reallocation\n"));
+	/* Only process requests that are currently pending.  More
+	 * might come in while the takeover run is in progress and
+	 * they will need to be processed later since they might
+	 * be in response flag changes.

CTDB repository

More information about the samba-cvs mailing list