[SCM] CTDB repository - branch master updated - ctdb-2.3-55-ge9ef93f

Amitay Isaacs amitay at samba.org
Wed Aug 14 00:18:31 MDT 2013


The branch, master has been updated
       via  e9ef93f7b6dad59eabaa32124df81f3e74c651ef (commit)
       via  b49c4f39666d5b1596213bf41bcdc47ed3c327ae (commit)
       via  ff5f0d1e29af2b293e30cdc54bed03a644be7038 (commit)
       via  11fbf4789d783dd0bac22754b374dd9ea4b03bad (commit)
       via  1d61988af9e4fa3621a3e2d06a859bcb53df2d67 (commit)
       via  f4ef83a256f59eeb00b9a5bc10c28347e1ad1031 (commit)
       via  e8b531405665885196c95fe1608db33a255bf761 (commit)
       via  3ba933d806106d12bc48b83b22d0f314d9d1e5e5 (commit)
       via  1a1be43f8466d46913dcdfe6dcedb94316cd28ad (commit)
       via  c7332526b1b488abefeb4be78a7cd3f2f9abc451 (commit)
       via  63be516673c5d9c0d543617bf1bb8bca919956a8 (commit)
       via  4e3bd06916bd3adac213fb18c7c2a24854b02d45 (commit)
       via  2fc6b6403707a292d134140fc0b9145b454992c5 (commit)
       via  ca61eb776ab862bd269e45ee0f9f96e7e1e0e001 (commit)
      from  5e9b1a7e24d058ff88aaa0563db36a804e866fa9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit e9ef93f7b6dad59eabaa32124df81f3e74c651ef
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Aug 12 11:36:25 2013 +1000

    eventscripts: Become unhealthy faster on nfsd failure
    
    Anecdotal evidence suggests that most nfsd RPC check failures are due
    to cluster filesystem or storage problem.  Apparently these are rarely
    helped by attempting to restart the NFS service because the restart
    tends to hang.
    
    Fail after 2 nfsd RPC check failures, instead of waiting for 6
    failures.  Restart on every 10th failure to try to bring the node back
    to good health.
    
    Update unit tests to match.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit b49c4f39666d5b1596213bf41bcdc47ed3c327ae
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Aug 9 11:56:29 2013 +1000

    tools/ctdb: Increase default control timeout to 10 seconds
    
    The current 3 second timeout is arbitrary and users trip over it
    sometimes.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit ff5f0d1e29af2b293e30cdc54bed03a644be7038
Author: Martin Schwenke <martin at meltin.net>
Date:   Thu Aug 8 16:02:44 2013 +1000

    eventscripts: Improve message logged when a counter hits a limit
    
    It should print the actual number of consecutive failures rather than
    the limit.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit 11fbf4789d783dd0bac22754b374dd9ea4b03bad
Author: Martin Schwenke <martin at meltin.net>
Date:   Tue Aug 6 12:42:13 2013 +1000

    eventscripts: Print a message when waiting for TCP connections to be killed
    
    This makes the gaps in the logs more obvious.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit 1d61988af9e4fa3621a3e2d06a859bcb53df2d67
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Aug 5 15:12:14 2013 +1000

    eventscripts: New configuration variable $CTDB_RPCINFO_LOCALHOST
    
    Passing "localhost" to the rpcinfo command causes overheads, like
    reading /etc/services multiple times.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>

commit f4ef83a256f59eeb00b9a5bc10c28347e1ad1031
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Aug 2 15:18:47 2013 +1000

    eventscripts: Add modulo (%) operator to ctdb_check_counter()
    
    Also add it to the corresponding eventscript unit test infrastructure.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit e8b531405665885196c95fe1608db33a255bf761
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Aug 2 16:05:46 2013 +1000

    eventscripts: Separate out RPC service restart code
    
    While doing this:
    
    * Explicitly assign RPC program and version information in
      _nfs_check_rpc_common().  This is more lines of code but is easier
      to read.
    
    * Don't print the options when starting a service.  Trying to print it
      makes the code messy for little benefit.
    
      Update the eventscript unit testing code and a Ganesha test to
      reflect this.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit 3ba933d806106d12bc48b83b22d0f314d9d1e5e5
Author: Martin Schwenke <martin at meltin.net>
Date:   Fri Aug 2 16:03:42 2013 +1000

    tests/eventscripts: Override background_with_logging(), just prepend "&"
    
    That is, output that goes through background_with_logging() just gets
    "&" prepended to each line.  This is cleaner than having the tests
    grovel through logs.
    
    Update some 49.winbind/50.samba tests to deal with this.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit 1a1be43f8466d46913dcdfe6dcedb94316cd28ad
Author: Martin Schwenke <martin at meltin.net>
Date:   Tue Jul 30 16:24:24 2013 +1000

    eventscripts: Remove support for RPC service 'q' and 's' restart flags
    
    They're hard to maintain and provide very little benefit.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit c7332526b1b488abefeb4be78a7cd3f2f9abc451
Author: Martin Schwenke <martin at meltin.net>
Date:   Tue Jul 30 16:21:36 2013 +1000

    eventscripts: When restarting the nfslock service only show output of start
    
    That is, /dev/null the "stop" output.  This is consistent with the way
    CTDB generally deals with the output when stopping a service.
    
    It also makes updating the eventscript unit tests easier.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>

commit 63be516673c5d9c0d543617bf1bb8bca919956a8
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Jul 29 15:27:24 2013 +1000

    tests/simple: Unreachable node test should wait for recovery to complete
    
    This should minimise the chances of a control timing out.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>

commit 4e3bd06916bd3adac213fb18c7c2a24854b02d45
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Jul 29 15:09:23 2013 +1000

    tests/simple: Fix the missing IP test
    
    Update the missing IP test to wait until restarts are complete.
    Otherwise a service restart can collide with the following monitor
    event and cause chaos.
    
    Also, do not disable 10.interface until it matters.  Disabling it too
    early can cause even more chaos if something goes wrong with the
    monitor step.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>

commit 2fc6b6403707a292d134140fc0b9145b454992c5
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Aug 13 14:02:46 2013 +1000

    recoverd: Use TDB_INCOMPATIBLE_HASH when creating volatile databases
    
    When creating missing databases either locally or remotely, recovery
    master calls ctdb_ctrl_createdb().  Recovery master always passes 0
    for tdb_flags.  For volatile databases, if TDB_INCOMPATIBLE_HASH is not
    specified, then they will be attached without using jenkins hash causing
    database corruption.
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

commit ca61eb776ab862bd269e45ee0f9f96e7e1e0e001
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Aug 13 13:55:47 2013 +1000

    Revert "recoverd: Use correct tdb flags when creating missing databases"
    
    This reverts commit 10a057d8e15c8c18e540598a940d3548c731b0b4.
    
    This approach would not work when creating local databases since currently
    there is no control to receive TDB flags for remote databases.
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>

-----------------------------------------------------------------------

Summary of changes:
 client/ctdb_client.c                               |   18 ++-
 config/ctdb.sysconfig                              |    5 +
 config/events.d/60.ganesha                         |    2 +-
 config/functions                                   |  131 ++++++++++++--------
 config/nfs-rpc-checks.d/10.statd.check             |    2 +-
 config/nfs-rpc-checks.d/20.nfsd.check              |    5 +-
 config/nfs-rpc-checks.d/30.lockd.check             |    2 +-
 include/ctdb_client.h                              |    2 +-
 server/ctdb_recoverd.c                             |    4 +-
 tests/eventscripts/10.interface.releaseip.011.sh   |    3 +
 tests/eventscripts/49.winbind.monitor.050.sh       |    9 +-
 tests/eventscripts/49.winbind.monitor.051.sh       |    9 +-
 tests/eventscripts/50.samba.monitor.050.sh         |    9 +-
 tests/eventscripts/50.samba.monitor.051.sh         |    9 +-
 tests/eventscripts/60.ganesha.monitor.141.sh       |    4 +-
 tests/eventscripts/60.nfs.monitor.112.sh           |    5 +-
 tests/eventscripts/60.nfs.monitor.113.sh           |    5 +-
 tests/eventscripts/60.nfs.monitor.114.sh           |    5 +-
 tests/eventscripts/etc-ctdb/rc.local               |    6 +
 tests/eventscripts/scripts/local.sh                |   45 ++++----
 ..._ctdb_config_check_error_on_unreachable_ctdb.sh |    2 +
 tests/simple/60_recoverd_missing_ip.sh             |   14 ++-
 tools/ctdb.c                                       |    2 +-
 23 files changed, 166 insertions(+), 132 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index ebd448c..764404e 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -1781,21 +1781,25 @@ int ctdb_ctrl_getdbhealth(struct ctdb_context *ctdb,
 /*
   create a database
  */
-int ctdb_ctrl_createdb(struct ctdb_context *ctdb, struct timeval timeout, uint32_t destnode,
-		       TALLOC_CTX *mem_ctx, const char *name, uint32_t tdb_flags)
+int ctdb_ctrl_createdb(struct ctdb_context *ctdb, struct timeval timeout, uint32_t destnode, 
+		       TALLOC_CTX *mem_ctx, const char *name, bool persistent)
 {
 	int ret;
 	int32_t res;
 	TDB_DATA data;
-	bool persistent;
+	uint64_t tdb_flags = 0;
 
 	data.dptr = discard_const(name);
 	data.dsize = strlen(name)+1;
 
-	persistent = (tdb_flags & CTDB_DB_FLAGS_PERSISTENT);
-	ret = ctdb_control(ctdb, destnode, 0,
-			   persistent?CTDB_CONTROL_DB_ATTACH_PERSISTENT:CTDB_CONTROL_DB_ATTACH,
-			   tdb_flags, data,
+	/* Make sure that volatile databases use jenkins hash */
+	if (!persistent) {
+		tdb_flags = TDB_INCOMPATIBLE_HASH;
+	}
+
+	ret = ctdb_control(ctdb, destnode, tdb_flags,
+			   persistent?CTDB_CONTROL_DB_ATTACH_PERSISTENT:CTDB_CONTROL_DB_ATTACH, 
+			   0, data, 
 			   mem_ctx, &data, &res, &timeout, NULL);
 
 	if (ret != 0 || res != 0) {
diff --git a/config/ctdb.sysconfig b/config/ctdb.sysconfig
index 7e775a2..6b7af67 100644
--- a/config/ctdb.sysconfig
+++ b/config/ctdb.sysconfig
@@ -148,6 +148,11 @@ CTDB_RECOVERY_LOCK="/some/place/on/shared/storage"
 # dump no stack traces.
 # CTDB_NFS_DUMP_STUCK_THREADS=5
 
+# Host to use for rpcinfo checks.  Using "localhost" causes
+# /etc/services to be read serveral times by rpcinfo, so this allows
+# optimisation.
+CTDB_RPCINFO_LOCALHOST="127.0.0.1"
+
 # When set to yes, the CTDB node will start in DISABLED mode and not host
 # any public ip addresses. The administrator needs to explicitely enable
 # the node with "ctdb enable"
diff --git a/config/events.d/60.ganesha b/config/events.d/60.ganesha
index 09860d0..70c4cdb 100755
--- a/config/events.d/60.ganesha
+++ b/config/events.d/60.ganesha
@@ -199,7 +199,7 @@ case "$1" in
 	    nfs_check_rpc_service "statd" \
 		-ge 6 "verbose unhealthy" \
 		-eq 4 "verbose restart" \
-		-eq 2 "restart:bs"
+		-eq 2 "restart:b"
 
 	if [ "$CTDB_SKIP_GANESHA_NFSD_CHECK" != "yes" ] ; then
 	    monitor_ganesha_nfsd
diff --git a/config/functions b/config/functions
index eabc940..d9cf127 100755
--- a/config/functions
+++ b/config/functions
@@ -281,34 +281,26 @@ _nfs_check_rpc_common ()
 	    which "rpc.${_prog_name}" >/dev/null 2>&1 || return 0
     esac
 
-    _version=1
-    _rpc_prog="$_prog_name"
-    _restart=""
-    _opts=""
     case "$_prog_name" in
 	nfsd)
 	    _rpc_prog=nfs
 	    _version=3
-	    _restart="echo 'Trying to restart NFS service'"
-	    _restart="${_restart}; startstop_nfs restart"
 	    ;;
 	mountd)
-	    _opts="${MOUNTD_PORT:+ -p }${MOUNTD_PORT}"
+	    _rpc_prog=mountd
+	    _version=1
 	    ;;
 	rquotad)
-	    _opts="${RQUOTAD_PORT:+ -p }${RQUOTAD_PORT}"
+	    _rpc_prog=rquotad
+	    _version=1
 	    ;;
 	lockd)
 	    _rpc_prog=nlockmgr
 	    _version=4
-	    _restart="echo 'Trying to restart lock manager service'"
-	    _restart="${_restart}; startstop_nfslock restart"
 	    ;;
 	statd)
 	    _rpc_prog=status
-	    _opts="${STATD_HOSTNAME:+ -n }${STATD_HOSTNAME}"
-	    _opts="${_opts}${STATD_PORT:+ -p }${STATD_PORT}"
-	    _opts="${_opts}${STATD_OUTGOING_PORT:+ -o }${STATD_OUTGOING_PORT}"
+	    _version=1
 	    ;;
 	*)
 	    echo "Internal error: unknown RPC program \"$_prog_name\"."
@@ -342,34 +334,11 @@ _nfs_check_rpc_action ()
 	    verbose)
 		echo "$ctdb_check_rpc_out"
 		;;
-	    restart|restart:*)
-		# No explicit command specified, construct rpc command.
-		if [ -z "$_restart" ] ; then
-		    _p="rpc.${_prog_name}"
-		    _restart="echo 'Trying to restart $_prog_name [${_p}${_opts}]'"
-		    _restart="${_restart}; killall -q -9 $_p"
-		    _restart="${_restart}; $_p $_opts"
-		fi
-
-		# Process restart flags...
-		_flags="${_action#restart:}"
-		# There may not have been a colon...
-		[ "$_flags" != "$_action" ] || _flags=""
-		# q=quiet - everything to /dev/null
-		if [ "${_flags#*q}" != "$_flags" ] ; then
-		    _restart="{ ${_restart} ; } >/dev/null 2>&1"
-		fi
-		# s=stealthy - last command to /dev/null
-		if [ "${_flags#*s}" != "$_flags" ] ; then
-		    _restart="${_restart} >/dev/null 2>&1"
-		fi
-		# b=background - the whole thing, easy and reliable
-		if [ "${_flags#*b}" != "$_flags" ] ; then
-		    _restart="{ ${_restart} ; } &"
-		fi
-
-		# Do it!
-		eval "${_restart}"
+	    restart)
+		_nfs_restart_rpc_service "$_prog_name"
+		;;
+	    restart:b)
+		_nfs_restart_rpc_service "$_prog_name" true
 		;;
 	    unhealthy)
 		exit 1
@@ -383,6 +352,52 @@ _nfs_check_rpc_action ()
     return 0
 }
 
+_nfs_restart_rpc_service ()
+{
+    _prog_name="$1"
+    _background="${2:-false}"
+
+    if $_background ; then
+	_maybe_background="background_with_logging"
+    else
+	_maybe_background=""
+    fi
+
+    _p="rpc.${_prog_name}"
+
+    case "$_prog_name" in
+	nfsd)
+	    echo "Trying to restart NFS service"
+	    $_maybe_background startstop_nfs restart
+	    ;;
+	mountd)
+	    echo "Trying to restart $_prog_name [${_p}]"
+	    killall -q -9 "$_p"
+	    $_maybe_background $_p ${MOUNTD_PORT:+-p} $MOUNTD_PORT
+	    ;;
+	rquotad)
+	    echo "Trying to restart $_prog_name [${_p}]"
+	    killall -q -9 "$_p"
+	    $_maybe_background $_p ${RQUOTAD_PORT:+-p} $RQUOTAD_PORT
+	    ;;
+	lockd)
+	    echo "Trying to restart lock manager service"
+	    $_maybe_background startstop_nfslock restart
+	    ;;
+	statd)
+	    echo "Trying to restart $_prog_name [${_p}]"
+	    killall -q -9 "$_p"
+	    $_maybe_background $_p \
+		${STATD_HOSTNAME:+-n} $STATD_HOSTNAME \
+		${STATD_PORT:+-p} $STATD_PORT \
+		${STATD_OUTGOING_PORT:+-o} $STATD_OUTGOING_PORT
+	    ;;
+	*)
+	    echo "Internal error: unknown RPC program \"$_prog_name\"."
+	    exit 1
+    esac
+}
+
 ######################################################
 # check that a rpc server is registered with portmap
 # and responding to requests
@@ -393,7 +408,9 @@ ctdb_check_rpc ()
     progname="$1"
     version="$2"
 
-    if ! ctdb_check_rpc_out=$(rpcinfo -u localhost $progname $version 2>&1) ; then
+    _localhost="${CTDB_RPCINFO_LOCALHOST:-127.0.0.1}"
+
+    if ! ctdb_check_rpc_out=$(rpcinfo -u $_localhost $progname $version 2>&1) ; then
 	ctdb_check_rpc_out="ERROR: $progname failed RPC check:
 $ctdb_check_rpc_out"
 	echo "$ctdb_check_rpc_out"
@@ -679,7 +696,9 @@ kill_tcp_connections ()
 
 	_count=0
 	while : ; do
-	    if [ -z "$(get_tcp_connections_for_ip $_ip)" ] ; then
+	    _remaining=$(get_tcp_connections_for_ip $_ip | wc -l)
+
+	    if [ $_remaining -eq 0 ] ; then
 		echo "Killed $_killcount TCP connections to released IP $_ip"
 		return
 	    fi
@@ -690,6 +709,7 @@ kill_tcp_connections ()
 		return
 	    fi
 
+	    echo "Waiting for $_remaining connections to be killed for IP $_ip"
 	    sleep 1
 	done
     }
@@ -861,7 +881,7 @@ startstop_nfslock() {
 			service nfsserver stop > /dev/null 2>&1
 			;;
 		restart)
-			service nfsserver stop
+			service nfsserver stop > /dev/null 2>&1
 			service nfsserver start
 			;;
 		esac
@@ -875,7 +895,7 @@ startstop_nfslock() {
 			service nfslock stop > /dev/null 2>&1
 			;;
 		restart)
-			service nfslock stop
+			service nfslock stop > /dev/null 2>&1
 			service nfslock start
 			;;
 		esac
@@ -1022,12 +1042,7 @@ drop_all_public_ips ()
 }
 
 ########################################################
-# some simple logic for counting events - per eventscript
-# usage: ctdb_counter_init
-#        ctdb_counter_incr
-#        ctdb_check_counter_limit <limit>
-# ctdb_check_counter_limit fails when count >= <limit>
-########################################################
+# Simple counters
 _ctdb_counter_common () {
     _service_name="${1:-${service_name:-${script_name}}}"
     _counter_file="$ctdb_fail_dir/$_service_name"
@@ -1053,9 +1068,19 @@ ctdb_check_counter () {
 
     # unary counting!
     _size=$(stat -c "%s" "$_counter_file" 2>/dev/null || echo 0)
-    if [ $_size $_op $_limit ] ; then
+    _hit=false
+    if [ "$_op" != "%" ] ; then
+	if [ $_size $_op $_limit ] ; then
+	    _hit=true
+	fi
+    else
+	if [ $(($_size $_op $_limit)) -eq 0 ] ; then
+	    _hit=true
+	fi
+    fi
+    if $_hit ; then
 	if [ "$_msg" = "error" ] ; then
-	    echo "ERROR: $_limit consecutive failures for $_service_name, marking node unhealthy"
+	    echo "ERROR: $_size consecutive failures for $_service_name, marking node unhealthy"
 	    exit 1		
 	else
 	    return 1
diff --git a/config/nfs-rpc-checks.d/10.statd.check b/config/nfs-rpc-checks.d/10.statd.check
index dd5e15d..d738a32 100644
--- a/config/nfs-rpc-checks.d/10.statd.check
+++ b/config/nfs-rpc-checks.d/10.statd.check
@@ -1,3 +1,3 @@
 -ge 6 verbose unhealthy
 -eq 4 verbose restart
--eq 2 restart:bs
+-eq 2 restart:b
diff --git a/config/nfs-rpc-checks.d/20.nfsd.check b/config/nfs-rpc-checks.d/20.nfsd.check
index dd5e15d..aa4a2e7 100644
--- a/config/nfs-rpc-checks.d/20.nfsd.check
+++ b/config/nfs-rpc-checks.d/20.nfsd.check
@@ -1,3 +1,2 @@
--ge 6 verbose unhealthy
--eq 4 verbose restart
--eq 2 restart:bs
+%   10 verbose restart:b unhealthy
+-ge  2 verbose unhealthy
diff --git a/config/nfs-rpc-checks.d/30.lockd.check b/config/nfs-rpc-checks.d/30.lockd.check
index 4bda6c3..95ae7b3 100644
--- a/config/nfs-rpc-checks.d/30.lockd.check
+++ b/config/nfs-rpc-checks.d/30.lockd.check
@@ -1,2 +1,2 @@
 -ge 15 verbose restart:b unhealthy
--eq 10 restart:bs
+-eq 10 restart:b
diff --git a/include/ctdb_client.h b/include/ctdb_client.h
index b2ae3bf..8739923 100644
--- a/include/ctdb_client.h
+++ b/include/ctdb_client.h
@@ -289,7 +289,7 @@ int ctdb_ctrl_getdbhealth(struct ctdb_context *ctdb,
 			  uint32_t destnode,
 			  uint32_t dbid, TALLOC_CTX *mem_ctx,
 			  const char **reason);
-int ctdb_ctrl_createdb(struct ctdb_context *ctdb, struct timeval timeout, uint32_t destnode, TALLOC_CTX *mem_ctx, const char *name, uint32_t tdb_flags);
+int ctdb_ctrl_createdb(struct ctdb_context *ctdb, struct timeval timeout, uint32_t destnode, TALLOC_CTX *mem_ctx, const char *name, bool persistent);
 
 int ctdb_ctrl_process_exists(struct ctdb_context *ctdb, uint32_t destnode, pid_t pid);
 
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index cb07339..a1513ee 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -468,7 +468,7 @@ static int create_missing_remote_databases(struct ctdb_context *ctdb, struct ctd
 			}
 			ctdb_ctrl_createdb(ctdb, CONTROL_TIMEOUT(), nodemap->nodes[j].pnn, 
 					   mem_ctx, name,
-					   dbmap->dbs[db].flags);
+					   dbmap->dbs[db].flags & CTDB_DB_FLAGS_PERSISTENT);
 			if (ret != 0) {
 				DEBUG(DEBUG_ERR, (__location__ " Unable to create remote db:%s\n", name));
 				return -1;
@@ -531,7 +531,7 @@ static int create_missing_local_databases(struct ctdb_context *ctdb, struct ctdb
 				return -1;
 			}
 			ctdb_ctrl_createdb(ctdb, CONTROL_TIMEOUT(), pnn, mem_ctx, name, 
-					   remote_dbmap->dbs[db].flags);
+					   remote_dbmap->dbs[db].flags & CTDB_DB_FLAGS_PERSISTENT);
 			if (ret != 0) {
 				DEBUG(DEBUG_ERR, (__location__ " Unable to create local db:%s\n", name));
 				return -1;
diff --git a/tests/eventscripts/10.interface.releaseip.011.sh b/tests/eventscripts/10.interface.releaseip.011.sh
index fbebadf..17b7421 100755
--- a/tests/eventscripts/10.interface.releaseip.011.sh
+++ b/tests/eventscripts/10.interface.releaseip.011.sh
@@ -31,6 +31,9 @@ while read dev ip bits ; do
     ok <<EOF
 Killing TCP connection 10.254.254.1:43210 ${ip}:445
 $out
+Waiting for 1 connections to be killed for IP ${ip}
+Waiting for 1 connections to be killed for IP ${ip}
+Waiting for 1 connections to be killed for IP ${ip}
 Timed out killing tcp connections for IP $ip
 EOF
 
diff --git a/tests/eventscripts/49.winbind.monitor.050.sh b/tests/eventscripts/49.winbind.monitor.050.sh
index d0d55f0..02589b3 100755
--- a/tests/eventscripts/49.winbind.monitor.050.sh
+++ b/tests/eventscripts/49.winbind.monitor.050.sh
@@ -9,9 +9,8 @@ setup_winbind "down"
 export CTDB_SERVICE_AUTOSTARTSTOP="yes"
 export CTDB_MANAGED_SERVICES="foo winbind bar"
 
-ok 'Starting service "winbind" - now managed'
+ok <<EOF
+Starting service "winbind" - now managed
+&Starting winbind: OK
+EOF
 simple_test
-
-# This depends on output in the log file from the above test
-ok 'Starting winbind: OK'
-check_ctdb_logfile
diff --git a/tests/eventscripts/49.winbind.monitor.051.sh b/tests/eventscripts/49.winbind.monitor.051.sh
index 1d68068..fbad928 100755
--- a/tests/eventscripts/49.winbind.monitor.051.sh
+++ b/tests/eventscripts/49.winbind.monitor.051.sh
@@ -10,9 +10,8 @@ export CTDB_SERVICE_AUTOSTARTSTOP="yes"
 export CTDB_MANAGED_SERVICES="foo"
 unset CTDB_MANAGES_WINBIND
 
-ok 'Stopping service "winbind" - no longer managed'
+ok <<EOF
+Stopping service "winbind" - no longer managed
+&Stopping winbind: OK
+EOF
 simple_test
-
-# This depends on output in the log file from the above test
-ok 'Stopping winbind: OK'
-check_ctdb_logfile
diff --git a/tests/eventscripts/50.samba.monitor.050.sh b/tests/eventscripts/50.samba.monitor.050.sh
index db66f66..69530f3 100755
--- a/tests/eventscripts/50.samba.monitor.050.sh
+++ b/tests/eventscripts/50.samba.monitor.050.sh
@@ -9,9 +9,8 @@ setup_samba "down"
 export CTDB_SERVICE_AUTOSTARTSTOP="yes"
 export CTDB_MANAGED_SERVICES="foo samba winbind bar"
 
-ok 'Starting service "samba" - now managed'
+ok <<EOF
+Starting service "samba" - now managed
+&Starting smb: OK
+EOF
 simple_test
-
-# This depends on output in the log file from the above test
-ok 'Starting smb: OK'
-check_ctdb_logfile
diff --git a/tests/eventscripts/50.samba.monitor.051.sh b/tests/eventscripts/50.samba.monitor.051.sh
index 1a9ab22..04c1fce 100755
--- a/tests/eventscripts/50.samba.monitor.051.sh
+++ b/tests/eventscripts/50.samba.monitor.051.sh
@@ -10,9 +10,8 @@ export CTDB_SERVICE_AUTOSTARTSTOP="yes"
 export CTDB_MANAGED_SERVICES="foo"
 unset CTDB_MANAGES_SAMBA
 
-ok 'Stopping service "samba" - no longer managed'
+ok <<EOF
+Stopping service "samba" - no longer managed
+&Stopping smb: OK
+EOF
 simple_test
-
-# This depends on output in the log file from the above test
-ok 'Stopping smb: OK'
-check_ctdb_logfile
diff --git a/tests/eventscripts/60.ganesha.monitor.141.sh b/tests/eventscripts/60.ganesha.monitor.141.sh
index 656a530..9cd82f8 100755
--- a/tests/eventscripts/60.ganesha.monitor.141.sh
+++ b/tests/eventscripts/60.ganesha.monitor.141.sh
@@ -13,7 +13,7 @@ ok_null
 simple_test || exit $?
 
 ok<<EOF
-Trying to restart statd [rpc.statd -n cluster1 -H /etc/ctdb/statd-callout ]
+Trying to restart statd [rpc.statd]
 EOF
 simple_test || exit $?
 
@@ -24,7 +24,7 @@ ok<<EOF
 ERROR: status failed RPC check:
 rpcinfo: RPC: Program not registered
 program status version 1 is not available
-Trying to restart statd [rpc.statd -n cluster1 -H /etc/ctdb/statd-callout ]
+Trying to restart statd [rpc.statd]
 EOF
 simple_test || exit $?
 
diff --git a/tests/eventscripts/60.nfs.monitor.112.sh b/tests/eventscripts/60.nfs.monitor.112.sh
index c5c39b2..49ee335 100755
--- a/tests/eventscripts/60.nfs.monitor.112.sh
+++ b/tests/eventscripts/60.nfs.monitor.112.sh
@@ -9,7 +9,4 @@ define_test "knfsd down, 6 iterations"
 setup_nfs
 rpc_services_down "nfs"
 
-iterate_test 6 'ok_null' \
-    2 'rpc_set_service_failure_response "nfsd"' \
-    4 'rpc_set_service_failure_response "nfsd"' \
-    6 'rpc_set_service_failure_response "nfsd"'
+iterate_test 10 'rpc_set_service_failure_response "nfsd"'
diff --git a/tests/eventscripts/60.nfs.monitor.113.sh b/tests/eventscripts/60.nfs.monitor.113.sh
index caa4989..505df1b 100755
--- a/tests/eventscripts/60.nfs.monitor.113.sh
+++ b/tests/eventscripts/60.nfs.monitor.113.sh
@@ -12,7 +12,4 @@ rpc_services_down "nfs"
 CTDB_NFS_DUMP_STUCK_THREADS=5
 FAKE_NFSD_THREAD_PIDS=""
 
-iterate_test 6 'ok_null' \
-    2 'rpc_set_service_failure_response "nfsd"' \
-    4 'rpc_set_service_failure_response "nfsd"' \
-    6 'rpc_set_service_failure_response "nfsd"'
+iterate_test 10 'rpc_set_service_failure_response "nfsd"'
diff --git a/tests/eventscripts/60.nfs.monitor.114.sh b/tests/eventscripts/60.nfs.monitor.114.sh
index 8279395..496f5e7 100755
--- a/tests/eventscripts/60.nfs.monitor.114.sh
+++ b/tests/eventscripts/60.nfs.monitor.114.sh
@@ -12,7 +12,4 @@ rpc_services_down "nfs"
 CTDB_NFS_DUMP_STUCK_THREADS=5
 FAKE_NFSD_THREAD_PIDS="1001 1002 1003"
 
-iterate_test 6 'ok_null' \
-    2 'rpc_set_service_failure_response "nfsd"' \
-    4 'rpc_set_service_failure_response "nfsd"' \
-    6 'rpc_set_service_failure_response "nfsd"'
+iterate_test 10 'rpc_set_service_failure_response "nfsd"'
diff --git a/tests/eventscripts/etc-ctdb/rc.local b/tests/eventscripts/etc-ctdb/rc.local
index 9cd4d55..b11c7ec 100755
--- a/tests/eventscripts/etc-ctdb/rc.local
+++ b/tests/eventscripts/etc-ctdb/rc.local


-- 
CTDB repository


More information about the samba-cvs mailing list