[SCM] Samba Shared Repository - branch master updated

Martin Schwenke martins at samba.org
Tue Sep 12 14:15:02 UTC 2017


The branch, master has been updated
       via  8d9a050 ctdb-tests: Add 31.clamd eventscript unit tests
       via  1eec073 ctdb-tests: Enhance ss stub to check for listening Unix domain sockets
       via  1aff2f8 ctdb-scripts: Switch ctdb_check_unix_socket() to use ss
       via  ff635f9 ctdb-scripts: Clean up ctdb_check_unix_socket()
       via  bff8d41 ctdb-daemon: Don't release all IPs before "startup" event
       via  6e58891 ctdb-recoverd: Abort recovery/takeover if recmaster changes
       via  f57d379 ctdb-daemon: GET_DB_SEQNUM should read database conditionally
       via  5d2f267 ctdb-daemon: Add a function to check if db access is allowed
       via  96aef23 ctdb-tests: Fix ctdb test binary name in path testing
       via  ff75f08 ctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd
      from  3ff1b83 s3: vfs: catia: compression get/set must act only on base file, and must cope with fsp==NULL.

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit 8d9a050a844822a0415cf4488c9ee687d613dd87
Author: Martin Schwenke <martin at meltin.net>
Date:   Sat Sep 2 20:59:32 2017 +1000

    ctdb-tests: Add 31.clamd eventscript unit tests
    
    These test that ctdb_check_unix_socket() is working.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>
    
    Autobuild-User(master): Martin Schwenke <martins at samba.org>
    Autobuild-Date(master): Tue Sep 12 16:14:12 CEST 2017 on sn-devel-144

commit 1eec073e8a997ff43b710189461ccef28872b039
Author: Martin Schwenke <martin at meltin.net>
Date:   Sat Sep 2 20:57:56 2017 +1000

    ctdb-tests: Enhance ss stub to check for listening Unix domain sockets
    
    Generalise command-line parsing, taking hints from old netstat stub,
    and use FAKE_NETSTAT_UNIX_LISTEN to specify listening Unix domain
    sockets.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit 1aff2f87469e11b30a45d3040b251335285e7bb3
Author: Martin Schwenke <martin at meltin.net>
Date:   Sat Mar 18 21:55:04 2017 +1100

    ctdb-scripts: Switch ctdb_check_unix_socket() to use ss
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit ff635f90edd8e012ac2838303cb0285ab09b8db6
Author: Martin Schwenke <martin at meltin.net>
Date:   Sat Mar 18 21:53:06 2017 +1100

    ctdb-scripts: Clean up ctdb_check_unix_socket()
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit bff8d410f93a0e50d16aff4896bea2d48e46f33a
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Sep 5 13:38:18 2016 +1000

    ctdb-daemon: Don't release all IPs before "startup" event
    
    This doesn't belong in the monitoring/startup code and it is already
    done in the 10.interface "init" event.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit 6e588913dd55f4b45bc7b60b46eed505c11b529c
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Fri Sep 8 11:24:27 2017 +1000

    ctdb-recoverd: Abort recovery/takeover if recmaster changes
    
    Recovery and takeover are run via helper from recovery daemon.  While the
    helpers are running, it's possible for the current node to lose election.
    If that happens, abort the currently running recovery/takeover helper.
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit f57d379446c551bca5906247c622e857c77089b0
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Thu Sep 7 17:21:03 2017 +1000

    ctdb-daemon: GET_DB_SEQNUM should read database conditionally
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021
    
    Once the recovery starts and databases are frozen, then all the record
    access is postponed till the recovery is complete except reading the
    database sequence number.  Database access for reading sequence number
    is done via a control which does not check if the databases are frozen
    or not.
    
    If the database is frozen and if the freeze transaction is not started
    (this can happen when a node is inactive, or during recovery when the
    database is frozen but the transaction has not yet started), then trying
    to read sequence number will cause ctdb daemon to deadlock.
    
    Before reading the sequence number, check if the database access is
    allowed.
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 5d2f2677de65a0fd6683bb759d80ebced604fa6b
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Thu Sep 7 17:18:18 2017 +1000

    ctdb-daemon: Add a function to check if db access is allowed
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13021
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 96aef2371c6c1e0c6bd13874a71583eb9609959b
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Sep 5 13:52:47 2017 +1000

    ctdb-tests: Fix ctdb test binary name in path testing
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit ff75f0836aef56476ec45a3bc8f3ca22c118e3a4
Author: Martin Schwenke <martin at meltin.net>
Date:   Tue Sep 12 11:51:19 2017 +1000

    ctdb-tests: Wait up to 30 seconds for process to be registered in ctdbd
    
    BUG: https://bugzilla.samba.org/show_bug.cgi?id=13012
    
    This avoids a potential race where the client is not properly
    registered before "ctdb process-exists" is called.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

-----------------------------------------------------------------------

Summary of changes:
 ctdb/config/functions                           |  22 +--
 ctdb/include/ctdb_private.h                     |   1 +
 ctdb/server/ctdb_freeze.c                       |  18 +++
 ctdb/server/ctdb_monitor.c                      |   3 -
 ctdb/server/ctdb_persistent.c                   |   5 +
 ctdb/server/ctdb_recoverd.c                     |   9 ++
 ctdb/tests/eventscripts/31.clamd.monitor.001.sh |  13 ++
 ctdb/tests/eventscripts/31.clamd.monitor.002.sh |  15 ++
 ctdb/tests/eventscripts/31.clamd.monitor.003.sh |  15 ++
 ctdb/tests/eventscripts/scripts/local.sh        |   7 +
 ctdb/tests/eventscripts/stubs/ss                | 177 +++++++++++++++++++-----
 ctdb/tests/scripts/test_wrap                    |   2 +-
 ctdb/tests/simple/07_ctdb_process_exists.sh     |   4 +-
 13 files changed, 241 insertions(+), 50 deletions(-)
 create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.001.sh
 create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.002.sh
 create mode 100755 ctdb/tests/eventscripts/31.clamd.monitor.003.sh


Changeset truncated at 500 lines:

diff --git a/ctdb/config/functions b/ctdb/config/functions
index 1131229..3805d48 100755
--- a/ctdb/config/functions
+++ b/ctdb/config/functions
@@ -376,16 +376,22 @@ ctdb_check_tcp_ports()
 
 ######################################################
 # check a unix socket
-# usage: ctdb_check_unix_socket SERVICE_NAME <socket_path>
+# usage: ctdb_check_unix_socket SOCKPATH
 ######################################################
-ctdb_check_unix_socket() {
-    socket_path="$1"
-    [ -z "$socket_path" ] && return
+ctdb_check_unix_socket()
+{
+	_sockpath="$1"
 
-    if ! netstat --unix -a -n | grep -q "^unix.*LISTEN.*${socket_path}$"; then
-        echo "ERROR: $service_name socket $socket_path not found"
-        return 1
-    fi
+	if [ -z "$_sockpath" ] ; then
+		echo "ERROR: ctdb_check_unix_socket() requires socket path"
+		return 1
+	fi
+
+	_out=$(ss -l -x "src ${_sockpath}" | tail -n +2)
+	if [ -z "$_out" ] ; then
+		echo "ERROR: ${service_name} not listening on ${_sockpath}"
+		return 1
+	fi
 }
 
 ################################################
diff --git a/ctdb/include/ctdb_private.h b/ctdb/include/ctdb_private.h
index 5b95b60..f5966cd 100644
--- a/ctdb/include/ctdb_private.h
+++ b/ctdb/include/ctdb_private.h
@@ -627,6 +627,7 @@ int32_t ctdb_control_wipe_database(struct ctdb_context *ctdb, TDB_DATA indata);
 
 bool ctdb_db_frozen(struct ctdb_db_context *ctdb_db);
 bool ctdb_db_all_frozen(struct ctdb_context *ctdb);
+bool ctdb_db_allow_access(struct ctdb_db_context *ctdb_db);
 
 /* from server/ctdb_keepalive.c */
 
diff --git a/ctdb/server/ctdb_freeze.c b/ctdb/server/ctdb_freeze.c
index d92f707..c41fc7d 100644
--- a/ctdb/server/ctdb_freeze.c
+++ b/ctdb/server/ctdb_freeze.c
@@ -874,3 +874,21 @@ bool ctdb_db_all_frozen(struct ctdb_context *ctdb)
 	}
 	return true;
 }
+
+bool ctdb_db_allow_access(struct ctdb_db_context *ctdb_db)
+{
+	if (ctdb_db->freeze_mode == CTDB_FREEZE_NONE) {
+		/* If database is not frozen, then allow access. */
+		return true;
+	} else if (ctdb_db->freeze_transaction_started) {
+		/* If database is frozen, allow access only if the
+		 * transaction is started.  This is required during
+		 * recovery.
+		 *
+		 * If a node is inactive, then transaction is not started.
+		 */
+		return true;
+	}
+
+	return false;
+}
diff --git a/ctdb/server/ctdb_monitor.c b/ctdb/server/ctdb_monitor.c
index 738acb1..223220c 100644
--- a/ctdb/server/ctdb_monitor.c
+++ b/ctdb/server/ctdb_monitor.c
@@ -270,9 +270,6 @@ static void ctdb_run_startup(struct tevent_context *ev,
 		return;
 	}
 
-	/* release any IPs we hold from previous runs of the daemon */
-	ctdb_release_all_ips(ctdb);
-
 	DEBUG(DEBUG_NOTICE,("Running the \"startup\" event.\n"));
 	ret = ctdb_event_script_callback(ctdb,
 					 ctdb->monitor->monitor_context,
diff --git a/ctdb/server/ctdb_persistent.c b/ctdb/server/ctdb_persistent.c
index 1811ae8..fc28655 100644
--- a/ctdb/server/ctdb_persistent.c
+++ b/ctdb/server/ctdb_persistent.c
@@ -344,6 +344,11 @@ static int32_t ctdb_get_db_seqnum(struct ctdb_context *ctdb,
 		goto done;
 	}
 
+	if (! ctdb_db_allow_access(ctdb_db)) {
+		ret = -1;
+		goto done;
+	}
+
 	key.dptr = (uint8_t *)discard_const(keyname);
 	key.dsize = strlen(keyname) + 1;
 
diff --git a/ctdb/server/ctdb_recoverd.c b/ctdb/server/ctdb_recoverd.c
index 9488bc2..2b94fed 100644
--- a/ctdb/server/ctdb_recoverd.c
+++ b/ctdb/server/ctdb_recoverd.c
@@ -1026,6 +1026,7 @@ static int helper_run(struct ctdb_recoverd *rec, TALLOC_CTX *mem_ctx,
 	struct tevent_fd *fde;
 	const char **args;
 	int nargs, ret;
+	uint32_t recmaster = rec->recmaster;
 
 	state = talloc_zero(mem_ctx, struct helper_state);
 	if (state == NULL) {
@@ -1085,6 +1086,14 @@ static int helper_run(struct ctdb_recoverd *rec, TALLOC_CTX *mem_ctx,
 
 	while (!state->done) {
 		tevent_loop_once(rec->ctdb->ev);
+
+		/* If recmaster changes, we have lost election */
+		if (recmaster != rec->recmaster) {
+			D_ERR("Recmaster changed to %u, aborting %s\n",
+			      rec->recmaster, type);
+			state->result = 1;
+			break;
+		}
 	}
 
 	close(state->fd[0]);
diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.001.sh b/ctdb/tests/eventscripts/31.clamd.monitor.001.sh
new file mode 100755
index 0000000..bd97057
--- /dev/null
+++ b/ctdb/tests/eventscripts/31.clamd.monitor.001.sh
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+. "${TEST_SCRIPTS_DIR}/unit.sh"
+
+define_test "Not managed, clamd not listening"
+
+export CTDB_MANAGES_CLAMD=no
+export CTDB_CLAMD_SOCKET="/var/run/clamd.sock"
+
+setup_generic
+
+ok_null
+simple_test
diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.002.sh b/ctdb/tests/eventscripts/31.clamd.monitor.002.sh
new file mode 100755
index 0000000..bfd7634
--- /dev/null
+++ b/ctdb/tests/eventscripts/31.clamd.monitor.002.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+. "${TEST_SCRIPTS_DIR}/unit.sh"
+
+define_test "Managed, clamd not listening"
+
+export CTDB_MANAGES_CLAMD=yes
+export CTDB_CLAMD_SOCKET="/var/run/clamd.sock"
+
+setup_generic
+
+required_result 1 <<EOF
+ERROR: clamd not listening on $CTDB_CLAMD_SOCKET
+EOF
+simple_test
diff --git a/ctdb/tests/eventscripts/31.clamd.monitor.003.sh b/ctdb/tests/eventscripts/31.clamd.monitor.003.sh
new file mode 100755
index 0000000..e57f025
--- /dev/null
+++ b/ctdb/tests/eventscripts/31.clamd.monitor.003.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+. "${TEST_SCRIPTS_DIR}/unit.sh"
+
+define_test "Managed, clamd listening"
+
+export CTDB_MANAGES_CLAMD=yes
+export CTDB_CLAMD_SOCKET="/var/run/clamd.sock"
+
+setup_generic
+
+unix_socket_listening "$CTDB_CLAMD_SOCKET"
+
+ok_null
+simple_test
diff --git a/ctdb/tests/eventscripts/scripts/local.sh b/ctdb/tests/eventscripts/scripts/local.sh
index 1d51232..83877c3 100644
--- a/ctdb/tests/eventscripts/scripts/local.sh
+++ b/ctdb/tests/eventscripts/scripts/local.sh
@@ -135,6 +135,13 @@ tcp_port_down ()
     done
 }
 
+unix_socket_listening ()
+{
+	_s="$1"
+
+	FAKE_NETSTAT_UNIX_LISTEN="${FAKE_NETSTAT_UNIX_LISTEN} ${_s}"
+}
+
 _tcp_connections ()
 {
 	_count="$1"
diff --git a/ctdb/tests/eventscripts/stubs/ss b/ctdb/tests/eventscripts/stubs/ss
index 1a3db0d..bb291b2 100755
--- a/ctdb/tests/eventscripts/stubs/ss
+++ b/ctdb/tests/eventscripts/stubs/ss
@@ -4,23 +4,73 @@ prog="ss"
 
 usage ()
 {
-    cat >&2 <<EOF
-Usage: $prog -tn state established [ '(' ip-filter ')' ] [ '(' port-filter ')' ]
+	cat >&2 <<EOF
+Usage: $prog { -t|--tcp | -x|--unix } [options] [ FILTER ]
 
 A fake ss stub that prints items depending on the variables
-FAKE_NETSTAT_TCP_ESTABLISHED and FAKE_NETSTAT_TCP_ESTABLISHED_FILE.
+FAKE_NETSTAT_TCP_ESTABLISHED, FAKE_TCP_LISTEN,
+FAKE_NETSTAT_UNIX_LISTEN, depending on command-line options.
 
-Note that "-tn state established" must be given.
+Note that -n is ignored.
 
 EOF
-    exit 1
+	exit 1
 }
 
-if [ "$1" != "-tn" -o "$2" != "state" -o "$3" != "established" ] ; then
-    usage
-fi
+not_supported ()
+{
+	echo "Options not supported in stub: $*" >&2
+	usage
+}
 
-shift 3
+############################################################
+
+#
+parse_filter ()
+{
+	# Very limited implementation:
+	# We only expect to find || inside parentheses
+	# We don't expect to see && - it is implied by juxtaposition
+	# Operator for port comparison is ignored and assumed to be ==
+
+	# Build lists of source ports and source IP addresses where
+	# each entry is surrounded by '|' characters.  These lists can
+	# be easily "searched" using the POSIX prefix and suffix
+	# removal operators.
+	in_parens=false
+	sports="|"
+	srcs="|"
+
+	while [ -n "$1" ] ; do
+		case "$1" in
+		\()
+			in_parens=true
+			shift
+			;;
+		\))
+			in_parens=false
+			shift
+			;;
+		\|\|)
+			if ! $in_parens ; then
+				not_supported "|| in parentheses"
+			fi
+			shift
+			;;
+		sport)
+			p="${3#:}" ; sports="${sports}${p}|"
+			shift 3
+			;;
+		src)
+			ip="${2#\[}" ; ip="${ip%\]}" ; srcs="${srcs}${ip}|"
+			shift 2
+			;;
+		*)
+			usage
+			;;
+		esac
+	done
+}
 
 # Check if socket has matches in both ok_ips and ok_ports
 filter_socket ()
@@ -44,31 +94,11 @@ filter_socket ()
 
 ss_tcp_established ()
 {
-    echo "Recv-Q Send-Q Local Address:Port Peer Address:Port"
-
-    # Very limited implementation:
-    # We only expect to find || inside parentheses
-    # We don't expect to see && - it is implied by juxtaposition
-    # Operator for port comparison is ignored and assumed to be ==
-
-    # Build lists of source ports and source IP addresses where each
-    # entry is surrounded by '|' characters.  These lists can be
-    # easily "searched" using the POSIX prefix and suffix removal
-    # operators.
-    in_parens=false
-    sports="|"
-    srcs="|"
-
-    while [ -n "$1" ] ; do
-	case "$1" in
-	    \() in_parens=true ; shift ;;
-	    \)) in_parens=false ; shift ;;
-	    \|\|) if ! $in_parens ; then usage ; fi ; shift ;;
-	    sport) p="${3#:}" ; sports="${sports}${p}|" ; shift 3 ;;
-	    src) ip="${2#\[}" ; ip="${ip%\]}" ; srcs="${srcs}${ip}|" ; shift 2 ;;
-	    *) usage ;;
-	esac
-    done
+    if $header ; then
+	echo "Recv-Q Send-Q Local Address:Port Peer Address:Port"
+    fi
+
+    parse_filter $*
 
     for i in $FAKE_NETSTAT_TCP_ESTABLISHED ; do
 	src="${i%|*}"
@@ -88,5 +118,80 @@ ss_tcp_established ()
     done <"$FAKE_NETSTAT_TCP_ESTABLISHED_FILE"
 }
 
-# Yes, lose the quoting so we can do a hacky parsing job
-ss_tcp_established $*
+############################################################
+
+unix_listen ()
+{
+	if $header ; then
+		cat <<EOF
+Netid State Recv-Q Send-Q   Local Address:Port   Peer Address:Port"
+EOF
+	fi
+
+	parse_filter $*
+
+	_n=12345
+	for _s in $FAKE_NETSTAT_UNIX_LISTEN ; do
+		# ss matches Unix domain sockets as either src or
+		# sport.
+		if filter_socket "$srcs" "$sports" "${_s}:" || \
+				filter_socket "$srcs" "$sports" ":${_s}" ; then
+			printf "u_str LISTEN 0 128   %s %d   * 0\n" "$_s" "$_n"
+			_n=$((_n + 1))
+		fi
+	done
+}
+
+############################################################
+
+# Defaults.
+tcp=false
+unix=false
+all=false
+listen=false
+header=true
+
+orig="$*"
+temp=$(getopt -n "$prog" -o "txnalHh" -l tcp -l unix -l help -- "$@")
+[ $? -eq 0 ] || usage
+
+eval set -- "$temp"
+
+while true ; do
+	case "$1" in
+	--tcp|-t) tcp=true ; shift ;;
+	--unix|-x) unix=true ; shift ;;
+	-l) listen=true ; shift ;;
+	-a) all=true ; shift ;;
+	-H) header=false ; shift ;;
+	-n) shift ;;
+	--) shift ; break ;;
+	-h|--help|*) usage ;;
+	esac
+done
+
+$tcp || $unix || not_supported "$*"
+
+if $tcp ; then
+	if [ "$1" != "state" -o "$2" != "established" ] || $listen ; then
+		usage
+	fi
+
+	shift 2
+
+	# Yes, lose the quoting so we can do a hacky parsing job
+	ss_tcp_established $*
+
+	exit
+fi
+
+if $unix ; then
+	if ! $listen ; then
+		not_supported "$orig"
+	fi
+
+	# Yes, lose the quoting so we can do a hacky parsing job
+	unix_listen $*
+
+	exit
+fi
diff --git a/ctdb/tests/scripts/test_wrap b/ctdb/tests/scripts/test_wrap
index 176310e..3db3180 100755
--- a/ctdb/tests/scripts/test_wrap
+++ b/ctdb/tests/scripts/test_wrap
@@ -10,7 +10,7 @@ TEST_SCRIPTS_DIR=$(dirname $0)
 # We need the test binaries (i.e. tests/bin/) to be in $PATH.  If they
 # aren't already in $PATH then we know that tests/bin/ sits alongside
 # tests/scripts/.
-f="ctdb_bench"
+f="fetch_ring"
 if [ ! $(which $f >/dev/null 2>&1) ] ; then
     d=$(dirname "$TEST_SCRIPTS_DIR")/bin
     [ -x "$d/$f" ] && PATH="$d:$PATH"
diff --git a/ctdb/tests/simple/07_ctdb_process_exists.sh b/ctdb/tests/simple/07_ctdb_process_exists.sh
index c44924b..f24e93a 100755
--- a/ctdb/tests/simple/07_ctdb_process_exists.sh
+++ b/ctdb/tests/simple/07_ctdb_process_exists.sh
@@ -52,9 +52,9 @@ cleanup ()
 
 ctdb_test_exit_hook_add cleanup
 
-echo "Checking for PID $client_pid on node $test_node"
+echo "Waiting until PID $client_pid is registered on node $test_node"
 status=0
-try_command_on_node $test_node \
+wait_until 30 try_command_on_node $test_node \
 	"$CTDB process-exists ${client_pid}" || status=$?
 echo "$out"
 


-- 
Samba Shared Repository



More information about the samba-cvs mailing list