[SCM] Samba Shared Repository - branch master updated

Amitay Isaacs amitay at samba.org
Tue Dec 18 09:14:02 UTC 2018


The branch, master has been updated
       via  944c92a15db ctdb-daemon: Modernise debug during record deletion for vacuuming
       via  cdca0d7e78a ctdb-daemon Add extra debug during record deletion for vacuuming
       via  2e3ad8c20d2 ctdb-tests: Minimise chances of test interfering with itself
       via  f1b594dce1c ctdb-daemon: Do not force full vacuum on first vacuuming run
       via  9bdd6814e4c ctdb-packaging: Update library versions to upstream versions
       via  59e244c9d04 ctdb-packaging: Match configure command as per spec file
       via  4443124fe8d ctdb-packaging: Call waf with python wrapper
       via  9912709eca2 ctdb-build: Use open() instead of file() for python3
       via  1e061ff1e3f ctdb-tool: Avoid data uninitialized warnings
      from  1ed91f0e102 ctdb-tests: Do not force TEST_VAR_DIR to be absolute

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit 944c92a15dbbcaaea1cd4e63bf2a109a33126437
Author: Martin Schwenke <martin at meltin.net>
Date:   Wed Oct 24 12:29:54 2018 +1100

    ctdb-daemon: Modernise debug during record deletion for vacuuming
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>
    
    Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
    Autobuild-Date(master): Tue Dec 18 10:13:50 CET 2018 on sn-devel-144

commit cdca0d7e78a4cad797ff457f860a37de78bcdb44
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Oct 15 21:21:25 2018 +1100

    ctdb-daemon Add extra debug during record deletion for vacuuming
    
    It isn't currently possible to distinguish these 2 cases.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit 2e3ad8c20d2964fba14d0d5b0b36c334ac22caa5
Author: Martin Schwenke <martin at meltin.net>
Date:   Tue Dec 18 14:31:24 2018 +1100

    ctdb-tests: Minimise chances of test interfering with itself
    
    Checking that the database contains 0 records cause a traverse.  This
    may take a lock and cause vacuuming to fail (or be deferred for a
    particular record/chain).  Minimise the chance of this happening by
    only checking for 0 records every 10 seconds.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit f1b594dce1c081924b16c52c8c50c984a16df098
Author: Martin Schwenke <martin at meltin.net>
Date:   Mon Oct 22 21:40:22 2018 +1100

    ctdb-daemon: Do not force full vacuum on first vacuuming run
    
    When the number of fast path vacuuming runs is 0 then a full vacuuming
    run is done.  This means the first one is a full run, which is almost
    certainly not what is intended.
    
    Combine the 2 conditionals to only flag a full vacuuming run when the
    count exceeds the configured limit.  This means that the
    full_vacuum_run flag is set in both parent and child, but this is
    harmless... and is better than getting it wrong.
    
    Also tweak the comparison to be less-than-or-equal, since the zeroth
    run needs to be counted.
    
    Signed-off-by: Martin Schwenke <martin at meltin.net>
    Reviewed-by: Amitay Isaacs <amitay at gmail.com>

commit 9bdd6814e4c64f3be04daa7e8739e141d2d664b7
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Dec 18 13:37:40 2018 +1100

    ctdb-packaging: Update library versions to upstream versions
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 59e244c9d0402062ae77c882e2af28d00da44449
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Dec 18 11:01:35 2018 +1100

    ctdb-packaging: Match configure command as per spec file
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 4443124fe8d7c12dfab62f9962939d14cbed1291
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Dec 18 11:03:51 2018 +1100

    ctdb-packaging: Call waf with python wrapper
    
    This allows to build packages even when python3 is not available by
    setting PYTHON variable.
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 9912709eca29d21cf56f342079db6b5cc02bbea2
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Dec 18 10:54:50 2018 +1100

    ctdb-build: Use open() instead of file() for python3
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

commit 1e061ff1e3fcfcd020df1a35c1c0697c114cb209
Author: Amitay Isaacs <amitay at gmail.com>
Date:   Tue Dec 18 11:33:30 2018 +1100

    ctdb-tool: Avoid data uninitialized warnings
    
    ../../tools/ctdb.c: In function 'str_to_data':
    ../../tools/ctdb.c:624: warning: 'data.dsize' may be used uninitialized in this function
    ../../tools/ctdb.c:624: warning: 'data.dptr' may be used uninitialized in this function
    
    Signed-off-by: Amitay Isaacs <amitay at gmail.com>
    Reviewed-by: Martin Schwenke <martin at meltin.net>

-----------------------------------------------------------------------

Summary of changes:
 ctdb/configure.rpm                                 |  4 +++-
 ctdb/packaging/RPM/ctdb.spec.in                    |  7 ++++---
 ctdb/server/ctdb_recover.c                         | 23 ++++++++++++----------
 ctdb/server/ctdb_vacuum.c                          | 13 ++++++------
 ctdb/tests/simple/69_recovery_resurrect_deleted.sh |  5 +++--
 ctdb/tools/ctdb.c                                  |  5 ++++-
 ctdb/wscript                                       |  2 +-
 7 files changed, 34 insertions(+), 25 deletions(-)


Changeset truncated at 500 lines:

diff --git a/ctdb/configure.rpm b/ctdb/configure.rpm
index 32e6673d310..e8ec3b24459 100755
--- a/ctdb/configure.rpm
+++ b/ctdb/configure.rpm
@@ -9,9 +9,11 @@ fi
 CFLAGS="-Wall -g -D_GNU_SOURCE" ./configure \
 	--builtin-libraries=replace,popt \
 	--bundled-libraries=!talloc,!tevent,!tdb \
-	--minimum-library-version=talloc:2.0.8,tdb:1.2.11,tevent:0.9.16 \
+	--minimum-library-version=talloc:2.1.14,tdb:1.3.17,tevent:0.9.37 \
 	--prefix=/usr \
+	--includedir=/usr/include/ctdb \
 	--libdir=${_libdir} \
+	--libexecdir=/usr/libexec \
 	--sysconfdir=/etc \
 	--mandir=/usr/man \
 	--localstatedir=/var \
diff --git a/ctdb/packaging/RPM/ctdb.spec.in b/ctdb/packaging/RPM/ctdb.spec.in
index 537f158b2d1..d25735e06e0 100644
--- a/ctdb/packaging/RPM/ctdb.spec.in
+++ b/ctdb/packaging/RPM/ctdb.spec.in
@@ -87,7 +87,8 @@ fi
 
 export CC
 
-CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" ./buildtools/bin/waf configure \
+CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" \
+$PYTHON ./buildtools/bin/waf configure \
 	--builtin-libraries=replace,popt \
 	--bundled-libraries=!talloc,!tevent,!tdb \
 	--minimum-library-version=talloc:%libtalloc_version,tdb:%libtdb_version,tevent:%libtevent_version \
@@ -102,7 +103,7 @@ CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" ./buildtools/bin/waf configure \
 	--mandir=%{_mandir} \
 	--localstatedir=%{_localstatedir}
 
-./buildtools/bin/waf build
+$PYTHON ./buildtools/bin/waf build
 
 %install
 # Clean up in case there is trash left from a previous build
@@ -111,7 +112,7 @@ rm -rf $RPM_BUILD_ROOT
 # Create the target build directory hierarchy
 mkdir -p $RPM_BUILD_ROOT%{_sysconfdir}/sudoers.d
 
-DESTDIR=$RPM_BUILD_ROOT ./buildtools/bin/waf install
+DESTDIR=$RPM_BUILD_ROOT $PYTHON ./buildtools/bin/waf install
 
 install -m644 config/ctdb.conf $RPM_BUILD_ROOT%{_sysconfdir}/ctdb
 install -m644 config/ctdb.tunables $RPM_BUILD_ROOT%{_sysconfdir}/ctdb
diff --git a/ctdb/server/ctdb_recover.c b/ctdb/server/ctdb_recover.c
index f05052e8466..cfe77f643a6 100644
--- a/ctdb/server/ctdb_recover.c
+++ b/ctdb/server/ctdb_recover.c
@@ -956,12 +956,12 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 	data.dptr = &rec->data[rec->keylen];
 
 	if (ctdb_lmaster(ctdb, &key) == ctdb->pnn) {
-		DEBUG(DEBUG_INFO,(__location__ " Called delete on record where we are lmaster\n"));
+		DBG_INFO("Called delete on record where we are lmaster\n");
 		return -1;
 	}
 
 	if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
-		DEBUG(DEBUG_ERR,(__location__ " Bad record size\n"));
+		DBG_ERR("Bad record size\n");
 		return -1;
 	}
 
@@ -969,6 +969,7 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 
 	/* use a non-blocking lock */
 	if (tdb_chainlock_nonblock(ctdb_db->ltdb->tdb, key) != 0) {
+		DBG_INFO("Failed to get non-blocking chain lock\n");
 		return -1;
 	}
 
@@ -981,10 +982,10 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 	if (data2.dsize < sizeof(struct ctdb_ltdb_header)) {
 		if (tdb_lock_nonblock(ctdb_db->ltdb->tdb, -1, F_WRLCK) == 0) {
 			if (tdb_delete(ctdb_db->ltdb->tdb, key) != 0) {
-				DEBUG(DEBUG_CRIT,(__location__ " Failed to delete corrupt record\n"));
+				DBG_ERR("Failed to delete corrupt record\n");
 			}
 			tdb_unlock(ctdb_db->ltdb->tdb, -1, F_WRLCK);
-			DEBUG(DEBUG_CRIT,(__location__ " Deleted corrupt record\n"));
+			DBG_ERR("Deleted corrupt record\n");
 		}
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
 		free(data2.dptr);
@@ -995,8 +996,9 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 
 	if (hdr2->rsn > hdr->rsn) {
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
-		DEBUG(DEBUG_INFO,(__location__ " Skipping record with rsn=%llu - called with rsn=%llu\n",
-			 (unsigned long long)hdr2->rsn, (unsigned long long)hdr->rsn));
+		DBG_INFO("Skipping record with rsn=%llu - called with rsn=%llu\n",
+			 (unsigned long long)hdr2->rsn,
+			 (unsigned long long)hdr->rsn);
 		free(data2.dptr);
 		return -1;
 	}
@@ -1004,26 +1006,27 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 	/* do not allow deleting record that have readonly flags set. */
 	if (hdr->flags & CTDB_REC_RO_FLAGS) {
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
-		DEBUG(DEBUG_INFO,(__location__ " Skipping record with readonly flags set\n"));
+		DBG_INFO("Skipping record with readonly flags set\n");
 		free(data2.dptr);
 		return -1;
 	}
 	if (hdr2->flags & CTDB_REC_RO_FLAGS) {
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
-		DEBUG(DEBUG_INFO,(__location__ " Skipping record with readonly flags set\n"));
+		DBG_INFO("Skipping record with readonly flags set locally\n");
 		free(data2.dptr);
 		return -1;
 	}
 
 	if (hdr2->dmaster == ctdb->pnn) {
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
-		DEBUG(DEBUG_INFO,(__location__ " Attempted delete record where we are the dmaster\n"));
+		DBG_INFO("Attempted delete record where we are the dmaster\n");
 		free(data2.dptr);
 		return -1;
 	}
 
 	if (tdb_lock_nonblock(ctdb_db->ltdb->tdb, -1, F_WRLCK) != 0) {
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
+		DBG_INFO("Failed to get non-blocking freelist lock\n");
 		free(data2.dptr);
 		return -1;
 	}
@@ -1031,7 +1034,7 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
 	if (tdb_delete(ctdb_db->ltdb->tdb, key) != 0) {
 		tdb_unlock(ctdb_db->ltdb->tdb, -1, F_WRLCK);
 		tdb_chainunlock(ctdb_db->ltdb->tdb, key);
-		DEBUG(DEBUG_INFO,(__location__ " Failed to delete record\n"));
+		DBG_INFO("Failed to delete record\n");
 		free(data2.dptr);
 		return -1;
 	}
diff --git a/ctdb/server/ctdb_vacuum.c b/ctdb/server/ctdb_vacuum.c
index 2194b7f4da7..9d086917f3c 100644
--- a/ctdb/server/ctdb_vacuum.c
+++ b/ctdb/server/ctdb_vacuum.c
@@ -1205,6 +1205,7 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
 	struct ctdb_context *ctdb = ctdb_db->ctdb;
 	struct ctdb_vacuum_child_context *child_ctx;
 	struct tevent_fd *fde;
+	bool full_vacuum_run = false;
 	int ret;
 
 	/* we don't vacuum if we are in recovery mode, or db frozen */
@@ -1247,7 +1248,11 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
 		return;
 	}
 
-	if (vacuum_handle->fast_path_count > ctdb->tunable.vacuum_fast_path_count) {
+	if (vacuum_handle->fast_path_count >=
+	    ctdb->tunable.vacuum_fast_path_count) {
+		if (ctdb->tunable.vacuum_fast_path_count > 0) {
+			full_vacuum_run = true;
+		}
 		vacuum_handle->fast_path_count = 0;
 	}
 
@@ -1266,7 +1271,6 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
 
 	if (child_ctx->child_pid == 0) {
 		char cc = 0;
-		bool full_vacuum_run = false;
 		close(child_ctx->fd[0]);
 
 		DEBUG(DEBUG_INFO,("Vacuuming child process %d for db %s started\n", getpid(), ctdb_db->db_name));
@@ -1276,11 +1280,6 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
 			_exit(1);
 		}
 
-		if ((ctdb->tunable.vacuum_fast_path_count > 0) &&
-		    (vacuum_handle->fast_path_count == 0))
-		{
-			full_vacuum_run = true;
-		}
 		cc = ctdb_vacuum_and_repack_db(ctdb_db, full_vacuum_run);
 
 		sys_write(child_ctx->fd[1], &cc, 1);
diff --git a/ctdb/tests/simple/69_recovery_resurrect_deleted.sh b/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
index b69bc5d2672..2c8fc12edef 100755
--- a/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
+++ b/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
@@ -63,8 +63,9 @@ echo "Wait until vacuuming deletes the record on active nodes"
 # 1. Original node processes delete queue, asks lmaster to fetch
 # 2. lmaster recoverd fetches
 # 3. lmaster processes delete queue
-# If vacuuming is just missed then need an extra interval.
-wait_until $((vacuum_interval * 4)) database_has_zero_records
+# If vacuuming is just missed then need an extra interval
+t=$((vacuum_interval * 4))
+wait_until "${t}/10" database_has_zero_records
 
 echo "Continue node ${first}"
 try_command_on_node $first $CTDB continue
diff --git a/ctdb/tools/ctdb.c b/ctdb/tools/ctdb.c
index b598a866c9b..eb4c684e8b0 100644
--- a/ctdb/tools/ctdb.c
+++ b/ctdb/tools/ctdb.c
@@ -626,6 +626,9 @@ static int str_to_data(const char *str, size_t len, TALLOC_CTX *mem_ctx,
 
 	if (strncmp(str, "0x", 2) == 0) {
 		ret = hex_to_data(str+2, len-2, mem_ctx, &data);
+		if (ret != 0) {
+			return ret;
+		}
 	} else {
 		data.dptr = talloc_memdup(mem_ctx, str, len);
 		if (data.dptr == NULL) {
@@ -635,7 +638,7 @@ static int str_to_data(const char *str, size_t len, TALLOC_CTX *mem_ctx,
 	}
 
 	*out = data;
-	return ret;
+	return 0;
 }
 
 static int run_helper(TALLOC_CTX *mem_ctx, const char *command,
diff --git a/ctdb/wscript b/ctdb/wscript
index 44a9d08acf9..30b09d6dc16 100644
--- a/ctdb/wscript
+++ b/ctdb/wscript
@@ -1177,7 +1177,7 @@ def distonly(ctx):
 
     version = get_version()
 
-    distfile = file('.distversion', 'w')
+    distfile = open('.distversion', 'w')
     for field in version.vcs_fields:
         distfile.write('%s=%s\n' % (field, str(version.vcs_fields[field])))
     distfile.close()


-- 
Samba Shared Repository



More information about the samba-cvs mailing list