[SCM] Samba Shared Repository - branch master updated
Amitay Isaacs
amitay at samba.org
Tue Dec 18 09:14:02 UTC 2018
The branch, master has been updated
via 944c92a15db ctdb-daemon: Modernise debug during record deletion for vacuuming
via cdca0d7e78a ctdb-daemon Add extra debug during record deletion for vacuuming
via 2e3ad8c20d2 ctdb-tests: Minimise chances of test interfering with itself
via f1b594dce1c ctdb-daemon: Do not force full vacuum on first vacuuming run
via 9bdd6814e4c ctdb-packaging: Update library versions to upstream versions
via 59e244c9d04 ctdb-packaging: Match configure command as per spec file
via 4443124fe8d ctdb-packaging: Call waf with python wrapper
via 9912709eca2 ctdb-build: Use open() instead of file() for python3
via 1e061ff1e3f ctdb-tool: Avoid data uninitialized warnings
from 1ed91f0e102 ctdb-tests: Do not force TEST_VAR_DIR to be absolute
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 944c92a15dbbcaaea1cd4e63bf2a109a33126437
Author: Martin Schwenke <martin at meltin.net>
Date: Wed Oct 24 12:29:54 2018 +1100
ctdb-daemon: Modernise debug during record deletion for vacuuming
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Tue Dec 18 10:13:50 CET 2018 on sn-devel-144
commit cdca0d7e78a4cad797ff457f860a37de78bcdb44
Author: Martin Schwenke <martin at meltin.net>
Date: Mon Oct 15 21:21:25 2018 +1100
ctdb-daemon Add extra debug during record deletion for vacuuming
It isn't currently possible to distinguish these 2 cases.
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 2e3ad8c20d2964fba14d0d5b0b36c334ac22caa5
Author: Martin Schwenke <martin at meltin.net>
Date: Tue Dec 18 14:31:24 2018 +1100
ctdb-tests: Minimise chances of test interfering with itself
Checking that the database contains 0 records cause a traverse. This
may take a lock and cause vacuuming to fail (or be deferred for a
particular record/chain). Minimise the chance of this happening by
only checking for 0 records every 10 seconds.
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit f1b594dce1c081924b16c52c8c50c984a16df098
Author: Martin Schwenke <martin at meltin.net>
Date: Mon Oct 22 21:40:22 2018 +1100
ctdb-daemon: Do not force full vacuum on first vacuuming run
When the number of fast path vacuuming runs is 0 then a full vacuuming
run is done. This means the first one is a full run, which is almost
certainly not what is intended.
Combine the 2 conditionals to only flag a full vacuuming run when the
count exceeds the configured limit. This means that the
full_vacuum_run flag is set in both parent and child, but this is
harmless... and is better than getting it wrong.
Also tweak the comparison to be less-than-or-equal, since the zeroth
run needs to be counted.
Signed-off-by: Martin Schwenke <martin at meltin.net>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 9bdd6814e4c64f3be04daa7e8739e141d2d664b7
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue Dec 18 13:37:40 2018 +1100
ctdb-packaging: Update library versions to upstream versions
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 59e244c9d0402062ae77c882e2af28d00da44449
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue Dec 18 11:01:35 2018 +1100
ctdb-packaging: Match configure command as per spec file
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 4443124fe8d7c12dfab62f9962939d14cbed1291
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue Dec 18 11:03:51 2018 +1100
ctdb-packaging: Call waf with python wrapper
This allows to build packages even when python3 is not available by
setting PYTHON variable.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 9912709eca29d21cf56f342079db6b5cc02bbea2
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue Dec 18 10:54:50 2018 +1100
ctdb-build: Use open() instead of file() for python3
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 1e061ff1e3fcfcd020df1a35c1c0697c114cb209
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue Dec 18 11:33:30 2018 +1100
ctdb-tool: Avoid data uninitialized warnings
../../tools/ctdb.c: In function 'str_to_data':
../../tools/ctdb.c:624: warning: 'data.dsize' may be used uninitialized in this function
../../tools/ctdb.c:624: warning: 'data.dptr' may be used uninitialized in this function
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
-----------------------------------------------------------------------
Summary of changes:
ctdb/configure.rpm | 4 +++-
ctdb/packaging/RPM/ctdb.spec.in | 7 ++++---
ctdb/server/ctdb_recover.c | 23 ++++++++++++----------
ctdb/server/ctdb_vacuum.c | 13 ++++++------
ctdb/tests/simple/69_recovery_resurrect_deleted.sh | 5 +++--
ctdb/tools/ctdb.c | 5 ++++-
ctdb/wscript | 2 +-
7 files changed, 34 insertions(+), 25 deletions(-)
Changeset truncated at 500 lines:
diff --git a/ctdb/configure.rpm b/ctdb/configure.rpm
index 32e6673d310..e8ec3b24459 100755
--- a/ctdb/configure.rpm
+++ b/ctdb/configure.rpm
@@ -9,9 +9,11 @@ fi
CFLAGS="-Wall -g -D_GNU_SOURCE" ./configure \
--builtin-libraries=replace,popt \
--bundled-libraries=!talloc,!tevent,!tdb \
- --minimum-library-version=talloc:2.0.8,tdb:1.2.11,tevent:0.9.16 \
+ --minimum-library-version=talloc:2.1.14,tdb:1.3.17,tevent:0.9.37 \
--prefix=/usr \
+ --includedir=/usr/include/ctdb \
--libdir=${_libdir} \
+ --libexecdir=/usr/libexec \
--sysconfdir=/etc \
--mandir=/usr/man \
--localstatedir=/var \
diff --git a/ctdb/packaging/RPM/ctdb.spec.in b/ctdb/packaging/RPM/ctdb.spec.in
index 537f158b2d1..d25735e06e0 100644
--- a/ctdb/packaging/RPM/ctdb.spec.in
+++ b/ctdb/packaging/RPM/ctdb.spec.in
@@ -87,7 +87,8 @@ fi
export CC
-CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" ./buildtools/bin/waf configure \
+CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" \
+$PYTHON ./buildtools/bin/waf configure \
--builtin-libraries=replace,popt \
--bundled-libraries=!talloc,!tevent,!tdb \
--minimum-library-version=talloc:%libtalloc_version,tdb:%libtdb_version,tevent:%libtevent_version \
@@ -102,7 +103,7 @@ CFLAGS="$RPM_OPT_FLAGS $EXTRA -D_GNU_SOURCE" ./buildtools/bin/waf configure \
--mandir=%{_mandir} \
--localstatedir=%{_localstatedir}
-./buildtools/bin/waf build
+$PYTHON ./buildtools/bin/waf build
%install
# Clean up in case there is trash left from a previous build
@@ -111,7 +112,7 @@ rm -rf $RPM_BUILD_ROOT
# Create the target build directory hierarchy
mkdir -p $RPM_BUILD_ROOT%{_sysconfdir}/sudoers.d
-DESTDIR=$RPM_BUILD_ROOT ./buildtools/bin/waf install
+DESTDIR=$RPM_BUILD_ROOT $PYTHON ./buildtools/bin/waf install
install -m644 config/ctdb.conf $RPM_BUILD_ROOT%{_sysconfdir}/ctdb
install -m644 config/ctdb.tunables $RPM_BUILD_ROOT%{_sysconfdir}/ctdb
diff --git a/ctdb/server/ctdb_recover.c b/ctdb/server/ctdb_recover.c
index f05052e8466..cfe77f643a6 100644
--- a/ctdb/server/ctdb_recover.c
+++ b/ctdb/server/ctdb_recover.c
@@ -956,12 +956,12 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
data.dptr = &rec->data[rec->keylen];
if (ctdb_lmaster(ctdb, &key) == ctdb->pnn) {
- DEBUG(DEBUG_INFO,(__location__ " Called delete on record where we are lmaster\n"));
+ DBG_INFO("Called delete on record where we are lmaster\n");
return -1;
}
if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_ERR,(__location__ " Bad record size\n"));
+ DBG_ERR("Bad record size\n");
return -1;
}
@@ -969,6 +969,7 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
/* use a non-blocking lock */
if (tdb_chainlock_nonblock(ctdb_db->ltdb->tdb, key) != 0) {
+ DBG_INFO("Failed to get non-blocking chain lock\n");
return -1;
}
@@ -981,10 +982,10 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
if (data2.dsize < sizeof(struct ctdb_ltdb_header)) {
if (tdb_lock_nonblock(ctdb_db->ltdb->tdb, -1, F_WRLCK) == 0) {
if (tdb_delete(ctdb_db->ltdb->tdb, key) != 0) {
- DEBUG(DEBUG_CRIT,(__location__ " Failed to delete corrupt record\n"));
+ DBG_ERR("Failed to delete corrupt record\n");
}
tdb_unlock(ctdb_db->ltdb->tdb, -1, F_WRLCK);
- DEBUG(DEBUG_CRIT,(__location__ " Deleted corrupt record\n"));
+ DBG_ERR("Deleted corrupt record\n");
}
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
free(data2.dptr);
@@ -995,8 +996,9 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
if (hdr2->rsn > hdr->rsn) {
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
- DEBUG(DEBUG_INFO,(__location__ " Skipping record with rsn=%llu - called with rsn=%llu\n",
- (unsigned long long)hdr2->rsn, (unsigned long long)hdr->rsn));
+ DBG_INFO("Skipping record with rsn=%llu - called with rsn=%llu\n",
+ (unsigned long long)hdr2->rsn,
+ (unsigned long long)hdr->rsn);
free(data2.dptr);
return -1;
}
@@ -1004,26 +1006,27 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
/* do not allow deleting record that have readonly flags set. */
if (hdr->flags & CTDB_REC_RO_FLAGS) {
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
- DEBUG(DEBUG_INFO,(__location__ " Skipping record with readonly flags set\n"));
+ DBG_INFO("Skipping record with readonly flags set\n");
free(data2.dptr);
return -1;
}
if (hdr2->flags & CTDB_REC_RO_FLAGS) {
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
- DEBUG(DEBUG_INFO,(__location__ " Skipping record with readonly flags set\n"));
+ DBG_INFO("Skipping record with readonly flags set locally\n");
free(data2.dptr);
return -1;
}
if (hdr2->dmaster == ctdb->pnn) {
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
- DEBUG(DEBUG_INFO,(__location__ " Attempted delete record where we are the dmaster\n"));
+ DBG_INFO("Attempted delete record where we are the dmaster\n");
free(data2.dptr);
return -1;
}
if (tdb_lock_nonblock(ctdb_db->ltdb->tdb, -1, F_WRLCK) != 0) {
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
+ DBG_INFO("Failed to get non-blocking freelist lock\n");
free(data2.dptr);
return -1;
}
@@ -1031,7 +1034,7 @@ static int delete_tdb_record(struct ctdb_context *ctdb, struct ctdb_db_context *
if (tdb_delete(ctdb_db->ltdb->tdb, key) != 0) {
tdb_unlock(ctdb_db->ltdb->tdb, -1, F_WRLCK);
tdb_chainunlock(ctdb_db->ltdb->tdb, key);
- DEBUG(DEBUG_INFO,(__location__ " Failed to delete record\n"));
+ DBG_INFO("Failed to delete record\n");
free(data2.dptr);
return -1;
}
diff --git a/ctdb/server/ctdb_vacuum.c b/ctdb/server/ctdb_vacuum.c
index 2194b7f4da7..9d086917f3c 100644
--- a/ctdb/server/ctdb_vacuum.c
+++ b/ctdb/server/ctdb_vacuum.c
@@ -1205,6 +1205,7 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
struct ctdb_context *ctdb = ctdb_db->ctdb;
struct ctdb_vacuum_child_context *child_ctx;
struct tevent_fd *fde;
+ bool full_vacuum_run = false;
int ret;
/* we don't vacuum if we are in recovery mode, or db frozen */
@@ -1247,7 +1248,11 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
return;
}
- if (vacuum_handle->fast_path_count > ctdb->tunable.vacuum_fast_path_count) {
+ if (vacuum_handle->fast_path_count >=
+ ctdb->tunable.vacuum_fast_path_count) {
+ if (ctdb->tunable.vacuum_fast_path_count > 0) {
+ full_vacuum_run = true;
+ }
vacuum_handle->fast_path_count = 0;
}
@@ -1266,7 +1271,6 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
if (child_ctx->child_pid == 0) {
char cc = 0;
- bool full_vacuum_run = false;
close(child_ctx->fd[0]);
DEBUG(DEBUG_INFO,("Vacuuming child process %d for db %s started\n", getpid(), ctdb_db->db_name));
@@ -1276,11 +1280,6 @@ static void ctdb_vacuum_event(struct tevent_context *ev,
_exit(1);
}
- if ((ctdb->tunable.vacuum_fast_path_count > 0) &&
- (vacuum_handle->fast_path_count == 0))
- {
- full_vacuum_run = true;
- }
cc = ctdb_vacuum_and_repack_db(ctdb_db, full_vacuum_run);
sys_write(child_ctx->fd[1], &cc, 1);
diff --git a/ctdb/tests/simple/69_recovery_resurrect_deleted.sh b/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
index b69bc5d2672..2c8fc12edef 100755
--- a/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
+++ b/ctdb/tests/simple/69_recovery_resurrect_deleted.sh
@@ -63,8 +63,9 @@ echo "Wait until vacuuming deletes the record on active nodes"
# 1. Original node processes delete queue, asks lmaster to fetch
# 2. lmaster recoverd fetches
# 3. lmaster processes delete queue
-# If vacuuming is just missed then need an extra interval.
-wait_until $((vacuum_interval * 4)) database_has_zero_records
+# If vacuuming is just missed then need an extra interval
+t=$((vacuum_interval * 4))
+wait_until "${t}/10" database_has_zero_records
echo "Continue node ${first}"
try_command_on_node $first $CTDB continue
diff --git a/ctdb/tools/ctdb.c b/ctdb/tools/ctdb.c
index b598a866c9b..eb4c684e8b0 100644
--- a/ctdb/tools/ctdb.c
+++ b/ctdb/tools/ctdb.c
@@ -626,6 +626,9 @@ static int str_to_data(const char *str, size_t len, TALLOC_CTX *mem_ctx,
if (strncmp(str, "0x", 2) == 0) {
ret = hex_to_data(str+2, len-2, mem_ctx, &data);
+ if (ret != 0) {
+ return ret;
+ }
} else {
data.dptr = talloc_memdup(mem_ctx, str, len);
if (data.dptr == NULL) {
@@ -635,7 +638,7 @@ static int str_to_data(const char *str, size_t len, TALLOC_CTX *mem_ctx,
}
*out = data;
- return ret;
+ return 0;
}
static int run_helper(TALLOC_CTX *mem_ctx, const char *command,
diff --git a/ctdb/wscript b/ctdb/wscript
index 44a9d08acf9..30b09d6dc16 100644
--- a/ctdb/wscript
+++ b/ctdb/wscript
@@ -1177,7 +1177,7 @@ def distonly(ctx):
version = get_version()
- distfile = file('.distversion', 'w')
+ distfile = open('.distversion', 'w')
for field in version.vcs_fields:
distfile.write('%s=%s\n' % (field, str(version.vcs_fields[field])))
distfile.close()
--
Samba Shared Repository
More information about the samba-cvs
mailing list