[SCM] Samba Shared Repository - branch master updated
Martin Schwenke
martins at samba.org
Mon Jul 7 08:07:04 MDT 2014
The branch, master has been updated
via eccce07 ctdb-tests: Add a test for ctdb restoredb
via 9c8c8a7 ctdb-tests: Check that ctdb wipedb cleans the database
via 2855173 ctdb-daemon: Do not thaw databases if recovery is active
via 28a1b75 ctdb-recoverd: Set recovery mode before freezing databases
via 72c6500 ctdb-tools: There is no need for forcing a recovery
via e5cd81d Revert "It was possible for ->recovery_mode to get out of sync with the new three db priorities in such a way that"
from 1be8564 ctdb-common: Use SCHED_RESET_ON_FORK when setting SCHED_FIFO
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit eccce073d084eceb4bfb5c25001b5873e2c0f2b2
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 17:44:24 2014 +1000
ctdb-tests: Add a test for ctdb restoredb
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Mon Jul 7 16:06:39 CEST 2014 on sn-devel-104
commit 9c8c8a7b0bfd4c1cafa3deaa012049b7f0851617
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 17:16:19 2014 +1000
ctdb-tests: Check that ctdb wipedb cleans the database
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 2855173dac5386bff655d1bb94c1848591b963e1
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 14:20:44 2014 +1000
ctdb-daemon: Do not thaw databases if recovery is active
This prevents ctdb tool from thawing databases prematurely in
thaw/wipedb/restoredb commands if recovery is active.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 28a1b75886fb4aea65e23bfd00b9f4c98780fdfd
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 14:24:52 2014 +1000
ctdb-recoverd: Set recovery mode before freezing databases
Setting recovery mode to active is the only correct way to inform recovery
daemon to run database recovery. Only freezing databases without setting
recovery mode should not trigger database recovery, as this mechanism
is used in tool to implement wipedb/restoredb commands.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit 72c6500ee440779819b9adb768a7022cc251f07e
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 14:15:45 2014 +1000
ctdb-tools: There is no need for forcing a recovery
This effectively reverts commit 442953c540424ad0c64f4264b5ee27c45a3130e8.
The correct way of telling recovery daemon to trigger a database recovery is
by setting recovery mode to active. There is no need to freeze databases as
recovery master will do that across the cluster anyway.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
commit e5cd81da77ef58992b7eb9ff7d972b499b946bb7
Author: Amitay Isaacs <amitay at gmail.com>
Date: Tue May 6 14:07:00 2014 +1000
Revert "It was possible for ->recovery_mode to get out of sync with the new three db priorities in such a way that"
This reverts commit 6578a97bd94fc14d5b6df85b84e50447f7bdb2e3.
This condition cannot happen since when recovery is triggered, all the
databases would get frozen and thawed in the order of priority. The only
other place where databases get frozen are for implementation of ctdb
wipedb/restoredb commands.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Martin Schwenke <martin at meltin.net>
-----------------------------------------------------------------------
Summary of changes:
ctdb/include/ctdb_private.h | 3 +-
ctdb/server/ctdb_banning.c | 2 +-
ctdb/server/ctdb_control.c | 13 +---
ctdb/server/ctdb_freeze.c | 13 +++-
ctdb/server/ctdb_recover.c | 2 +-
ctdb/server/ctdb_recoverd.c | 42 ++++++------
ctdb/tests/simple/58_ctdb_restoredb.sh | 118 ++++++++++++++++++++++++++++++++
ctdb/tests/simple/71_ctdb_wipedb.sh | 10 +++
ctdb/tools/ctdb.c | 13 ----
9 files changed, 165 insertions(+), 51 deletions(-)
create mode 100755 ctdb/tests/simple/58_ctdb_restoredb.sh
Changeset truncated at 500 lines:
diff --git a/ctdb/include/ctdb_private.h b/ctdb/include/ctdb_private.h
index de7880b..aeebed1 100644
--- a/ctdb/include/ctdb_private.h
+++ b/ctdb/include/ctdb_private.h
@@ -1059,7 +1059,8 @@ void ctdb_request_control_reply(struct ctdb_context *ctdb, struct ctdb_req_contr
TDB_DATA *outdata, int32_t status, const char *errormsg);
int32_t ctdb_control_freeze(struct ctdb_context *ctdb, struct ctdb_req_control *c, bool *async_reply);
-int32_t ctdb_control_thaw(struct ctdb_context *ctdb, uint32_t priority);
+int32_t ctdb_control_thaw(struct ctdb_context *ctdb, uint32_t priority,
+ bool check_recmode);
int ctdb_start_recoverd(struct ctdb_context *ctdb);
void ctdb_stop_recoverd(struct ctdb_context *ctdb);
diff --git a/ctdb/server/ctdb_banning.c b/ctdb/server/ctdb_banning.c
index 13d97c8..a9d1891 100644
--- a/ctdb/server/ctdb_banning.c
+++ b/ctdb/server/ctdb_banning.c
@@ -70,11 +70,11 @@ void ctdb_local_node_got_banned(struct ctdb_context *ctdb)
*/
ctdb->vnn_map->generation = INVALID_GENERATION;
+ ctdb->recovery_mode = CTDB_RECOVERY_ACTIVE;
for (i=1; i<=NUM_DB_PRIORITIES; i++) {
ctdb_start_freeze(ctdb, i);
}
ctdb_release_all_ips(ctdb);
- ctdb->recovery_mode = CTDB_RECOVERY_ACTIVE;
}
int32_t ctdb_control_set_ban_state(struct ctdb_context *ctdb, TDB_DATA indata)
diff --git a/ctdb/server/ctdb_control.c b/ctdb/server/ctdb_control.c
index b0220de..1f9e4bc 100644
--- a/ctdb/server/ctdb_control.c
+++ b/ctdb/server/ctdb_control.c
@@ -198,16 +198,7 @@ static int32_t ctdb_control_dispatch(struct ctdb_context *ctdb,
return ctdb_control_push_db(ctdb, indata);
case CTDB_CONTROL_GET_RECMODE: {
- int i;
- if (ctdb->recovery_mode == CTDB_RECOVERY_ACTIVE) {
- return CTDB_RECOVERY_ACTIVE;
- }
- for (i=1; i<=NUM_DB_PRIORITIES; i++) {
- if (ctdb->freeze_mode[i] == CTDB_FREEZE_FROZEN) {
- return CTDB_RECOVERY_ACTIVE;
- }
- }
- return CTDB_RECOVERY_NORMAL;
+ return ctdb->recovery_mode;
}
case CTDB_CONTROL_SET_RECMASTER: {
@@ -327,7 +318,7 @@ static int32_t ctdb_control_dispatch(struct ctdb_context *ctdb,
case CTDB_CONTROL_THAW:
CHECK_CONTROL_DATA_SIZE(0);
- return ctdb_control_thaw(ctdb, (uint32_t)c->srvid);
+ return ctdb_control_thaw(ctdb, (uint32_t)c->srvid, true);
case CTDB_CONTROL_SET_RECMODE:
CHECK_CONTROL_DATA_SIZE(sizeof(uint32_t));
diff --git a/ctdb/server/ctdb_freeze.c b/ctdb/server/ctdb_freeze.c
index fee44d4..42a1285 100644
--- a/ctdb/server/ctdb_freeze.c
+++ b/ctdb/server/ctdb_freeze.c
@@ -267,11 +267,18 @@ static void thaw_priority(struct ctdb_context *ctdb, uint32_t priority)
/*
thaw the databases
*/
-int32_t ctdb_control_thaw(struct ctdb_context *ctdb, uint32_t priority)
+int32_t ctdb_control_thaw(struct ctdb_context *ctdb, uint32_t priority,
+ bool check_recmode)
{
-
if (priority > NUM_DB_PRIORITIES) {
- DEBUG(DEBUG_ERR,(__location__ " Invalid db priority : %u\n", priority));
+ DEBUG(DEBUG_ERR,(__location__ " Invalid db priority : %u\n",
+ priority));
+ return -1;
+ }
+
+ if (check_recmode && ctdb->recovery_mode == CTDB_RECOVERY_ACTIVE) {
+ DEBUG(DEBUG_ERR, ("Failing to thaw databases while "
+ "recovery is active\n"));
return -1;
}
diff --git a/ctdb/server/ctdb_recover.c b/ctdb/server/ctdb_recover.c
index 7249f50..aa6abbe 100644
--- a/ctdb/server/ctdb_recover.c
+++ b/ctdb/server/ctdb_recover.c
@@ -623,7 +623,7 @@ int32_t ctdb_control_set_recmode(struct ctdb_context *ctdb,
/* force the databases to thaw */
for (i=1; i<=NUM_DB_PRIORITIES; i++) {
if (ctdb->freeze_handles[i] != NULL) {
- ctdb_control_thaw(ctdb, i);
+ ctdb_control_thaw(ctdb, i, false);
}
}
diff --git a/ctdb/server/ctdb_recoverd.c b/ctdb/server/ctdb_recoverd.c
index f553f67..ab73e88 100644
--- a/ctdb/server/ctdb_recoverd.c
+++ b/ctdb/server/ctdb_recoverd.c
@@ -379,8 +379,23 @@ static int set_recovery_mode(struct ctdb_context *ctdb, struct ctdb_recoverd *re
tmp_ctx = talloc_new(ctdb);
CTDB_NO_MEMORY(ctdb, tmp_ctx);
- /* freeze all nodes */
nodes = list_of_active_nodes(ctdb, nodemap, tmp_ctx, true);
+
+ data.dsize = sizeof(uint32_t);
+ data.dptr = (unsigned char *)&rec_mode;
+
+ if (ctdb_client_async_control(ctdb, CTDB_CONTROL_SET_RECMODE,
+ nodes, 0,
+ CONTROL_TIMEOUT(),
+ false, data,
+ NULL, NULL,
+ NULL) != 0) {
+ DEBUG(DEBUG_ERR, (__location__ " Unable to set recovery mode. Recovery failed.\n"));
+ talloc_free(tmp_ctx);
+ return -1;
+ }
+
+ /* freeze all nodes */
if (rec_mode == CTDB_RECOVERY_ACTIVE) {
int i;
@@ -399,21 +414,6 @@ static int set_recovery_mode(struct ctdb_context *ctdb, struct ctdb_recoverd *re
}
}
-
- data.dsize = sizeof(uint32_t);
- data.dptr = (unsigned char *)&rec_mode;
-
- if (ctdb_client_async_control(ctdb, CTDB_CONTROL_SET_RECMODE,
- nodes, 0,
- CONTROL_TIMEOUT(),
- false, data,
- NULL, NULL,
- NULL) != 0) {
- DEBUG(DEBUG_ERR, (__location__ " Unable to set recovery mode. Recovery failed.\n"));
- talloc_free(tmp_ctx);
- return -1;
- }
-
talloc_free(tmp_ctx);
return 0;
}
@@ -3654,17 +3654,17 @@ static void main_loop(struct ctdb_context *ctdb, struct ctdb_recoverd *rec,
if (ctdb->recovery_mode == CTDB_RECOVERY_NORMAL) {
DEBUG(DEBUG_ERR,("Node is stopped or banned but recovery mode is not active. Activate recovery mode and lock databases\n"));
- ret = ctdb_ctrl_freeze_priority(ctdb, CONTROL_TIMEOUT(), CTDB_CURRENT_NODE, 1);
- if (ret != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to freeze node in STOPPED or BANNED state\n"));
- return;
- }
ret = ctdb_ctrl_setrecmode(ctdb, CONTROL_TIMEOUT(), CTDB_CURRENT_NODE, CTDB_RECOVERY_ACTIVE);
if (ret != 0) {
DEBUG(DEBUG_ERR,(__location__ " Failed to activate recovery mode in STOPPED or BANNED state\n"));
return;
}
+ ret = ctdb_ctrl_freeze(ctdb, CONTROL_TIMEOUT(), CTDB_CURRENT_NODE);
+ if (ret != 0) {
+ DEBUG(DEBUG_ERR,(__location__ " Failed to freeze node in STOPPED or BANNED state\n"));
+ return;
+ }
}
/* If this node is stopped or banned then it is not the recovery
diff --git a/ctdb/tests/simple/58_ctdb_restoredb.sh b/ctdb/tests/simple/58_ctdb_restoredb.sh
new file mode 100755
index 0000000..31b81a6
--- /dev/null
+++ b/ctdb/tests/simple/58_ctdb_restoredb.sh
@@ -0,0 +1,118 @@
+#!/bin/bash
+
+test_info()
+{
+ cat <<EOF
+The command 'ctdb restoredb' is used to restore a database across the
+whole cluster.
+
+Prerequisites:
+
+* An active CTDB cluster with at least 2 active nodes.
+
+Steps:
+
+1. Verify that the status on all of the ctdb nodes is 'OK'.
+2. Create a persistent test database
+3. Add some records to test database
+4. Backup database
+5. Wipe database and verify the database is empty on all nodes
+6. Restore database and make sure all the records are restored
+7. Make sure no recovery has been triggered
+
+Expected results:
+
+* Database operations should not cause a recovery
+
+EOF
+}
+
+. "${TEST_SCRIPTS_DIR}/integration.bash"
+
+ctdb_test_init "$@"
+
+set -e
+
+cluster_is_healthy
+
+try_command_on_node 0 $CTDB status
+generation=$(echo "$out" | sed -n -e 's/^Generation:\([0-9]*\)/\1/p')
+
+try_command_on_node 0 "$CTDB listnodes"
+num_nodes=$(echo "$out" | wc -l)
+
+# 2.
+test_db="restoredb_test.tdb"
+test_dump=$(mktemp)
+echo $test_dump
+echo "Create persistent test database \"$test_db\""
+try_command_on_node 0 $CTDB attach "$test_db" persistent
+try_command_on_node 0 $CTDB wipedb "$test_db"
+
+# 3.
+# add 10,000 records to database
+echo "Adding 10000 records to database"
+(
+for i in $(seq 1 10000) ; do
+ echo "\"key$i\" \"value$i\""
+done
+) | try_command_on_node -i 0 $CTDB ptrans "$test_db"
+
+num_records=$(db_ctdb_cattdb_count_records 1 "$test_db")
+if [ $num_records = "10000" ] ; then
+ echo "OK: Records added"
+else
+ echo "BAD: We did not end up with 10000 records"
+ echo "num records = $num_records"
+ exit 1
+fi
+
+ctdb_test_exit_hook_add "rm -f $test_dump"
+
+# 4.
+echo "Backup database"
+try_command_on_node 0 $CTDB backupdb "$test_db" "$test_dump"
+
+# 5.
+echo "Wipe database"
+try_command_on_node 0 $CTDB wipedb "$test_db"
+
+# check that the database is restored
+num_records=$(db_ctdb_cattdb_count_records 1 "$test_db")
+if [ $num_records = "0" ] ; then
+ echo "OK: Database was wiped"
+else
+ echo "BAD: We did not end up with an empty database"
+ echo "num records = $num_records"
+ exit 1
+fi
+
+# 6.
+echo "Restore database"
+try_command_on_node 0 $CTDB restoredb "$test_dump" "$test_db"
+
+# check that the database is restored
+num_records=$(db_ctdb_cattdb_count_records 1 "$test_db")
+if [ $num_records = "10000" ] ; then
+ echo "OK: Database was restored"
+else
+ echo "BAD: We did not end up with 10000 records"
+ echo "num records = $num_records"
+ exit 1
+fi
+
+# 7.
+wait_until_ready
+
+try_command_on_node 0 $CTDB status
+new_generation=$(echo "$out" | sed -n -e 's/^Generation:\([0-9]*\)/\1/p')
+
+echo "Old generation = $generation"
+echo "New generation = $new_generation"
+
+if [ "$generation" = "$new_generation" ]; then
+ echo "OK: Database recovery not triggered."
+else
+ echo "BAD: Database recovery triggered."
+ exit 1
+fi
diff --git a/ctdb/tests/simple/71_ctdb_wipedb.sh b/ctdb/tests/simple/71_ctdb_wipedb.sh
index 886e5e6..1919ea7 100755
--- a/ctdb/tests/simple/71_ctdb_wipedb.sh
+++ b/ctdb/tests/simple/71_ctdb_wipedb.sh
@@ -52,6 +52,16 @@ db_ctdb_tstore 1 "$test_db" "DEF" "DEF"
# 4.
echo "Wipe database"
try_command_on_node 0 $CTDB wipedb "$test_db"
+
+# check that the database is wiped
+num_records=$(db_ctdb_cattdb_count_records 1 "$test_db")
+if [ $num_records = "0" ] ; then
+ echo "OK: Database was wiped"
+else
+ echo "BAD: We did not end up with an empty database"
+ exit 1
+fi
+
echo "Force a recovery"
try_command_on_node 0 $CTDB recover
diff --git a/ctdb/tools/ctdb.c b/ctdb/tools/ctdb.c
index aa0e4e3..3b158fd 100644
--- a/ctdb/tools/ctdb.c
+++ b/ctdb/tools/ctdb.c
@@ -3463,23 +3463,10 @@ static int control_recover(struct ctdb_context *ctdb, int argc, const char **arg
{
int ret;
uint32_t generation, next_generation;
- bool force;
-
- /* "force" option ignores freeze failure and forces recovery */
- force = (argc == 1) && (strcasecmp(argv[0], "force") == 0);
/* record the current generation number */
generation = get_generation(ctdb);
- ret = ctdb_ctrl_freeze_priority(ctdb, TIMELIMIT(), options.pnn, 1);
- if (ret != 0) {
- if (!force) {
- DEBUG(DEBUG_ERR, ("Unable to freeze node\n"));
- return ret;
- }
- DEBUG(DEBUG_WARNING, ("Unable to freeze node but proceeding because \"force\" option given\n"));
- }
-
ret = ctdb_ctrl_setrecmode(ctdb, TIMELIMIT(), options.pnn, CTDB_RECOVERY_ACTIVE);
if (ret != 0) {
DEBUG(DEBUG_ERR, ("Unable to set recovery mode\n"));
--
Samba Shared Repository
More information about the samba-cvs
mailing list