[SCM] CTDB repository - branch 1.0.114 updated - ctdb-1.0.114.6-50-gef1a530
Michael Adam
obnox at samba.org
Wed Sep 4 07:20:33 MDT 2013
The branch, 1.0.114 has been updated
via ef1a5308271ef33b0f059265b08ad12e9a6aa0d4 (commit)
via 148427d9b4d233183ed9e828abdd5b877b66ba49 (commit)
via 527c44c1b710cd74772d70890d79b2831db39e40 (commit)
via c403c88a499b4f45dc9569695ac9281372622aff (commit)
via f134db968b87f41291e4ae31accb1f1a3bf18639 (commit)
via 1721dd295f82906985fd5e0fbf0b99c9e185436a (commit)
via 94901a1273eb36220cd4a95e80fa24f7352d523c (commit)
via d142dd04a81ebbe0ecb43559c87519291c4262de (commit)
via c102eefead7cc3a3486229e89362f440d2f5fb4e (commit)
via f4d0c8a9da4613fe8740e67d2a886fbb16680a41 (commit)
via 91391d83399704ac78f7a332158adef495d85821 (commit)
via 46ecf6f7311ac1584393a96a152002cc62ec38f1 (commit)
via 6c5d3640e8ef3949b1707724feb1c739b94c4b89 (commit)
via 54731e435582ae00fcc38eacecbfe569bd30055f (commit)
via 0fea80054ea3ec1cef8d0b100e8b9631b8aac753 (commit)
via 38479a245d2b43defef0249de85f3b0c76c7a4f2 (commit)
via 2f3103af6cfbc7165785b432bff5e4162b547090 (commit)
via 2e357793720ceb770ac87346899eb5dfaa93d679 (commit)
via 26fadcfcca71cbfb14470737691f2f0528acc1ca (commit)
via ab23be493829d7a65f1c2165442b05f618dfe7f4 (commit)
via 031a0d991739db45b14c9ddbca9c37e46c4d02e8 (commit)
via 35a60e42a140885307bf24c06619b35f993f54c1 (commit)
via 555d21a28e95d653cc5e66f1790bc029a86c8a9c (commit)
via 5503a9489b79878f79ddc02be0d3f6d43f0958c0 (commit)
via 0cef0c99cfe27f67777c0c7471d7f596fea2d51b (commit)
via add9c07e7ab70d2307faaae77e69781be866c1a8 (commit)
via c560b2fb00dd0e3caf7dd2ab794ffd12dd67f1a9 (commit)
via 9c35bc67038bab299c0baa7e1da6e0f00d7359b1 (commit)
via 670b6707feed785993d8a1fc95a5f346ccb61bbc (commit)
via 4d87f6de6c497e58ace4b4ad25ea90a7f351ef7d (commit)
via f330cf0b465c7b7de47ae2cc9240b718cafb6106 (commit)
via 111fd27ff926970e033353d47df706f0abd317ae (commit)
via 24493d052d66ab4309f33983aaa78ddaa565cdad (commit)
via 9eeab362666dbb3099096eb09ccf8ca6efe2e12f (commit)
via 58a4e3d58e51dc9830de9956cc168a978f4cca54 (commit)
via c25bb14f9c8ad2dee9020d85ba0188544ee3495e (commit)
via d5fdae399b945e013b21b8f61b31f1362d0090db (commit)
via 9f0602327f20dec49b9b9806b2e8b2ef68e7c8e0 (commit)
from 11a20ecbd949bd45410189d7b7e6348b42a9729e (commit)
http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.0.114
- Log -----------------------------------------------------------------
commit ef1a5308271ef33b0f059265b08ad12e9a6aa0d4
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 17:46:43 2013 +1000
doc: The second half of monitoring is only for recovery master
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit fcd5e1f04c5fe6c98399429b8f0918b8779acba6)
commit 148427d9b4d233183ed9e828abdd5b877b66ba49
Author: Michael Adam <obnox at samba.org>
Date: Wed Jun 26 09:23:22 2013 +0200
recoverd: when the recmaster is banned, use that information when forcing an election
When we trigger an election because the recmaster considers itself inactive,
update our local nodemap with the recmaster's flags before calling
force_election(). This way, we don't send the inactive node freeze commands
(e.g.) that may fail and then lead to ourselves getting banned.
The theory is that this should help avoiding banning loops.
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit 932360992b08a5483d90c0590218ba0fd756119e)
commit 527c44c1b710cd74772d70890d79b2831db39e40
Author: Michael Adam <obnox at samba.org>
Date: Wed Jun 26 07:11:51 2013 +0200
recoverd: fix a comment typo
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit 741944f118e98f178b860194eecb215180949d18)
commit c403c88a499b4f45dc9569695ac9281372622aff
Author: Michael Adam <obnox at samba.org>
Date: Fri Jun 21 17:57:37 2013 +0200
recoverd: fix a comment in main_loop
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit ac06c46e4a80c635f6094b5ac6f0bf3e3a02db95)
commit f134db968b87f41291e4ae31accb1f1a3bf18639
Author: Michael Adam <obnox at samba.org>
Date: Fri Jun 21 14:06:22 2013 +0200
recoverd: eliminate some trailing spaces from ctdb_election_win()
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit df30c0a05ed908fc2a997c56ff5484736b23b70f)
commit 1721dd295f82906985fd5e0fbf0b99c9e185436a
Author: Martin Schwenke <martin at meltin.net>
Date: Fri Jun 28 16:31:07 2013 +1000
recoverd: Don't continue if the current node gets banned
Can not continue with recovery or monitoring cluster.
Signed-off-by: Martin Schwenke <martin at meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 14399de1dd0bd8dabf1f48b1457e3ccb37589d8a)
commit 94901a1273eb36220cd4a95e80fa24f7352d523c
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri Jun 28 14:31:02 2013 +1000
recoverd: Refactor code to ban misbehaving nodes
Since we have nodemap information, there is no need to hardcode the
limit of 20.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Pair-Programmed-With: Martin Schwenke <martin at meltin.net>
(cherry picked from commit aea12dce83ef385e9fb3bc03ac7ace0874a0e3fe)
Conflicts:
server/ctdb_recoverd.c
commit d142dd04a81ebbe0ecb43559c87519291c4262de
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 16:01:16 2013 +1000
recoverd: Move code to ban other nodes after we get local node flags
If a node gets banned first, then it should not ban other nodes.
This code was moved up in main_loop to avoid waiting for nodemap
from other nodes (commit 83b0261f2cb453195b86f547d360400103a8b795).
To prevent a banned node from banning other nodes, we need to first get
nodemap information from local node, so trying to ban other nodes can
fail if we are already banned.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit ae1693905036ecdbc4594fde1f12500faae4a554)
commit c102eefead7cc3a3486229e89362f440d2f5fb4e
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 15:44:27 2013 +1000
recoverd: Delay the initial election if node is started in stopped state
Since there is an early exit if a node is stopped or banned, we can wait till
the node becomes active to start initial election.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 593a17678fbd3109e118154b034d43b852659518)
commit f4d0c8a9da4613fe8740e67d2a886fbb16680a41
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 15:33:49 2013 +1000
recoverd: Update capabilities only if the current node is active
Since we do an early return if a node is stopped or banned, move update
capabilities code below the early return and just before we check the
capabilities of current recovery master.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 93bcb6617e1024f810533e12390a572f51703ca0)
commit 91391d83399704ac78f7a332158adef495d85821
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 15:46:04 2013 +1000
recoverd: No need to check if node is recovery master when inactive
If a node is stopped or banned, it will cause early return from the
main_loop, so this check is redundent. The election will called by an
active node.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 815ddd3341b7e9db39e05a3a3fcd9a1420f053bc)
commit 46ecf6f7311ac1584393a96a152002cc62ec38f1
Author: Amitay Isaacs <amitay at gmail.com>
Date: Thu Jun 27 15:39:15 2013 +1000
recoverd: Always do an early exit from main_loop if node is stopped or banned
A stopped or banned node cannot do anything useful. So do not participate
in any cluster activity and do not cause any unnecessary network traffic.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 2396981c4bcf30530aeb7f4395093cc202105b50)
commit 6c5d3640e8ef3949b1707724feb1c739b94c4b89
Author: Martin Schwenke <martin at meltin.net>
Date: Tue Jul 3 10:30:29 2012 +1000
recoverd: main_loop() should not verify local IPs if node is stopped
Doing these checks is pointless and potentially causes unnecessary log
messages.
Signed-off-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit a0c30c820fd47d4f8620dc060c825be10754f5d1)
(cherry picked from commit d181a5dadffacc5bfe04dcab6595b03499e613ad)
commit 54731e435582ae00fcc38eacecbfe569bd30055f
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri Jun 28 14:10:47 2013 +1000
recoverd: Do not set banning credits on a node if current node is inactive
If the current node is banned or stopped, then it should not assign banning
credits to other nodes since the current node will not have up-to-date flags
of other nodes.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 38304f88e0c634e97d4687c25adef975f71537b8)
commit 0fea80054ea3ec1cef8d0b100e8b9631b8aac753
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jul 1 17:40:36 2013 +1000
banning: Do not come out of ban if databases are not frozen
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit a60f228f8380f222f838eb619d2ab55f96f11ac2)
commit 38479a245d2b43defef0249de85f3b0c76c7a4f2
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jun 24 14:33:32 2013 +1000
banning: No need to check if banned pnn is for local node
If the banned pnn is not the local node, the function returns early.
So no need for additional check.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 297d93cecc3c0655e72ecac38508e113bdbeab9c)
commit 2f3103af6cfbc7165785b432bff5e4162b547090
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri Jun 28 14:04:18 2013 +1000
banning: Make ctdb_local_node_got_banned() a void function
When this function is called, we are already committed to banning
and there is no point in failing this function. In case, freezing of
databases fails, it will be fixed from recovery daemon.
(cherry picked from commit bb178338658b4ae32382a1f62f7c21cee1d4878f)
commit 2e357793720ceb770ac87346899eb5dfaa93d679
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri Jun 28 14:02:44 2013 +1000
recoverd: Also check if current node is in recovery when it is banned
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 6a9dbb8fb0f1f6e8c206189cdc2d33bb371ea2a8)
commit 26fadcfcca71cbfb14470737691f2f0528acc1ca
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri Jun 28 14:09:35 2013 +1000
recoverd: Set node_flags information as soon as we get nodemap
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 8d622660a14c929e365d306147b378ea6ab92175)
commit ab23be493829d7a65f1c2165442b05f618dfe7f4
Author: Amitay Isaacs <amitay at gmail.com>
Date: Wed Jun 26 16:02:23 2013 +1000
recovered: Remove old comment as the code corresponding to that has gone away
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 34af2cdf686d5d77854cbaa7bbcd8f878e9171c7)
commit 031a0d991739db45b14c9ddbca9c37e46c4d02e8
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jun 24 14:31:50 2013 +1000
banning: Log ban state changes for other nodes at higher debug level
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit c6f8407648abb37f2ed781afa5171dad8c9f59e9)
commit 35a60e42a140885307bf24c06619b35f993f54c1
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jul 1 16:28:04 2013 +1000
freeze: Make ctdb_start_freeze() a void function
If this function fails due to memory errors, there is no way to recover.
The best course of action is to abort.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 46efe7a886f8c4c56f19536adc98a73c22db906a)
Conflicts:
server/ctdb_freeze.c
commit 555d21a28e95d653cc5e66f1790bc029a86c8a9c
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jul 1 16:21:00 2013 +1000
freeze: If priority is invalid here, it's time to abort
ctdb_start_freeze() is called from ctdb_control_freeze() which fixes the
priority if it's 0 and return error if it's invalid. Other callers of
ctdb_start_freeze() are internal to CTDB. So if priority is invalid in
ctdb_start_freeze(), definitely something is seriously wrong.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 87716e8f504d659515d3dbcf93badbf106873bc8)
Conflicts:
server/ctdb_freeze.c
commit 5503a9489b79878f79ddc02be0d3f6d43f0958c0
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jul 1 13:26:33 2013 +1000
freeze: Log message from ctdb_start_freeze() and ctdb_control_freeze()
This ensures that whenever databases are frozen either via sending
control or by calling ctdb_start_freeze(), the action is logged.
Since ctdb_control_freeze() calls ctdb_start_freeze(), move logging of
message in early return condition if databases are already frozen.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 478e24bceda3fedfba54ccb48faa115df726b819)
commit 0cef0c99cfe27f67777c0c7471d7f596fea2d51b
Author: Amitay Isaacs <amitay at gmail.com>
Date: Mon Jun 24 14:18:58 2013 +1000
recoverd: Print banning message only after verifying pnn
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 4be8dff3a4451192f838497b4747273685959bed)
commit add9c07e7ab70d2307faaae77e69781be866c1a8
Author: Amitay Isaacs <amitay at gmail.com>
Date: Wed Jun 26 15:22:46 2013 +1000
recoverd: When updating flags on nodes, send updated flags and not old flags
This was broken by commit a9a1156ea4e10483a4bf4265b8e9203f0af033aa.
Instead of a SRVID_SET_NODE_FLAGS message to recovery daemon, a control
was sent to the local daemon which in turn informed the recovery daemon.
And while doing this change old flags were sent via CONTROL_MODIFY_FLAGS.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
(cherry picked from commit 7eb2f89979360b6cc98ca9b17c48310277fa89fc)
commit c560b2fb00dd0e3caf7dd2ab794ffd12dd67f1a9
Author: Amitay Isaacs <amitay at gmail.com>
Date: Fri May 24 18:07:39 2013 +1000
vacuum: Reduce the priority of non-critical error
Since the complete database is not locked when the receive_records
control is received, it's possible that we may not be able to obtain
lock on a chain. We will try again to store this record.
Signed-off-by: Amitay Isaacs <amitay at gmail.com>
Reviewed-by: Michael Adam <obnox at samba.org>
(cherry picked from commit 32723c9efdad1c6ca4aa53f308ccd9bef1aadfff)
commit 9c35bc67038bab299c0baa7e1da6e0f00d7359b1
Author: Michael Adam <obnox at samba.org>
Date: Fri May 17 11:01:31 2013 +0200
ctdbd: remove a nonempty blank line
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit d9e24782a90d9ce29c0e6584b75d2b186142174d)
commit 670b6707feed785993d8a1fc95a5f346ccb61bbc
Author: Michael Adam <obnox at samba.org>
Date: Fri May 17 11:00:32 2013 +0200
ctdbd: update comment describing ctdb_call_send_redirect()
Signed-off-by: Michael Adam <obnox at samba.org>
(cherry picked from commit 9a21d417c51fb9cad8f2e87e00ca54d379aef860)
Conflicts:
server/ctdb_call.c
commit 4d87f6de6c497e58ace4b4ad25ea90a7f351ef7d
Author: Martin Schwenke <martin at meltin.net>
Date: Thu Oct 11 15:59:00 2012 +1100
recoverd: Clarify some misleading log messages
Signed-off-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit 14589bf7c16ba017fe00d4e8bea8cc501546c60f)
commit f330cf0b465c7b7de47ae2cc9240b718cafb6106
Author: Martin Schwenke <martin at meltin.net>
Date: Fri Jul 6 20:43:46 2012 +1000
recoverd: All inactive nodes should yield recovery master role
Not just stopped nodes. In reality, this means that banned nodes will
also yield, since nodes in the other inactive states won't be running
a daemon.
This seems sensible since if another node notices that an inactive
node is the recovery master then it will force an election anyway.
Signed-off-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit fc18188b7b63eb0dafbc47e3abf80e306e1dfc31)
commit 111fd27ff926970e033353d47df706f0abd317ae
Author: Martin Schwenke <martin at meltin.net>
Date: Wed Nov 9 14:45:01 2011 +1100
Clean up warnings: remove changed_flags in monitor_helper
Signed-off-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit 3e4fa518f02db75e4e4a7f326a71df226913f8a8)
commit 24493d052d66ab4309f33983aaa78ddaa565cdad
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Tue Jun 22 22:50:23 2010 +0930
speed startup: alter recovery loop
We do a recovery on startup. But the code does:
Sleep for ctdb->tunable.recover_interval.
Check for recovery.
We want to do it in the other order. This is best done by extracting
the loop into a separate "main_loop" function.
Seconds between ctdbd first log message and node healthy:
BEFORE: 24.09
AFTER: 23.58
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(cherry picked from commit 097046025176b9fcb670839d1a9f100f890e7ed2)
commit 9eeab362666dbb3099096eb09ccf8ca6efe2e12f
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date: Wed Jun 2 09:51:47 2010 +1000
rename ctdb_set_message_handler to ctdb_client_set_message_handler
to avoid a colission with the function of the same name in libctdb
(cherry picked from commit 41dbdd4fc0ab560420fb0e24a3179ff7c94c5bb7)
Conflicts:
include/ctdb_client.h
tests/src/ctdb_fetch.c
commit 58a4e3d58e51dc9830de9956cc168a978f4cca54
Author: Martin Schwenke <martin at meltin.net>
Date: Fri Nov 11 12:41:24 2011 +1100
Added some #ifndefs to stop files being included multiple times.
Signed-off-by: Martin Schwenke <martin at meltin.net>
(cherry picked from commit fdca12c25e6fce6206135b994dedf44265e4eb09)
commit c25bb14f9c8ad2dee9020d85ba0188544ee3495e
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date: Wed Apr 28 15:43:11 2010 +1000
The recent change to the recovery daemon to keep track of and
verify that all nodes agree on the most recent ip address assignments
broke "ctdb moveip ..." since that call would never trigger
a full takeover run and thus would immediately trigger an inconsistency.
Add a new message to the recovery daemon where we can tell the recovery daemon to update its assignments.
BZ62782
(cherry picked from commit e7069082e5f0380dcddee247db8754218ce18cab)
commit d5fdae399b945e013b21b8f61b31f1362d0090db
Author: Michael Adam <obnox at samba.org>
Date: Wed Aug 21 09:16:47 2013 +0200
Fix the build after backporting f3bf2ab61f8dbbc806ec23a68a87aaedd458e712.
This patch (keeping track of public IP assignment in recovery daemon)
which was backported to 1.0.0114 as 9640e2bb889bd99389d9fb247191a19785a75104
renamed "struct _trbt_tree_t" to "struct trbt_tree".
In master, this patch came before the introduction of the delete queue
to the db context. So in the 1.0.114 branch we need to fix up afterwards.
Signed-off-by: Michael Adam <obnox at samba.org>
commit 9f0602327f20dec49b9b9806b2e8b2ef68e7c8e0
Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
Date: Thu Apr 8 14:07:57 2010 +1000
In the recovery daemon, keep track of which node we have assigned public ip
addresses and verify that the remote nodes have/keep a consistent view of
assigned addresses.
If a remote node has an inconsistent view of addresses visavi the recovery
master this will trigger a full ip reallocation.
(cherry picked from commit f3bf2ab61f8dbbc806ec23a68a87aaedd458e712)
Conflicts:
include/ctdb_private.h
-----------------------------------------------------------------------
Summary of changes:
client/ctdb_client.c | 4 +-
common/rb_tree.h | 19 ++-
doc/recovery-process.txt | 4 +-
include/ctdb.h | 8 +-
include/ctdb_private.h | 15 ++-
include/includes.h | 5 +
lib/util/db_wrap.h | 4 +
server/ctdb_banning.c | 37 +++--
server/ctdb_call.c | 14 +--
server/ctdb_freeze.c | 30 +---
server/ctdb_monitor.c | 2 +-
server/ctdb_recover.c | 2 +-
server/ctdb_recoverd.c | 436 +++++++++++++++++++++++++---------------------
server/ctdb_takeover.c | 78 ++++++++-
tcp/ctdb_tcp.h | 6 +-
tests/src/ctdb_bench.c | 2 +-
tests/src/ctdb_fetch.c | 4 +-
tools/ctdb.c | 17 ++-
18 files changed, 411 insertions(+), 276 deletions(-)
Changeset truncated at 500 lines:
diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 45c9fad..e78c60a 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -466,7 +466,7 @@ int ctdb_call(struct ctdb_db_context *ctdb_db, struct ctdb_call *call)
tell the daemon what messaging srvid we will use, and register the message
handler function in the client
*/
-int ctdb_set_message_handler(struct ctdb_context *ctdb, uint64_t srvid,
+int ctdb_client_set_message_handler(struct ctdb_context *ctdb, uint64_t srvid,
ctdb_message_fn_t handler,
void *private_data)
@@ -1857,7 +1857,7 @@ static int ctdb_traverse_ext(struct ctdb_db_context *ctdb_db,
state.fn = fn;
state.listemptyrecords = withemptyrecords;
- ret = ctdb_set_message_handler(ctdb_db->ctdb, srvid, traverse_handler, &state);
+ ret = ctdb_client_set_message_handler(ctdb_db->ctdb, srvid, traverse_handler, &state);
if (ret != 0) {
DEBUG(DEBUG_ERR,("Failed to setup traverse handler\n"));
return -1;
diff --git a/common/rb_tree.h b/common/rb_tree.h
index 9846666..b5ddbb2 100644
--- a/common/rb_tree.h
+++ b/common/rb_tree.h
@@ -17,21 +17,22 @@
along with this program; if not, see <http://www.gnu.org/licenses/>.
*/
-
-
+#ifndef _RB_TREE_H
+#define _RB_TREE_H
+
#define TRBT_RED 0x00
#define TRBT_BLACK 0x01
-typedef struct _trbt_node_t {
- struct _trbt_tree_t *tree;
- struct _trbt_node_t *parent;
- struct _trbt_node_t *left;
- struct _trbt_node_t *right;
+typedef struct trbt_node {
+ struct trbt_tree *tree;
+ struct trbt_node *parent;
+ struct trbt_node *left;
+ struct trbt_node *right;
uint32_t rb_color;
uint32_t key32;
void *data;
} trbt_node_t;
-typedef struct _trbt_tree_t {
+typedef struct trbt_tree {
trbt_node_t *root;
/* automatically free the tree when the last node has been deleted */
#define TRBT_AUTOFREE 0x00000001
@@ -85,3 +86,5 @@ int trbt_traversearray32(trbt_tree_t *tree, uint32_t keylen, int (*callback)(voi
/* Lookup the first node in the tree with a key based on an array of uint32
and return a pointer to data or NULL */
void *trbt_findfirstarray32(trbt_tree_t *tree, uint32_t keylen);
+
+#endif /* _RB_TREE_H */
diff --git a/doc/recovery-process.txt b/doc/recovery-process.txt
index 7780d84..7cfc678 100644
--- a/doc/recovery-process.txt
+++ b/doc/recovery-process.txt
@@ -112,8 +112,8 @@ These tests are performed on all nodes in the cluster which is why it is optimiz
as few network calls to other nodes as possible.
Each node only performs 1 call to the recovery master in each loop and to no other nodes.
-NORMAL NODE CLUSTER MONITORING
-------------------------------
+RECOVERY MASTER CLUSTER MONITORING
+-----------------------------------
The recovery master performs a much more extensive test. In addition to tests 1-9 above
the recovery master also performs the following tests:
diff --git a/include/ctdb.h b/include/ctdb.h
index 7f0ff2f..ec1e5e6 100644
--- a/include/ctdb.h
+++ b/include/ctdb.h
@@ -76,6 +76,12 @@ struct ctdb_call_info {
*/
#define CTDB_SRVID_SET_NODE_FLAGS 0xF400000000000000LL
+/*
+ a message ID to ask the recovery daemon to update the expected node
+ assignment for a public ip
+ */
+#define CTDB_SRVID_RECD_UPDATE_IP 0xF500000000000000LL
+
/*
a message to tell the recovery daemon to fetch a set of records
*/
@@ -259,7 +265,7 @@ uint32_t ctdb_get_num_nodes(struct ctdb_context *ctdb);
/* setup a handler for ctdb messages */
typedef void (*ctdb_message_fn_t)(struct ctdb_context *, uint64_t srvid,
TDB_DATA data, void *);
-int ctdb_set_message_handler(struct ctdb_context *ctdb, uint64_t srvid,
+int ctdb_client_set_message_handler(struct ctdb_context *ctdb, uint64_t srvid,
ctdb_message_fn_t handler,
void *private_data);
diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index a2af9bb..355f361 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -443,7 +443,7 @@ struct ctdb_context {
struct ctdb_call_state *pending_calls;
struct ctdb_client_ip *client_ip_list;
bool do_checkpublicip;
- struct _trbt_tree_t *server_ids;
+ struct trbt_tree *server_ids;
const char *event_script_dir;
const char *notification_script;
const char *default_public_interface;
@@ -472,6 +472,9 @@ struct ctdb_context {
/* mapping from pid to ctdb_client * */
struct ctdb_client_pid_list *client_pids;
+
+ /* used in the recovery daemon to remember the ip allocation */
+ struct trbt_tree *ip_tree;
};
struct ctdb_db_context {
@@ -491,7 +494,7 @@ struct ctdb_db_context {
struct ctdb_vacuum_handle *vacuum_handle;
char *unhealthy_reason;
struct ctdb_persistent_state *persistent_state;
- struct _trbt_tree_t *delete_queue;
+ struct trbt_tree *delete_queue;
int (*ctdb_ltdb_store_fn)(struct ctdb_db_context *ctdb_db,
TDB_DATA key,
struct ctdb_ltdb_header *header,
@@ -1475,7 +1478,7 @@ int ctdb_ctrl_get_all_tunables(struct ctdb_context *ctdb,
uint32_t destnode,
struct ctdb_tunable *tunables);
-int ctdb_start_freeze(struct ctdb_context *ctdb, uint32_t priority);
+void ctdb_start_freeze(struct ctdb_context *ctdb, uint32_t priority);
bool parse_ip_mask(const char *s, const char *iface, ctdb_sock_addr *addr, unsigned *mask);
bool parse_ip_port(const char *s, ctdb_sock_addr *addr);
@@ -1629,7 +1632,7 @@ int ctdb_vacuum_init(struct ctdb_db_context *ctdb_db);
int32_t ctdb_control_enable_script(struct ctdb_context *ctdb, TDB_DATA indata);
int32_t ctdb_control_disable_script(struct ctdb_context *ctdb, TDB_DATA indata);
-int32_t ctdb_local_node_got_banned(struct ctdb_context *ctdb);
+void ctdb_local_node_got_banned(struct ctdb_context *ctdb);
int32_t ctdb_control_set_ban_state(struct ctdb_context *ctdb, TDB_DATA indata);
int32_t ctdb_control_get_ban_state(struct ctdb_context *ctdb, TDB_DATA *outdata);
int32_t ctdb_control_set_db_priority(struct ctdb_context *ctdb, TDB_DATA indata);
@@ -1678,6 +1681,10 @@ void ctdb_run_notification_script(struct ctdb_context *ctdb, const char *event);
void ctdb_fault_setup(void);
+int verify_remote_ip_allocation(struct ctdb_context *ctdb,
+ struct ctdb_all_public_ips *ips);
+int update_ip_assignment_tree(struct ctdb_context *ctdb,
+ struct ctdb_public_ip *ip);
/**
* structure to pass to a schedule_for_deletion_control
diff --git a/include/includes.h b/include/includes.h
index 43a6a5f..667c12f 100644
--- a/include/includes.h
+++ b/include/includes.h
@@ -1,3 +1,6 @@
+#ifndef _CTDB_INCLUDES_H
+#define _CTDB_INCLUDES_H
+
#define HAVE_UNIXSOCKET 1
#include "replace.h"
@@ -58,3 +61,5 @@ _PUBLIC_ int set_blocking(int fd, BOOL set);
#include "lib/util/debug.h"
#include "lib/util/util.h"
+
+#endif /* _CTDB_INCLUDES_H */
diff --git a/lib/util/db_wrap.h b/lib/util/db_wrap.h
index 6ab9362..5ae5fd1 100644
--- a/lib/util/db_wrap.h
+++ b/lib/util/db_wrap.h
@@ -19,6 +19,8 @@
along with this program; if not, see <http://www.gnu.org/licenses/>.
*/
+#ifndef _DB_WRAP_H
+#define _DB_WRAP_H
struct tdb_wrap {
struct tdb_context *tdb;
@@ -30,3 +32,5 @@ struct tdb_wrap {
struct tdb_wrap *tdb_wrap_open(TALLOC_CTX *mem_ctx,
const char *name, int hash_size, int tdb_flags,
int open_flags, mode_t mode);
+
+#endif /* _DB_WRAP_H */
diff --git a/server/ctdb_banning.c b/server/ctdb_banning.c
index 35f1784..0984050 100644
--- a/server/ctdb_banning.c
+++ b/server/ctdb_banning.c
@@ -32,6 +32,21 @@ ctdb_ban_node_event(struct event_context *ev, struct timed_event *te,
struct timeval t, void *private_data)
{
struct ctdb_context *ctdb = talloc_get_type(private_data, struct ctdb_context);
+ bool freeze_failed = false;
+ int i;
+
+ /* Make sure we were able to freeze databases during banning */
+ for (i=1; i<=NUM_DB_PRIORITIES; i++) {
+ if (ctdb->freeze_mode[i] != CTDB_FREEZE_FROZEN) {
+ freeze_failed = true;
+ break;
+ }
+ }
+ if (freeze_failed) {
+ DEBUG(DEBUG_ERR, ("Banning timedout, but still unable to freeze databases\n"));
+ ctdb_ban_self(ctdb);
+ return;
+ }
DEBUG(DEBUG_ERR,("Banning timedout\n"));
ctdb->nodes[ctdb->pnn]->flags &= ~NODE_FLAGS_BANNED;
@@ -42,7 +57,7 @@ ctdb_ban_node_event(struct event_context *ev, struct timed_event *te,
}
}
-int32_t ctdb_local_node_got_banned(struct ctdb_context *ctdb)
+void ctdb_local_node_got_banned(struct ctdb_context *ctdb)
{
uint32_t i;
@@ -57,14 +72,10 @@ int32_t ctdb_local_node_got_banned(struct ctdb_context *ctdb)
ctdb->vnn_map->generation = INVALID_GENERATION;
for (i=1; i<=NUM_DB_PRIORITIES; i++) {
- if (ctdb_start_freeze(ctdb, i) != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to freeze db priority %u\n", i));
- }
+ ctdb_start_freeze(ctdb, i);
}
ctdb_release_all_ips(ctdb);
ctdb->recovery_mode = CTDB_RECOVERY_ACTIVE;
-
- return 0;
}
int32_t ctdb_control_set_ban_state(struct ctdb_context *ctdb, TDB_DATA indata)
@@ -79,12 +90,16 @@ int32_t ctdb_control_set_ban_state(struct ctdb_context *ctdb, TDB_DATA indata)
return -1;
}
if (bantime->time == 0) {
- DEBUG(DEBUG_INFO,("unbanning node %d\n", bantime->pnn));
+ DEBUG(DEBUG_NOTICE,("unbanning node %d\n", bantime->pnn));
ctdb->nodes[bantime->pnn]->flags &= ~NODE_FLAGS_BANNED;
} else {
- DEBUG(DEBUG_INFO,("banning node %d\n", bantime->pnn));
+ DEBUG(DEBUG_NOTICE,("banning node %d\n", bantime->pnn));
if (ctdb->tunable.enable_bans == 0) {
- DEBUG(DEBUG_INFO,("Bans are disabled - ignoring ban of node %u\n", bantime->pnn));
+ /* FIXME: This is bogus. We really should be
+ * taking decision based on the tunables on
+ * the banned node and not local node.
+ */
+ DEBUG(DEBUG_WARNING,("Bans are disabled - ignoring ban of node %u\n", bantime->pnn));
return 0;
}
@@ -121,10 +136,8 @@ int32_t ctdb_control_set_ban_state(struct ctdb_context *ctdb, TDB_DATA indata)
ctdb->nodes[bantime->pnn]->flags |= NODE_FLAGS_BANNED;
event_add_timed(ctdb->ev, ctdb->banning_ctx, timeval_current_ofs(bantime->time,0), ctdb_ban_node_event, ctdb);
- if (bantime->pnn == ctdb->pnn) {
- return ctdb_local_node_got_banned(ctdb);
- }
+ ctdb_local_node_got_banned(ctdb);
return 0;
}
diff --git a/server/ctdb_call.c b/server/ctdb_call.c
index 4812776..0072fcd 100644
--- a/server/ctdb_call.c
+++ b/server/ctdb_call.c
@@ -108,27 +108,15 @@ static void ctdb_send_error(struct ctdb_context *ctdb,
* to its local ctdb (ctdb_request_call). If the node is not itself
* the record's DMASTER, it first redirects the packet to the
* record's LMASTER. The LMASTER then redirects the call packet to
- * the current DMASTER. But there is a race: The record may have
- * been migrated off the DMASTER while the redirected packet is
- * on the wire (or in the local queue). So in case the record has
- * migrated off the new destinaton of the call packet, instead of
- * going back to the LMASTER to get the new DMASTER, we try to
- * reduce rountrips by fist chasing the record a couple of times
- * before giving up the direct chase and finally going back to the
- * LMASTER (again). Note that this works because of this: When
+ * the current DMASTER. Note that this works because of this: When
* a record is migrated off a node, then the new DMASTER is stored
* in the record's copy on the former DMASTER.
- *
- * The maxiumum number of attempts for direct chase to make before
- * going back to the LMASTER is configurable by the tunable
- * "MaxRedirectCount".
*/
static void ctdb_call_send_redirect(struct ctdb_context *ctdb,
TDB_DATA key,
struct ctdb_req_call *c,
struct ctdb_ltdb_header *header)
{
-
uint32_t lmaster = ctdb_lmaster(ctdb, &key);
if (ctdb->pnn == lmaster) {
c->hdr.destnode = header->dmaster;
diff --git a/server/ctdb_freeze.c b/server/ctdb_freeze.c
index 74952e1..1f4ebdf 100644
--- a/server/ctdb_freeze.c
+++ b/server/ctdb_freeze.c
@@ -250,34 +250,29 @@ static int ctdb_freeze_waiter_destructor(struct ctdb_freeze_waiter *w)
/*
start the freeze process for a certain priority
*/
-int ctdb_start_freeze(struct ctdb_context *ctdb, uint32_t priority)
+void ctdb_start_freeze(struct ctdb_context *ctdb, uint32_t priority)
{
- if (priority == 0) {
- DEBUG(DEBUG_ERR,("Freeze priority 0 requested, remapping to priority 1\n"));
- priority = 1;
- }
-
if ((priority < 1) || (priority > NUM_DB_PRIORITIES)) {
DEBUG(DEBUG_ERR,(__location__ " Invalid db priority : %u\n", priority));
- return -1;
+ ctdb_fatal(ctdb, "Internal error");
}
if (ctdb->freeze_mode[priority] == CTDB_FREEZE_FROZEN) {
/* we're already frozen */
- return 0;
+ return;
}
+ DEBUG(DEBUG_ERR, ("Freeze priority %u\n", priority));
+
/* Stop any vacuuming going on: we don't want to wait. */
ctdb_stop_vacuuming(ctdb);
/* if there isn't a freeze lock child then create one */
if (ctdb->freeze_handles[priority] == NULL) {
ctdb->freeze_handles[priority] = ctdb_freeze_lock(ctdb, priority);
- CTDB_NO_MEMORY(ctdb, ctdb->freeze_handles[priority]);
+ CTDB_NO_MEMORY_FATAL(ctdb, ctdb->freeze_handles[priority]);
ctdb->freeze_mode[priority] = CTDB_FREEZE_PENDING;
}
-
- return 0;
}
/*
@@ -290,8 +285,6 @@ int32_t ctdb_control_freeze(struct ctdb_context *ctdb, struct ctdb_req_control *
priority = (uint32_t)c->srvid;
- DEBUG(DEBUG_ERR, ("Freeze priority %u\n", priority));
-
if (priority == 0) {
DEBUG(DEBUG_ERR,("Freeze priority 0 requested, remapping to priority 1\n"));
priority = 1;
@@ -303,14 +296,12 @@ int32_t ctdb_control_freeze(struct ctdb_context *ctdb, struct ctdb_req_control *
}
if (ctdb->freeze_mode[priority] == CTDB_FREEZE_FROZEN) {
+ DEBUG(DEBUG_ERR, ("Freeze priority %u\n", priority));
/* we're already frozen */
return 0;
}
- if (ctdb_start_freeze(ctdb, priority) != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to start freezing databases with priority %u\n", priority));
- return -1;
- }
+ ctdb_start_freeze(ctdb, priority);
/* add ourselves to list of waiters */
if (ctdb->freeze_handles[priority] == NULL) {
@@ -341,10 +332,7 @@ bool ctdb_blocking_freeze(struct ctdb_context *ctdb)
int i;
for (i=1; i<=NUM_DB_PRIORITIES; i++) {
- if (ctdb_start_freeze(ctdb, i)) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to freeze databases of prio %u\n", i));
- continue;
- }
+ ctdb_start_freeze(ctdb, i);
/* block until frozen */
while (ctdb->freeze_mode[i] == CTDB_FREEZE_PENDING) {
diff --git a/server/ctdb_monitor.c b/server/ctdb_monitor.c
index 81ea69c..24791f7 100644
--- a/server/ctdb_monitor.c
+++ b/server/ctdb_monitor.c
@@ -479,7 +479,7 @@ int32_t ctdb_control_modflags(struct ctdb_context *ctdb, TDB_DATA indata)
/* if we have become banned, we should go into recovery mode */
if ((node->flags & NODE_FLAGS_BANNED) && !(c->old_flags & NODE_FLAGS_BANNED) && (node->pnn == ctdb->pnn)) {
- return ctdb_local_node_got_banned(ctdb);
+ ctdb_local_node_got_banned(ctdb);
}
return 0;
diff --git a/server/ctdb_recover.c b/server/ctdb_recover.c
index 4794e63..ed6fe41 100644
--- a/server/ctdb_recover.c
+++ b/server/ctdb_recover.c
@@ -1120,7 +1120,7 @@ static int store_tdb_record(struct ctdb_context *ctdb,
/* use a non-blocking lock */
if (tdb_chainlock_nonblock(ctdb_db->ltdb->tdb, key) != 0) {
- DEBUG(DEBUG_ERR, (__location__ " Failed to lock chain\n"));
+ DEBUG(DEBUG_INFO, (__location__ " Failed to lock chain in non-blocking mode\n"));
return -1;
}
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index db2a2ca..0426258 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -80,13 +80,13 @@ static void ctdb_ban_node(struct ctdb_recoverd *rec, uint32_t pnn, uint32_t ban_
struct ctdb_context *ctdb = rec->ctdb;
struct ctdb_ban_time bantime;
- DEBUG(DEBUG_NOTICE,("Banning node %u for %u seconds\n", pnn, ban_time));
-
if (!ctdb_validate_pnn(ctdb, pnn)) {
DEBUG(DEBUG_ERR,("Bad pnn %u in ctdb_ban_node\n", pnn));
return;
}
+ DEBUG(DEBUG_NOTICE,("Banning node %u for %u seconds\n", pnn, ban_time));
+
bantime.pnn = pnn;
bantime.time = ban_time;
@@ -141,6 +141,12 @@ static void ctdb_set_culprit_count(struct ctdb_recoverd *rec, uint32_t culprit,
return;
}
+ /* If we are banned or stopped, do not set other nodes as culprits */
+ if (rec->node_flags & NODE_FLAGS_INACTIVE) {
+ DEBUG(DEBUG_NOTICE, ("This node is INACTIVE, cannot set culprit node %d\n", culprit));
+ return;
+ }
+
if (ctdb->nodes[culprit]->ban_state == NULL) {
ctdb->nodes[culprit]->ban_state = talloc_zero(ctdb->nodes[culprit], struct ctdb_banning_state);
CTDB_NO_MEMORY_VOID(ctdb, ctdb->nodes[culprit]->ban_state);
@@ -966,7 +972,7 @@ static int update_local_flags(struct ctdb_recoverd *rec, struct ctdb_node_map *n
Since we are the recovery master we can just as
well update the flags on all nodes.
*/
- ret = ctdb_ctrl_modflags(ctdb, CONTROL_TIMEOUT(), nodemap->nodes[j].pnn, nodemap->nodes[j].flags, ~nodemap->nodes[j].flags);
+ ret = ctdb_ctrl_modflags(ctdb, CONTROL_TIMEOUT(), nodemap->nodes[j].pnn, remote_nodemap->nodes[j].flags, ~remote_nodemap->nodes[j].flags);
if (ret != 0) {
DEBUG(DEBUG_ERR, (__location__ " Unable to update nodeflags on remote nodes\n"));
return -1;
@@ -1234,6 +1240,7 @@ static void reload_nodes_file(struct ctdb_context *ctdb)
}
static int ctdb_reload_remote_public_ips(struct ctdb_context *ctdb,
+ struct ctdb_recoverd *rec,
struct ctdb_node_map *nodemap,
uint32_t *culprit)
{
@@ -1280,6 +1287,11 @@ static int ctdb_reload_remote_public_ips(struct ctdb_context *ctdb,
return -1;
}
+ if (verify_remote_ip_allocation(ctdb, ctdb->nodes[j]->known_public_ips)) {
+ DEBUG(DEBUG_ERR,("Node %d has inconsistent public ip allocation and needs update.\n", ctdb->nodes[j]->pnn));
+ rec->need_takeover_run = true;
+ }
+
/* grab a new shiny list of public ips from the node */
ret = ctdb_ctrl_get_public_ips_flags(ctdb,
CONTROL_TIMEOUT(),
@@ -1300,6 +1312,37 @@ static int ctdb_reload_remote_public_ips(struct ctdb_context *ctdb,
return 0;
}
+
+static void ban_misbehaving_nodes(struct ctdb_recoverd *rec, bool *self_ban)
+{
+ struct ctdb_context *ctdb = rec->ctdb;
+ int i;
+ struct ctdb_banning_state *ban_state;
+
+ *self_ban = false;
--
CTDB repository
More information about the samba-cvs
mailing list