[SCM] CTDB repository - branch 2.5 updated - ctdb-2.5.3-62-g7f3613c
Amitay Isaacs
amitay at samba.org
Fri Jun 20 00:07:04 MDT 2014
The branch, 2.5 has been updated
via 7f3613c510c4549e381a78291ca87c76ece91710 (commit)
via baff290202f8c72f5f218a534a41c20ee003862b (commit)
via bc633b143b440f8881f34165d2d6eee00fbec588 (commit)
via ad471d68a29a48a298ca70d1b3d89a3608e9b6a9 (commit)
via c2247f4a4824be2d5a96e7cafb9e8c6de9236d1c (commit)
via 7ac180a2efd0ea52c29e9f2e33f3e032466e91d5 (commit)
via a3a2034ef34aef3ac11fd623161e2ff307847188 (commit)
via fe7705bcc50b8769d6ee5966943996bd2beb52f7 (commit)
via 4d6f3660d91f12002118fec266b0a898a248b1ba (commit)
via f815281b5d69cf36bb653685295a78243696fac2 (commit)
via 6415edcc7190b15fc513e011836c09b6bef9181d (commit)
via 3dc9b4ada14eb0e830e0604e0758a273aef7a93f (commit)
from c6f91dd3df0f8604a2eadef3c44025e07a1c1a03 (commit)
http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=2.5
- Log -----------------------------------------------------------------
commit 7f3613c510c4549e381a78291ca87c76ece91710
Author: Michael Adam <obnox at samba.org>
Date: Sat Feb 15 01:36:06 2014 +0100
vacuum: always run freelist_size again
and not only if repack_limit != 0. This partially reverts
commit 48f2d1158820bfb063ba0a0bbfb6f496a8e7522.
With the new tdb code this defragments the
free list by merging adjacent records.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 5334881afab42eae77bb2015ec21cbfe1df87807)
commit baff290202f8c72f5f218a534a41c20ee003862b
Author: Michael Adam <obnox at samba.org>
Date: Tue Apr 22 22:09:35 2014 +0200
vacuum: add missing return to ctdb_vacuum_traverse_db() error path.
This got lost in commit 19948702992c94553e1a611540ad398de9f9d8b9
("ctdb-vacuum: make ctdb_vacuum_traverse_db() void.")
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 026d79cb009beba6987da6a6dd5fd98609140136)
commit bc633b143b440f8881f34165d2d6eee00fbec588
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 03:36:49 2014 +0200
vacuum: remove now unused talloc ctx argument from ctdb_vacuum_db()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit b8658b395921a5400c9f794a07748f5ad18991f8)
commit ad471d68a29a48a298ca70d1b3d89a3608e9b6a9
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 03:34:05 2014 +0200
vacuum: move init of vdata into init_vdata funcion
This is a small code cleanup.
vdata is only used in ctdb_vacuum_db() and not in
ctdb_vacuum_and_repack_db() where it is currently initialized.
This patch moves creation and all previously scattered
initialization of vacuum_data into ctdb_vacuum_init_vacuum_data
which is called from ctdb_vacuum_db.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit c3cb8c277a02a8a68c11ef8d341c8116172e989b)
commit c2247f4a4824be2d5a96e7cafb9e8c6de9236d1c
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 03:08:20 2014 +0200
vacuum: remove vacuum limit from vdata - not used
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 3cf018935e057c1748ab44491135c632c023de9f)
commit 7ac180a2efd0ea52c29e9f2e33f3e032466e91d5
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 03:02:42 2014 +0200
vacuum: remove a superfluous comment.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit a99035a4c52f68a4a4f1862c74c1c71273a47d5b)
commit a3a2034ef34aef3ac11fd623161e2ff307847188
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:59:51 2014 +0200
vacuum: untangle assignmend and check for return of tdb_repack()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 385e2236fbc2a4dca97c3b2f83c592f99abaee25)
commit fe7705bcc50b8769d6ee5966943996bd2beb52f7
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:57:25 2014 +0200
vacuum: remove a superfluous and misleading comment
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit e0a11369cf85b111e0966f2677315345ddaaac76)
commit 4d6f3660d91f12002118fec266b0a898a248b1ba
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:57:00 2014 +0200
vacuum: remove now unused ctdb_repack_tdb().
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 413f99f86e26a8b12a807d0c2436c78805983566)
commit f815281b5d69cf36bb653685295a78243696fac2
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:53:29 2014 +0200
vacuum: use plain tdb_repack() instead of ctdb_repack_tdb()
Since we usually have 0 records left for repack-deletion,
repacking is essentially used for the purpose of defragmenting
the freelist, we can use the vanilla tdb_repack function.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 368683d7af747e3d67c48a702ff315a2022c6f85)
commit 6415edcc7190b15fc513e011836c09b6bef9181d
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:47:21 2014 +0200
vacuum: remove a comment in ctdb_vacuum_and_repack_db()
The repack operation now mainly defragments the freelist
and does not usually delete any records any more.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit ec2f1abe486c77f8a27ebc731917337efc9cd04f)
commit 3dc9b4ada14eb0e830e0604e0758a273aef7a93f
Author: Michael Adam <obnox at samba.org>
Date: Sat Apr 19 02:51:39 2014 +0200
vacuum: adapt debug message for repacking.
Now we usually have records to delete == 0 after the preceding
vacuum run. Anyways, deletion is not a major aspect any more
of the repack run and will vanish soon.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
(Imported from commit 67e5746b236cca860d02f741411b567a37a9cf4e)
-----------------------------------------------------------------------
Summary of changes:
server/ctdb_vacuum.c | 238 +++++++++++--------------------------------------
1 files changed, 54 insertions(+), 184 deletions(-)
Changeset truncated at 500 lines:
diff --git a/server/ctdb_vacuum.c b/server/ctdb_vacuum.c
index 574ad87..ce3c600 100644
--- a/server/ctdb_vacuum.c
+++ b/server/ctdb_vacuum.c
@@ -53,7 +53,6 @@ struct ctdb_vacuum_handle {
/* a list of records to possibly delete */
struct vacuum_data {
- uint32_t repack_limit;
struct ctdb_context *ctdb;
struct ctdb_db_context *ctdb_db;
struct tdb_context *dest_db;
@@ -745,6 +744,7 @@ static void ctdb_vacuum_traverse_db(struct ctdb_db_context *ctdb_db,
if (ret == -1 || vdata->traverse_error) {
DEBUG(DEBUG_ERR, (__location__ " Traverse error in vacuuming "
"'%s'\n", ctdb_db->db_name));
+ return;
}
if (vdata->count.db_traverse.total > 0) {
@@ -1171,11 +1171,29 @@ done:
/**
* initialize the vacuum_data
*/
-static int ctdb_vacuum_init_vacuum_data(struct ctdb_db_context *ctdb_db,
- struct vacuum_data *vdata)
+static struct vacuum_data *ctdb_vacuum_init_vacuum_data(
+ struct ctdb_db_context *ctdb_db,
+ TALLOC_CTX *mem_ctx)
{
int i;
struct ctdb_context *ctdb = ctdb_db->ctdb;
+ struct vacuum_data *vdata;
+
+ vdata = talloc_zero(mem_ctx, struct vacuum_data);
+ if (vdata == NULL) {
+ DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
+ return NULL;
+ }
+
+ vdata->ctdb = ctdb_db->ctdb;
+ vdata->ctdb_db = ctdb_db;
+ vdata->delete_list = trbt_create(vdata, 0);
+ if (vdata->delete_list == NULL) {
+ DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
+ goto fail;
+ }
+
+ vdata->start = timeval_current();
vdata->count.delete_queue.added_to_delete_list = 0;
vdata->count.delete_queue.added_to_vacuum_fetch_list = 0;
@@ -1200,7 +1218,7 @@ static int ctdb_vacuum_init_vacuum_data(struct ctdb_db_context *ctdb_db,
ctdb->num_nodes);
if (vdata->vacuum_fetch_list == NULL) {
DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
- return -1;
+ goto fail;
}
for (i = 0; i < ctdb->num_nodes; i++) {
vdata->vacuum_fetch_list[i] = (struct ctdb_marshall_buffer *)
@@ -1208,12 +1226,17 @@ static int ctdb_vacuum_init_vacuum_data(struct ctdb_db_context *ctdb_db,
offsetof(struct ctdb_marshall_buffer, data));
if (vdata->vacuum_fetch_list[i] == NULL) {
DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
- return -1;
+ talloc_free(vdata);
+ return NULL;
}
vdata->vacuum_fetch_list[i]->db_id = ctdb_db->db_id;
}
- return 0;
+ return vdata;
+
+fail:
+ talloc_free(vdata);
+ return NULL;
}
/**
@@ -1249,11 +1272,12 @@ static int ctdb_vacuum_init_vacuum_data(struct ctdb_db_context *ctdb_db,
* This executes in the child context.
*/
static int ctdb_vacuum_db(struct ctdb_db_context *ctdb_db,
- struct vacuum_data *vdata,
bool full_vacuum_run)
{
struct ctdb_context *ctdb = ctdb_db->ctdb;
int ret, pnn;
+ struct vacuum_data *vdata;
+ TALLOC_CTX *tmp_ctx;
DEBUG(DEBUG_INFO, (__location__ " Entering %s vacuum run for db "
"%s db_id[0x%08x]\n",
@@ -1274,9 +1298,16 @@ static int ctdb_vacuum_db(struct ctdb_db_context *ctdb_db,
ctdb->pnn = pnn;
- ret = ctdb_vacuum_init_vacuum_data(ctdb_db, vdata);
- if (ret != 0) {
- return ret;
+ tmp_ctx = talloc_new(ctdb_db);
+ if (tmp_ctx == NULL) {
+ DEBUG(DEBUG_ERR, ("Out of memory!\n"));
+ return -1;
+ }
+
+ vdata = ctdb_vacuum_init_vacuum_data(ctdb_db, tmp_ctx);
+ if (vdata == NULL) {
+ talloc_free(tmp_ctx);
+ return -1;
}
if (full_vacuum_run) {
@@ -1289,186 +1320,34 @@ static int ctdb_vacuum_db(struct ctdb_db_context *ctdb_db,
ctdb_process_delete_list(ctdb_db, vdata);
+ talloc_free(tmp_ctx);
+
/* this ensures we run our event queue */
ctdb_ctrl_getpnn(ctdb, TIMELIMIT(), CTDB_CURRENT_NODE);
return 0;
}
-
-/*
- * traverse function for repacking
- */
-static int repack_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data,
- void *private_data)
-{
- struct vacuum_data *vdata = (struct vacuum_data *)private_data;
-
- if (vdata->vacuum) {
- uint32_t hash = ctdb_hash(&key);
- struct delete_record_data *kd;
- /*
- * check if we can ignore this record because it's in the delete_list
- */
- kd = (struct delete_record_data *)trbt_lookup32(vdata->delete_list, hash);
- /*
- * there might be hash collisions so we have to compare the keys here to be sure
- */
- if (kd && kd->key.dsize == key.dsize && memcmp(kd->key.dptr, key.dptr, key.dsize) == 0) {
- struct ctdb_ltdb_header *hdr = (struct ctdb_ltdb_header *)data.dptr;
- /*
- * we have to check if the record hasn't changed in the meantime in order to
- * savely remove it from the database
- */
- if (data.dsize == sizeof(struct ctdb_ltdb_header) &&
- hdr->dmaster == kd->ctdb->pnn &&
- ctdb_lmaster(kd->ctdb, &(kd->key)) == kd->ctdb->pnn &&
- kd->hdr.rsn == hdr->rsn) {
- vdata->count.repack.vacuumed++;
- return 0;
- }
- }
- }
- if (tdb_store(vdata->dest_db, key, data, TDB_INSERT) != 0) {
- vdata->traverse_error = true;
- return -1;
- }
- vdata->count.repack.copied++;
- return 0;
-}
-
-/*
- * repack a tdb
- */
-static int ctdb_repack_tdb(struct tdb_context *tdb, TALLOC_CTX *mem_ctx, struct vacuum_data *vdata)
-{
- struct tdb_context *tmp_db;
-
- if (tdb_transaction_start(tdb) != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to start transaction\n"));
- return -1;
- }
-
- tmp_db = tdb_open("tmpdb", tdb_hash_size(tdb),
- TDB_INTERNAL|TDB_DISALLOW_NESTING,
- O_RDWR|O_CREAT, 0);
- if (tmp_db == NULL) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to create tmp_db\n"));
- tdb_transaction_cancel(tdb);
- return -1;
- }
-
- vdata->traverse_error = false;
- vdata->dest_db = tmp_db;
- vdata->vacuum = true;
- vdata->count.repack.vacuumed = 0;
- vdata->count.repack.copied = 0;
-
- /*
- * repack and vacuum on-the-fly by not writing the records that are
- * no longer needed
- */
- if (tdb_traverse_read(tdb, repack_traverse, vdata) == -1) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to traverse copying out\n"));
- tdb_transaction_cancel(tdb);
- tdb_close(tmp_db);
- return -1;
- }
-
- DEBUG(DEBUG_INFO,(__location__ " %u records vacuumed\n",
- vdata->count.repack.vacuumed));
-
- if (vdata->traverse_error) {
- DEBUG(DEBUG_ERR,(__location__ " Error during traversal\n"));
- tdb_transaction_cancel(tdb);
- tdb_close(tmp_db);
- return -1;
- }
-
- if (tdb_wipe_all(tdb) != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to wipe database\n"));
- tdb_transaction_cancel(tdb);
- tdb_close(tmp_db);
- return -1;
- }
-
- vdata->traverse_error = false;
- vdata->dest_db = tdb;
- vdata->vacuum = false;
- vdata->count.repack.copied = 0;
-
- if (tdb_traverse_read(tmp_db, repack_traverse, vdata) == -1) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to traverse copying back\n"));
- tdb_transaction_cancel(tdb);
- tdb_close(tmp_db);
- return -1;
- }
-
- if (vdata->traverse_error) {
- DEBUG(DEBUG_ERR,(__location__ " Error during second traversal\n"));
- tdb_transaction_cancel(tdb);
- tdb_close(tmp_db);
- return -1;
- }
-
- tdb_close(tmp_db);
-
-
- if (tdb_transaction_commit(tdb) != 0) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to commit\n"));
- return -1;
- }
- DEBUG(DEBUG_INFO,(__location__ " %u records copied\n",
- vdata->count.repack.copied));
-
- return 0;
-}
-
/*
* repack and vaccum a db
* called from the child context
*/
static int ctdb_vacuum_and_repack_db(struct ctdb_db_context *ctdb_db,
- TALLOC_CTX *mem_ctx,
bool full_vacuum_run)
{
uint32_t repack_limit = ctdb_db->ctdb->tunable.repack_limit;
const char *name = ctdb_db->db_name;
int freelist_size = 0;
- struct vacuum_data *vdata;
-
- vdata = talloc_zero(mem_ctx, struct vacuum_data);
- if (vdata == NULL) {
- DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
- return -1;
- }
-
- vdata->ctdb = ctdb_db->ctdb;
- vdata->repack_limit = repack_limit;
- vdata->delete_list = trbt_create(vdata, 0);
- vdata->ctdb_db = ctdb_db;
- if (vdata->delete_list == NULL) {
- DEBUG(DEBUG_ERR,(__location__ " Out of memory\n"));
- talloc_free(vdata);
- return -1;
- }
+ int ret;
- vdata->start = timeval_current();
-
- /*
- * gather all records that can be deleted in vdata
- */
- if (ctdb_vacuum_db(ctdb_db, vdata, full_vacuum_run) != 0) {
+ if (ctdb_vacuum_db(ctdb_db, full_vacuum_run) != 0) {
DEBUG(DEBUG_ERR,(__location__ " Failed to vacuum '%s'\n", name));
}
- if (repack_limit != 0) {
- freelist_size = tdb_freelist_size(ctdb_db->ltdb->tdb);
- if (freelist_size == -1) {
- DEBUG(DEBUG_ERR,(__location__ " Failed to get freelist size for '%s'\n", name));
- talloc_free(vdata);
- return -1;
- }
+ freelist_size = tdb_freelist_size(ctdb_db->ltdb->tdb);
+ if (freelist_size == -1) {
+ DEBUG(DEBUG_ERR,(__location__ " Failed to get freelist size for '%s'\n", name));
+ return -1;
}
/*
@@ -1476,22 +1355,17 @@ static int ctdb_vacuum_and_repack_db(struct ctdb_db_context *ctdb_db,
*/
if ((repack_limit == 0 || (uint32_t)freelist_size < repack_limit))
{
- talloc_free(vdata);
return 0;
}
- DEBUG(DEBUG_INFO,("Repacking %s with %u freelist entries and %u records to delete\n",
- name, freelist_size, vdata->count.delete_list.left));
+ DEBUG(DEBUG_INFO, ("Repacking %s with %u freelist entries\n",
+ name, freelist_size));
- /*
- * repack and implicitely get rid of the records we can delete
- */
- if (ctdb_repack_tdb(ctdb_db->ltdb->tdb, mem_ctx, vdata) != 0) {
+ ret = tdb_repack(ctdb_db->ltdb->tdb);
+ if (ret != 0) {
DEBUG(DEBUG_ERR,(__location__ " Failed to repack '%s'\n", name));
- talloc_free(vdata);
return -1;
}
- talloc_free(vdata);
return 0;
}
@@ -1641,16 +1515,12 @@ ctdb_vacuum_event(struct event_context *ev, struct timed_event *te,
_exit(1);
}
- /*
- * repack the db
- */
if ((ctdb->tunable.vacuum_fast_path_count > 0) &&
(vacuum_handle->fast_path_count == 0))
{
full_vacuum_run = true;
}
- cc = ctdb_vacuum_and_repack_db(ctdb_db, child_ctx,
- full_vacuum_run);
+ cc = ctdb_vacuum_and_repack_db(ctdb_db, full_vacuum_run);
write(child_ctx->fd[1], &cc, 1);
_exit(0);
--
CTDB repository
More information about the samba-cvs
mailing list