[SCM] Samba Shared Repository - branch master updated
Amitay Isaacs
amitay at samba.org
Wed Mar 5 19:33:04 MST 2014
The branch, master has been updated
via d8e110e ctdb-vacuum: fix delete list counts in delete_marshall_traverse_first
via 5b81848 ctdb-vacuum: fix possible cause for delelete_list processing counts left records > 0
via 551e9d7 ctdb-vacuum: systematize counters into various structs
via ec3f33c ctdb-vacuum: remove unused counter vdata->total
via 49eb503 ctdb-vacuum: make ctdb_process_delete_list() void
via 776d4e8 ctdb-vacuum: make ctdb_process_vacuum_fetch_lists() void.
via 1994870 ctdb-vacuum: make ctdb_vacuum_traverse_db() void.
via 7983946 ctdb-vacuum: don't stop in process_vacuum_fetch_lists when sending to one node fails.
via f3483de ctdb-vacuum: catch and log errors to traverse the delete list in ctdb_process_delete_list()
via 72ea975 ctdb-vacuum: catch and log error of traverse in ctdb_process_delete_queue()
via 83fa09e ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_marshall_traverse_first()
via 974aa73 ctdb-vacuum: simplify delete_marshall_traverse_first: use tdb_null
via 16837bc ctdb-vacuum: remove VacuumLimit criterion for triggering a repack
via 6fdd250 ctdb-vacuum: treat value 0 of tunable VacuumLimit as turning off repacking
via 1b2fc1f ctdb-vacuum: add consistency check for counts at end of process_delete_list()
via cf407d2 ctdb-vacuum: log error if records are left for deletion after ctdb_process_delete_list()
via 23be632 ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_record_traverse()
via be2f1a0 ctdb-vacuum: update comment for ctdb_process_delete_queue
via cd877b6 ctdb-vacuum: rename ctdb_vacuum_db_fast --> ctdb_process_delete_queue
via c4478bd ctdb-vacuum: update comment for ctdb_vacuum_traverse_db
via 0309d5e ctdb-vacuum: rename ctdb_vacuum_db_full --> ctdb_vacuum_traverse_db
via ba49deb ctdb-vacuum: change full db traverse vacuuming to fill delete queue
via d0b7b38 ctdb-vacuum: run the fast vacuum after the db traverse
via 04c2115 ctdb-vacuum: rename private->private_data in repack_traverse
via 951efa1 ctdb-vacuum: rename private->private_data in vacuum_traverse
via 01f359c ctdb-vacuum: extract check for full vacuum run out of ctdb_vacuum_db_full()
via c88fd19 ctdb-vacuum: add consistency check for counts to ctdb_vacuum_db_fast()
via 5d5907c ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_queue_traverse()
via fe68b3c ctdb-vacuum: simplify delete_record_traverse() - free treats NULL
via 593bddf ctdb-vacuum: simplify delete_queue_traverse() - free treats NULL pointers.
via 24bec3d ctdb-vacuum: reduce indentation in delete_queue_traverse
via 48f2d11 ctdb-vacuum: treat value 0 of tunable RepackLimit as turned off.
via af5568b ctdb-vacuum: fix treatment of remaining records and statistics in delete_record_traverse()
via b4e0b01 ctdb-vacuum: cast freelist_size in comparison.
via 6a46a25 ctdb-vacuum: improve output of delete list statistics
from 9869358 s3:smb2_server: avoid talloc_zero_array() in smbd_smb2_request_setup_out()
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit d8e110ed7dacda18860cce0c86e4e44f0b83dd42
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 21 23:43:17 2014 +0100
ctdb-vacuum: fix delete list counts in delete_marshall_traverse_first
when bumping skipped, decrement left, so the sum is correct
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay at samba.org>
Autobuild-Date(master): Thu Mar 6 03:32:33 CET 2014 on sn-devel-104
commit 5b81848e50b43b7ab7889f5217e05ca42e452c8f
Author: Michael Adam <obnox at samba.org>
Date: Thu Feb 20 00:58:17 2014 +0100
ctdb-vacuum: fix possible cause for delelete_list processing counts left records > 0
We need to have left records == 0 at the end of the delete list processing.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 551e9d791c146473b45f8a9fb1574e0ad7cca6b2
Author: Michael Adam <obnox at samba.org>
Date: Thu Feb 20 00:32:08 2014 +0100
ctdb-vacuum: systematize counters into various structs
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit ec3f33c314207f93529d7c9a2bfb82bf05e3a08b
Author: Michael Adam <obnox at samba.org>
Date: Thu Feb 20 00:29:47 2014 +0100
ctdb-vacuum: remove unused counter vdata->total
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 49eb503c5d3133e1476a15f9d11ce4269407e6c6
Author: Michael Adam <obnox at samba.org>
Date: Sun Feb 16 01:08:18 2014 +0100
ctdb-vacuum: make ctdb_process_delete_list() void
The overall return code was not really used anyways.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 776d4e88f2a6068016dedd37003cdde10f8090a5
Author: Michael Adam <obnox at samba.org>
Date: Sun Feb 16 00:37:43 2014 +0100
ctdb-vacuum: make ctdb_process_vacuum_fetch_lists() void.
This constantly returns 0 anyways.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 19948702992c94553e1a611540ad398de9f9d8b9
Author: Michael Adam <obnox at samba.org>
Date: Sun Feb 16 00:35:34 2014 +0100
ctdb-vacuum: make ctdb_vacuum_traverse_db() void.
Failure in traversal of the DB should not
prevent further processing.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 7983946680ac0de8f82dfee6f0f849a11653d042
Author: Michael Adam <obnox at samba.org>
Date: Sun Feb 16 00:26:00 2014 +0100
ctdb-vacuum: don't stop in process_vacuum_fetch_lists when sending to one node fails.
We should try to continue vacuuming as much as possible.
Failure to send records to one lmaster doesn't mean the
others will fail too.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit f3483de240987f05cc20f747ac467c8ed81bb03e
Author: Michael Adam <obnox at samba.org>
Date: Sat Feb 15 18:06:09 2014 +0100
ctdb-vacuum: catch and log errors to traverse the delete list in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 72ea9759930bd29b59c0f831d2cb2f4f1e2e643d
Author: Michael Adam <obnox at samba.org>
Date: Sat Feb 15 17:59:22 2014 +0100
ctdb-vacuum: catch and log error of traverse in ctdb_process_delete_queue()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 83fa09e78c6ca8e08cb2659f013a05b4b340f0aa
Author: Michael Adam <obnox at samba.org>
Date: Sat Feb 15 13:03:51 2014 +0100
ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_marshall_traverse_first()
Spare malloc and free.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 974aa73160d50b7cf63b4a5e6dd7a7b1408ece42
Author: Michael Adam <obnox at samba.org>
Date: Sat Feb 15 13:01:33 2014 +0100
ctdb-vacuum: simplify delete_marshall_traverse_first: use tdb_null
we know anyways the record to store is empty at this point.
So skip pointer calculations.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 16837bc309aa9a86fc21d7f59a8fce0b947428a3
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 22:05:21 2014 +0100
ctdb-vacuum: remove VacuumLimit criterion for triggering a repack
With the new vacuuming, we consider it an error if there are
records left for deletion after processing the various lists.
All records that can be deleted should have been deleted by
tdb_delete calls.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 6fdd25008f384408dfc103b90ab40b8e64ce18b0
Author: Michael Adam <obnox at samba.org>
Date: Wed Feb 12 17:41:28 2014 +0100
ctdb-vacuum: treat value 0 of tunable VacuumLimit as turning off repacking
I.e. no number of records found to delete will trigger the
repacking.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 1b2fc1f096f80db5974eab021f12f0ad9af24882
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 22:02:41 2014 +0100
ctdb-vacuum: add consistency check for counts at end of process_delete_list()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit cf407d208afdc70b31ce5013591c869b36f588f1
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 22:01:38 2014 +0100
ctdb-vacuum: log error if records are left for deletion after ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 23be632449524e74d73fcb6fd3875a6d5a428d89
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 21:50:59 2014 +0100
ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_record_traverse()
Spare malloc and free.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit be2f1a0c790d08571dd757fb7b2941d367175008
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:48:02 2014 +0100
ctdb-vacuum: update comment for ctdb_process_delete_queue
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit cd877b6a21a5080b3d9ae8ee7ac8cf27c4fd9512
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:47:25 2014 +0100
ctdb-vacuum: rename ctdb_vacuum_db_fast --> ctdb_process_delete_queue
This describes more precisely what this does.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit c4478bd40daee22bbe5696bb17d0a1bff164c7f7
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:46:49 2014 +0100
ctdb-vacuum: update comment for ctdb_vacuum_traverse_db
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 0309d5eda4f28c7e99e93e0da6c02757ea0cae8f
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:42:37 2014 +0100
ctdb-vacuum: rename ctdb_vacuum_db_full --> ctdb_vacuum_traverse_db
This describes more precisely what it actually is, nowadays.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit ba49deb2344c0a9a8f76c9fd0136bdeadad6af89
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:38:31 2014 +0100
ctdb-vacuum: change full db traverse vacuuming to fill delete queue
This lets the "fast vacuum" delete queue traverse do the actual work.
On the positive side, we note that this lets the "full vacuuming"
treat the records that have never been migrated with data correctly.
These had previously been added to the delete list for complicated
cross-node deletion instead of directly deleting them.
On the other hand side, there might be a slight overhead
since the records are read again in the delete queu traverse,
but this is OK because this change is in preparation of
untangling the db traverse altogether from the vacuum run,
making it independent.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit d0b7b3882511769b1bfc1d0d4fdc0dba288e6ccd
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:27:14 2014 +0100
ctdb-vacuum: run the fast vacuum after the db traverse
This in preparation of modifying the db traverse to
fill the delete_queue that is processed by the fast
vacuum run, instead of filling the same lists as the
fast vacuum run for further processing.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 04c2115b606f2346fc7315a503b3dae0189e0737
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:08:20 2014 +0100
ctdb-vacuum: rename private->private_data in repack_traverse
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 951efa1097a113910c06ce78d1c9fb70e3f4d75e
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:07:55 2014 +0100
ctdb-vacuum: rename private->private_data in vacuum_traverse
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 01f359cafccb5ae3bea312d628dad92746520527
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 18:03:02 2014 +0100
ctdb-vacuum: extract check for full vacuum run out of ctdb_vacuum_db_full()
This is more consistent.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit c88fd19714b98769887dbff59d8c1d077cf351d5
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 17:58:01 2014 +0100
ctdb-vacuum: add consistency check for counts to ctdb_vacuum_db_fast()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 5d5907c7cf09567e73092578917624c8789c7471
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 15:28:22 2014 +0100
ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_queue_traverse()
this spares malloc and free
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit fe68b3c4942a4660c9b35c6316856644c32f5631
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 15:35:01 2014 +0100
ctdb-vacuum: simplify delete_record_traverse() - free treats NULL
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 593bddf2e82fcb9666449c40625b972ff9c7961c
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 15:34:23 2014 +0100
ctdb-vacuum: simplify delete_queue_traverse() - free treats NULL pointers.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 24bec3d31b16c4c83b5ed76ecffccbfda53858fd
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 15:30:08 2014 +0100
ctdb-vacuum: reduce indentation in delete_queue_traverse
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 48f2d1158820bfb063ba0a0bbfb6f496a8e7522d
Author: Michael Adam <obnox at samba.org>
Date: Wed Feb 12 17:40:31 2014 +0100
ctdb-vacuum: treat value 0 of tunable RepackLimit as turned off.
I.e. when RepackLimit is set to 0, no size of the freelist
should trigger a repack in vacuuming.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit af5568b26761dadbb652d92f8c8ced477b38c7cc
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 01:55:39 2014 +0100
ctdb-vacuum: fix treatment of remaining records and statistics in delete_record_traverse()
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit b4e0b01a8c8415bec9c7dbbe4494813917dddfe5
Author: Michael Adam <obnox at samba.org>
Date: Wed Feb 12 17:38:56 2014 +0100
ctdb-vacuum: cast freelist_size in comparison.
At this point, it is >= 0 anyways.
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
commit 6a46a255307a070c887525ee1d79810ba12442bb
Author: Michael Adam <obnox at samba.org>
Date: Fri Feb 14 00:53:23 2014 +0100
ctdb-vacuum: improve output of delete list statistics
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Amitay Isaacs <amitay at gmail.com>
-----------------------------------------------------------------------
Summary of changes:
ctdb/server/ctdb_vacuum.c | 579 +++++++++++++++++++++++----------------------
1 files changed, 293 insertions(+), 286 deletions(-)
Changeset truncated at 500 lines:
diff --git a/ctdb/server/ctdb_vacuum.c b/ctdb/server/ctdb_vacuum.c
index 22603b8..574ad87 100644
--- a/ctdb/server/ctdb_vacuum.c
+++ b/ctdb/server/ctdb_vacuum.c
@@ -53,36 +53,43 @@ struct ctdb_vacuum_handle {
/* a list of records to possibly delete */
struct vacuum_data {
- uint32_t vacuum_limit;
uint32_t repack_limit;
struct ctdb_context *ctdb;
struct ctdb_db_context *ctdb_db;
struct tdb_context *dest_db;
trbt_tree_t *delete_list;
- uint32_t delete_count;
struct ctdb_marshall_buffer **vacuum_fetch_list;
struct timeval start;
bool traverse_error;
bool vacuum;
- uint32_t total;
- uint32_t vacuumed;
- uint32_t copied;
- uint32_t fast_added_to_vacuum_fetch_list;
- uint32_t fast_added_to_delete_list;
- uint32_t fast_deleted;
- uint32_t fast_skipped;
- uint32_t fast_error;
- uint32_t fast_total;
- uint32_t full_added_to_vacuum_fetch_list;
- uint32_t full_added_to_delete_list;
- uint32_t full_skipped;
- uint32_t full_error;
- uint32_t full_total;
- uint32_t delete_left;
- uint32_t delete_remote_error;
- uint32_t delete_local_error;
- uint32_t delete_deleted;
- uint32_t delete_skipped;
+ struct {
+ struct {
+ uint32_t added_to_vacuum_fetch_list;
+ uint32_t added_to_delete_list;
+ uint32_t deleted;
+ uint32_t skipped;
+ uint32_t error;
+ uint32_t total;
+ } delete_queue;
+ struct {
+ uint32_t scheduled;
+ uint32_t skipped;
+ uint32_t error;
+ uint32_t total;
+ } db_traverse;
+ struct {
+ uint32_t total;
+ uint32_t remote_error;
+ uint32_t local_error;
+ uint32_t deleted;
+ uint32_t skipped;
+ uint32_t left;
+ } delete_list;
+ struct {
+ uint32_t vacuumed;
+ uint32_t copied;
+ } repack;
+ } count;
};
/* this structure contains the information for one record to be deleted */
@@ -99,6 +106,10 @@ struct delete_records_list {
struct vacuum_data *vdata;
};
+static int insert_record_into_delete_queue(struct ctdb_db_context *ctdb_db,
+ const struct ctdb_ltdb_header *hdr,
+ TDB_DATA key);
+
/**
* Store key and header in a tree, indexed by the key hash.
*/
@@ -158,7 +169,7 @@ static int add_record_to_delete_list(struct vacuum_data *vdata, TDB_DATA key,
return -1;
}
- vdata->delete_count++;
+ vdata->count.delete_list.total++;
return 0;
}
@@ -200,8 +211,6 @@ static int add_record_to_vacuum_fetch_list(struct vacuum_data *vdata,
memcpy(old_size+(uint8_t *)vfl, rec, rec->length);
talloc_free(rec);
- vdata->total++;
-
return 0;
}
@@ -209,23 +218,39 @@ static int add_record_to_vacuum_fetch_list(struct vacuum_data *vdata,
static void ctdb_vacuum_event(struct event_context *ev, struct timed_event *te,
struct timeval t, void *private_data);
+static int vacuum_record_parser(TDB_DATA key, TDB_DATA data, void *private_data)
+{
+ struct ctdb_ltdb_header *header =
+ (struct ctdb_ltdb_header *)private_data;
+
+ if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
+ return -1;
+ }
+
+ *header = *(struct ctdb_ltdb_header *)data.dptr;
+
+ return 0;
+}
/*
* traverse function for gathering the records that can be deleted
*/
-static int vacuum_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data, void *private)
+static int vacuum_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data,
+ void *private_data)
{
- struct vacuum_data *vdata = talloc_get_type(private, struct vacuum_data);
+ struct vacuum_data *vdata = talloc_get_type(private_data,
+ struct vacuum_data);
struct ctdb_context *ctdb = vdata->ctdb;
+ struct ctdb_db_context *ctdb_db = vdata->ctdb_db;
uint32_t lmaster;
struct ctdb_ltdb_header *hdr;
int res = 0;
- vdata->full_total++;
+ vdata->count.db_traverse.total++;
lmaster = ctdb_lmaster(ctdb, &key);
if (lmaster >= ctdb->num_nodes) {
- vdata->full_error++;
+ vdata->count.db_traverse.error++;
DEBUG(DEBUG_CRIT, (__location__
" lmaster[%u] >= ctdb->num_nodes[%u] for key"
" with hash[%u]!\n",
@@ -237,42 +262,29 @@ static int vacuum_traverse(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data,
if (data.dsize != sizeof(struct ctdb_ltdb_header)) {
/* it is not a deleted record */
- vdata->full_skipped++;
+ vdata->count.db_traverse.skipped++;
return 0;
}
hdr = (struct ctdb_ltdb_header *)data.dptr;
if (hdr->dmaster != ctdb->pnn) {
- vdata->full_skipped++;
+ vdata->count.db_traverse.skipped++;
return 0;
}
- if (lmaster == ctdb->pnn) {
- /*
- * We are both lmaster and dmaster, and the record is empty.
- * So we should be able to delete it.
- */
- res = add_record_to_delete_list(vdata, key, hdr);
- if (res != 0) {
- vdata->full_error++;
- } else {
- vdata->full_added_to_delete_list++;
- }
+ /*
+ * Add the record to this process's delete_queue for processing
+ * in the subsequent traverse in the fast vacuum run.
+ */
+ res = insert_record_into_delete_queue(ctdb_db, hdr, key);
+ if (res != 0) {
+ vdata->count.db_traverse.error++;
} else {
- /*
- * We are not lmaster.
- * Add the record to the blob ready to send to the nodes.
- */
- res = add_record_to_vacuum_fetch_list(vdata, key);
- if (res != 0) {
- vdata->full_error++;
- } else {
- vdata->full_added_to_vacuum_fetch_list++;
- }
+ vdata->count.db_traverse.scheduled++;
}
- return res;
+ return 0;
}
/*
@@ -319,8 +331,7 @@ static int delete_marshall_traverse_first(void *param, void *data)
struct delete_records_list *recs = talloc_get_type(param, struct delete_records_list);
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data, ctdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
uint32_t hash = ctdb_hash(&(dd->key));
int res;
@@ -331,7 +342,8 @@ static int delete_marshall_traverse_first(void *param, void *data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- recs->vdata->delete_skipped++;
+ recs->vdata->count.delete_list.skipped++;
+ recs->vdata->count.delete_list.left--;
talloc_free(dd);
return 0;
}
@@ -341,26 +353,13 @@ static int delete_marshall_traverse_first(void *param, void *data)
* changed and that we are still its lmaster and dmaster.
*/
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] does not exist or is not"
- " a ctdb-record. skipping.\n",
- hash, ctdb_db->db_name));
- goto skip;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] has been recycled. "
- "skipping.\n",
- hash, ctdb_db->db_name));
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
goto skip;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->flags & CTDB_REC_RO_FLAGS) {
+ if (header.flags & CTDB_REC_RO_FLAGS) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has read-only flags. "
"skipping.\n",
@@ -368,7 +367,7 @@ static int delete_marshall_traverse_first(void *param, void *data)
goto skip;
}
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has been migrated away. "
"skipping.\n",
@@ -376,7 +375,7 @@ static int delete_marshall_traverse_first(void *param, void *data)
goto skip;
}
- if (header->rsn != dd->hdr.rsn) {
+ if (header.rsn != dd->hdr.rsn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] seems to have been "
"migrated away and back again (with empty "
@@ -404,10 +403,7 @@ static int delete_marshall_traverse_first(void *param, void *data)
* on the record's dmaster.
*/
- ctdb_data.dptr = tdb_data.dptr + sizeof(struct ctdb_ltdb_header);
- ctdb_data.dsize = tdb_data.dsize - sizeof(struct ctdb_ltdb_header);
-
- res = ctdb_ltdb_store(ctdb_db, dd->key, header, ctdb_data);
+ res = ctdb_ltdb_store(ctdb_db, dd->key, &header, tdb_null);
if (res != 0) {
DEBUG(DEBUG_ERR, (__location__ ": Failed to store record with "
"key hash [0x%08x] on database db[%s].\n",
@@ -422,15 +418,12 @@ static int delete_marshall_traverse_first(void *param, void *data)
skip:
tdb_chainunlock(ctdb_db->ltdb->tdb, dd->key);
- recs->vdata->delete_skipped++;
+ recs->vdata->count.delete_list.skipped++;
+ recs->vdata->count.delete_list.left--;
talloc_free(dd);
dd = NULL;
done:
- if (tdb_data.dptr != NULL) {
- free(tdb_data.dptr);
- }
-
if (dd == NULL) {
return 0;
}
@@ -466,12 +459,11 @@ static int delete_queue_traverse(void *param, void *data)
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb; /* or dd->ctdb ??? */
int res;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
uint32_t hash = ctdb_hash(&(dd->key));
- vdata->fast_total++;
+ vdata->count.delete_queue.total++;
res = tdb_chainlock(ctdb_db->ltdb->tdb, dd->key);
if (res != 0) {
@@ -479,29 +471,22 @@ static int delete_queue_traverse(void *param, void *data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
return 0;
}
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- /* Does not exist or not a ctdb record. Skip. */
- goto skipped;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- /* The record has been recycled (filled with data). Skip. */
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
goto skipped;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
/* The record has been migrated off the node. Skip. */
goto skipped;
}
- if (header->rsn != dd->hdr.rsn) {
+ if (header.rsn != dd->hdr.rsn) {
/*
* The record has been migrated off the node and back again.
* But not requeued for deletion. Skip it.
@@ -527,9 +512,9 @@ static int delete_queue_traverse(void *param, void *data)
DEBUG(DEBUG_ERR,
(__location__ " Error adding record to list "
"of records to send to lmaster.\n"));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
} else {
- vdata->fast_added_to_vacuum_fetch_list++;
+ vdata->count.delete_queue.added_to_vacuum_fetch_list++;
}
goto done;
}
@@ -542,9 +527,9 @@ static int delete_queue_traverse(void *param, void *data)
DEBUG(DEBUG_ERR,
(__location__ " Error adding record to list "
"of records for deletion on lmaster.\n"));
- vdata->fast_error++;
+ vdata->count.delete_queue.error++;
} else {
- vdata->fast_added_to_delete_list++;
+ vdata->count.delete_queue.added_to_delete_list++;
}
} else {
res = tdb_delete(ctdb_db->ltdb->tdb, dd->key);
@@ -554,25 +539,23 @@ static int delete_queue_traverse(void *param, void *data)
(__location__ " Error deleting record with key "
"hash [0x%08x] from local data base db[%s].\n",
hash, ctdb_db->db_name));
- vdata->fast_error++;
- } else {
- DEBUG(DEBUG_DEBUG,
- (__location__ " Deleted record with key hash "
- "[0x%08x] from local data base db[%s].\n",
- hash, ctdb_db->db_name));
- vdata->fast_deleted++;
+ vdata->count.delete_queue.error++;
+ goto done;
}
+
+ DEBUG(DEBUG_DEBUG,
+ (__location__ " Deleted record with key hash "
+ "[0x%08x] from local data base db[%s].\n",
+ hash, ctdb_db->db_name));
+ vdata->count.delete_queue.deleted++;
}
goto done;
skipped:
- vdata->fast_skipped++;
+ vdata->count.delete_queue.skipped++;
done:
- if (tdb_data.dptr != NULL) {
- free(tdb_data.dptr);
- }
tdb_chainunlock(ctdb_db->ltdb->tdb, dd->key);
return 0;
@@ -591,10 +574,8 @@ static int delete_record_traverse(void *param, void *data)
struct ctdb_db_context *ctdb_db = dd->ctdb_db;
struct ctdb_context *ctdb = ctdb_db->ctdb;
int res;
- struct ctdb_ltdb_header *header;
- TDB_DATA tdb_data;
+ struct ctdb_ltdb_header header;
uint32_t lmaster;
- bool deleted = false;
uint32_t hash = ctdb_hash(&(dd->key));
res = tdb_chainlock(ctdb_db->ltdb->tdb, dd->key);
@@ -603,7 +584,9 @@ static int delete_record_traverse(void *param, void *data)
(__location__ " Error getting chainlock on record with "
"key hash [0x%08x] on database db[%s].\n",
hash, ctdb_db->db_name));
- vdata->delete_local_error++;
+ vdata->count.delete_list.local_error++;
+ vdata->count.delete_list.left--;
+ talloc_free(dd);
return 0;
}
@@ -612,46 +595,29 @@ static int delete_record_traverse(void *param, void *data)
* changed and that we are still its lmaster and dmaster.
*/
- tdb_data = tdb_fetch(ctdb_db->ltdb->tdb, dd->key);
- if (tdb_data.dsize < sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] does not exist or is not"
- " a ctdb-record. skipping.\n",
- hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
- }
-
- if (tdb_data.dsize > sizeof(struct ctdb_ltdb_header)) {
- DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
- "on database db[%s] has been recycled. "
- "skipping.\n",
- hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ res = tdb_parse_record(ctdb_db->ltdb->tdb, dd->key,
+ vacuum_record_parser, &header);
+ if (res != 0) {
+ goto skip;
}
- header = (struct ctdb_ltdb_header *)tdb_data.dptr;
-
- if (header->flags & CTDB_REC_RO_FLAGS) {
+ if (header.flags & CTDB_REC_RO_FLAGS) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has read-only flags. "
"skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
- if (header->dmaster != ctdb->pnn) {
+ if (header.dmaster != ctdb->pnn) {
DEBUG(DEBUG_INFO, (__location__ ": record with hash [0x%08x] "
"on database db[%s] has been migrated away. "
"skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
- if (header->rsn != dd->hdr.rsn + 1) {
+ if (header.rsn != dd->hdr.rsn + 1) {
/*
* The record has been migrated off the node and back again.
* But not requeued for deletion. Skip it.
@@ -663,8 +629,7 @@ static int delete_record_traverse(void *param, void *data)
"migrated away and back again (with empty "
"data). skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
+ goto skip;
}
lmaster = ctdb_lmaster(ctdb_db->ctdb, &dd->key);
@@ -674,8 +639,7 @@ static int delete_record_traverse(void *param, void *data)
"delete list (key hash [0x%08x], db[%s]). "
"Strange! skipping.\n",
hash, ctdb_db->db_name));
- vdata->delete_skipped++;
- goto done;
--
Samba Shared Repository
More information about the samba-cvs
mailing list