[PATCH] CTDB vacuuming updates

[Amitay Isaacs]

Hi,

Here are some CTDB vacuuming changes to improve performance.

[PATCH 1/6] ctdb-vacuum: Track time for vacuuming in database

Add vacuuming latency to database statistics.

[PATCH 2/6] ctdb-vacuum: Stagger vacuuming child processes

On a highly loaded system, forking multiple vacuuming child processes
simultaneously can be expensive.  So delay vacuuming event till there are
no active vacuuming child processes.

[PATCH 3/6] ctdb-vacuum: Use non-blocking lock when traversing delete
[PATCH 4/6] ctdb-vacuum: Use non-blocking lock when traversing delete

These change tdb_chainlock() to tdb_chainlock_nonblock().  Vacuuming needs
more work to re-add skipped records back into scheduled delete list.
However, these patches still prevent vacuuming getting in the way of ctdb
daemon.

[PATCH 5/6] ctdb-vacuum: Do not delete VACUUM MIGRATED records

While processing records scheduled for deletion, any record for which the
current node is not lmaster, is migrated to the lmaster node.  For this
migration recovery deamon creates a call_request with
CTDB_CALL_FLAG_VACUUM_MIGRATION flag.  When such a record is stored on
lmaster, the record is marked with CTDB_REC_FLAG_VACUUM_MIGRATED.  such
records must be stored locally and scheduled for deletion locally.

[PATCH 6/6] ctdb-recoverd: Process all the records for vacuum fetch

Recovery daemon sends migration call requests one after another only after
receiving reply.  This is very slow and can take longer than the
VacuumInterval.  Any subsequent migration request sent to recovery daemon
will be ignored if the previous run has not completed.


Please review and push if ok.

Amitay.