[PATCH] CTDB vacuuming updates

[Amitay Isaacs]

On Fri, Nov 7, 2014 at 4:44 AM, Michael Adam <obnox at samba.org> wrote:

> Hi Amitay,
>
> the commit messages look promising...
> ...but where are the patches? :-)
>
> Cheers - Michael
>
> On 2014-11-06 at 23:28 +1100, Amitay Isaacs wrote:
> > Hi,
> >
> > Here are some CTDB vacuuming changes to improve performance.
> >
> > [PATCH 1/6] ctdb-vacuum: Track time for vacuuming in database
> >
> > Add vacuuming latency to database statistics.
> >
> > [PATCH 2/6] ctdb-vacuum: Stagger vacuuming child processes
> >
> > On a highly loaded system, forking multiple vacuuming child processes
> > simultaneously can be expensive.  So delay vacuuming event till there are
> > no active vacuuming child processes.
> >
> > [PATCH 3/6] ctdb-vacuum: Use non-blocking lock when traversing delete
> > [PATCH 4/6] ctdb-vacuum: Use non-blocking lock when traversing delete
> >
> > These change tdb_chainlock() to tdb_chainlock_nonblock().  Vacuuming
> needs
> > more work to re-add skipped records back into scheduled delete list.
> > However, these patches still prevent vacuuming getting in the way of ctdb
> > daemon.
> >
> > [PATCH 5/6] ctdb-vacuum: Do not delete VACUUM MIGRATED records
> >
> > While processing records scheduled for deletion, any record for which the
> > current node is not lmaster, is migrated to the lmaster node.  For this
> > migration recovery deamon creates a call_request with
> > CTDB_CALL_FLAG_VACUUM_MIGRATION flag.  When such a record is stored on
> > lmaster, the record is marked with CTDB_REC_FLAG_VACUUM_MIGRATED.  such
> > records must be stored locally and scheduled for deletion locally.
> >
> > [PATCH 6/6] ctdb-recoverd: Process all the records for vacuum fetch
> >
> > Recovery daemon sends migration call requests one after another only
> after
> > receiving reply.  This is very slow and can take longer than the
> > VacuumInterval.  Any subsequent migration request sent to recovery daemon
> > will be ignored if the previous run has not completed.
> >
> >
> > Please review and push if ok.
> >
> > Amitay.
>


Oops. :-)

Here are the patches!

Amitay.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctdb.patches
Type: application/octet-stream
Size: 8360 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141107/410de4e5/attachment.obj>