tdb performance

[tridge at samba.org]

Volker,

 > We'd have to be more careful with a correct tdb_close() on
 > the db's that do it. In tdb_close() I would clean up the
 > records I did not really delete but just mark.

still relies on us doing a clean close, no? :)

 > Yep, that's a different possibility. But with the
 > per-process in-memory freelist we could in theory reduce
 > freelist contention as well as get rid of the fcntl calls.
 > 
 > Not sure which one is better, I'll look at some
 > benchmarking.

it could help on delete, but won't help on allocate.

It might be worth measuring what level of contention we really have,
and constructing a benchmark that really stresses the contention.

The simplest way I can think of to measure contention is this:

 1) change tdb_brlock() to always try F_SETLK first before any use of
    F_SETLKW. (the only place contention can happen in tdb is via a
    F_SETLKW lock call).

 2) when a F_SETLK fails then we know we have hit contention. In that
    case, use gettimeofday() to get the time, then use F_SETLKW to
    wait, and use gettimeofday() again to record the time in
    contention.

 3) add that time, and report the number of times locks are contended,
    and the average and max times.

If we can then find a benchmark that is both realistic and trigger
high contention then we will have something to measure how much we
have improved things.

Cheers, Tridge