tdb performance

[tridge at samba.org]

Volker,

 > An obvious extension we would have to do is to properly
 > clean up the records marked as dead in case a process dies.

This is the bit that would concern me most. If we had to do a full
scan in order to do that cleanup then we could end up with a very
nasty scan problem. All our previous attempts at preiodic scan for
cleanup have been disasters, and been removed :-)

Are you sure this would be a real win? The way I usually test this
sort of idea is this:

 1) remove the lock/unlock pair, leaving the code unsafe

 2) benchmark it, see what improvement you get

 3) if its a significant improvement then try to find a way to remove
    the lock safely

This relies on the fact that in most benchmark runs you don't hit the
race that the lock prevents. So it gives you a optimistic estimate of
how much you could gain from the change. My guess is that it won't be
a worthwhile improvement, but doing the above testing will tell you if
all the work of making this safe is worthwhile.

Cheers, Tridge