TDB locking overhead and performance...

[Jeremy Allison]

On Sat, Apr 22, 2006 at 06:15:31PM +0100, Daniel J Blueman wrote:
> I did some smbd profiling, and was seeing quite a lot of time spent
> (un)locking [1] TDB files.
> 
> A quick workload of opening a few directory trees, we see a _lot_ of
> activity with a few TDB files:
> 
> locking.tdb: 11943 accesses
> brlock.tdb: 690 accesses
> group_mapping.tdb: 126 accesses
> account_policy.tdb: 20 accesses
> 
> As a test, I changed the opening of the locking.tdb and brlock.tdb
> files to use the TDB_INTERNAL flag, avoiding use of the fcntl(F_SETLK)
> syscall for locking individual database records. Performance was a lot
> snappier, with quite a bit less system time used.
> 
> What is the scope of implementing shared memory TDBs, where locking
> could be done simply on structures?

tdb's *are* shared memory, but shared memory needs locking for
concurrent access. fcntl locks are the only safe, portable way
of doing this (and by "safe" I mean will dissapear if the holding
process aborts).

I have done some work on this in the code that will be released
as 3.0.23 - I'd suggest you download this (current SAMBA_3_0)
and check, I think you might find the results may be a little
better.

Jeremy.