process shared robust mutexes for tdb

[Rusty Russell]

Volker Lendecke <Volker.Lendecke at SerNet.DE> writes:

> Hi!
>
> Under
>
> http://git.samba.org/?p=vl/samba.git/.git;a=shortlog;h=refs/heads/tdb
>
> find my current work in progress to augment the fcntl locks
> in tdb with mutexes for the freelist and hash chain locks.
> The reason for this is that fcntl under load pretty much
> collapses. I don't have exact results yet, but the initial
> tests for tdbtorture are spectacular. One test went down
> from 8m47.470s to 0m13.376s. The tdb_open_ex patches are far
> from done. I am still thinking how to protect tdb with
> mutexes from other use. The idea is that this scheme is only
> used for highly contended tdb like locking.tdb where we
> don't do transactions for example.
>
> Comments?

I like the performance improvements, which implies we're doing *way* too
much locking; WTF is going on?  If it's due to multiple processes
locking the same records, which is the worst case for fcntl locks as
implemented in Linux today, we need mutexes to improve like this.

Implementing the global lock is pretty difficult, especially since Linux
has an arbitrary limit (2048) on how many locks it will recover.   We
might group chains such that there are only 2048 locks, then lock all of
them?  It wouldn't be all that slow in practice, I think, but it risks
increasing contention (there's still a benefit to having more hash
buckets, due to shorter chains, but it's not as clear).

Thoughts?
Rusty.