process shared robust mutexes for tdb

Ira Cooper ira at
Mon Dec 24 17:50:22 MST 2012

On Mon, Dec 24, 2012 at 6:33 PM, simo <idra at> wrote:

> On Mon, 2012-12-24 at 15:39 -0500, Ira Cooper wrote:
> > The chainlocks are easier! :).  The problem is, you can lock all of a
> > database in one fcntl call now.  Where with mutexes, you need to use N
> > calls to cover each mutex.  So there is actually a crossover point where
> > mutexes can slow you down.  Having to take 10,000 mutexes vs. 1 fcntl
> > call... well, the fcntl call can win ;).  Locking every record in a 1M
> > record DB... You see the issue.  The whole concept of a
> > "semi-transactional" database like TDB is a bit wacky.  Mind you I see
> why
> > each part ended up the way it did... but still.
> Well, you can clearly take shortcuts if you need to lock the whole DB,
> by using a special mutex I guess.
> You could actually use a completely separate mutex space to represent
> locking data if you really needed to.
> Of course locking each record seprately makes it for multiple
> read/writers at the same time which is what you want on something like
> locking.tdb You do not want to have to always take the mutex for every
> operation or you just serialize everything.

Well.. we do it already (locking the vnode, in fact what you are suggesting
is already implemented in the kernel in a sense)  For locking.tdb it'd be
nice to stop.  But locking.tdb isn't the evil one for me.  serverid.tdb is,
believe it or not.  (I've actually gone through with dtrace to figure that

I have code in need of cleanup to basically remove all locking from the
common case from serverid.tdb.  (I have tested it, I am using it.  So it
isn't vapor, it just needs cleaning up so others can use it ;). )

In fact using it I've removed fcntl from the "critical paths" in my
use-case, allowing us to scale with CPU power, and also go faster in
general.  I'll try to turn that code in early January.  I've already sent
out pre-review copies.  So, it is a matter of dealing with the comments
I've gotten, so I can get it ready to go in.  (At least for 3.6.  For
master/4.0 it may not be as needed.  For 3.6 users... it is quite nice.)

> > > simo: Does LDB use transactions?  I thought it did?
> > >
> > > It does by default on every write operation.
> >
> >
> > So until someone does the work to support transactions, and make them
> fast,
> > it wouldn't see any improvements.  I strongly encourage Volker's initial
> > approach!
> Well the idea with transactions is that you can have only one of them
> running at any given time. It turns your code into serializing writes
> which means you need a single mutex to represent a global write lock.
> Basically the point is that it is an either-or thing. So there isn't
> really much to do to 'make transactions faster' as transaction (from the
> pov of locking) are a write lock on the while db.

Worse, you really have a reader/writer lock... Which is a bit more

Locking is a complicated thing.. and all optimizations need measuring, in
real scenarios.  Micro-benchmarks only tell part of the story :(.


More information about the samba-technical mailing list