tdb performance

Volker Lendecke Volker.Lendecke at SerNet.DE
Mon Feb 5 00:09:02 GMT 2007


On Mon, Feb 05, 2007 at 10:49:51AM +1100, tridge at samba.org wrote:
>  > Hmmmm. Can't tdb_free() consider these records as ones to
>  > merge if they happen to be next to a record to be deleted?
> 
> Aren't you planning to only record the location of these records in
> memory, not in the tdb? (or perhaps I misunderstood your proposal).

The current plan would be to only change the magic field of
the record to "I'm deleted but a process still has my
position in memory".

> If so, then the problem would be if the process exits while there are
> some of these pending. Then we'd only find them with a traverse.

We'd have to be more careful with a correct tdb_close() on
the db's that do it. In tdb_close() I would clean up the
records I did not really delete but just mark.

Looking at tdb_free() it does not look unreasonable to also
merge in the delete pending records if they happen to be
left or right of a record to be freed. This would then mean
that we have to check if a record is still pending when we
want to re-use it.

> A completely different approach would be to add support for futxes as
> an alternative to fcntl locks in tdb. A futex has essentially zero

The cost of fcntl() is non-zero, but that is not the main
reason why I'm thinking about this. It is mainly that I've
seen considerable contention on the freelist on real
machines. 

> We should also revive the proposal for a hash of freelists at some
> stage. That would make a big difference for some loads. The idea is to
> have N freelists (maybe 8 or so as default), plus a master freelist
> that the other freelists feed off when empty. We'd hash on pid (or
> similar id) to choose the freelist.

Yep, that's a different possibility. But with the
per-process in-memory freelist we could in theory reduce
freelist contention as well as get rid of the fcntl calls.

Not sure which one is better, I'll look at some
benchmarking.

Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.samba.org/archive/samba-technical/attachments/20070205/52c702cb/attachment.bin


More information about the samba-technical mailing list