tdb memory consumption

[Jeremy Allison]

On Mon, Feb 12, 2007 at 05:34:36PM +0100, Volker Lendecke wrote:
> Hi!
> 
> Looking at smbd3 with the valgrind heap profiler I noticed
> that a significant amount of heap memory is used by
> tdb_open_ex(), and it turns out that the tdb->locked[] array
> takes up this space. For each hash chain we allocate 8
> bytes, and with brlock.tdb/locking.tdb now having a default
> of 10007 chains and group_mapping.ldb being at 10000 chains
> we end up at about 250k memory consumption. This array
> should only be used very sparsely, in practice we only have
> very few chains locked. More than one application-level lock
> is forbidden due to deadlock danger, and tdb_traverse does
> two. Add the freelist, and we're at four at most.
> 
> The following patch adds a dynamic allocation of the
> tdb->locked[] array. I'm not sure if we can max it at a
> certain number, so that we could change it to a static
> array.
> 
> 250k seems not much, but this is memory we jump around in,
> and for a 1000 user server this is 1/4GB, and I think
> getting rid of that might be worthwile. It might become more
> when we go to more ldb databases and bigger hash chains.
> 
> What do you think?

Genious out of the box thinking. +1 on applying from me.

Thanks,

	Jeremy.