Samba4: LDB size limit and memory leak

[simo]

On Mon, 2009-06-01 at 18:05 +1000, tridge at samba.org wrote:
> Hi Ronnie,
> 
>  > Yes, it would make reading the entire index-object a lot slower, since it
>  > would only read one entry per tdb_fetch(), but then again the full
>  > tdb_traverse that is needed (twice) for the tdb_repack() is not exactly
>  > cheap either.
> 
> The repack is only done when the underlying file grows. When adding
> 20k users as fast as we can, this happens about 10-15 times over a
> period of an hour or so. (remember that when we expand a tdb, we
> always expand by at least 1.25 in size - see tdb_expand())
> 
> If we made indexing have to read a linked list, we would be following
> those lists several hundred times per second.
> 
> I'm currently considering changing the indexing so that when a single
> attribute has more than 100 DNs, it will be split across multiple
> records. That would mean we would have to do multiple fetches (one for
> each 100 DNs), but insert/delete of indexes won't deal with ever
> expanding records.

Tridge,
if you make this change, please make it so that it is an ldb option so
that newer and older ldb can still read a file using the old format. And
only an explicit change in an ldb special attribute triggers this new
behavior.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Principal Software Engineer at Red Hat, Inc. <simo at redhat.com>