Samba4: LDB size limit and memory leak

[tridge at samba.org]

Hi Ronnie,

 > Yes, it would make reading the entire index-object a lot slower, since it
 > would only read one entry per tdb_fetch(), but then again the full
 > tdb_traverse that is needed (twice) for the tdb_repack() is not exactly
 > cheap either.

The repack is only done when the underlying file grows. When adding
20k users as fast as we can, this happens about 10-15 times over a
period of an hour or so. (remember that when we expand a tdb, we
always expand by at least 1.25 in size - see tdb_expand())

If we made indexing have to read a linked list, we would be following
those lists several hundred times per second.

I'm currently considering changing the indexing so that when a single
attribute has more than 100 DNs, it will be split across multiple
records. That would mean we would have to do multiple fetches (one for
each 100 DNs), but insert/delete of indexes won't deal with ever
expanding records.

Cheers, Tridge