[TDB] Patches for file and memory usage growth issues

[Jeremy Allison]

On Sat, Apr 09, 2011 at 11:01:15PM -0400, simo wrote:
> Hi all,
> during this week I have been working on a memory growth issue affecting
> sssd when storing large group of users in the LDB based cache.
> 
> It turns out that a lot of memory is being used by TDB.
> 
> Please take a look at the last 3 patches in this tree:
> http://git.samba.org/?p=idra/samba.git;a=shortlog;h=refs/heads/tdb-memsize
> 
> The first one should be uncontroversial, the original intent makes sense
> in a tdb database when you use small records, but when a rare huge
> record is saved in, growing the db by 100 times its size really is
> overkill. I also reduced the minimum overhead from 25% to 10% as with
> large TDBs (in the order of 500MB/1GB, growing by 25% is quite a lot).

I had to do a similar fix for an OEM. Your heuristic looks a little
cleaner (I added an additional tunable). Something like this is
definitely needed though.

Jeremy.