Samba4: LDB size limit and memory leak

[Andrew Bartlett]

On Mon, 2009-05-25 at 08:30 +0200, Marcel Ritter wrote:
> Hi,
> 
> I'm running several samba 4 tests at the moment and I found two things,
> that may need some attention:
> 
> 1. LDB size seems to be limited to 4GB (both on i586 and x86_64). I tried
>     to create a lot of new users using setup/newuser and somewhere in
> between
>     20000 and 40000 users, setup/newuser crashes:

Correct.  If a 64 bit varient of TDB were developed, that limit would be
removed. 

> ltdb: tdb(/var/lib/samba/private/users.ldb):
> tdb_transaction_setup_recovery: transaction data over new region boundary
> ltdb: tdb(/var/lib/samba/private/users.ldb):
> tdb_transaction_prepare_commit: failed to setup recovery data
> ltdb: tdb(/var/lib/samba/private/sam.ldb): tdb_transaction_cancel: no
> transaction
> commit called but no ldb transactions are active!Traceback (most recent
> call last):
>   File "/usr/share/samba/setup/newuser", line 60, in <module>
>     samdb.newuser(username, opts.unixname, password)
>   File "/usr/lib/python2.6/site-packages/samba/samdb.py", line 153, in
> newuser
>     self.transaction_commit()
> _ldb.LdbError: (1, 'ldb transaction commit: Operations error (1)')
> 
>    The resulting filesize of users.ldb is exactly 4GB + 4096 bytes
> (4294971392 bytes)
> 
> 2. As mentioned above I used setup/newuser to create the samba4 users. To
>     speed things up a little I tried to modify newuser to internally
> loop over
>     function "samdb.newuser(username, opts.unixname, password)".
>     However the amount of memory needed increases dramatically when using
>     this version (and finally the python process dies when out of memory).
>     I also tried to loop over a "less demanding" function like
> "samdb.domain_dn()"
>     with the same effect on memory consumption.
>     Looks like a memory leak to me.

It is quite possible that the python code has not recovered the moment
at this point.  

What version are you working with?

We have significantly improved ldb's memory use and disk space
requirements in the past few months. 

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.                  http://redhat.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20090525/24921bdc/attachment.bin