Samba and LDB at 150k+ (was: Re: [Samba] 32 bits limit?)

Andrew Bartlett abartlet at samba.org
Tue Jun 9 16:49:43 MDT 2015


(dropping the samba list)

On Mon, 2015-06-08 at 13:48 +0200, mathias dufresne wrote:
> Hi all,
> 
> 
> I tried tdbdump / tdbrestore and it saved some space but not solved
> the main issue which is 4GB is not large enough for a large
> organization.

I totally agree.

> I had a look on LMDB which sounds really good, unfortunately the
> module is not finished, not enough at least to use it on a large AD in
> production (large as 120k users + 150k computers + thousands of groups
> which are not yet defined). As you know the larger is the company the
> colder they are regarding not fully stable products.
> 
> 
> This leads my to one question: is it possible to split the LDB
> database?
> 
> What I meant is if I create an OU named "persons" to host all users,
> is it possible move this OU and its contents into another LDB file?

No, it isn't possible to split it up that way, because then we would
need complex logic to allow move operations between LDB files, as well
as logic to allow a subtree search correctly.  The split of one ldb file
per partition works very well for a number of operations, because it
matches the LDAP semantics we need.

On the other hand, changing the key-value store under the database, but
keeping the layer above the key-value store largely the same seems like
a massive change, but both layers are themselves quite mature, they just
have not been used together before.

What I'm getting at is that with the right testing, I think this (or
perhaps an alternative using ntdb) is the less risky approach, and the
one the follows the direction the Samba Team would like to go in.  

The point of database It would also be the appropriate way to introduce
other efficiencies in the database format (such as a dual
normalised/original data store, improved indexes or SD caching).

I'm really excited to hear about deployments at this scale, and I'm
always impressed by the tasks folks put Samba to, beyond my wildest
dreams.  I would suggest that in going to this scale, using the results
of the testing you are doing to make an investment in ensuring Samba
works well from Day One will be critical.

Finally, I don't like talking money or resources here, but I think it is
important to suggest that given the potential savings in moving 150,000
users to Samba AD, that investing a portion of that in your own
developers or via a Samba support company would not only ensure a
smoother transition, it would support the improvements that your
suggested scale will need.

Andrew Bartlett
-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba






More information about the samba-technical mailing list