Roughly how many users should Samba 4.0 be able to handle?

[Matthieu Patou]

On 01/19/2013 03:20 AM, Michael Wood wrote:
> Hi
>
> Thanks for the reply.
>
> On 17 January 2013 22:03, Matthieu Patou <mat at samba.org> wrote:
>> On 01/17/2013 06:47 AM, Michael Wood wrote:
>>> I'm trying to get an idea of how many users a Samba 4.0 server should be
>>> able to handle.
>>>
>>> In the past (one of the alphas) I tried adding about 40000 users and Samba
>>> slowed to a crawl.  I can't remember the details.  Just that I had to roll
>>> back to by last backup to fix the performance again.
>> So due to indexing 40 000 users will result into 200 000 ldb entries at
>> least and I suspect 400 000 worst case.
> Ouch!  That sounds like a lot.
Indexes ... 1 entry has a couple of indexed attributes and still we are 
not honoring all the indexing flags (ie. index things like cn=mat*)
>
>> I think it highly depend on the workload that you have.
>> If you have more than 1 DC it will be slow quite quickly because DCs keeps
>> on asking each other for changes (I think it's every 15s but it's tuneable)
>> and this kind of search is not indexed yet (we are doing something like
>> '(usnchanged>=max_usn_from_last_replication)').
> OK, I currently have only one DC.  This is only for authentication.
> There are no workstations joined to the domain or anything like that.
> Authentication is done from a few services using LDAP.
>
> There are very few groups with just about all users in one of the
> groups and a few of the users in some other groups in addition.
>
> So the authentication process generally just does a bind as the user
> (all users are in the same OU) and then checks that they are in the
> group needed for accessing the service.

>
>> The good news is that I have a clear idea on how to implement indexing for
>> this and this should start right after the first set of optimization are
>> pushed.
>>
>> If you have 1 DC and very selective query (ideal is just basedn query) then
>> I don't think that doing tdb search in a 400 000 will be that bad but I will
>> let tdb expert(s) give their pov but you are rarely in this case, and
>> queries that looked selective (ie. base=MYOUWITH2USERS,DC=domain,DC=tld) are
>> currently not so selective (but my patches are changing this).
> OK, thanks.  I think a simple bind of a DN should be quite selective :)
>
>> I would love to do some tunning on big databases if someone can provide us a
>> script to create a realistic big database (ie. not all users are called
>> fooxxxx and not just 2 attributes are set) it will for sure help the
>> process.
> Well, I'm not sure how "realistic" mine would be, despite being quite real :)
>
>>> I've compiled 4.0.1 and am busy adding 5000 users to my 1200 existing
>>> users
>>> (but on a test machine) to see what happens, but if anyone is running
>>> Samba
>>> 4.0 with large numbers of users I'd like to hear about it.
>>>
>>> Also, I know Matthieu's been working on optimising the ldb indexing code.
>>> Has that made it to master yet?
>> No but it's in process, feel free to try my ldb_perfs branch in my git repo
>> to see.
> I tested adding 5000 users to the existing 1200 and it did not seem to
> have an impact on the speed.  I am not planning to try 40000 again
> soon, though.
No impact on speed ? you mean no big difference ?

For the adding part I'm not surprised actually because I took the 
decision to even potentially make write a bit slower in order to make 
read quicker.
And also you need to reindex the database to feel the difference ...

One more question are you adding all the users in 1 transaction ?

Matthieu.

-- 
Matthieu Patou
Samba Team
http://samba.org