ldbadd --no-sync implications?

[Michael Wood]

Hi metze

On 3 March 2011 18:32, Stefan (metze) Metzmacher <metze at samba.org> wrote:
> Hi Michael,
>
>> By the way, adding batches of 1000 users to the group takes longer and long.
>>
>> e.g. if I have LDIF files like this:
>>
>> dn: CN=TestGroup,CN=Users,DC=example,DC=com
>> changetype: modify
>> -
>> add: member
>> member: CN=one,CN=Users,DC=example,DC=com
>> -
>> add: member
>> member: CN=two,CN=Users,DC=example,DC=com
>> -
>> add: member
>> member: CN=three,CN=Users,DC=example,DC=com
>> -
>> add: member
>> member: CN=four,CN=Users,DC=example,DC=com
>> -
>
> My ldif looks like this:
>
> dn: CN=TestGroup,CN=Users,DC=example,DC=com
> changetype: modify
> add: member
> member: CN=one,CN=Users,DC=example,DC=com
> member: CN=two,CN=Users,DC=example,DC=com
> member: CN=three,CN=Users,DC=example,DC=com
> member: CN=four,CN=Users,DC=example,DC=com
> -
>
> But I guess it doesn't make a difference.

I'll give it a try and see if it helps.

>> with 1000 users in each, and then run that through ldbmodify --nosync,
>> I get the following times.  As you can see, each batch of 1000 users
>> gets slower and slower:
>
> I think I have fixed that with commits:
>
> 05088fb855a1fc043c3f75d01742cdbbfbb3330e s4:dsdb/samldb: avoid nested
> unindexed searches in samldb_member_check()
> 60691c13224656e6a889eb01246582f10c056b7b s4:dsdb/samdb: optimize
> samldb_prim_group_change()

Unfortunately the slowing down of ldbmodify for adding more and more
batches of 1000 users to a group with a large number of users in it
still seems to be a problem.

I'm now running Version 4.0.0alpha15-GIT-ba77274 on a test box, and
since the upgrade it's still getting slower:

batch-csv.bd-group.ldif
Modified 1 records with 0 failures

real	9m12.581s
user	8m58.730s
sys	0m8.960s
batch-csv.be-group.ldif
Modified 1 records with 0 failures

real	9m19.655s
user	9m11.270s
sys	0m7.020s
batch-csv.bf-group.ldif
Modified 1 records with 0 failures

real	9m45.768s
user	9m37.520s
sys	0m6.900s

I haven't tried yet to see whether OS X is any happier with the new
version, though.

-- 
Michael Wood <esiotrot at gmail.com>