[PATCH] Measure parallel performance and use standard process model in ldap

[Stefan Metzmacher]

Am 05.04.2017 um 12:42 schrieb Andrew Bartlett:
> On Wed, 2017-04-05 at 12:00 +0200, Stefan Metzmacher wrote:
>> Am 05.04.2017 um 06:36 schrieb Andrew Bartlett via samba-technical:
>>> The attached patches make our LDAP server multi-process.
>>>
>>> This brings performance gains and losses.  Essentially if we have
>>> one
>>> client binding and going away, strictly one at a time, then there
>>> is a
>>> significant overhead.  However, with multiple clients we get to use
>>> multiple CPUs and it is worth it if there are multiple incoming
>>> connections or the connections are long-lasting. 
>>>
>>> We hope to fix up and restore the prefork process model in the
>>> future,
>>> but this isn't on the cards right now, and our users need the extra
>>> speed.
>>>
>>> Please review/push the attached patch and the performance tests. 
>>
>> I guess this should be on top of the ldb/tdb patches?
> 
> I've tested it with and without, and it is viable both ways.  Naturally
> I want both in, the perf tests first ideally. 
> 
>> Is a server with these and the ldb/tdb patches able to handle
>> a zarafa server which serves about 500-1000 users which make use of
>> the webmail interface? We have a customer were we used a OpenLDAP
>> proxy in front of the samba servers to handle the load.
>> The proxy was able to deliver responses under 10 msecs,
>> while samba needed about 350 msecs. The problem is that the proxy
>> db gets corrupted every few days.
> 
> I've not tested it with that, but it was the thread from a similar user
> that made me pick this back up (it has been in my backlog for a while).
> 
> What I will say is this.  On my workstation, the tests did:
> 
> test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
> 94.185378
> 
> that is 180ms per search
> 
> With just this patch I get:
> test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
> 52.460954,
> 
> That is 100ms per search.  Still not 10ms, but then there is also a
> bind in there.  Also still not scaling linearly with CPUs. 
> 
> With the ldb patch, we get:
> test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
> 13.458421,
> 
> That is 26ms per search, much closer to what we need. 
> 
> If I had a client suffering the consequences of the corrupted DB on the
> Zarafa proxy, I would want to try these patches :-)

Not in this form, I'll write a mail in the other thread...

Do you also have numbers for indexed searches?
With a complex expressions like:

(&(objectClass=user)(|(cn=user1)(cn=user2)(cn=user3)(cn=user4)(cn=user5)))
or more complex things.

metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170405/7d35ccc7/signature.sig>