[PATCH] Measure parallel performance and use standard process model in ldap

[Andrew Bartlett]

On Wed, 2017-04-05 at 12:00 +0200, Stefan Metzmacher wrote:
> Am 05.04.2017 um 06:36 schrieb Andrew Bartlett via samba-technical:
> > The attached patches make our LDAP server multi-process.
> > 
> > This brings performance gains and losses.  Essentially if we have
> > one
> > client binding and going away, strictly one at a time, then there
> > is a
> > significant overhead.  However, with multiple clients we get to use
> > multiple CPUs and it is worth it if there are multiple incoming
> > connections or the connections are long-lasting. 
> > 
> > We hope to fix up and restore the prefork process model in the
> > future,
> > but this isn't on the cards right now, and our users need the extra
> > speed.
> > 
> > Please review/push the attached patch and the performance tests. 
> 
> I guess this should be on top of the ldb/tdb patches?

I've tested it with and without, and it is viable both ways.  Naturally
I want both in, the perf tests first ideally. 

> Is a server with these and the ldb/tdb patches able to handle
> a zarafa server which serves about 500-1000 users which make use of
> the webmail interface? We have a customer were we used a OpenLDAP
> proxy in front of the samba servers to handle the load.
> The proxy was able to deliver responses under 10 msecs,
> while samba needed about 350 msecs. The problem is that the proxy
> db gets corrupted every few days.

I've not tested it with that, but it was the thread from a similar user
that made me pick this back up (it has been in my backlog for a while).

What I will say is this.  On my workstation, the tests did:

test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
94.185378

that is 180ms per search

With just this patch I get:
test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
52.460954,

That is 100ms per search.  Still not 10ms, but then there is also a
bind in there.  Also still not scaling linearly with CPUs. 

With the ldb patch, we get:
test_500_concurrent_unindexed_search_10_workers(ad_dc_ntvfs)":
13.458421,

That is 26ms per search, much closer to what we need. 

If I had a client suffering the consequences of the corrupted DB on the
Zarafa proxy, I would want to try these patches :-)

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba