AD DC LDAP server multi-process - slower!

[Andrew Bartlett]

On Tue, 2016-12-13 at 08:51 -0800, Jeremy Allison wrote:
> On Tue, Dec 13, 2016 at 04:43:19PM +1300, Andrew Bartlett wrote:
> > 
> > G'Day,
> > 
> > We had a simple patch to change the single process LDAP server into
> > multi-process, in the standard model like smbd uses, one fork per
> > connection. 
> > 
> > The performance on a test of 
> >  - connect
> >  - NTLM binds
> >  - drop socket
> > showed using multiple processes this way, with the high exit cost,
> > is
> > actually slower.  We expected that having mutiple CPUs available
> > would
> > make it at least a little faster, and we would optimise from there
> > with
> > the talloc_for_exit() or similar.
> > 
> > We will re-investigate the prefork process model for the ldap
> > server. 
> 
> Yes, interestingly enough Volker predicted that to me in a phone
> call due to the case below.
> 
> Are you sure this is the exit cost ? When we have many shared
> libraries the cost of setting up the vm mappings on fork can also
> be very expensive.

The perf graph of the testing we did without parallel workers showed
the extra CPU time went 50/50 into the setup and tear-down.  That
showed the worse case slowing down to 140% the unpatched.  I'm still
surprised that multiple CPUs didn't recover that, so there may be other
costs as well. 

The same probably applies for NETLOGON, but connections there tend to
be much longer lived, and the connect/bind/drop pattern from web
clients won't be a major part of the load.

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba