[PATCH][WIP] Make the Samba AD DC multi-process

Mon Nov 21 04:01:56 UTC 2016

On Fri, 2016-10-14 at 20:01 +1300, Andrew Bartlett wrote:
> G'Day,
> 
> Attached is a WIP set of patches to make Samba connect to ldb faster,
> for the @ATTRIBUTES load, used until we can read the full schema.
> 
> This avoids some O(n^2) behaviour for the 600 attributes in the
> default
> schema, which was taking 5% of the time to run a simple ldbsearch.
> 
> Please consider, but don't push until I run the beachmarks.

Attached is some work in progress to do this, and to allow the LDAP and
NETLOGON server to be multi-process. 

This breaks the RPC protocol by not checking the assoc_group when we
accept a bind to the NETLOGON rpc server in the AD DC.

It also breaks the current link between the lsarpc services and
netlogon, which currently allow a bind on either pipe to access these
services. 

I've tried to make that all as generic as possible.

Please comment.  I don't plan to push this without coming back to the
list. 

To address the cost of the additional processes, particularly for LDAP,
we hope to have some improvments to talloc() to make short-lived child
processes less costly at exit() time. 

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-perf-Add-simple-tests-for-the-open-close-a-database-.patch
Type: text/x-patch
Size: 5101 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0001-perf-Add-simple-tests-for-the-open-close-a-database-.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-ldb-Add-helper-function-ldb_schema_attribute_remove_.patch
Type: text/x-patch
Size: 3064 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0002-ldb-Add-helper-function-ldb_schema_attribute_remove_.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-ldb-Reduce-scope-of-allocation-and-de-allocation-of-.patch
Type: text/x-patch
Size: 5397 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0003-ldb-Reduce-scope-of-allocation-and-de-allocation-of-.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-ldb-Reduce-per-attribute-memory-allocation-during-AT.patch
Type: text/x-patch
Size: 1146 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0004-ldb-Reduce-per-attribute-memory-allocation-during-AT.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-ldb-Add-helper-function-ldb_schema_attribute_fill_wi.patch
Type: text/x-patch
Size: 2618 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0005-ldb-Add-helper-function-ldb_schema_attribute_fill_wi.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0006-ldb-load-ATTRIBUTES-faster-by-sorting-once-not-at-ea.patch
Type: text/x-patch
Size: 5209 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0006-ldb-load-ATTRIBUTES-faster-by-sorting-once-not-at-ea.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0007-ldb-bump-ABI-version-for-new-private-functions.patch
Type: text/x-patch
Size: 23236 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0007-ldb-bump-ABI-version-for-new-private-functions.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0008-s4-rpc_server-Show-what-RPC-interfaces-are-listening.patch
Type: text/x-patch
Size: 1146 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0008-s4-rpc_server-Show-what-RPC-interfaces-are-listening.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0009-dsdb-specify-attributes-when-loading-schema.patch
Type: text/x-patch
Size: 3746 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0009-dsdb-specify-attributes-when-loading-schema.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0010-ldap-Run-the-LDAP-server-with-the-default-typically-.patch
Type: text/x-patch
Size: 1421 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0010-ldap-Run-the-LDAP-server-with-the-default-typically-.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0011-ldb-Avoid-individual-memory-allocations-when-searchi.patch
Type: text/x-patch
Size: 1150 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0011-ldb-Avoid-individual-memory-allocations-when-searchi.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0012-torture-Remove-access-to-LSARPC-via-pipe-netlogon-in.patch
Type: text/x-patch
Size: 4220 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0012-torture-Remove-access-to-LSARPC-via-pipe-netlogon-in.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0013-idl-Do-not-listen-for-lsarpc-on-pipe-netlogon.patch
Type: text/x-patch
Size: 1717 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0013-idl-Do-not-listen-for-lsarpc-on-pipe-netlogon.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0014-rpc_server-netlogon-Move-from-memcache-to-a-tdb-cach.patch
Type: text/x-patch
Size: 18089 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0014-rpc_server-netlogon-Move-from-memcache-to-a-tdb-cach.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0015-s4-rpc_server-Allow-each-interface-to-declare-if-it-.patch
Type: text/x-patch
Size: 4480 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0015-s4-rpc_server-Allow-each-interface-to-declare-if-it-.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0016-s4-rpc_server-Allow-listener-for-RPC-servers-to-use-.patch
Type: text/x-patch
Size: 10532 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0016-s4-rpc_server-Allow-listener-for-RPC-servers-to-use-.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0017-s4-rpc_server-Do-not-check-association-groups-for-NE.patch
Type: text/x-patch
Size: 3018 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0017-s4-rpc_server-Do-not-check-association-groups-for-NE.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0018-selftest-Use-rpc-server-port-netlogon-smb.conf-optio.patch
Type: text/x-patch
Size: 994 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0018-selftest-Use-rpc-server-port-netlogon-smb.conf-optio.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0019-s4-netlogon-Push-the-netlogon-server-in-the-AD-DC-in.patch
Type: text/x-patch
Size: 1525 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161121/c8e9d8a9/0019-s4-netlogon-Push-the-netlogon-server-in-the-AD-DC-in.bin>