Filtering links by the uptodateness vector, sorting them client-side (was: Re: [PATCH] drs exop test & refactoring)

Andrew Bartlett abartlet at samba.org
Mon Dec 19 10:48:40 UTC 2016


On Mon, 2016-12-19 at 14:02 +1300, Bob Campbell wrote:
> Hi,
> 
> The first attached patch is some refactoring & extension to support
> the
> next two, and upcoming, patches.
> 
> The other patches are a test to show that the given
> uptodateness_vector
> is not considered when figuring out which links to replicate, and a
> fix
> for that.
> 
> Metze, you should add your copyright to getnc_exop and drs_base.
> 
> Please review and/or push if appropriate.

G'Day Bob,

I want to emphisisie how important this patch is.  We have been asked
by clients to put significant effort into making Samba not get into
O(n^2) loops when processing linked attributes, because at scale it
becomes quite an issue.  In particular, when a new DC is installed,
absent this patch, all the links it knows about get replicated back out
to the partner DCs. 

That is harmless, but takes considerable processing time to resolve. 

For that angle, I wanted to foreshadow that we are also working on a
means to store linked attributes sorted by the GUID, and that we are
very keen to merge this patch, once complete, for 4.6.

The aim is very simple: get_parsed_dns() in Samba's repl_meta_data code
represents the bulk of the CPU time spent during link replication and
link value modification.  We have already pushed changes to make the
GUID parsing faster, but we can avoid parsing many of the DNs at all by
using a bisection search.  That however needs us to keep the links
sorted in the ldb message.

The links are already implicitly sorted in many cases, as new replicas
will have them applied in the same sorted order that the initial DC
sends in GetNCChanges, so sorting them consistently won't break
reliable assumptions. 

We hope to present this patch series soon, and hope that Bob's patches
can be merged for 4.6.  

(I have very high hopes for how Samba 4.6 will scale, given the work
already done). 

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba




More information about the samba-technical mailing list