Filtering links by the uptodateness vector, sorting them client-side (was: Re: [PATCH] drs exop test & refactoring)

Andrew Bartlett abartlet at samba.org
Wed Jan 4 08:39:24 UTC 2017


On Thu, 2016-12-22 at 22:36 +1300, Andrew Bartlett wrote:
> On Mon, 2016-12-19 at 23:48 +1300, Andrew Bartlett wrote:
> 
> > 
> > For that angle, I wanted to foreshadow that we are also working on
> > a
> > means to store linked attributes sorted by the GUID, and that we
> > are
> > very keen to merge this patch, once complete, for 4.6.
> > 
> > The aim is very simple: get_parsed_dns() in Samba's repl_meta_data
> > code
> > represents the bulk of the CPU time spent during link replication
> > and
> > link value modification.  We have already pushed changes to make
> > the
> > GUID parsing faster, but we can avoid parsing many of the DNs at
> > all
> > by
> > using a bisection search.  That however needs us to keep the links
> > sorted in the ldb message.
> > 
> > The links are already implicitly sorted in many cases, as new
> > replicas
> > will have them applied in the same sorted order that the initial DC
> > sends in GetNCChanges, so sorting them consistently won't break
> > reliable assumptions. 
> > 
> > We hope to present this patch series soon, and hope that Bob's
> > patches
> > can be merged for 4.6.  
> 
> I'm glad to say Bob's changes are in.  A patch set for the link
> sorting
> is at:
> 
> http://git.catalyst.net.nz/gw?p=samba.git;a=shortlog;h=refs/heads/ldb
> -s
> orted-links

This work as continued, and it is almost finished, for the case of
adding links.  Delete and replace are still inefficient however.

> Features include:
>  - using a magic value in @IDXATTR to detect if the DB has been
> opened
> in an old Samba version
>  - new compatibleFeatures and incompatibleFeatures on @SAMBA_DSDB
>  - sorting the links and a bisection search on link addition
> 
> TODO:
>  - efficient handling of delete and replace of links
>  - an upgrade path (currently this is only enabled on new domains or
> replicas)
>  - sorting by the binary GUID, not the GUID_compare() to match DRS
> and
> eventually perhaps allow the sorting in GetNCChanges to be removed. 

This has now been done.

>  - fixing tests that are too sensitive to link order (diff of
> ldbsearch
> output, needs piping via sort).

This is done in a patch at:

https://git.samba.org/?p=abartlet/samba.git/.git;a=commit;h=2e9aa2b08ae
601f97f2aef6711271c7e1befa486

A branch with this fix in it is at 
https://git.samba.org/?p=abartlet/samba.git/.git;a=shortlog;h=refs/head
s/ldb-sorted-links

> We hope to improve it tomorrow and get it in to master very soon, so
> feedback on the whole diff most welcome while we tidy up the commit
> order.

I'm not expecting this to make 4.6, but it is almost ready for master,
and so may make it depending on when the release it cut.  We are doing
the performance graphs to see what difference it actually makes right
now.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba




More information about the samba-technical mailing list