Filtering links by the uptodateness vector, sorting them client-side (was: Re: [PATCH] drs exop test & refactoring)

[Andrew Bartlett]

On Mon, 2016-12-19 at 23:48 +1300, Andrew Bartlett wrote:

> For that angle, I wanted to foreshadow that we are also working on a
> means to store linked attributes sorted by the GUID, and that we are
> very keen to merge this patch, once complete, for 4.6.
> 
> The aim is very simple: get_parsed_dns() in Samba's repl_meta_data
> code
> represents the bulk of the CPU time spent during link replication and
> link value modification.  We have already pushed changes to make the
> GUID parsing faster, but we can avoid parsing many of the DNs at all
> by
> using a bisection search.  That however needs us to keep the links
> sorted in the ldb message.
> 
> The links are already implicitly sorted in many cases, as new
> replicas
> will have them applied in the same sorted order that the initial DC
> sends in GetNCChanges, so sorting them consistently won't break
> reliable assumptions. 
> 
> We hope to present this patch series soon, and hope that Bob's
> patches
> can be merged for 4.6.  

I'm glad to say Bob's changes are in.  A patch set for the link sorting
is at:

http://git.catalyst.net.nz/gw?p=samba.git;a=shortlog;h=refs/heads/ldb-s
orted-links

It is very much a draft, but we hope to give it a very big tidy-up
tomorrow.  Features include:
 - using a magic value in @IDXATTR to detect if the DB has been opened
in an old Samba version
 - new compatibleFeatures and incompatibleFeatures on @SAMBA_DSDB
 - sorting the links and a bisection search on link addition

TODO:
 - efficient handling of delete and replace of links
 - an upgrade path (currently this is only enabled on new domains or
replicas)
 - sorting by the binary GUID, not the GUID_compare() to match DRS and
eventually perhaps allow the sorting in GetNCChanges to be removed. 
 - fixing tests that are too sensitive to link order (diff of ldbsearch
output, needs piping via sort).

We hope to improve it tomorrow and get it in to master very soon, so
feedback on the whole diff most welcome while we tidy up the commit
order.

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ldb-sorted-links.diff
Type: text/x-patch
Size: 63155 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20161222/50317801/ldb-sorted-links.bin>