replPropertyMetaData sorting

[Andrew Bartlett]

On Fri, 2016-06-03 at 11:57 +0200, Stefan Metzmacher wrote:
> Hi Andrew,
> 
> > I've been looking, as part of my work at Catalyst, at issues around
> > corruption of our DB due to replication issues.  Specifically,
> > rename
> > handling, particularly with subtrees. 
> > 
> > I found and fixed a number of bugs, which you will have seen in the
> > braches I've proposed.  However, I've been less than satisfied with
> > the
> > tests for this area, so have been extending them.
> > 
> > I figured you might be very interested in the latest test findings,
> > regarding replPropertyMetaData.
> > 
> > As far as I can tell, in windows 2012r2, the special sort order for
> > replPropertyMetaData, that is with the RDN last, simply doesn't
> > exist. 
> >  Indeed, I wonder if it has always been an artifact of our DRS ->
> > ldb
> > conversion code in replicated_objects.c
> 
> Could be :-)
> I'm also unable to see the rdn attribute metadata in any of LDAP
> searches
> or ndrdumps of replication traces.

Good.

> I guess the important thing is that we keep the rdn value uptodate/in
> sync
> with the DN and the name attribute.

Yep.

> > I now have tests that query the server for the replPropertyMetaData
> > and
> > the DRS equivalent, after each operation, and it shows no such
> > special
> > sort order.
> > 
> > The only special thing I can find is that the RDN (eg CN) is given
> > a
> > local invocationID and a local version quite often, while remaining
> > excluded from replication (name is replicated instead).
> > 
> > See 
> > http://git.catalyst.net.nz/gw?p=samba.git;a=shortlog;h=refs/heads/r
> > epl_move-replPropertyMetaData
> > 
> > Thankfully because this is excluded over DRS, this has not caused
> > replication corruption.
> > 
> > Finally, I do expect that the patches I have already written will
> > cover
> > the dbcheck flapping test, which was a genuine DB corruption, due
> > to
> > mishandling of renamed/deleted objects.
> 
> I don't have time for a deep review, but it looks
> promising. Thanks a lot!
> 
> One comment regarding
> https://git.samba.org/?p=abartlet/samba.git/.git;a=commitdiff;h=4f90f
> 343fcbde0ace10e3b6cf65a78eba5b65762
> 
> From reading just the diff I'm wondering how this pass against Samba.

I have a list of password attributes to exclude, so as not to open the
can of worms you are handling here. :-)

> I had to add the following trick to get meta data stamps for empty
> attributes in order to get
> https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=340900584
> b39386cbb47655069bc15b93520a101
> to pass see
> https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=a95c629bd
> 05f618cfcf7047a724b04231b842309
> and
> https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=c002e4724
> f8f36105b907d28381ec476a8f73ff9
> 
> This is part of the
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/
> master4-smart-ok
> and
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/
> master4-smart-tmp
> branches. I'll write more about that branches later....

Thanks.  

I thought I was ready to submit these, but during the review process we
seem to have broken something, so I'm debugging that, looking over the
diff between the branches that passed and what should be much the same
code now.  

I'll be very glad when this is locked into autobuild. :-)

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba