[PATCH] samba-tool dbcheck: handle missing objectClass

[Andrew Bartlett]

On Fri, 2014-03-21 at 16:34 +1300, Andrew Bartlett wrote:
> On Wed, 2014-03-12 at 07:50 +1300, Andrew Bartlett wrote:
> > On Tue, 2014-03-11 at 16:18 +0100, Felix Botner wrote:
> > > > Felix:  Any news on testing this with a genuinely corrupted network?
> > > 
> > > Unfortunately not, but I think your analysis is correct with regard to the USN 
> > > ordering related attribute skip. We had a case in our internal testing 
> > > environments which looked like a replication issue related to USN ordering, 
> > > for details see https://forge.univention.org/bugzilla/show_bug.cgi?id=33904 .
> > > 
> > > > Are all the failing installations all pure Samba?
> > > 
> > > Yes.
> > > 
> > > > Does this only happen on domains with additional objectclasses, or also
> > > > on domains with stock schema?  Is the corruption across ALL DCs, or just
> > > > some?
> > > 
> > > This happened in a domain without schema extensions. 
> > 
> > Great.  This rules out the attribute sorting issue as a distinct thing
> > (while still valid and correct, it would only happen with schema
> > extensions).
> > 
> > > The corruption may vary 
> > > between DCs, but I'm not 100% sure about this. Cannot confirm this currently.
> > 
> > Please confirm this as soon as you can - it is vital to the
> > understanding, and to a way to resolve this.  I'll follow up more in
> > Arvid's mail as to why. 
> > 
> > > > Looking at the bug:
> > > > https://bugzilla.samba.org/show_bug.cgi?id=10398
> > > > and the code, I think I've found a possible cause.
> > > > 
> > > > get_nc_changes_build_object() has this code in it:
> > > > 
> > > >                 /* if the attribute has not changed, and it is not the
> > > >                    instanceType then don't include it */
> > > >                 if (md.ctr.ctr1.array[i].local_usn < highest_usn &&
> > > >                     extended_op != DRSUAPI_EXOP_REPL_SECRET &&
> > > >                     md.ctr.ctr1.array[i].attid !=
> > > > DRSUAPI_ATTID_instanceType) continue;
> > > > 
> > > > The purpose of this chunk is to avoid re-sending changes that the client
> > > > already has.  This would be particularly important for jpegPhoto, for
> > > > example.
> > > > 
> > > > However, if the highest USN the client thought it had seen was actually
> > > > higher than the one for which it had seen all objects, then a 'new'
> > > > object could be downloaded without all it's attributes.
> > > > 
> > > > I think that is how this corruption happens.  While disturbing, this
> > > > also suggests that the fix is to force re-replication, not to delete the
> > > > object, as on at least one DC, the whole correct object exists. 
> > > 
> > > Yes, this would pretty much explain the behavior we have seen.
> > 
> > Thanks.  At least now we have a possible direction to explore. 
> 
> If you still have access to a 'broken' environment, does:
> 
> samba-tool drs replicate --full-sync [--local] <destinationDC>
> <sourceDC> <NC> 
> 
> fix the issue?
> 
> Also, if you can please confirm that it still works even with the new
> patches we added to master this week.

Also, can I please see any replUpToDateVector values present in the
broken environment. 

My best guess is that the USN in this has got 'ahead' of where our
database really is replicated up to.

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba