Need urgent help with samba4 DC re-join

[Andrew Bartlett]

On Thu, 2012-06-28 at 15:16 +0200, Andreas Oster wrote:
> Am 28.06.2012 09:20, schrieb Andrew Bartlett:
> > On Thu, 2012-06-28 at 07:26 +0200, Andreas Oster wrote:
> >> Am 28.06.2012 00:00, schrieb Andrew Bartlett:
> >>> On Wed, 2012-06-27 at 19:27 +0200, Andreas Oster wrote:
> >>>> Am 27.06.2012 15:43, schrieb Andreas Oster:
> >>>>> Am 27.06.2012 15:35, schrieb Andrew Bartlett:
> >>>>>> On Wed, 2012-06-27 at 15:28 +0200, Andreas Oster wrote:
> >>>>>>> Am 27.06.2012 15:21, schrieb Andrew Bartlett:
> >>>>>>>> On Wed, 2012-06-27 at 15:09 +0200, Andreas Oster wrote:
> >>>>>>>>> Hello Andrew,
> >>>>>>>>>
> >>>>>>>>> i think the only differences when doing a "ldbsearch -H sam.ldb -s base
> >>>>>>>>> -b DC=DomainDnsZones,DC=novanetwork,DC=loc" are:
> >>>>>>>>>
> >>>>>>>>> objectClass: domain
> >>>>>>>>> objectClass: domainDNS
> >>>>>>>>>
> >>>>>>>>> and
> >>>>>>>>>
> >>>>>>>>> objectCategory: CN=Top,CN=Schema,CN=Configuration,DC=novanetwork,DC=loc
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I do not know if this was correct before demoting the second DC.
> >>>>>>>>> It did not come into my mind to check for errors because everything
> >>>>>>>>> worked like a charm and I was/am really happy with samba4.
> >>>>>>>>>
> >>>>>>>>> here the output of:
> >>>>>>>>>
> >>>>>>>>> ../bin/ldbsearch -H sam.ldb -s base -b
> >>>>>>>>> dc=domaindnszones,DC=novanetwork,DC=loc --reveal --show-binary
> >>>>>>>>> replPropertyMetaData
> >>>>>>>>
> >>>>>>>> Thanks.  This gives us a very good clue as to what has gone on:
> >>>>>>>>
> >>>>>>>> I'm assuming that 61f36cfd-ba7d-4702-87d3-7e861bb32cfe is PDC and
> >>>>>>>> fd9ca123-ed33-483a-a735-ff41940789a2 was the BDC?
> >>>>>>>>
> >>>>>>>> The key attributes changed that you mention are objectClass and
> >>>>>>>> objectCategory.  Both need to be fixed.  The incorrect values seem to
> >>>>>>>> have been written at Sun Apr 22 16:07:06 2012 CEST compared with Sun Apr
> >>>>>>>> 22 16:03:41 2012 CEST for the good ones.
> >>>>>>>>
> >>>>>>>> My guess is that in attempting to replicate the DNS to the slave with
> >>>>>>>> the samba-tool drs commands, and running samba_upgradedns on that
> >>>>>>>> server, have somehow sent back a corrupted version of the same object.
> >>>>>>>>
> >>>>>>>> Andrew Bartlett
> >>>>>>>>
> >>>>>>
> >>>>>>> Hello Andrew,
> >>>>>>>
> >>>>>>> this is absolute possible. In a prior try to replicate the
> >>>>>>> DomainDnsZones and ForestDnsZones I used the samba-tool drs command but
> >>>>>>> this did not succeed and, if I do remember correct, quit with an error
> >>>>>>> message. As everything kept on working as before, it did not come to my
> >>>>>>> mind that it might have broken anything.
> >>>>>>>
> >>>>>>> Do you have an idea how to fix this ?
> >>>>>>
> >>>>>> ldbedit -H sam.ldb -s base -b dc=domaindnszones,DC=novanetwork,DC=loc
> >>>>>>
> >>>>>> Then set:
> >>>>>>
> >>>>>> objectClass: domainDNS 
> >>>>>> objectCategory:
> >>>>>> CN=Domain-DNS,CN=Schema,CN=Configuration,DC=novanetwork,DC=loc
> >>>>>>
> >>>>>> That should fix it (I hope).
> >>>>>>
> >>>>>> This is the end for me for tonight, but I'll follow up tomorrow.
> >>>>>> Hopefully others here can help you with any remaining details. 
> >>>>>>
> >>>>>> KEEP GOOD BACKUPS.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Andrew Bartlett
> >>>>>>
> >>>>> Hello Andrew,
> >>>>>
> >>>>> thank you very much for your help. I appreciate very much that you use
> >>>>> your limited time to help guys like me.
> >>>>>
> >>>>> I will create a backup and do the proposed changes with ldbedit. I will
> >>>>> report here if joining works again afterwards.
> >>>>>
> >>>>> best regards
> >>>>>
> >>>>> Andreas
> >>>>>
> >>>>>
> >>>> Hello Andrew,
> >>>>
> >>>> unfortunately, I have been unable to modify/add the settings via
> >>>> ldbedit. I got the following error message when committing the
> >>>> modifications:
> >>>>
> >>>> ../bin/ldbedit -H sam.ldb -s base -b dc=domaindnszones,DC=novanetwork,DC=loc
> >>>> failed to modify DC=DomainDnsZones,DC=novanetwork,DC=loc - cannot change
> >>>> replicated attribute on partial replica at
> >>>> ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:1408
> >>>>
> >>>> Any idea what could be causing it ?
> >>>
> >>> When Amitay first wrote samba_dnsupgrade, he misunderstood about the
> >>> difference between a partial and a full replica.  A partition does not
> >>> start as one, and then become another.  We will need to correct your
> >>> database to record the DNS partition as being a full replica. 
> >>>
> >>>> Luckily, I did a vmware snapshot before demoting the second DC, I was so
> >>>> upset that I forget about that. I have now reverted back to the old
> >>>> snapshots and second DC is functional again.
> >>>> I have done the tests with ldbsearch on the DomainDnsZones and
> >>>> ForestDnsZones and realized, that the faulty entries already existed
> >>>> before demoting. So I guess before I can demote the second DC again I
> >>>> will have to fix those errors.
> >>>
> >>> It will also be required before any modifications can be made.  This may
> >>> explain why DNS entries appear to be 'stuck' - Samba is refusing to
> >>> change anything in that partition, because it wrongly believes that
> >>> someone else is the master for that data. 
> >>>
> >>> Andrew Bartlett
> >>>
> >> Hello Andrew,
> >>
> >> do you have an idea what needs to be changed ? Is it only the
> >> DomainDnsZones and ForestDnsZones part or are there other places where
> >> changes need to be made ? Yesterday I have tried to change the
> >> DomainDnsZones stuff but got an error message when trying to commit the
> >> modifications.
> > 
> > That is what I was trying to explain.  The fact that the NTDS Settings
> > for your DC lists these as partialReplica partitions is the cause of the
> > problem. 
> > 
> > We need to correct that in your instance, and if we find that many folks
> > have run the buggy version of the samba_dnsupgrade script, we may need
> > to add a special case to dbcheck for this.  I'm already thinking a
> > schema compliance check would be very worthwhile, so this can be found
> > before modifications are made.
> > 
> > Andrew Bartlett
> > 
> Hello Andrew,
> 
> so, how should I proceed on from here ? What can I do to fix those issues ?
> 
> best regards
> 
> Andreas

Can you please run:

ldbsearch -H sam.ldb -s sub --cross-ncs objectclass=ntdsdsa

I need to see what is in your NTDS Setting entry for each DC so I can
figure out how to fix this.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org