Need urgent help with samba4 DC re-join

[Andrew Bartlett]

On Thu, 2012-06-28 at 07:26 +0200, Andreas Oster wrote:
> Am 28.06.2012 00:00, schrieb Andrew Bartlett:
> > On Wed, 2012-06-27 at 19:27 +0200, Andreas Oster wrote:
> >> Am 27.06.2012 15:43, schrieb Andreas Oster:
> >>> Am 27.06.2012 15:35, schrieb Andrew Bartlett:
> >>>> On Wed, 2012-06-27 at 15:28 +0200, Andreas Oster wrote:
> >>>>> Am 27.06.2012 15:21, schrieb Andrew Bartlett:
> >>>>>> On Wed, 2012-06-27 at 15:09 +0200, Andreas Oster wrote:
> >>>>>>> Hello Andrew,
> >>>>>>>
> >>>>>>> i think the only differences when doing a "ldbsearch -H sam.ldb -s base
> >>>>>>> -b DC=DomainDnsZones,DC=novanetwork,DC=loc" are:
> >>>>>>>
> >>>>>>> objectClass: domain
> >>>>>>> objectClass: domainDNS
> >>>>>>>
> >>>>>>> and
> >>>>>>>
> >>>>>>> objectCategory: CN=Top,CN=Schema,CN=Configuration,DC=novanetwork,DC=loc
> >>>>>>>
> >>>>>>>
> >>>>>>> I do not know if this was correct before demoting the second DC.
> >>>>>>> It did not come into my mind to check for errors because everything
> >>>>>>> worked like a charm and I was/am really happy with samba4.
> >>>>>>>
> >>>>>>> here the output of:
> >>>>>>>
> >>>>>>> ../bin/ldbsearch -H sam.ldb -s base -b
> >>>>>>> dc=domaindnszones,DC=novanetwork,DC=loc --reveal --show-binary
> >>>>>>> replPropertyMetaData
> >>>>>>
> >>>>>> Thanks.  This gives us a very good clue as to what has gone on:
> >>>>>>
> >>>>>> I'm assuming that 61f36cfd-ba7d-4702-87d3-7e861bb32cfe is PDC and
> >>>>>> fd9ca123-ed33-483a-a735-ff41940789a2 was the BDC?
> >>>>>>
> >>>>>> The key attributes changed that you mention are objectClass and
> >>>>>> objectCategory.  Both need to be fixed.  The incorrect values seem to
> >>>>>> have been written at Sun Apr 22 16:07:06 2012 CEST compared with Sun Apr
> >>>>>> 22 16:03:41 2012 CEST for the good ones.
> >>>>>>
> >>>>>> My guess is that in attempting to replicate the DNS to the slave with
> >>>>>> the samba-tool drs commands, and running samba_upgradedns on that
> >>>>>> server, have somehow sent back a corrupted version of the same object.
> >>>>>>
> >>>>>> Andrew Bartlett
> >>>>>>
> >>>>
> >>>>> Hello Andrew,
> >>>>>
> >>>>> this is absolute possible. In a prior try to replicate the
> >>>>> DomainDnsZones and ForestDnsZones I used the samba-tool drs command but
> >>>>> this did not succeed and, if I do remember correct, quit with an error
> >>>>> message. As everything kept on working as before, it did not come to my
> >>>>> mind that it might have broken anything.
> >>>>>
> >>>>> Do you have an idea how to fix this ?
> >>>>
> >>>> ldbedit -H sam.ldb -s base -b dc=domaindnszones,DC=novanetwork,DC=loc
> >>>>
> >>>> Then set:
> >>>>
> >>>> objectClass: domainDNS 
> >>>> objectCategory:
> >>>> CN=Domain-DNS,CN=Schema,CN=Configuration,DC=novanetwork,DC=loc
> >>>>
> >>>> That should fix it (I hope).
> >>>>
> >>>> This is the end for me for tonight, but I'll follow up tomorrow.
> >>>> Hopefully others here can help you with any remaining details. 
> >>>>
> >>>> KEEP GOOD BACKUPS.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Andrew Bartlett
> >>>>
> >>> Hello Andrew,
> >>>
> >>> thank you very much for your help. I appreciate very much that you use
> >>> your limited time to help guys like me.
> >>>
> >>> I will create a backup and do the proposed changes with ldbedit. I will
> >>> report here if joining works again afterwards.
> >>>
> >>> best regards
> >>>
> >>> Andreas
> >>>
> >>>
> >> Hello Andrew,
> >>
> >> unfortunately, I have been unable to modify/add the settings via
> >> ldbedit. I got the following error message when committing the
> >> modifications:
> >>
> >> ../bin/ldbedit -H sam.ldb -s base -b dc=domaindnszones,DC=novanetwork,DC=loc
> >> failed to modify DC=DomainDnsZones,DC=novanetwork,DC=loc - cannot change
> >> replicated attribute on partial replica at
> >> ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:1408
> >>
> >> Any idea what could be causing it ?
> > 
> > When Amitay first wrote samba_dnsupgrade, he misunderstood about the
> > difference between a partial and a full replica.  A partition does not
> > start as one, and then become another.  We will need to correct your
> > database to record the DNS partition as being a full replica. 
> > 
> >> Luckily, I did a vmware snapshot before demoting the second DC, I was so
> >> upset that I forget about that. I have now reverted back to the old
> >> snapshots and second DC is functional again.
> >> I have done the tests with ldbsearch on the DomainDnsZones and
> >> ForestDnsZones and realized, that the faulty entries already existed
> >> before demoting. So I guess before I can demote the second DC again I
> >> will have to fix those errors.
> > 
> > It will also be required before any modifications can be made.  This may
> > explain why DNS entries appear to be 'stuck' - Samba is refusing to
> > change anything in that partition, because it wrongly believes that
> > someone else is the master for that data. 
> > 
> > Andrew Bartlett
> > 
> Hello Andrew,
> 
> do you have an idea what needs to be changed ? Is it only the
> DomainDnsZones and ForestDnsZones part or are there other places where
> changes need to be made ? Yesterday I have tried to change the
> DomainDnsZones stuff but got an error message when trying to commit the
> modifications.

That is what I was trying to explain.  The fact that the NTDS Settings
for your DC lists these as partialReplica partitions is the cause of the
problem. 

We need to correct that in your instance, and if we find that many folks
have run the buggy version of the samba_dnsupgrade script, we may need
to add a special case to dbcheck for this.  I'm already thinking a
schema compliance check would be very worthwhile, so this can be found
before modifications are made.

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org