[Samba] samba4 replication issues | sam.ldb inconsistency

[mourik jan heupink - merit]

Hi all,

We seem to have some issues with our samba4 ad setup. I posted about 
this last week already but had received no replies at all so far. :-(

What is our situation:

two domain controllers (dc1 and dc2), one (separate) fileserver, all 
running sernet-4.1.7. From the workstations perspective, everything is 
running as it should, there appear to be no issues.

However: something in my replication has gone wrong... on dc2:

==== INBOUND NEIGHBORS ====

DC=DomainDnsZones,DC=samba,DC=company,DC=com
         Default-First-Site-Name\DC1 via RPC
                 DSA object GUID: 81a27497-bdfb-4977-9874-675bbfba490f
                 Last attempt @ Tue Jul  8 17:12:09 2014 CEST failed, 
result 8442 (WERR_DS_DRA_INTERNAL_ERROR)
                 3252 consecutive failure(s).
                 Last success @ Tue Jul  1 16:34:57 2014 CEST

CN=Configuration,DC=samba,DC=company,DC=com
         Default-First-Site-Name\DC1 via RPC
                 DSA object GUID: 81a27497-bdfb-4977-9874-675bbfba490f
                 Last attempt @ Tue Jul  8 17:12:10 2014 CEST was successful
                 0 consecutive failure(s).
                 Last success @ Tue Jul  8 17:12:10 2014 CEST
(the rest all replicates succesfully)

Then, to verify integrity of DC=DomainDnsZones on dc1, I type:

root at dc1:/var/log/samba# samba-tool dbcheck --cross-ncs
ltdb: 
tdb(/var/lib/samba/private/sam.ldb.d/DC=DOMAINDNSZONES,DC=SAMBA,DC=COMPANY,DC=COM.ldb): 
tdb_rec_read bad magic 0x198 at offset=1044437120
ERROR(ldb): uncaught exception - Indexed and full searches both failed!

On dc2 the same "samba-tool dbcheck cross-ncs" says: "checking 187478 
objects". Has been running for many hours now, I have no idea how far it 
is. The server is pretty buzy doing it.

So, my working conclusion is that on DC1 the 
DC=DomainDnsZones,DC=samba,DC=company,DC=com has become corrupted, and 
therefore fails to replicate to dc2.

Does the list agree with this?

I hope that dc2 is still having the correct DC=DomainDnsZones. But, 
since replication seems to be only from dc1 TO dc2, I'm unsure how to 
import the healthy dc2 database into dc1.

Does the above make any sense? How to make both dc's happy and fully 
functional again?

Any help would be VERY much appreciated... Hopefully I'll get some 
replies this time!

Kind regards,
MJ