[Samba] replication problems in samba4 ad domain

[Jon Detert]

I started with one dc, 'dc1', running samba v4.0.21, in subnet1.

I successfully added two more dc's, 'dc2' and 'dc3', both running samba v4.0.24, both in subnet2.

There are several firewalls between subnets 1 & 2.

I continued to make firewall holes on behalf of msad after I added dc's 2 & 3.  I.e. when they were added, there were patterns of communication between the dcs that weren't yet allowed.

Replication is not fully working, and I don't know how to fix the situation.

Suggestions?  Thanks!

Replication Status is this:
---------------------------
     a) Changes made to dc1 replicate to dc2&3, but changes made to either dc2 or 3, do not replicate to dc1 (but do replicate to the other - i.e. if made on dc2, it replicates to dc3, and vice versa).

     b) The output of "samba-tool drs showrepl" :

        1) on all 3 dcs, says "Warning: No NC replicated for Connection!" in the "KCC CONNECTION OBJECTS" section for each of the other 2 dcs 
        2) on all 3 dcs, shows success for all 5 branches listed under the "OUTBOUND NEIGHBORS" section.
        3) on dcs 2 & 3, shows success for all 5 branches listed under the "INBOUND NEIGHBORS" section.
        4) on dc1, shows success for dc3, and failure for dc2, for all 5 branches under the "INBOUND NEIGHBORS" section.
     
Attempts I've made to resolve:
------------------------------
1) manually start the replication from dc2 -> dc1, by typing this on dc2:
   # samba-tool drs replicate dc1.infinity.local dc2.infinity.local dc=infinity,dc=local
ldb_wrap open of secrets.ldb
GENSEC backend 'gssapi_spnego' registered
GENSEC backend 'gssapi_krb5' registered
GENSEC backend 'gssapi_krb5_sasl' registered
GENSEC backend 'sasl-DIGEST-MD5' registered
GENSEC backend 'schannel' registered
GENSEC backend 'spnego' registered
GENSEC backend 'ntlmssp' registered
GENSEC backend 'krb5' registered
GENSEC backend 'fake_gssapi_krb5' registered
Using binding ncacn_ip_tcp:dc1.infinity.local[,seal]
ERROR(<class 'samba.drs_utils.drsException'>): DsReplicaSync failed - drsException: DsReplicaSync failed (-1073610723, 'NT_STATUS_RPC_PROTOCOL_ERROR')
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/drs.py", line 334, in run
    drs_utils.sendDsReplicaSync(self.drsuapi, self.drsuapi_handle, source_dsa_guid, NC, req_options)
  File "/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line 83, in sendDsReplicaSync
    raise drsException("DsReplicaSync failed %s" % estr)
#
As you can see, it fails.

2) demote dc2 from being a dc by typing this on dc2:
# samba-tool domain demote -Uadministrator
GENSEC backend 'gssapi_spnego' registered
GENSEC backend 'gssapi_krb5' registered
GENSEC backend 'gssapi_krb5_sasl' registered
GENSEC backend 'sasl-DIGEST-MD5' registered
GENSEC backend 'schannel' registered
GENSEC backend 'spnego' registered
GENSEC backend 'ntlmssp' registered
GENSEC backend 'krb5' registered
GENSEC backend 'fake_gssapi_krb5' registered
Using dc1.infinity.local as partner server for the demotion
Using binding ncacn_ip_tcp:dc1.infinity.local[,seal]
Password for [INFINITY\administrator]:
Desactivating inbound replication
Asking partner server dc1.infinity.local to synchronize from us
Error while demoting, re-enabling inbound replication
ERROR(<class 'samba.drs_utils.drsException'>): Error while sending a DsReplicaSync for partion CN=Schema,CN=Configuration,DC=infinity,DC=local - drsException: DsReplicaSync failed (-1073610723, 'NT_STATUS_RPC_PROTOCOL_ERROR')
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py", line 647, in run
    sendDsReplicaSync(drsuapiBind, drsuapi_handle, ntds_guid, str(part), drsuapi.DRSUAPI_DRS_WRIT_REP)
  File "/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line 83, in sendDsReplicaSync
    raise drsException("DsReplicaSync failed %s" % estr)
#
-- 
Jon Detert
Sr. Systems Administrator
Infinity Healthcare
Milwaukee, Wisconsin
414-290-6759