[Samba] Replication between Samba DCs (on different sites)?
Lorenzo Milesi
lorenzo.milesi at yetopen.com
Tue Feb 7 18:19:15 UTC 2023
>> gives some wire traces (or looks like), it ends up like the remote is returning
>> WERR_FILE_NOT_FOUND. And I don't see this error on the remote, all what I see
>> remote reporting on the logs is WERR_OK.
>
> So, after recompiling samba multiple times adding numerous debugging messages
> into *_DsReplicaSync and below, I found out the database on the "primary" DC
> contained a few references to the objects I had to remove before, for example:
>
> NOTE: old (due to rename or delete) DN string component for rIDSetReferences in
> object CN=SVDCM\0ADEL:a1a97bca-fbdf-429a-966e-cb8d71da606c,CN=Deleted
> Objects,DC=tls,DC=msk,DC=ru - CN=RID Set,CN=SVDCM,OU=Domain
> Controllers,DC=tls,DC=msk,DC=ru
>
> (note the CN=Deleted Objects).
>
> It was a long and painful debugging which lasted 2 complete days.
>
> After all this, when trying to find a way to get a dump of ldb - I found
> (by a chance) samba-tool dbcheck. Which found all these objects (but
> displayed "0 errors" anyway). And after removing these "Deleted Objects"
> things, it started working fine.
>
> There are just 329 objects in the db now.
>
> So, basically, samba-tool dbcheck for the rescue at the very least,
> and note that renames/deletes in samba does not quite work.
I'm stuck in a similar situation. I've upgraded a Samba4 network (3 DCs) from 4.13 to 4.17 using demote/join. Now dc1 and dc3 are fine, while dc2 is reporting WERR_FILE_NOT_FOUND on all sync items.
I ran samba-tool dbcheck and samba-tool dbcheck --cross-ncs several times, no errors are reported (anymore).
But whenever I attempt a replication, either of the full domain or of a single item, I get:
ERROR(<class 'samba.drs_utils.drsException'>): DsReplicaSync failed - drsException: DsReplicaSync failed (2, 'WERR_FILE_NOT_FOUND')
File "/usr/lib/python3/dist-packages/samba/netcmd/drs.py", line 570, in run
drs_utils.sendDsReplicaSync(server_bind, server_bind_handle, source_dsa_guid, NC, req_options)
File "/usr/lib/python3/dist-packages/samba/drs_utils.py", line 100, in sendDsReplicaSync
raise drsException("DsReplicaSync failed %s" % estr)
`samba-tool drs showrepl` on dc1 shows:
root at dc1:/var/lib/samba/bind-dns/dns# samba-tool drs showrepl
Default-First-Site-Name\DC1
DSA Options: 0x00000001
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
DSA invocationId: e6cb3930-897e-4ba9-952d-28802ace401d
==== INBOUND NEIGHBORS ====
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC2 via RPC
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC2 via RPC
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
Last attempt @ Tue Feb 7 19:01:08 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:08 2023 CET
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:01:08 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:08 2023 CET
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC2 via RPC
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC2 via RPC
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC2 via RPC
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
Last attempt @ Tue Feb 7 19:01:08 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:08 2023 CET
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:01:09 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 19:01:09 2023 CET
==== OUTBOUND NEIGHBORS ====
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
==== KCC CONNECTION OBJECTS ====
Connection --
Connection name: b058383e-10b2-4d00-a87d-30f88dd41db3
Enabled : TRUE
Server DNS name : dc2.wdc.domain.it
Server DN name : CN=NTDS Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
TransportType: RPC
options: 0x00000001
Warning: No NC replicated for Connection!
Connection --
Connection name: e5bbfdc3-9b2f-4bdb-82a5-52d2a3b73d9f
Enabled : TRUE
Server DNS name : exmedc.wdc.domain.it
Server DN name : CN=NTDS Settings,CN=DC3,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
TransportType: RPC
options: 0x00000001
Warning: No NC replicated for Connection!
While on (failing) dc2:
root at dc2:/var/lib/samba/bind-dns# samba-tool drs showrepl
Default-First-Site-Name\DC2
DSA Options: 0x00000001
DSA object GUID: 39a77331-7665-49bf-8dd4-89e19a1b1709
DSA invocationId: c39e6ca3-e46d-4994-be0a-6aa647b6934b
==== INBOUND NEIGHBORS ====
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 18:59:10 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 18:59:10 2023 CET
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:00:14 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
18 consecutive failure(s).
Last success @ NTTIME(0)
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 19:05:10 2023 CET failed, result 121 (WERR_SEM_TIMEOUT)
1 consecutive failure(s).
Last success @ Tue Feb 7 18:59:10 2023 CET
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:05:10 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
14 consecutive failure(s).
Last success @ NTTIME(0)
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 18:59:10 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 18:59:10 2023 CET
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 18:59:10 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
13 consecutive failure(s).
Last success @ NTTIME(0)
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 18:59:11 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 18:59:11 2023 CET
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:00:25 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
20 consecutive failure(s).
Last success @ NTTIME(0)
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ Tue Feb 7 18:59:10 2023 CET was successful
0 consecutive failure(s).
Last success @ Tue Feb 7 18:59:10 2023 CET
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 18:59:10 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
13 consecutive failure(s).
Last success @ NTTIME(0)
==== OUTBOUND NEIGHBORS ====
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:04:06 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
36 consecutive failure(s).
Last success @ NTTIME(0)
CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:04:06 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
36 consecutive failure(s).
Last success @ NTTIME(0)
DC=DomainDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:04:06 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
36 consecutive failure(s).
Last success @ NTTIME(0)
CN=Schema,CN=Configuration,DC=wdc,DC=domain,DC=it [13/1937]
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:04:06 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
36 consecutive failure(s).
Last success @ NTTIME(0)
DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC1 via RPC
DSA object GUID: 5b1453be-b074-4d33-9169-796f85eed444
Last attempt @ Tue Feb 7 19:04:06 2023 CET failed, result 2 (WERR_FILE_NOT_FOUND)
36 consecutive failure(s).
Last success @ NTTIME(0)
DC=ForestDnsZones,DC=wdc,DC=domain,DC=it
Default-First-Site-Name\DC3 via RPC
DSA object GUID: c3dcc23e-d2b3-4899-a615-3f4559b3a647
Last attempt @ NTTIME(0) was successful
0 consecutive failure(s).
Last success @ NTTIME(0)
==== KCC CONNECTION OBJECTS ====
Connection --
Connection name: ef26d1b1-771e-4feb-9049-e2dbf9ab6f64
Enabled : TRUE
Server DNS name : dc1.wdc.domain.it
Server DN name : CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
TransportType: RPC
options: 0x00000001
Warning: No NC replicated for Connection!
Connection --
Connection name: 219691ab-12bd-49ab-8a92-0570cabb3589
Enabled : TRUE
Server DNS name : exmedc.wdc.domain.it
Server DN name : CN=NTDS Settings,CN=DC3,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
TransportType: RPC
options: 0x00000001
Warning: No NC replicated for Connection!
There must be some stale record but I have no idea how to dig them out.
One ODD thing I found via RSAT: dc2 used to be into a separate site, while with DC demotion and rejoin has fallen into "Default-First-Site-Name". But Active Directory Sites and Services shows TWO DC2 records, one in the aforementioned site and one in the "original" one. The "original" one doesn't have a "NTDS Settings" child entry into the tree. I deleted it but apparently didn't have any impact. I moved DC2 back to the original site.
DC2 is also now hodling all FSMO roles, and if I try to take them back on DC1:
# samba-tool fsmo transfer --role=all -U administrator
ERROR: Transfer of 'rid' role failed: Failed FSMO transfer: WERR_NETNAME_DELETED
A second attempt transferred some of the roles, but still resulted in an error:
root at dc1:/var/lib/samba/bind-dns/dns# samba-tool fsmo transfer --role=all -U administrator
This DC already has the 'rid' FSMO role
This DC already has the 'pdc' FSMO role
This DC already has the 'naming' FSMO role
This DC already has the 'infrastructure' FSMO role
This DC already has the 'schema' FSMO role
Password for [WDC\administrator]:
ERROR: Failed to add role 'domaindns': LDAP error 16 LDAP_NO_SUCH_ATTRIBUTE - <attribute 'fSMORoleOwner': no matching attribute value while deleting attribute on 'CN=Infrastructure,DC=DomainDnsZones,DC=wdc,DC=domain,DC=it'> <>
Now all roles except DomainDnsZonesMasterRole and ForestDnsZonesMasterRole shows as residing on DC1:
root at dc1:/var/lib/samba/bind-dns/dns# samba-tool fsmo show
SchemaMasterRole owner: CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
InfrastructureMasterRole owner: CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
RidAllocationMasterRole owner: CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
PdcEmulationMasterRole owner: CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
DomainNamingMasterRole owner: CN=NTDS Settings,CN=DC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
DomainDnsZonesMasterRole owner: CN=NTDS Settings,CN=DC2,CN=Servers,CN=ARUBA-DataCenter1,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
ForestDnsZonesMasterRole owner: CN=NTDS Settings,CN=DC2,CN=Servers,CN=ARUBA-DataCenter1,CN=Sites,CN=Configuration,DC=wdc,DC=domain,DC=it
All servers run 4.17.5-Ubuntu (from mjt).
If matters:
root at dc1:/var/lib/samba/bind-dns/dns# samba-tool domain level show
Domain and forest function level for domain 'DC=wdc,DC=domain,DC=it'
Forest function level: (Windows) 2008 R2
Domain function level: (Windows) 2008 R2
Lowest function level of a DC: (Windows) 2008 R2
Thanks
--
Lorenzo Milesi - lorenzo.milesi at yetopen.com
CTO @ YetOpen Srl
Corso Martiri della Liberazione 114 - 23900 Lecco - ITALY - | 4801 Glenwood Avenue - Suite 200 - Raleigh, NC 27612 - USA -
Tel +39 0341 220 205 - info.it at yetopen.com | Phone +1 919-817-8106 - info.us at yetopen.com
Think green - Non stampare questa e-mail se non necessario / Don't print this email unless necessary
-------- D.Lgs. 196/2003 e GDPR 679/2016 --------
Tutte le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario.
Tutte le informazioni ivi contenute, compresi eventuali allegati, sono da ritenere confidenziali e riservate secondo i termini
del vigente D.Lgs. 196/2003 in materia di privacy e del Regolamento europeo 679/2016 - GDPR - e quindi ne e' proibita l'utilizzazione ulteriore non autorizzata.
Nel caso in cui questo messaggio Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo, stamparlo, a non inoltrarlo a terzi e ad avvertirci non appena possibile.
Grazie.
Confidentiality notice: this email message including any attachment is for the sole use of the intended recipient and may contain confidential and privileged information;
pursuant to Legislative Decree 196/2003 and the European General Data Protection Regulation 679/2016 - GDPR - any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recepient please delete this message without copying, printing or forwarding it to others, and alert us as soon as possible.
Thank you.
More information about the samba
mailing list