[Samba] Upgrading samba DC with "DC Rejoin" fails "Failed to setup database for BIND, AD based DNS cannot be used"

Wed Dec 8 19:09:37 UTC 2021

Hello

I tried to upgrade one of my lab domain controlers running raspbian buster with samba (Version 4.9.5-Debian) to Raspbian Bullseye with samba Version 4.13.13-Debian. I tried to follow the wiki article (https://wiki.samba.org/index.php/Upgrading_a_Samba_AD_DC#Rejoining_the_upgraded_DC) as close as possible and tried the "DC rejoin" approach as I am upgrading over several major releases. 

Unfortunately the rejoining failed 

sudo samba-tool domain join DOMAIN.int DC -U"DOMAIN\administrator" --dns-backend=BIND9_DLZ:

INFO 2021-12-08 16:55:22,835 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #107: Finding a writeable DC for domain 'DOMAIN.int'
INFO 2021-12-08 16:55:22,874 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #109: Found DC dc01.DOMAIN.int
Password for [DOMAIN\administrator]:
INFO 2021-12-08 16:55:29,005 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1543: workgroup is DOMAIN
INFO 2021-12-08 16:55:29,006 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1546: realm is DOMAIN.int
Adding CN=DC02,OU=Domain Controllers,DC=DOMAIN,DC=de
Adding CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=DOMAIN,DC=de
Adding CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=DOMAIN,DC=de
Adding SPNs to CN=DC02,OU=Domain Controllers,DC=DOMAIN,DC=de
Setting account password for DC02$
Enabling account
Adding DNS account CN=dns-DC02,CN=Users,DC=DOMAIN,DC=de with dns/ SPN
Setting account password for dns-DC02
Calling bare provision
INFO 2021-12-08 16:55:32,678 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2122: Looking up IPv4 addresses
INFO 2021-12-08 16:55:32,684 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2139: Looking up IPv6 addresses
WARNING 2021-12-08 16:55:32,690 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2144: More than one IPv6 address found. Using IPv6_GLOBAL
INFO 2021-12-08 16:55:34,466 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2290: Setting up share.ldb
INFO 2021-12-08 16:55:34,555 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2294: Setting up secrets.ldb
INFO 2021-12-08 16:55:34,626 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2299: Setting up the registry
INFO 2021-12-08 16:55:34,872 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2302: Setting up the privileges database
INFO 2021-12-08 16:55:35,015 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2305: Setting up idmap db
INFO 2021-12-08 16:55:35,108 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2312: Setting up SAM db
INFO 2021-12-08 16:55:35,138 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #897: Setting up sam.ldb partitions and settings
INFO 2021-12-08 16:55:35,140 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #909: Setting up sam.ldb rootDSE
INFO 2021-12-08 16:55:35,160 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #1322: Pre-loading the Samba 4 and AD schema
Unable to determine the DomainSID, can not enforce uniqueness constraint on local domainSIDs

INFO 2021-12-08 16:55:35,436 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2364: A Kerberos configuration suitable for Samba AD has been generated at /var/lib/samba/private/krb5.conf
INFO 2021-12-08 16:55:35,437 pid:4874 /usr/lib/python3/dist-packages/samba/provision/__init__.py #2366: Merge the contents of this file with your system krb5.conf or replace it with this one. Do not create a symlink!
Provision OK for domain DN DC=DOMAIN,DC=de
Starting replication
Schema-DN[CN=Schema,CN=Configuration,DC=DOMAIN,DC=de] objects[402/1550] linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=DOMAIN,DC=de] objects[804/1550] linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=DOMAIN,DC=de] objects[1206/1550] linked_values[0/0]
Schema-DN[CN=Schema,CN=Configuration,DC=DOMAIN,DC=de] objects[1550/1550] linked_values[0/0]
Analyze and apply schema objects
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[402/1642] linked_values[0/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[804/1642] linked_values[0/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[1206/1642] linked_values[0/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[1608/1642] linked_values[0/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[1642/1642] linked_values[46/46]
Failed to commit objects: DOS code 0x000021bf
Missing target object - retrying with DRS_GET_TGT
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[2044/1642] linked_values[47/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[2446/1642] linked_values[47/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[2848/1642] linked_values[47/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[3250/1642] linked_values[47/1]
Partition[CN=Configuration,DC=DOMAIN,DC=de] objects[3284/1642] linked_values[92/46]
Replicating critical objects from the base DN of the domain
Partition[DC=DOMAIN,DC=de] objects[98/98] linked_values[23/23]
Partition[DC=DOMAIN,DC=de] objects[311/311] linked_values[31/31]
Done with always replicated NC (base, config, schema)
Replicating DC=DomainDnsZones,DC=DOMAIN,DC=de
Partition[DC=DomainDnsZones,DC=DOMAIN,DC=de] objects[87/87] linked_values[0/0]
Replicating DC=ForestDnsZones,DC=DOMAIN,DC=de
Partition[DC=ForestDnsZones,DC=DOMAIN,DC=de] objects[26/26] linked_values[0/0]
Exop on[CN=RID Manager$,CN=System,DC=DOMAIN,DC=de] objects[3] linked_values[0]
Committing SAM database
Repacking database from v1 to v2 format (first record CN=Text-Encoded-OR-Address,CN=Schema,CN=Configuration,DC=DOMAIN,DC=de)
Repack: re-packed 10000 records so far
Repacking database from v1 to v2 format (first record CN=IntellimirrorSCP-Display,CN=413,CN=DisplaySpecifiers,CN=Configuration,DC=DOMAIN,DC=de)
Repacking database from v1 to v2 format (first record DC=DC02\0ADEL:9c0906f0-58cf-4947-9a93-8525ea7ecd0d,CN=Deleted Objects,DC=DomainDnsZones,DC=DOMAIN,DC=de)
Repacking database from v1 to v2 format (first record DC=_ldap._tcp.intfault-First-Site-Name._sites.dc,DC=_msdcs.DOMAIN.int,CN=MicrosoftDNS,DC=ForestDnsZones,DC=DOMAIN,DC=de)
Repacking database from v1 to v2 format (first record CN=DOMAIN,CN=hosts,CN=ypServ30,CN=RpcServices,CN=System,DC=DOMAIN,DC=de)
INFO 2021-12-08 16:56:23,666 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1116: Adding 3 remote DNS records for DC02.DOMAIN.int
INFO 2021-12-08 16:56:23,838 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1175: Adding DNS AAAA record DC02.DOMAIN.int for IPv6 IP: IPv6_GLOBAL
INFO 2021-12-08 16:56:23,936 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1175: Adding DNS AAAA record DC02.DOMAIN.int for IPv6 IP: IPv6_LOCAL
INFO 2021-12-08 16:56:24,012 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1179: Adding DNS A record DC02.DOMAIN.int for IPv4 IP: 172.19.173.32
INFO 2021-12-08 16:56:24,144 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1207: Adding DNS CNAME record 3810997d-5854-4572-a87f-a5a1ae81366a._msdcs.DOMAIN.int for DC02.DOMAIN.int
INFO 2021-12-08 16:56:24,285 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1232: All other DNS records (like _ldap SRV records) will be created samba_dnsupdate on first startup
INFO 2021-12-08 16:56:24,287 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1238: Replicating new DNS records in DC=DomainDnsZones,DC=DOMAIN,DC=de
Partition[DC=DomainDnsZones,DC=DOMAIN,DC=de] objects[2/2] linked_values[0/0]
INFO 2021-12-08 16:56:24,424 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1238: Replicating new DNS records in DC=ForestDnsZones,DC=DOMAIN,DC=de
Partition[DC=ForestDnsZones,DC=DOMAIN,DC=de] objects[2/2] linked_values[0/0]
INFO 2021-12-08 16:56:24,521 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1253: Sending DsReplicaUpdateRefs for all the replicated partitions
INFO 2021-12-08 16:56:24,690 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1283: Setting isSynchronized and dsServiceName
INFO 2021-12-08 16:56:24,733 pid:4874 /usr/lib/python3/dist-packages/samba/join.py #1298: Setting up secrets database
ERROR 2021-12-08 16:56:25,350 pid:4874 /usr/lib/python3/dist-packages/samba/provision/sambadns.py #888: Failed to setup database for BIND, AD based DNS cannot be used
Join failed - cleaning up
ERROR(<class 'samba.join.DCJoinException'>): uncaught exception - Can't join, error: Not removing account DC02$ which looks like a Samba DC account matching the password we already have.  To override, remove secrets.ldb and secrets.tdb
 File "/usr/lib/python3/dist-packages/samba/netcmd/__init__.py", line 186, in _run
   return self.run(*args, **kwargs)
 File "/usr/lib/python3/dist-packages/samba/netcmd/domain.py", line 661, in run
   join_DC(logger=logger, server=server, creds=creds, lp=lp, domain=domain,
 File "/usr/lib/python3/dist-packages/samba/join.py", line 1559, in join_DC
   ctx.do_join()
 File "/usr/lib/python3/dist-packages/samba/join.py", line 1469, in do_join
   ctx.cleanup_old_join()
 File "/usr/lib/python3/dist-packages/samba/join.py", line 288, in cleanup_old_join
   ctx.cleanup_old_accounts(force=force)
 File "/usr/lib/python3/dist-packages/samba/join.py", line 253, in cleanup_old_accounts
   raise DCJoinException("Not removing account %s which "

When I delete the files secrets.ldb and secrets.tdb on the to be joined DC the result of another attempt is still the same. Deleting the same files on the primary results in problems with winbind not starting up.

Any Ideas? Any help highly appreciated

Henning