[Samba] Joining Samba to Upgraded 2003 domain failing

[Travis Wenks]

join attempted via
sudo samba-tool domain join net.example.com DC
-U'NET.EXAMPLE.COM\administrator'
--option='idmap_ldb:use rfc2307 = yes

failure lines

gensec_gssapi: NO credentials were delegated
GSSAPI Connection will be cryptographically signed
INFO 2020-11-13 09:00:44,891 pid:12210
/usr/local/samba/lib/python3.8/site-packages/samba/join.py #1178: Adding
DNS A record TLA-DC06.NET.EXAMPLE.COM for IPv4 IP: 10.74.20.69
Join failed - cleaning up
ldb_wrap open of secrets.ldb
Could not find machine account in secrets database: Failed to fetch machine
account password for EXAMPLE from both secrets.ldb (Could not find entry to
match filter: '(&(flatname=EXAMPLE)(objectclass=primaryDomain))' base:
'cn=Primary Domains': No such object: dsdb_search at
../../source4/dsdb/common/util.c:4760) and from
/usr/local/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO
Deleted CN=TLA-DC06,OU=Domain Controllers,DC=NET,DC=EXAMPLE,DC=COM
Deleted CN=NTDS
Settings,CN=TLA-DC06,CN=Servers,CN=NBG,CN=Sites,CN=Configuration,DC=NET,DC=EXAMPLE,DC=COM

Deleted
CN=TLA-DC06,CN=Servers,CN=NBG,CN=Sites,CN=Configuration,DC=NET,DC=EXAMPLE,DC=COM

ERROR(runtime): uncaught exception - (9714,
'WERR_DNS_ERROR_NAME_DOES_NOT_EXIST')
File
"/usr/local/samba/lib/python3.8/site-packages/samba/netcmd/__init__.py",
line 186, in _run
return self.run(*args, **kwargs)
File "/usr/local/samba/lib/python3.8/site-packages/samba/netcmd/domain.py",
line 661, in run
join_DC(logger=logger, server=server, creds=creds, lp=lp, domain=domain,
File "/usr/local/samba/lib/python3.8/site-packages/samba/join.py", line
1558, in join_DC
ctx.do_join()
File "/usr/local/samba/lib/python3.8/site-packages/samba/join.py", line
1455, in do_join
ctx.join_add_dns_records()
File "/usr/local/samba/lib/python3.8/site-packages/samba/join.py", line
1196, in join_add_dns_records
= ctx.samdb.dns_lookup("%s.%s" % (name, zone),
File "/usr/local/samba/lib/python3.8/site-packages/samba/samdb.py", line
1245, in dns_lookup
return dsdb_dns.lookup(self, dns_name,


This happens when trying to join a DC from packages or sources to a
existing domain that started as a 2003 server, was upgraded to 2008r2 them
migrated to samba. The FQDN is NET.EXAMPLE.COM here and the workgroup is
EXAMPLE
All servers are using bind9 for the backend and I have tried to join with
both the bind and samba dns backends
The lmhosts file is primed with ip address and server for all hosts.
The krbconf looks like this
[libdefaults]
default_realm = NET.EXAMPLE.COM
dns_lookup_realm = false
dns_lookup_kdc = true
[realms]
NET.EXAMPLE.COM = {
kdc = TLA-DC06 (NEW ubuntu server)
kdc = TLA-DC03 (working ubuntu server)
kdc = TLA-DC10 (working ubuntu server)
kdc = TLA-DC30 (working ubuntu server)
}
[domain_realm]
.net.example.com = NET.EXAMPLE.COM
Hosts file has all dc's and the domain in it.
The named.conf is based on the wiki and it is working well to my knowledge


This big thing that is stumping me is all our sites we build from the
ground up are named based on the wiki so we use
net.customer-owned-domain.com and the workgroup is net. While this site
preceded us and the workgroup is the customer-owned-domain

We found this
https://lists.samba.org/archive/samba/2020-February/228112.html

But I don't know where to go from here to fix this.
Is this the problem?
If so, what is our path to fix?