[Samba] Unable to contact RPC server on a new DC

[Andrey Repin]

Greetings, All!

I've added a new DC to the working AD, transferred FSMO roles (checked, all 7
are ok') and (supposedly) correctly demoted the old DC.

SchemaMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=
InfrastructureMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
RidAllocationMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Si
PdcEmulationMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sit
DomainNamingMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sit
DomainDnsZonesMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ForestDnsZonesMasterRole owner: CN=NTDS
Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=S

Now, I'm unable to connect to the domain using RSAT - the error is "RPC server
unavailable".

Tried to forcefully remove old DC from the new one, just to be sure. It did
some additional cleanup, judging by the wall of debug messages, but that did
not help in the slightest.
Googled a bug https://bugzilla.samba.org/show_bug.cgi?id=12534 and manually
removed the last DNS record.
Now the DNS check reporting only new DC, but it is still not working like it
should.

So far, I've made these checks:

I can impersonate domain users and login with SSH using domain users on
domain members, even new logins with homedir creation, but…

Domain users/groups listing works partially.

One member:
groups: cannot find name for group ID 10008
groups: cannot find name for group ID 10009
groups: cannot find name for group ID 10010
# getent group | grep -P "1\\d{4}"
domain sudoers:x:10006:
domain admins:x:10000:
domain users:x:10001:
cvs:x:10005:

Another member:

# getent group | grep -P "1\\d{4}"
domain computers:x:10003:
domain sudoers:x:10006:
domain admins:x:10000:
domain guests:x:10002:
cloud admins:x:10010:
domain users:x:10001:
remote users:x:10009:
cloud users:x:10008:
git admins:x:10007:
cvs:x:10005:

Even wbinfo lists groups/users incorrectly on different machines.

anrdaemon at pubserver64:xterm:~
$ wbinfo -g | wc -l
20
anrdaemon at hosting64:xterm:~
$ wbinfo -g | wc -l
18

anrdaemon at daemon1:screen&0:~
$ wbinfo -u | wc -l
12
anrdaemon at hosting64:xterm:~
$ wbinfo -u | wc -l
11

Domain authorization works… not.

anrdaemon at hosting64:xterm:~
$ sudo -iH
[sudo] password for anrdaemon:
Sorry, try again.
sudo: 1 incorrect password attempt

anrdaemon at pubserver64:xterm:~
$ sudo -iH
[sudo] password for anrdaemon:
Domain Controller unreachable, using cached credentials instead.
Network resources may be unavailable

At the same time LDAP works like a charm and top-level tests pass

# net ads testjoin
Join is OK

# net ads info
LDAP server: 192.168.1.19
LDAP server name: dc2.ads.darkdragon.lan
Realm: ADS.DARKDRAGON.LAN
Bind Path: dc=ADS,dc=DARKDRAGON,dc=LAN
LDAP port: 389
Server time: Вс, 07 май 2023 18:03:26 MSK
KDC server: 192.168.1.19
Server time offset: 0

But…

# wbinfo -t
checking the trust secret for domain DARKDRAGON via RPC calls failed
wbcCheckTrustCredentials(DARKDRAGON): error code was
NT_STATUS_DOMAIN_CONTROLLER_NOT_FOUND (0xc0000233)
failed to call wbcCheckTrustCredentials: WBC_ERR_AUTH_ERROR
Could not check secret

And DC log is spammed with messages like

: [2023/05/07 18:05:49.693988,  0]
../../source4/auth/unix_token.c:95(security_token_to_unix_toke>
:   Unable to convert first SID
(S-1-5-21-2269650170-3990761244-2407083512-1106) in user token to>
: [2023/05/07 18:05:49.694080,  0]
../../libcli/security/security_token.c:56(security_token_debug)
:   Security token SIDs (8):
:     SID[  0]: S-1-5-21-2269650170-3990761244-2407083512-1106
:     SID[  1]: S-1-5-21-2269650170-3990761244-2407083512-515
:     SID[  2]: S-1-1-0
:     SID[  3]: S-1-5-2
:     SID[  4]: S-1-5-11
:     SID[  5]: S-1-5-64-10
:     SID[  6]: S-1-5-32-554
:     SID[  7]: S-1-5-32-545
:    Privileges (0x          800000):
:     Privilege[  0]: SeChangeNotifyPrivilege
:    Rights (0x             400):
:     Right[  0]: SeRemoteInteractiveLogonRight

DC smb.conf attached, if needed.

I have a full offline backup of both DC's prior to transfer and already tried
to redo the process, but to the same effect.

So, what can I do now?

----------

Update since the original message failed to be delivered to the list,
and a month passed already:
Now, even DC is unable to confirm its connection to the domain.

(DC2)root at dc2:screen:~
# net ads testjoin
kerberos_kinit_password DARKDRAGON at ADS.DARKDRAGON.LAN failed: Client
not found in Kerberos database
Join to domain is not valid: The name provided is not a properly
formed account name.

The question remains the same: what can I do now? Short of restarting
the entire domain.
What checks can I run to see where the culprit is, and how I can cure it?


-- 
With best regards,
Andrey Repin
Saturday, May 6, 2023 22:22:05

Sorry for my terrible english...