[Samba] kerberos issue (SPN not found) with windows Hyper-V (samba 4.5.3 AD)

Thu Mar 16 10:35:52 UTC 2017

Hello,

I've setup over 6 months ago samba 4 AD on centos 7.3 (self compiled 
from source). Up until now I didn't encounter any undocumented errors. I 
have 3 DC's (all samba 4.5.3) which are working pretty nice with over 60 
windows clients.

The issue I've stumbled upon is when I added Windows server Hyper-V 
hosts to the domain. Tried with Hyper-V from 2012, 2012r2 and new 2016 - 
all exact same problem.

I've searched and googled and found one old topic with the same issue in 
samba lists, but no help was given, but also - not enough info was supplied.

The main issue is that Hyper-v Hosts are unable to authenticate each 
other using kerberos for live migration and replication (only two 
features that require kerberos) - windows host gives well documented 
error, that it's unable to authenticate using kerberos.

I've gathered all the logs, which I think explain the issue quite 
clearly and hopefully someone will be able to give a viable solution.

domain/realm let's call it:
mydomain.com.xyz @ MYDOMAIN.COM.XYZ
hyper-v hosts:
BM-SRV-5 and BMSRV-WIN10 (both with windows server 2016 standard with 
hyper-v host role installed)
DC1, DC2, DC3 are my 3 domain controllers (names not really original :) )

Microsoft Hyper-V requires specific SPN's registered for hosts:

*Microsoft Virtual Console Service**
**Hyper-V Replica Service**
**Microsoft Virtual System Migration Service*

The SPN's should be automatically registered in the AD machine account 
by the windows, but this fails with windows error 14050. This error is 
well documented, but none of the solutions helped, and I think the error 
is with samba AD as I'll try to explain.

I added the  SPN's manually via windows setpsn (for both hyper-v hosts 
of course, mydomain.com.xyz is of course bogus name, real domain is 
something different)

/setspn -S "Hyper-V Replica Service/BMSRV-WIN10" BMSRV-WIN10//
//setspn -S "Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz" 
BMSRV-WIN10//
//
//setspn -S "Microsoft Virtual System Migration Service/BMSRV-WIN10" 
BMSRV-WIN10//
//setspn -S "Microsoft Virtual System Migration 
Service/BMSRV-WIN10.mydomain.com.xyz" BMSRV-WIN10//
//
//setspn -S "Microsoft Virtual Console Service/BMSRV-WIN10" BMSRV-WIN10"//
//setspn -S "Microsoft Virtual Console 
Service/BMSRV-WIN10.mydomain.com.xyz" BMSRV-WIN10//
/
Both windows and samba when queried show correct SPN's:
output of windows query:

spn -l BMSRV-WIN10

Registered ServicePrincipalNames for 
CN=BMSRV-WIN10,CN=Computers,DC=mydomain,DC=com,DC=xyz:
     HOST/BMSRV-WIN10
     HOST/BMSRV-WIN10.mydomain.com.xyz
     Hyper-V Replica Service/BMSRV-WIN10
     Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz
     Microsoft Virtual Console Service/BMSRV-WIN10
     Microsoft Virtual Console Service/BMSRV-WIN10.mydomain.com.xyz
     Microsoft Virtual System Migration Service/BMSRV-WIN10
     Microsoft Virtual System Migration Service/BMSRV-WIN10.mydomain.com.xyz
     RestrictedKrbHost/BMSRV-WIN10
     RestrictedKrbHost/BMSRV-WIN10.mydomain.com.xyz
     TERMSRV/BMSRV-WIN10
     TERMSRV/BMSRV-WIN10.mydomain.com.xyz
     WSMAN/BMSRV-WIN10
     WSMAN/BMSRV-WIN10.mydomain.com.xyz

output of samba-tool query:
samba-tool spn list BMSRV-WIN10$

samba-tool spn list BMSRV-WIN10$
schema_fsmo_init: we are master[no] updates allowed[no]
User CN=BMSRV-WIN10,CN=Computers,DC=mydomain,DC=com,DC=xyz has the 
following servicePrincipalName:
          HOST/BMSRV-WIN10
          HOST/BMSRV-WIN10.mydomain.com.xyz
          Hyper-V Replica Service/BMSRV-WIN10
          Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz
          Microsoft Virtual Console Service/BMSRV-WIN10
          Microsoft Virtual Console Service/BMSRV-WIN10.mydomain.com.xyz
          Microsoft Virtual System Migration Service/BMSRV-WIN10
          Microsoft Virtual System Migration 
Service/BMSRV-WIN10.mydomain.com.xyz
          RestrictedKrbHost/BMSRV-WIN10
          RestrictedKrbHost/BMSRV-WIN10.mydomain.com.xyz
          TERMSRV/BMSRV-WIN10
          TERMSRV/BMSRV-WIN10.mydomain.com.xyz
          WSMAN/BMSRV-WIN10
          WSMAN/BMSRV-WIN10.mydomain.com.xyz

It looks all fine and well (the SPN names are 100% correct verified).

For the hyper-v features to work (replica and live migration) with 
kerberos I need to setup delegation (it's set - verified it a milion 
times over it's set the right way, just like MS wants it).

I know that I can obtain tickets to other SPN
(from windows: *klist cifs/BMSRV-WIN10* grants me a valid ticket for 
example)

Now cometh the error:
When I try to run hyper-v replica it fails with error concerning 
kerberos and SPN  not being there

Log from samba DC3 (when trying to start Hyper-V replica from BM-SRV-5 
to BMSRV-WIN.10)

  Kerberos: TGS-REQ BM-SRV-5$@MYDOMAIN.COM.XYZ from 
ipv4:192.168.1.10:56993 for Hyper-V\ Replica\ 
Service/BMSRV-WIN10.mydomain.com.xyz at MYDOMAIN.COM.XYZ [canonicalize, 
renewable, forwardable]
[2017/03/16 10:55:07.246904,  4] 
../source4/dsdb/samdb/cracknames.c:169(LDB_lookup_spn_alias)
   LDB_lookup_spn_alias: no alias for service Hyper-V Replica Service 
applicable
[2017/03/16 10:55:07.246971,  3] 
../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
   Kerberos: Searching referral for BMSRV-WIN10.mydomain.com.xyz
[2017/03/16 10:55:07.247028,  3] 
../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
   Kerberos: Server not found in database: Hyper-V\ Replica\ 
Service/BMSRV-WIN10.mydomain.com.xyz at MYDOMAIN.COM.XYZ: no such entry 
found in hdb
[2017/03/16 10:55:07.247053,  3] 
../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
   Kerberos: Failed building TGS-REP to ipv4:192.168.1.10:56993

log from wireshark (earlier attempt but same issue, this time when 
trying to start live migration from BM-SRV-5 to BMSRV-WIN10):

req-body
     Padding: 0
     kdc-options: 40810000 (forwardable, renewable, canonicalize)
     realm: MYDOMAIN.COM.XYZ
     sname
         name-type: kRB5-NT-SRV-INST (2)
         sname-string: 2 items
             SNameString: Microsoft Virtual System Migration Service
             SNameString: BMSRV-WIN10
     till: 2037-09-13 02:48:05 (UTC)
     nonce: 17847174
     etype: 5 items
     enc-authorization-data

error:
krb-error
     pvno: 5
     msg-type: krb-error (30)
     ctime: 2017-03-16 08:01:23 (UTC)
     cusec: 128
     stime: 2017-03-16 08:01:23 (UTC)
     susec: 66964
     error-code: eRR-S-PRINCIPAL-UNKNOWN (7)
     realm: <unspecified realm>
     sname
         name-type: kRB5-NT-UNKNOWN (0)
         sname-string: 0 items

Same errors are when going the other way round,

So the SPN's are clearly there (both setspn -l and samba-tool spn list 
outputs confirm that), the client sends correct request (as seen by 
wireshark and/or samba log), but suddenly samba is unable to find the SPN.
I'm a complete newbie (well, sort-of) when it comes to kerberos and 
samba, but maybe because the SPN is with spaces, as it's pretty unusual, 
but that's what Microsoft wants/needs?
I don't know, just a guess :-) . The features offered by hyper-v in AD 
are obviously beneficial and I would love to get them working.
Any help, workaround or tip - I will be very, very thankful. If more 
info is needed I'll gladly supply logs/whatever is needed.

Kacper Wirski