[Samba] Again NFSv4 and Kerberos at the 'samba way'...

L.P.H. van Belle belle at bazuin.nl
Thu Oct 25 13:20:51 UTC 2018


Hai marco, 

I left you original mail a bit intact and commented inbetween lines. 

> 
> > The nfs-server needs to be able to delegate the servers 
> with kerberos. (obligated for nfsv4 with kerberos mounts ) 
> > Start - ADUC, enable advanced features - goto  CN=Computers  
> >  get the member server's properties, tab Delegation, enable 
> "Trust this computer for delegation to any service (kerberos only) 
> >  I have set this on both NFS server and NFS client, thats 
> more because of the use of my servers. 
> 
> OK. Done. The same can be achived with:
> 
> 	samba-tool delegation for-any-service vdmpp1$ on
Great, saves me searching, i'll add the to my scripts. Thanks ;-) 
I still need todo more with samba-tool and drop the windows ADUC. 

> 
> > And obligated in smb.conf for this setup. 
> >     kerberos method = secrets and keytab
> OK.

> >     dedicated keytab file = /etc/krb5.keytab
> Seems not needed. smb.conf manpage say explicitly that this is needed
> only if 'kerberos method = dedicated keytab'; if 'kerberos 
> method = secrets and keytab' is set, system keytab are used.
Yes, thats exact what i want here, i want to see this in the configs.
Even its the default, it make the readablity of smb.conf better. 

And this is why you dont see this on the DC's. smb.conf
For the DC's its: 
/var/lib/samba/private/dns.keytab
/var/lib/samba/private/secrets.keytab


> 
> >     # Renew the kerberos ticket
> >     winbind refresh tickets = yes
> 
> Mmmmhhhh... manapage says about 'pam_winbind' tickets, so seems a
> 'user' part, not a system keytab one... anyway, in doubt, setted.
No this is needed. computer$ a user. 
If the computer$ changes its password its handled by winbind refresh tickets = yes
This keep you member working and in sync with the ad password for the computer. 
Out of sync, your server losses ad access. 

> 
> 
> > Check the spn/upn in the AD with the RSAT's ADUC, this is why i do.
> 
> Ok, added the nfs/ SPN:
> 	samba-tool spn add nfs/vdmpp1.ad.fvg.lnf.it vdmpp1$

On my own DC ( samba 4.8.6) , im adding the nfs/FQDN to hostname$ 
samba-tool spn add nfs/$(hostname -f) $(hostname -s)\$ 

And what is my result. 
samba-tool spn list $(hostname -s)\$ | grep nfs
Result :   nfs/hostname.internal.domain.tld

> 
> clearly you can check it also with:
> 
>  root at vdcsv1:~# samba-tool spn list vdmpp1$
>  vdmpp1$
>  User 
> CN=VDMPP1,OU=Computers,OU=Pasian,OU=FVG,DC=ad,DC=fvg,DC=lnf,DC
> =it has the following servicePrincipalName: 
> 	 HOST/VDMPP1
> 	 HOST/vdmpp1.ad.fvg.lnf.it
> 	 HOST/filepp.ad.fvg.lnf.it
> 	 HOST/FILEPP
> 	 HOST/cupspp.ad.fvg.lnf.it
> 	 HOST/CUPSPP
> 	 HOST/homepp.ad.fvg.lnf.it
> 	 HOST/HOMEPP
> 	 nfs/vdmpp1.ad.fvg.lnf.it
> 	 nfs/vdmpp1.ad.fvg.lnf.it/vdmpp1
> 	 nfs/vdmpp1.ad.fvg.lnf.it/vdmpp1.ad.fvg.lnf.it

Hmm, 
> 	 nfs/vdmpp1.ad.fvg.lnf.it   << correct 

And these are wrong. 
> 	 nfs/vdmpp1.ad.fvg.lnf.it/vdmpp1
> 	 nfs/vdmpp1.ad.fvg.lnf.it/vdmpp1.ad.fvg.lnf.it
Remove these 2. 


What is the output of : 
dig a  vdmpp1.ad.fvg.lnf.it
dig a  vdmpp2.ad.fvg.lnf.it
dig a  filepp.ad.fvg.lnf.it
ping -c1 vdmpp1.ad.fvg.lnf.it
ping -c1 vdmpp2.ad.fvg.lnf.it
ping -c1 filepp.ad.fvg.lnf.it
dig -x $(hostname -i)


> 
> 
> Still i get:
> 
> 	root at vdmpp2:~# mount -t nfs4 -o sec=krb5 
> vdmpp1.ad.fvg.lnf.it:/home /home
> 	mount.nfs4: access denied by server while mounting 
> vdmpp1.ad.fvg.lnf.it:/home
> 

On the NFS server post the output of 
exportfs
getfacl /home

> on server and client now i got no logs at all, even if i've added
> '-vvv' to GSS options and 'Verbosity = 5' to idmap.
The -vvv on nfs-comon or nfs-kernel-server.... 
Shows what happend with the connections, while connection. 

The idmap Verbose, if your mount was sucessfull, it shows the imapping logs.  By example from my server. : 
An ls in /home/users  ( as user root )
Oct 25 13:09:21 member3 nfsidmap[22443]: key: 0x2c4e3178 type: uid value: root at internal.example.tld timeout 600
Oct 25 13:09:21 member3 nfsidmap[22443]: nfs4_name_to_uid: calling nsswitch->name_to_uid
Oct 25 13:09:21 member3 nfsidmap[22443]: nss_getpwnam: name 'root at internal.example.tld' domain 'internal.example.tld': resulting localname 'root'
Oct 25 13:09:21 member3 nfsidmap[22443]: nfs4_name_to_uid: nsswitch->name_to_uid returned 0
Oct 25 13:09:21 member3 nfsidmap[22443]: nfs4_name_to_uid: final return value is 0
Oct 25 13:09:21 member3 nfsidmap[22444]: key: 0x324af16a type: gid value: root at internal.example.tld timeout 600
Oct 25 13:09:21 member3 nfsidmap[22444]: nfs4_name_to_gid: calling nsswitch->name_to_gid
Oct 25 13:09:21 member3 nfsidmap[22444]: nfs4_name_to_gid: nsswitch->name_to_gid returned 0
Oct 25 13:09:21 member3 nfsidmap[22444]: nfs4_name_to_gid: final return value is 0

And my (username) login in with ssh with a kerberized automounted homedir.
Oct 25 13:12:47 member3 systemd[1]: Started Session 30 of user username.
Oct 25 13:12:48 member3 nfsidmap[22518]: key: 0x2a737fa8 type: user value: 10002 timeout 600
Oct 25 13:12:48 member3 nfsidmap[22518]: nfs4_uid_to_name: calling nsswitch->uid_to_name
Oct 25 13:12:48 member3 nfsidmap[22518]: nfs4_uid_to_name: nsswitch->uid_to_name returned 0
Oct 25 13:12:48 member3 nfsidmap[22518]: nfs4_uid_to_name: final return value is 0
Oct 25 13:12:48 member3 nfsidmap[22518]: nfs4_uid_to_name: final return value is 0
Oct 25 13:12:48 member3 nfsidmap[22519]: key: 0xae3b2ad type: group value: 10000 timeout 600
Oct 25 13:12:48 member3 nfsidmap[22519]: nfs4_gid_to_name: calling nsswitch->gid_to_name
Oct 25 13:12:48 member3 nfsidmap[22519]: nfs4_gid_to_name: nsswitch->gid_to_name returned 0
Oct 25 13:12:48 member3 nfsidmap[22519]: nfs4_gid_to_name: final return value is 0

id username
uid=10002(username) gid=10000(domain users) groups=10000(domain users)  (and more)..  

! returned 0 is good ! 
nsswitch->uid_to_name returned 0 , nsswitch was able to resolve my name from uid. 

> 
> 
> > You nfs stalled, then it gets mask to prevent other errors. 
> > systemctl unmask nfs-common
> > systemctl enable nfs-common
> > If you keep hitting problems with the nfs server/client 
> 
> I've treid on client. purged 'nfs-common', reinstall, restore
> configuration on /etc/default/nfs-common and /etc/idmapd.conf, but:
> 
> 	root at vdmpp2:~# systemctl unmask nfs-common
> 	root at vdmpp2:~# systemctl start nfs-common
> 	Failed to start nfs-common.service: Unit 
> nfs-common.service is masked.
> 
> there's no /usr/sbin/rpc.gssd run, only idmap. Mount fail:
> 
> 	root at vdmpp2:~# mount -t nfs4 -o sec=krb5 
> vdmpp1.ad.fvg.lnf.it:/home /home
> 	mount.nfs4: an incorrect mount option was specified

And if you test with 
mount -t nfs4 -o sec=sys vdmpp1.ad.fvg.lnf.it:/home /home
Or 
mount -t nfs4 -o sec=krb5,vers=4.1 vdmpp1.ad.fvg.lnf.it:/home /home

Does that work or one of these work? If sys works then its not firewalling. 

I can tell more about this after the asked outputs.. 
The command : exportfs ( on the nfs server ) should tell me more. 

> 
> i've tried to run by hand with '-vvv' and i got:
> 
>  Oct 25 11:52:57 vdmpp2 rpc.gssd[13790]: doing a full rescan
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: 
> #012handle_gssd_upcall: 'mech=krb5 uid=0 service=* 
> enctypes=18,17,16,23,3,1,2 ' (nfs/clnt28)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
There are types for NFS Kerberos only and do not cover CIFS Kerberos support. 
And visa versa. 
Have you set the encryption types i suggested in /etc/krb5.conf ?
The one i posted support CIFS and NFS both. 


>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: 
> krb5_use_machine_creds: uid 0 tgtname (null)
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: Full hostname for 
> 'vdmpp1.ad.fvg.lnf.it' is 'vdmpp1.ad.fvg.lnf.it'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: Full hostname for 
> 'vdmpp2.ad.fvg.lnf.it' is 'vdmpp2.ad.fvg.lnf.it'

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for vdmpp2$@AD.FVG.LNF.IT while getting keytab entry 
> for 'vdmpp2$@AD.FVG.LNF.IT'
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
? No key table table entry??  Hmm.. 
Check this with : klist -ke | grep "vdmpp2\\$"

My servers output to compair with : 
   2 HOSTNAME1$@MY_REALM (des-cbc-crc)
   2 HOSTNAME1$@MY_REALM (des-cbc-md5)
   2 HOSTNAME1$@MY_REALM (arcfour-hmac)
   2 HOSTNAME1$@MY_REALM (aes128-cts-hmac-sha1-96)
   2 HOSTNAME1$@MY_REALM (aes256-cts-hmac-sha1-96)

Just to be sure its there. 

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for VDMPP2$@AD.FVG.LNF.IT while getting keytab entry 
> for 'VDMPP2$@AD.FVG.LNF.IT'

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for root/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT while 
> getting keytab entry for 'root/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT'


>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: Success getting 
> keytab entry for 'nfs/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT'

Ok, post the output of vdmpp2. 
>From the DC:  samba-tool spn list vdmpp2$

Looks like the local keytab is having problems. 
Run  on vdmpp2 :
klist -ke
kinit nfs/$(hostname -f) -kt /etc/krb5.keytab
klist | grep "Default principal"
That should show :
Default principal: nfs/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT

And run: kdestroy

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: 
> gssd_get_single_krb5_cred: principal 
> 'nfs/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT' 
> ccache:'FILE:/tmp/krb5ccmachine_AD.FVG.LNF.IT'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: INFO: Credentials in 
> CC 'FILE:/tmp/krb5ccmachine_AD.FVG.LNF.IT' are good until 1540497198

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: creating tcp client 
> for server vdmpp1.ad.fvg.lnf.it
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: DEBUG: port already 
> set to 2049
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: creating context 
> with server nfs at vdmpp1.ad.fvg.lnf.it
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: WARNING: Failed to 
> create krb5 context for user with uid 0 for server 
> nfs at vdmpp1.ad.fvg.lnf.it

>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: WARNING: Failed to 
> create machine krb5 context with cred cache 
> FILE:/tmp/krb5ccmachine_AD.FVG.LNF.IT for server vdmpp1.ad.fvg.lnf.it


>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: WARNING: Machine 
> cache prematurely expired or corrupted trying to recreate 
> cache for server vdmpp1.ad.fvg.lnf.it
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: Full hostname for 
> 'vdmpp1.ad.fvg.lnf.it' is 'vdmpp1.ad.fvg.lnf.it'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: Full hostname for 
> 'vdmpp2.ad.fvg.lnf.it' is 'vdmpp2.ad.fvg.lnf.it'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for vdmpp2$@AD.FVG.LNF.IT while getting keytab entry 
> for 'vdmpp2$@AD.FVG.LNF.IT'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for VDMPP2$@AD.FVG.LNF.IT while getting keytab entry 
> for 'VDMPP2$@AD.FVG.LNF.IT'
>  Oct 25 11:53:18 vdmpp2 rpc.gssd[13790]: No key table entry 
> found for root/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT while 
> getting keytab entry for 'root/vdmpp2.ad.fvg.lnf.it at AD.FVG.LNF.IT'

Side note here.
If you dont use the idmap.conf to map the computername to root.
You can set the root/spn also, root is used by the client to mount from a users request. 
I used the computer$ to root mapping in idmap.conf for this. 

See how far you get, and let me know. 

At least you questions, helped me also to locate one of my last nfs problems.
Which was in the end also a simple, missing SPN in the AD, but existed localy. 
It was there some time, and only one user noticed it (me), so very very low prio. 
But fixed now. :-)) so thank you for fresshing up my memory. :-) 

Take this slow, make sure you have tested every step before you go to the next. 


Greetz, 

Louis






More information about the samba mailing list