issue -- cifs automounts stopped working
Xiaoli Feng
xifeng at redhat.com
Fri Jul 31 03:54:32 UTC 2020
Hello,
I'm not sure if your problem is like this. But I know there is
a bug for selinux. It will prevent to mount cifs with krb5 when
krb5 upgrade from 1.17 to 1.18 in RHEL8. Disabled selinux or
executing "setsebool samba_export_all_rw=1" will fix this issue.
https://bugzilla.redhat.com/show_bug.cgi?id=1847221
Thanks.
----- Original Message -----
> From: "Steve French via samba-technical" <samba-technical at lists.samba.org>
> To: "Michael Keane, K1MK" <mkeane at arrl.org>
> Cc: "Shyam Prasad N" <nspmangalore at gmail.com>, "CIFS" <linux-cifs at vger.kernel.org>, "samba-technical"
> <samba-technical at lists.samba.org>
> Sent: Wednesday, July 22, 2020 12:03:10 PM
> Subject: Re: issue -- cifs automounts stopped working
>
> There are two common Kerberos libraries, MIT and Heimdal (Heimdal is
> also packaged as an optional part of Samba as an alternative to MIT
> Kerberos) ... would be interesting if there is a way to work around
> this by switching to Heimdal krb5 libraries.
>
> Adding samba-technical as some of the developers on samba-technical
> know the Kerberos maintainers and may have suggestions on how to track
> down this possible regression in MIT kerberos.
>
> On Tue, Jul 21, 2020 at 10:48 PM Michael Keane, K1MK <mkeane at arrl.org> wrote:
> >
> > Tried setting KRB5_CCNAME to KEYRING:persistent:1000 with no change
> > (Also tried KRB5CCNAME as well, as that's the environment variable
> > which seems to work with kinit & klist)
> >
> > Also tried setting default_ccahe_name in /etc/krb5.conf to a legacy
> > value of
> >
> > default_ccache_name = /tmp/krb5_%{uid}
> >
> > in order to force it to use the filesystem rather than the kernel space
> > for the credentials cache. And it still fails in the same way. I think
> > that tends to makes this more likely a regression in krb5 than the kernel.
> >
> > I believe that I may have verified this by reverting from
> > krb5-*/-/1.18.2-10 to krb5-*-1.17.1-5 which resolves the issue
> >
> >
> >
> > On 07/21/2020 3:59 PM, Steve French wrote:
> > > Would setting KRB5_CCNAME (as would be expected in the past) for that
> > > process fix the issue?
> > >
> > > On Tue, Jul 21, 2020 at 2:52 PM Michael Keane, K1MK <mkeane at arrl.org>
> > > wrote:
> > >> KRB5_CCNAME is not set in the environment of the user for whom the
> > >> automount is failing.
> > >>
> > >> In /etc/krb5.conf the default is set to:
> > >>
> > >> default_ccache_name = KEYRING:persistent:%{uid}
> > >>
> > >> The calls to keyctl() in the strace for cifs.upcall indicate that it's
> > >> successfully locating the user's keyring in the kernel and that it is
> > >> able to read key data from that keying
> > >>
> > >> I haven't gone through and tried unpacking and decoding the contents of
> > >> those reads, but the reads aren't failing outright like it couldn't find
> > >> the credentials cache or the contents that it's looking for
> > >>
> > >>
> > >> On 07/21/2020 1:36 PM, Steve French wrote:
> > >>> That is plausible but I also wonder about other whether other parts of
> > >>> krb5 configuration are changed/broken eg KRB5_CCNAME environment
> > >>> variable
> > >>>
> > >>> See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
> > >>>
> > >>> On Tue, Jul 21, 2020, 12:24 Michael Keane, K1MK <mkeane at arrl.org
> > >>> <mailto:mkeane at arrl.org>> wrote:
> > >>>
> > >>> Thanks
> > >>>
> > >>> In trying to further debug this problem, I've stood up a separate
> > >>> Fedora 31 instance for testing with the result that the same
> > >>> configuration that is failing on Fedora 32 is working under Fedora
> > >>> 31
> > >>>
> > >>> Using strace it would appear the deviation occurs at the point of
> > >>> "handle_krb5_mech" where the Fedora 31 instance proceeds though
> > >>> several keyctl() calls to a successful return while the Fedora 32
> > >>> instance goes through a similar sequence of keyctl() calls but
> > >>> rather than finishing up with a final call to keyctl() to
> > >>> instantiate the key it appears that the krb5 library and/or sssd
> > >>> is going through a series of steps to locate the KDC (and failing)
> > >>> even though the KDCs have always been explicitly configured in
> > >>> /etc/krb5.conf
> > >>>
> > >>> So the root cause of this issue may not be anything in cifs.upcall
> > >>> but rather something in the sssd or kerberos that got changed as a
> > >>> result of moving from Fedora 31 to Fedora 32
> > >>>
> > >>> On 07/18/2020 8:40 PM, Steve French wrote:
> > >>>> Looks like error obtaining the key (either keyutils package not
> > >>>> installed or no Kerberos ticket found).
> > >>>>
> > >>>> This page has debug instructions for cifs.upcall. see of that
> > >>>> information is helpful.
> > >>>>
> > >>>> http://sprabhu.blogspot.com/2014/12/debugging-calls-to-cifsupcall.html?m=1
> > >>>>
> > >>>> Also try kinit as a local user and then mount with cruid mount
> > >>>> option pointing to their uid to see if that helps.
> > >>>>
> > >>>> On Fri, Jul 17, 2020, 09:04 Michael Keane, K1MK <mkeane at arrl.org
> > >>>> <mailto:mkeane at arrl.org>> wrote:
> > >>>>
> > >>>> CIFS automounts to local NAS devices have stopped working
> > >>>> recently
> > >>>>
> > >>>> Fedora 32
> > >>>>
> > >>>> kernel 5.7.8-200.fc32.x86_64
> > >>>>
> > >>>> mount.cifs version: 6.9
> > >>>>
> > >>>> auto.misc: it-share
> > >>>> -fstype=cifs,multiuser,cruid=${UID},rw,vers=default,sec=krb5
> > >>>> ://filer5/IT_Share
> > >>>>
> > >>>> dmesg:
> > >>>>
> > >>>> [ 3428.883661] fs/cifs/cifsfs.c: Devname:
> > >>>> //filer5.arrlhq.org/IT_Share
> > >>>> <http://filer5.arrlhq.org/IT_Share> flags: 0
> > >>>> [ 3428.883702] fs/cifs/connect.c: Username: root
> > >>>> [ 3428.883706] fs/cifs/connect.c: file mode: 0755 dir
> > >>>> mode: 0755
> > >>>> [ 3428.883709] fs/cifs/connect.c: CIFS VFS: in
> > >>>> mount_get_conns as
> > >>>> Xid: 12 with uid: 0
> > >>>> [ 3428.883710] fs/cifs/connect.c: UNC:
> > >>>> \\filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3428.883721] fs/cifs/connect.c: Socket created
> > >>>> [ 3428.883723] fs/cifs/connect.c: sndbuf 16384 rcvbuf
> > >>>> 131072
> > >>>> rcvtimeo 0x1b58
> > >>>> [ 3428.884126] fs/cifs/fscache.c:
> > >>>> cifs_fscache_get_client_cookie:
> > >>>> (0x000000006d2b3226/0x0000000090edec3a)
> > >>>> [ 3428.884130] fs/cifs/connect.c: CIFS VFS: in
> > >>>> cifs_get_smb_ses as
> > >>>> Xid: 13 with uid: 0
> > >>>> [ 3428.884131] fs/cifs/connect.c: Existing smb sess not
> > >>>> found
> > >>>> [ 3428.884135] fs/cifs/smb2pdu.c: Negotiate protocol
> > >>>> [ 3428.884141] fs/cifs/connect.c: Demultiplex PID: 5778
> > >>>> [ 3428.884156] fs/cifs/transport.c: Sending smb:
> > >>>> smb_len=252
> > >>>> [ 3429.010818] fs/cifs/connect.c: RFC1002 header 0x11c
> > >>>> [ 3429.010830] fs/cifs/smb2misc.c: SMB2 data length 96
> > >>>> offset 128
> > >>>> [ 3429.010832] fs/cifs/smb2misc.c: SMB2 len 224
> > >>>> [ 3429.010835] fs/cifs/smb2misc.c: length of negcontexts
> > >>>> 60 pad 0
> > >>>> [ 3429.010871] fs/cifs/transport.c: cifs_sync_mid_result:
> > >>>> cmd=0
> > >>>> mid=0 state=4
> > >>>> [ 3429.010885] fs/cifs/misc.c: Null buffer passed to
> > >>>> cifs_small_buf_release
> > >>>> [ 3429.010891] fs/cifs/smb2pdu.c: mode 0x1
> > >>>> [ 3429.010893] fs/cifs/smb2pdu.c: negotiated smb3.1.1
> > >>>> dialect
> > >>>> [ 3429.010903] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
> > >>>> 0x348 0xbb92
> > >>>> [ 3429.010907] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
> > >>>> 0x348 0x1bb92
> > >>>> [ 3429.010910] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3
> > >>>> 0x6 0x1
> > >>>> [ 3429.010912] fs/cifs/smb2pdu.c: decoding 2 negotiate
> > >>>> contexts
> > >>>> [ 3429.010914] fs/cifs/smb2pdu.c: decode SMB3.11
> > >>>> encryption neg
> > >>>> context of len 4
> > >>>> [ 3429.010916] fs/cifs/smb2pdu.c: SMB311 cipher type:1
> > >>>> [ 3429.010921] fs/cifs/connect.c: Security Mode: 0x1
> > >>>> Capabilities:
> > >>>> 0x300046 TimeAdjust: 0
> > >>>> [ 3429.010923] fs/cifs/smb2pdu.c: Session Setup
> > >>>> [ 3429.010926] fs/cifs/smb2pdu.c: sess setup type 5
> > >>>> [ 3429.010949] fs/cifs/cifs_spnego.c: key description =
> > >>>> ver=0x2;host=filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
> > >>>> [ 3429.025053] CIFS VFS: \\filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org> Send error in SessSetup
> > >>>> = -126
> > >>>> [ 3429.025056] fs/cifs/connect.c: CIFS VFS: leaving
> > >>>> cifs_get_smb_ses
> > >>>> (xid = 13) rc = -126
> > >>>> [ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
> > >>>> full_path=\\filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
> > >>>> full_path=\\filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025060] fs/cifs/connect.c: build_unc_path_to_root:
> > >>>> full_path=\\filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025062] fs/cifs/dfs_cache.c: __dfs_cache_find:
> > >>>> search path:
> > >>>> \filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025063] fs/cifs/dfs_cache.c: get_dfs_referral: get
> > >>>> an DFS
> > >>>> referral for \filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025065] fs/cifs/dfs_cache.c: dfs_cache_noreq_find:
> > >>>> path:
> > >>>> \filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> > >>>> [ 3429.025071] fs/cifs/fscache.c:
> > >>>> cifs_fscache_release_client_cookie:
> > >>>> (0x000000006d2b3226/0x0000000090edec3a)
> > >>>> [ 3429.025076] fs/cifs/connect.c: CIFS VFS: leaving
> > >>>> mount_put_conns
> > >>>> (xid = 12) rc = 0
> > >>>> [ 3429.025077] CIFS VFS: cifs_mount failed w/return code
> > >>>> = -2
> > >>>>
> > >>>> journalctl:
> > >>>>
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: key
> > >>>> description:
> > >>>> cifs.spnego;0;0;39010000;ver=0x2;host=filer5.arrlhq.org
> > >>>> <http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ver=2
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> host=filer5.arrlhq.org <http://filer5.arrlhq.org>
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ip=10.1.123.38
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: sec=1
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: uid=0
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: creduid=1000
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: user=root
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: pid=5776
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> get_cachename_from_process_env:
> > >>>> pathname=/proc/5776/environ
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> get_existing_cc: default ccache is
> > >>>> KEYRING:persistent:1000:1000
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> handle_krb5_mech: getting service ticket for
> > >>>> filer5.arrlhq.org <http://filer5.arrlhq.org>
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> cifs_krb5_get_req: unable to get credentials for
> > >>>> filer5.arrlhq.org <http://filer5.arrlhq.org>
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> > >>>> handle_krb5_mech: failed to obtain service ticket
> > >>>> (-1765328370)
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Unable to
> > >>>> obtain service ticket
> > >>>> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> > >>>> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Exit status
> > >>>> -1765328370
> > >>>>
> > >>>> $ klist
> > >>>> Ticket cache: KEYRING:persistent:1000:1000
> > >>>> Default principal: mkeane at ARRLHQ.ORG
> > >>>> <mailto:mkeane at ARRLHQ.ORG>
> > >>>>
> > >>>> Valid starting Expires Service principal
> > >>>> 07/17/2020 09:43:06 07/17/2020 19:43:06
> > >>>> krbtgt/ARRLHQ.ORG at ARRLHQ.ORG <mailto:ARRLHQ.ORG at ARRLHQ.ORG>
> > >>>> renew until 07/24/2020 09:42:57
> > >>>>
> > >>>> Filer5 is a QNAP TS-870U-RP Version 4.3.6.1070 (2019/09/10)
> > >>>> NAS device
> > >>>> but having this issue with other NAS device on LAN
> > >>>>
> > >>>> --
> > >>>> Michael Keane, K1MK
> > >>>> IT Manager
> > >>>> ARRL, The National Association for Amateur Radio™
> > >>>> 225 Main Street, Newington, CT 06111-1494 USA
> > >>>> Telephone: (860) 594-0285
> > >>>> email: mkeane at arrl.org <mailto:mkeane at arrl.org>
> > >>>>
> > >>> --
> > >>> Michael Keane, K1MK
> > >>> IT Manager
> > >>> ARRL, The National Association for Amateur Radio™
> > >>> 225 Main Street, Newington, CT 06111-1494 USA
> > >>> Telephone: (860) 594-0285
> > >>> email:mkeane at arrl.org <mailto:mkeane at arrl.org>
> > >>>
> > >> --
> > >> Michael Keane, K1MK
> > >> IT Manager
> > >> ARRL, The National Association for Amateur Radio™
> > >> 225 Main Street, Newington, CT 06111-1494 USA
> > >> Telephone: (860) 594-0285
> > >> email: mkeane at arrl.org
> > >>
> > >
> >
> > --
> > Michael Keane, K1MK
> > IT Manager
> > ARRL, The National Association for Amateur Radio™
> > 225 Main Street, Newington, CT 06111-1494 USA
> > Telephone: (860) 594-0285
> > email: mkeane at arrl.org
> >
>
>
> --
> Thanks,
>
> Steve
>
>
More information about the samba-technical
mailing list