[Samba] multiuser/cifscreds not functioning on newer Ubuntu releases

[Nate Collins]

This is a problem that's plagued us for a while that we haven't been
able to resolve due to the lack of familiarity with cifscreds and keyring
debugging, so I thought I'd ask here to see if there's anything obviously
wrong with our setup, or if there's any cifscreds/keyring debugging
advice that could help us.

We currently use multiuser CIFS mounts on a handful of domain-joined
Ubuntu 16.04 servers to permit users to access CIFS shares with their AD
credentials. The CIFS shares in question are mounted by a separate service
account. Everything has been working as expected on the 16.04 servers:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"

$ uname -a
Linux host1 4.4.0-206-generic #238-Ubuntu SMP Tue Mar 16 07:52:37 UTC
2021 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg --list | grep cifs
ii  cifs-utils                            2:6.4-1ubuntu1.1
amd64        Common Internet File System utilities

$ keyctl show
Session Keyring
 [key] --alswrv      0     0  keyring: _ses
 [key] ---lswrv      0 65534   \_ keyring: _uid.0

$ cifscreds add share
Password:

$ ls /mnt/share/
Content
...

$ keyctl show
Session Keyring
 [key] --alswrv      0     0  keyring: _ses
 [key] ---lswrv      0 65534   \_ keyring: _uid.0
 [key] ----sw-v  [uid] [gid]   \_ logon: cifs:a:[share IP]

$ mount | grep multiuser
//share/share on /mnt/share type cifs
(rw,relatime,vers=3.0,sec=ntlmsspi,cache=strict,multiuser,domain=[domain],
uid=0,noforceuid,gid=0,noforcegid,addr=[share IP],file_mode=0755,
dir_mode=0755,nounix,serverino,mapposix,noperm,rsize=1048576,
wsize=1048576,echo_interval=60,actimeo=1)

$ grep cifs /etc/pam.d/*
/etc/pam.d/common-auth:auth    optional            pam_cifscreds.so
/etc/pam.d/common-session:session optional    pam_cifscreds.so
host=[domain controller]

$

However, with the exact same setup on an 18.04 and 20.04 server, I am
unable to access the CIFS mount after running cifscreds add share:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"

$ uname -a
Linux host2 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC
2018 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg --list | grep cifs
ii  cifs-utils                            2:6.8-1ubuntu1.1
amd64        Common Internet File System utilities

$ keyctl show
Session Keyring
 [key] --alswrv      0     0  keyring: _ses
 [key] ---lswrv      0 65534   \_ keyring: _uid.0

$ cifscreds add share
Password:

$ ls /mnt/share
ls: cannot access '/mnt/share': Permission denied

$ keyctl show
Session Keyring
 [key] --alswrv      0     0  keyring: _ses
 [key] ---lswrv      0 65534   \_ keyring: _uid.0
 [key] ----sw-v  [uid] [gid]   \_ logon: cifs:a:[share IP]

$ mount | grep multiuser
//share/share on /mnt/share type cifs
(rw,relatime,vers=3.0,sec=ntlmsspi,cache=strict,multiuser,domain=[domain],
uid=0,noforceuid,gid=0,noforcegid,addr=[share IP],file_mode=0755,
dir_mode=0755,soft,nounix,serverino,mapposix,noperm,rsize=1048576,
wsize=1048576,echo_interval=60,actimeo=1)

$

The following error appears in dmesg whenever I try to interact with
/mnt/share:

[   44.660510] CIFS VFS: signing requested but authenticated as guest
[   44.663053] CIFS VFS: SMB signature verification returned error = -13
[   44.663172] CIFS VFS: failed to connect to IPC (rc=-13)
[   44.664501] CIFS VFS: SMB signature verification returned error = -13
[   44.665361] CIFS VFS: SMB signature verification returned error = -13
[   44.665442] CIFS VFS: cifs_put_smb_ses: Session Logoff failure rc=-13

I can't tell if these mssages are significant or not -- I recall reading
that this was just a change in logging between releases. I tried using
ntlm through ntlmssp for the mount, but none of those protocols resulted
in the mountpoint being accessable by my user (they did however get rid of
the dmesg message). I've tried adjusting other mount options to no avail.
I can recreate this issue on multiple shares across multiple servers.

There's been other cifscreds strangeness that we've noticed, like access
to mounts not being immediately revoked when `cifscreds clear[all]`
is ran, but this is the main, easily reproducible issue we've been seeing
that's preventing us from using multiuser on CIFS mounts.

We use Samba DCs on version 4.12.3-8, and the CIFS share hosts are
up-to-date TrueNAS servers.