[Samba] CTDB and locking issues reloaded

[Giuseppe Lo Presti]

Hi there,

I'm following your list for several months, as we run a CTDB cluster of 
Samba gateways over our distributed filesystem. I decided to post my 
first message to resume an old thread 
(https://lists.samba.org/archive/samba/2016-October/204043.html), hoping 
to progress and reach some conclusion.

Essentially, we observe and can reproduce issues when attempting to lock 
a given file from two clients connecting to two DIFFERENT samba 
gateways, whereas everything works as expected when the two clients 
happen to connect to the same samba gateway.

The issue is that both clients are able to take an exclusive lock on the 
same file, as if the lock information is not propagated across the 
cluster. And indeed I could see the following on the server side:

# smbstatus -L | grep lopresti
2:8324 4179 DENY_ALL 0x3 RDWR NONE /eos home-l/lopresti/testsambalock 
Tue Apr 14 12:00:45 2020
3:29245 4179 DENY_ALL 0x3 RDWR NONE /eos home-l/lopresti/testsambalock 
Tue Apr 14 12:02:38 2020

I obtained the above by patching the smbclient code from a 4.12 branch 
following the above mentioned thread, and starting two separate sessions as:

$ ./bin/smbclient //128.142.32.44/eos
smb: \> cd home-l/lopresti
smb: \home-l\lopresti\> open testsambalock
open file \home-l\lopresti\testsambalock: for read/write fnum 1

and:

$ ./bin/smbclient //128.142.32.45/eos
smb: \> cd home-l/lopresti
smb: \home-l\lopresti\> open testsambalock
open file \home-l\lopresti\testsambalock: for read/write fnum 1

[The IPs being two of the four public IPs of our cluster]

The servers are running Samba 4.9.1-10 on CentOS 7.7. Locking settings 
are as follows:

# grep lock /etc/samba/smb.conf
     locking = yes
     posix locking = no
     strict locking = no
     oplocks = no
     level2 oplocks = no
     kernel oplocks = no

We've tried different combinations for the locking settings, following 
different (even contradicting) advices, but nothing changed with respect 
to this issue.

So the question is do you acknowledge the correct behaviour should be to 
prevent the second client from getting the lock? What would you advice 
as locking settings?

To further debug the case, I've collected full (level 10) logs of the 
two sessions on the two gateways, and can make them available, as well 
as the full configuration or anything else you may need.

Thanks in advance for your help,
Cheers,

Giuseppe