samba within nfs, locking mechanism fails

Tue Feb 26 14:36:13 MST 2013

Hi Vincenzo,

if I got you right, you have the following setup:

- local directory /test on serverA is exported with Samba and NFS
  simultaneously
- serverB imports /test from serverA via NFS an re-exports it with Samba
- SMB clients access the same files on Samba share test on both
  serverA and serverB simultaneously.

In that case, you are bound for trouble because you are
essentially serving the same file system from two different
hosts with two instances of Samba that don't know of each other,
hence lack the necessary interprocess communication (IPC) on the SMB
level. You will probably get data corruption. In theory, the
fix for this kind of problem is indeed using CTDB: it makes two
(or more) independent instances of samba one clustered instance
by providing the necessary IPC.

This would mean that you use the nfs mount to create a
kind of clustered file system. In principle this can work,
although I have to say that to my experience, using NFS
as a backend file system for clustered Samba does not
work very well. And your situation is asymmetric, which
might create extra difficulties.

I think your chances are best to use CTDB in this situation
when you turn off Samba-Posix interaction as much as possible,
i.e. setting "posix locking = no" and "kernel oplocks = no".
With Samba 4.0 you can also set "kernel share modes = no".

But you still have to solve the split brain problem which is
usually done with the CTDB recovery lock. You need to have
POSIX fcntl byte range locking working between serverA and
the nfs mount on serverB in order to configure the usual way.
This may or may not very well. I have not tried such a
configuration.

All that being said, if you seriously want to serve the same
file space with samba from more than one server simultaneously,
you should probably consider using a proper clustered file
system.

Cheers - Michael

On 2013-02-26 at 10:51 +0100, Vincenzo De Sanctis wrote:
> 2013/2/25 Vincenzo De Sanctis <vincenzo.desanctis at gmail.com>:
> > this is the case:
> >
> > serverA [ CentOs 5.6 kernel 2.6.18-238.12.1.el5.centos.plus, Samba ver. 3.5.21 ]
> > serverB [ CentOS 5.6 kernel 2.6.18-348.1.1.el5.centos.plus, Samba ver.
> > 3.6.6-0.129.el5 ]
> > clientA [ WindowsXP ]
> > clientB [ WindowsXP ]
> >
> >
> > The serverA shares via Samba the resource [test]
> >
> >
> > [global]
> >
> >    workgroup = DMIT
> >    netbios name = SAMBA
> >    server string = DMIT domain server
> >    interfaces = eth0
> >    smb ports = 445
> >    encrypt passwords = yes
> >     smb passwd file = /etc/samba/smbpasswd
> >    passdb backend = smbpasswd
> >    username map = /etc/samba/smbusers
> >    log file = /var/log/samba/pc/%m.log
> >    time server = Yes
> >    logon script = logon.bat
> >    logon path =
> >    logon drive = M:
> >    logon home = \\%L\%U
> >    domain logons = yes
> >    os level = 33
> >    preferred master = yes
> >    domain master = yes
> >    local master = yes
> >    printjob username = %M\%U
> >    hide dot files = No[netlogon]
> >    path = /etc/samba/netlogon
> > ;   max protocol = smb2
> >
> >
> > [test]
> >    comment = test
> >    path = /test
> >    read only = no
> >    writable = yes
> >    create mode = 0775
> >    force create mode = 0775
> >    directory mode = 02775
> >    force directory mode = 02775
> >    public = no
> >    oplocks = no
> >
> >
> > il serverB monta tramite client nfs la risorsa /test  (mount
> > serverA:/test /test)
> > Queta e' il semplicissimo file di configurazione smb.conf di serverB:
> >
> > [global]
> >
> >    workgroup = DMIT
> >    domain master = no
> >    domain logons = no
> >    encrypt passwords = yes
> >    security = server
> >    password server = serverA
> >    interfaces = eth0
> >    smb ports = 445
> >
> > [test]
> >    comment = test
> >    path = /test
> >    read only = no
> >    writable = yes
> >    create mode = 0775
> >    force create mode = 0775
> >    directory mode = 02775
> >    force directory mode = 02775
> >    public = no
> >    oplocks = no
> >
> >
> >
> > Now on the clientA I open an excel2003 file from \\serverA\test and on
> > clientB i open the same file but from \\serverB\test (consider that
> > test is the same directory mounter from serverA via nfs)
> >
> >
> > This is what happens:
> >
> > 1) I can open without problem the file on clientA from \\serverA\test,
> > instead I have problem to open the the same file from \\serverB\test
> > (after 5min later it goes in timeout)
> >
> >
> > 2) If I add "posix locking = no" on serverA and on serverB both
> > excel2003 files open without the locking mechanism.
> >
> > 3) I tried various combinations changing kernel oplocks, oplocks,
> > level2 oplocks, posix locking, locking, strict locking, nt acl support
> > but nothing changed.
> >
> >
> > 4) I tried to open the same file from the same serverA (from clientA
> > and from clientB) without nfs and now the locking works well (both
> > from \\serverA\test)
> >
> >
> > The strange thing is that on my company newtwork there are many old
> > samba servers (samba 2.3) and they works well within nfs.
> > The proper way to use samba like a cluser is DFS insead of NFS, but
> > now I can not consider a migration or an upgrade to all the newtork,
> > so the best way at the moment is to use nfs, like the prevoiis
> > sysadmin did.
> >
> >
> > Have you had experience about this strange case?
> > Are there known bugs regarding the new samba versions + nfs ?
> >
> > --
> > Vincenzo De Sanctis
> 
> I forgot to say that CentOS is virtualized, perhaps the problem is on
> VMware ESXi 4.1
> -- 
> Vincenzo De Sanctis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 206 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20130226/d4863a35/attachment.pgp>