Treating share modes as advisory when set by Mac clients

[Brian Campbell]

On Mon, Mar 12, 2012 at 3:53 PM, Jeremy Allison <jra at samba.org> wrote:

> On Mon, Mar 12, 2012 at 03:27:01PM -0400, Brian Campbell wrote:
> > I work on network attached storage for video editing. Our customers use
> > both Mac and Windows workstations, and on the Mac, they generally use
> > Thursby Software's DAVE as a client, as it works much better than the
> > built-in smbfs on Mac OS X.
> >
> > On Mac OS X, when opening a file, you can request a shared or exclusive
> > lock on the file using the O_SHLOCK or O_EXLOCK flags in the open() call.
> > On local filesystems, these are implemented as locks with flock()
> semantics.
> >
> > Thursby recently added support in DAVE to implement these flags using
> share
> > modes (DENY_WRITE/DENY_ALL). They apparently did this because Microsoft
> > Word depends on these locks working, and was getting confused when two
> > people could edit the same file at the same time. This also appears to be
> > how smbfs on Mac OS X implements these open() flags.
> >
> > This has caused a problem for our customers using Final Cut (and possibly
> > some other video editing software). When Final Cut writes files out in
> > certain situations (renders, and log and transfer), it opens the file
> with
> > the O_EXLOCK flag, and holds the file open with that exclusive lock even
> > after it's done writing, until you close out the entire project. Locally,
> > that doesn't cause any problems, since the lock is advisory and most
> > software that needs to read the file doesn't try to obtain any locks on
> the
> > file.
> >
> > With DAVE now implementing these locks as share modes, which are
> mandatory
> > and enforced by the server, we're running into trouble. When Final Cut
> > writes the file out, and keeps the file open with the lock, this means
> the
> > file continues to have the share mode DENY_ALL on the server, which
> > prevents anyone else from reading the file.
> >
> > I was wondering about whether it would be possible to recognize this
> > situation on the server, and when we see a share mode being set by DAVE
> or
> > Mac OS X's smbfs, to interpret that as an advisory lock, and not a
> > mandatory lock.
> >
> > How to do this is a bit of an open question. You would probably still
> want
> > share modes to be mandatory for Windows clients. But you would want them
> to
> > be advisory for Mac clients. I don't believe other Unix clients have
> locks
> > that can be set on open(), and I haven't seen any other clients that
> > actually implement flock(). I was thinking that you could add an extra
> bit
> > somewhere in share_mode_entry indicating whether a share mode is
> mandatory
> > or advisory. Windows clients would open files with a mandatory share
> mode.
> > Mac clients would open them with an advisory share mode. If a file has a
> > mandatory share mode that is incompatible with your requested access, it
> > would fail. If a file has an advisory share mode, opening a file will
> only
> > fail if you are requesting a mandatory or advisory share mode that is
> > incompatible with the advisory share mode, but will not fail if you are
> > opening the file with a share mode of DENY_NONE even if the type of
> access
> > you are requesting would be incompatible with the share mode were it
> > mandatory.
> >
> > Does this make any sense? Does it sound like something that would be
> > reasonable to implement? Are there any gotchas here that I'm missing? I'm
> > completely new the the Samba code base and SMB, so please feel free to
> > educate me if I'm missing something.
>
> So there is a very old parameter called "share modes = [yes|no]" that
> is share-specific that was designed to do what you want (i.e. you'd
> redirect all the Mac clients to a specific share with this set).
>
> Having said that it hasn't really been tested or used for many
> a year, and now only exists as one check inside smbd/open.c
> that always causes open_mode_check() to return true (i.e. don't
> prevent opens) - which is almost certainly wrong and will break
> other logic.
>
> What you'd need is to resurrect that code correctly.
>

I was thinking of doing something a bit more nuanced than turning share
modes on or off. My proposal was to make share modes advisory, at least for
certain clients. That would give the expected semantics for Mac clients; if
they didn't try to obtain any locks, they could still open the file. But if
they tried to obtain shared or exclusive locks, that would fail depending
on whether their lock was compatible with any existing share modes.

Turning off share modes entirely may cause problems for anyone who actually
depends on the advisory locking semantics that these locks have on the Mac.
Now, that may be OK for us; as far as I know, none of our customers had any
problems due to open locks not working before Thursby implemented this
feature. But I was thinking that in general, allowing for share modes to be
advisory for Mac clients would get you closer to the semantics that Mac
applications are actually expecting, instead of being off entirely.

And yeah, I tried using the "share modes = no" parameter, and it does not
appear to work. I could resurrect that feature, and that would probably
work for our use case. We have also asked Thursby to give us a build of
DAVE that allows you to toggle off the implementation of open locks as
share modes. Both of these will put us back to where we were before, with
open locks not actually doing anything on the server. I was just hoping to
do a little better than that, and implement the semantics that Mac clients
actually expect.


> The real question is why is Dave keeping files open beyond
> their application open lifetime ? That would cause problems
> for lots of software.
>

DAVE is not keeping the files open beyond their lifetime (actually, I
believe that one version did for certain files created by Avid, but they
have fixed that bug now). It is Final Cut that is keeping the file open. If
you have a project open in Final Cut, and do a "log and transfer" (import a
video clip from your camera, tape, or memory card), the new file that it
creates upon import is opened with an exclusive lock. Final Cut keeps that
file open as long as you have the project open, with the exclusive lock the
entire time, even once it is finished doing any writing. If you close the
project, it will close all of its open files, and when you open it up again
it will open the media clip without any locks; it's just a problem with
files which were open for writing with an exclusive lock and stay open with
said lock even after Final Cut is done writing them.

-- Brian