[distcc] Local vs. NFS lock files.

[Donohue, Michael]

It looks like a common workaround to broken or missing NFS file locks is
the use of dot-lockfiles.   The claim is that file creation on NFS is
atomic, so creating a file with a well known name is a suitable
replacement for advisory locking.  Given that distcc isn't actually
writing data into the lock files, then they are just advisory locks.
Can we perhaps substitute file creation for file locks, as a more
conservative way of handling this?    Has this been considered before?

This may also solve one of our outstanding problems with lock files on
NFS, where they remain locked even though there are no processes holding
them locked.  With file creation, instead of file locking, we can look
at the date the file was created, and choose to ignore its existence if
it is too old.  With file locks, you get no other information besides
the fact that it is locked.

Michael

-----Original Message-----
From: Martin Pool [mailto:mbp at sourcefrog.net] 
Sent: Monday, April 04, 2005 6:18 AM
To: Donohue, Michael
Cc: Daniel Kegel; distcc at lists.samba.org
Subject: Re: [distcc] Local vs. NFS lock files.

On Wed, 2005-03-30 at 12:49 -0700, Donohue, Michael wrote:

> This is enough to convince me that NFS locking isn't hurting us at
> PayPal, anyway.   What exactly are the issues that arise elsewhere?

The main problem is that many people have NFS half-working, so that file
IO works but locks don't work.  This might be because NFS locks are
broken in your software (old Unix?) or because you forgot to run the
lock server (easily done on linux). Because locks and IO are handled by
different daemons and different protocols it's easy to have this and not
notice.  (In NFS4 they're a single protocol.)  Furthermore it seems that
if locks aren't working, the client OS will often just grant all locks.

In general NetApp servers seem to be the one case where NFS does work
pretty reliably... which is nice for netapp owners, but not so good for
conservative design. :-)

There is also this:

http://www.mail-archive.com/distcc@lists.samba.org/msg01325.html

If you know that NFS locks work then it's fine for you to put DISTCC_DIR
there.  It's not a great long-term solution for coordinating between
machines because obviously not everyone has a shared disk.

-- 
Martin