[Samba] Can Samba lock files until copy is complete?

[Michael Wood]

On 20 August 2010 22:49, Whit Blauvelt <whit+samba at transpect.com> wrote:
> Hi,
>
> We've got a system where staff use Samba mounts on their Windows desktops to
> drop files into a Linux directory for further processing. Some of those
> files are large, and take time for the file copy across Samba to complete.
>
> The problem is that looking at the directory from the Linux side, to see if
> there are new files to process, the directory listing for the
> files-copied-across-by-Samba looks the same for complete files as for
> partial ones - same file name, same perms. We have been handling this by a
> script which checks for files whose size hasn't increased in the last X
> seconds. That's not only an ugly kludge, but fails if system load or network
> congestion stalls the file transfer for too long - the partial file then
> gets "recognized" as complete when its not, and taken for further processing
> when it shouldn't be yet.
>
> There's got to be a better way. Looking at the docs I see options for
> different ways to lock files against being written to by two users at once.
> But I don't see anything to prevent a file from being read by one user as
> it's being written to by another - or in the initial process of being
> written to the directory location through copying, as in our case.
>
> I've tried, of course, googling the list archives. Maybe my search terms are
> poorly chosen. This has to be a problem that's been solved before, right?

I think the simplest way to do this, if you have some control over the
clients, is to get them to upload to a temporary name and rename to
the real name once the upload has completed.  Then get the process
that looks for files to ignore the ones with the temporary names.

If you can't control the filenames, try using something like incron to
process the files only when they have been closed.  Or use lsof to
check if samba still has the files open.  I don't know how reliable
that will be, but perhaps it's another heuristic to add to your "not
changed size for a while" check.

Otherwise, you'll have to do as Jeremy suggests, of course.

-- 
Michael Wood <esiotrot at gmail.com>