[Samba] Can Samba lock files until copy is complete?

[Nico Kadel-Garcia]

On Sat, Aug 21, 2010 at 8:17 AM, Michael Wood <esiotrot at gmail.com> wrote:
> On 20 August 2010 22:49, Whit Blauvelt <whit+samba at transpect.com> wrote:
>> Hi,
>>
>> We've got a system where staff use Samba mounts on their Windows desktops to
>> drop files into a Linux directory for further processing. Some of those
>> files are large, and take time for the file copy across Samba to complete.
>>
>> The problem is that looking at the directory from the Linux side, to see if
>> there are new files to process, the directory listing for the
>> files-copied-across-by-Samba looks the same for complete files as for
>> partial ones - same file name, same perms. We have been handling this by a
>> script which checks for files whose size hasn't increased in the last X
>> seconds. That's not only an ugly kludge, but fails if system load or network
>> congestion stalls the file transfer for too long - the partial file then
>> gets "recognized" as complete when its not, and taken for further processing
>> when it shouldn't be yet.
>>
>> There's got to be a better way. Looking at the docs I see options for
>> different ways to lock files against being written to by two users at once.
>> But I don't see anything to prevent a file from being read by one user as
>> it's being written to by another - or in the initial process of being
>> written to the directory location through copying, as in our case.
>>
>> I've tried, of course, googling the list archives. Maybe my search terms are
>> poorly chosen. This has to be a problem that's been solved before, right?
>
> I think the simplest way to do this, if you have some control over the
> clients, is to get them to upload to a temporary name and rename to
> the real name once the upload has completed.  Then get the process
> that looks for files to ignore the ones with the temporary names.
>
> If you can't control the filenames, try using something like incron to
> process the files only when they have been closed.  Or use lsof to
> check if samba still has the files open.  I don't know how reliable
> that will be, but perhaps it's another heuristic to add to your "not
> changed size for a while" check.

"rsync", available in various forms for numerous operating systems,
does this quite efficiently. It's also good at using temporary files
that are noticeable as temporary and can be flushed if the
transmission for bulky files fails.