Multiple concurrent rsyncs: an idea...
wayned at samba.org
Tue Sep 21 07:16:04 GMT 2004
On Sun, Sep 19, 2004 at 07:54:47AM -0600, Sean Reifschneider wrote:
> My thought is that it could be implemented fairly inexpensively by
> mostly relying on the temporary files that are already written. If the
> temp files were given a common extension (even if it were just common
> among a concurrent set of rsyncs), the processes could use the temporary
> files to determine which daemon works on which file.
I don't think that's the right way to add some locking. For one thing,
the file needs to be locked from the moment the generator decides that
the file needs to be transferred (which signals the start of the
transfer work), but the temp file doesn't get created until the receiver
starts to receive the update information from the sender.
I think a better approach would be to add an option that uses a lock dir
to note which inodes are currently being handled by an existing rsync
(along with the generator's pid to allow for easy cleanup of the dir if
an rsync dies). Then, another rsync that was told to use the same lock
dir could skip any changed files that had a lock-file in the lock dir.
Another alternative would be to use a real file-locking call on the file
when the generator opens the file, and leave the file open until we get
a (brand-new) signal from the receiver that the file has been finished.
If someone wants to work up something like this, let me know. (It would
go into the patches dir for now.)
More information about the rsync